Patch Moogsoft AIOps Tarball

Ensure the patch tarball is available to each server being patched:

Stop all processes on each host:

$MOOGSOFT_HOME/bin/utils/process_cntl moog_farmd stop;
$MOOGSOFT_HOME/bin/utils/process_cntl apache-tomcat stop;
kill -9 $(ps -ef | grep java | grep lam | awk '{print $2}') 2>/dev/null;
(crontab -l | sed -e 's/^\(.*events_analyser.*\)$/#\1/') | crontab -
(crontab -l | sed -e 's/^\(.*keepalive.*\)$/#\1/') | crontab -
ps -ef | grep java | egrep 'events_analyser|keepalive' | awk '{print $2}' | xargs kill 2>/dev/null;
$MOOGSOFT_HOME/bin/utils/process_cntl nginx stop;

FOR DEPLOYMENTS BEING UPGRADED FROM 7.3.1.6 OR EARLIER ONLY:: Perform the database upgrade procedure documented here: Percona Cluster 5.7 Tarball Minor Version Upgrade

Gather the required variables:

OLD_DIST_PATH=$(readlink -f $MOOGSOFT_HOME/bin|awk -F '/' 'sub(FS $NF,x)');
CERT_REAL_PATH_PEM=$(readlink -f $(grep -h 'ssl_certificate ' $OLD_DIST_PATH/cots/nginx/config/conf.d/moog-ssl.conf|awk '{print $2}'|tr -d ';'));
CERT_REAL_PATH_KEY=$(readlink -f $(grep -h 'ssl_certificate_key' $OLD_DIST_PATH/cots/nginx/config/conf.d/moog-ssl.conf|awk '{print $2}'|tr -d ';'));
CERT_PATH_PEM=$(grep -h 'ssl_certificate ' $OLD_DIST_PATH/cots/nginx/config/conf.d/moog-ssl.conf);
CERT_PATH_KEY=$(grep -h 'ssl_certificate_key' $OLD_DIST_PATH/cots/nginx/config/conf.d/moog-ssl.conf);

Place the downloaded tarball file into an empty directory called patch_directory and execute the following commands:

VERSION=7.3.1.8;
cd patch_directory
tar -xf moogsoft-aiops-${VERSION}.tgz
bash moogsoft-aiops-install-${VERSION}.sh

FOR DEPLOYMENTS BEING UPGRADED FROM 7.3.1.3 OR EARLIER ONLY: Upgrade the apache-tomcat:

$MOOGSOFT_HOME/bin/utils/moog_init_ui.sh -tfz $($MOOGSOFT_HOME/bin/utils/moog_config_reader -k mooms.zone)

Identify the config files that have changed between the previously installed version and upgraded version. For example:

Run the following command to find files with differences:

diff -rq $OLD_DIST_PATH/config $MOOGSOFT_HOME/dist/7.3.1.8/config | grep -i 'differ'

Update files in $MOOGSOFT_HOME/config with any changes introduced in the upgraded version of these files.

There is a change in “controller.conf” and "security.conf". Identify the config file that has changed between the previously installed version and this version. For example:

diff -rq $OLD_DIST_PATH/config $MOOGSOFT_HOME/dist/7.3.1.8/config | grep -i 'differ'

Update files in $MOOGSOFT_HOME/config with any changes introduced in the v8.0.0.5 version of these files.

Identify the bot files that have changed between the previously installed version and upgraded version. For example:

Run the following command to find files with differences:

diff -rq $OLD_DIST_PATH/bots $MOOGSOFT_HOME/dist/7.3.1.8/bots | grep -i 'differ'

Update files in $MOOGSOFT_HOME/bots with any changes introduced in the upgraded version of these files.

FOR DEPLOYMENTS BEING UPGRADED FROM 7.3.1.4 OR EARLIER ONLY: Execute following schema update; to enable auto-pruning of the process_logs table:

mysql -u root moogdb -e "UPDATE system_config SET properties = JSON_SET(properties, '$.aged_logs_period', 90) WHERE config_type = 'Splitter';"

FOR DEPLOYMENTS BEING UPGRADED FROM 7.3.1.4 OR EARLIER ONLY:

Execute the following schema change to add tables new in this version:

mysql -u root moogdb < $MOOGSOFT_HOME/etc/moog/moog_sigdb/tables/process_logs.sql
mysql -u root moogdb < $MOOGSOFT_HOME/etc/moog/moog_sigdb/tables/split_prc_alerts.sql

FOR DEPLOYMENTS BEING UPGRADED FROM 7.3.1.0 OR EARLIER ONLY: Confirm that the MySQL variable log_bin_trust_function_creators is enabled using the following command:

mysql -e "show variables like '%log_bin_trust_function_creators%';"

The 'log_bin_trust_function_creators' value needs to be ON or 1. If the value is OFF or 0, one of the following steps must be followed depending on how your database is deployed:

Apply schema changes by running below command: Replace 'moogdb' and 'historic_moogdb' if the database names are different in the current deployment.

mysql -u root moogdb -e "ALTER TABLE alerts ADD KEY idx_max_severity(alert_id, severity);"
mysql -u root historic_moogdb -e "ALTER TABLE alerts ADD KEY idx_max_severity(alert_id, severity);"

FOR DEPLOYMENTS BEING UPGRADED FROM 7.3.1.1 OR EARLIER ONLY:

FOR DEPLOYMENTS BEING UPGRADED FROM 7.3.1.7 OR EARLIER ONLY:

Execute the following schema change to add tables new in this version:

mysql -u root moogdb < $MOOGSOFT_HOME/etc/moog/moog_sigdb/tables/active_splitter.sql

FOR ALL DEPLOYMENTS:

Refresh all stored procedures (provide database password when prompted):

DB_VERSION=$($MOOGSOFT_HOME/bin/utils/moog_mysql_client -e "select moog_version from schema_version" -B --skip-column-names 2>/dev/null | sed 's/\(.....\).*/\1/' | sed '/^7.3.*/! s/.*/7.3.0/');
$MOOGSOFT_HOME/bin/utils/moog_db_auto_upgrader -t ${DB_VERSION} -u root

Rebuild Apache Tomcat and update the Nginx configuration:

$MOOGSOFT_HOME/bin/utils/moog_init_ui.sh -tnfwz $($MOOGSOFT_HOME/bin/utils/moog_config_reader -k mooms.zone) --accept-eula
cp -f $CERT_REAL_PATH_PEM $MOOGSOFT_HOME/dist/7.3.1.8/cots/nginx/ssl/
cp -f $CERT_REAL_PATH_KEY $MOOGSOFT_HOME/dist/7.3.1.8/cots/nginx/ssl/
sed -i "s|.*ssl_certificate .*|${CERT_PATH_PEM}|" $MOOGSOFT_HOME/dist/7.3.1.8/cots/nginx/config/conf.d/moog-ssl.conf
sed -i "s|.*ssl_certificate_key.*|${CERT_PATH_KEY}|" $MOOGSOFT_HOME/dist/7.3.1.8/cots/nginx/config/conf.d/moog-ssl.conf
$MOOGSOFT_HOME/bin/utils/process_cntl nginx restart

FOR DEPLOYMENTS BEING UPGRADED FROM 7.3.0.1 OR EARLIER ONLY:

Re-start non-UI integrations or LAMs

Verify the remaining processes are working properly and update the crontab:

$MOOGSOFT_HOME/bin/utils/process_cntl apache-tomcat status;
$MOOGSOFT_HOME/bin/utils/process_cntl nginx status;
$MOOGSOFT_HOME/bin/utils/process_cntl elasticsearch status;
(crontab -l | sed -e 's/^\#\+\(.*events_analyser.*\)/\1/') | crontab -
(crontab -l | sed -e 's/^\#\+\(.*keepalive.*\)/\1/') | crontab -

Validate the patch:

$MOOGSOFT_HOME/bin/utils/moog_install_validator.sh
$MOOGSOFT_HOME/bin/utils/tomcat_install_validator.sh
$MOOGSOFT_HOME/bin/utils/moog_db_validator.sh

Re-install the latest 'Addons' pack: /AIOps.7.3.0/install-moogsoft-add-ons.html

If no errors from the validation, restart moogfarmd:

$MOOGSOFT_HOME/bin/utils/process_cntl moog_farmd start;

Clear the browser cache and log in to the UI.

Restart any event feeds if they were stopped.