RPM - Upgrade UI components
Follow these steps to perform an RPM upgrade on the Moogsoft AIOps UI components to v7.3.x from v7.0.x, v7.1.x, or 7.2.x:
Nginx
Apache Tomcat
UI integrations
These components should always reside on the same server.
Refer to Upgrade Moogsoft AIOps for general information and upgrade instructions for other components and versions.
Stop services and processes
Run the following command as root to stop the default tomcat service on any servers with the moogsoft-ui package installed and where the Apache Tomcat service is running. Change the service name if you are not using the default.
service apache-tomcat stop
Run the following commands as root to query/stop the default LAM/integrations services on any servers with the moogsoft-integrations/moogsoft-integrations-ui package installed and where the LAMs/integrations are running.
Check for running LAM and integration processes and stop them using the relevant service scripts:
systemctl status | grep lamd service <lam_service_name> stop
Run the following command to stop any remaining active LAM and integration processes:
kill -9 $(ps -ef | grep java | grep _lam | awk '{print $2}') 2>/dev/nullNote
Complete the Elasticsearch steps below if you have installed Elasticsearch on the same server as your UI components. Moogsoft AIOps recommends that you move Elasticsearch to your Core server (the server running Moogfarmd) to optimize index performance.
Delete the Elasticsearch indexes
Run this command on the moogsoft-search/Elasticsearch server to remove the old Elasticsearch indexes:
curl -XDELETE 'http://localhost:9200/alerts/' && curl -XDELETE 'http://localhost:9200/situations/'
If the command completes successfully, the following message is displayed:
{"acknowledged":true}{"acknowledged":true}Modify the Elasticsearch repo
Note
You can skip this section if you are following the 'Offline RPM' upgrade process, as the Elasticsearch package is obtained from the local Yum repository instead.
Run the following command to modify the Elasticsearch Yum repository to point to v6 instead of v5:
sed -i 's/5.x/6.x/g' $(grep 'artifacts.elastic' /etc/yum.repos.d/* | awk -F: '{ print $1 }' | sort -u | head -1)
Upgrade Moogsoft AIOps
To upgrade Moogsoft AIOps, run the upgrade command below that corresponds to your chosen upgrade mechanism.
If you have already run this step on the current host as part of this upgrade (for single-host upgrade for example), you can skip this step.
If you are using a remote or offline Yum repository, run the following command on every host where a Moogsoft AIOps RPM package is installed:
yum -y upgrade $(rpm -qa --qf '%{NAME}\n' | grep moogsoft | sed 's/$/-7.3.0/')If you are using downloaded RPM files on a host, run the following command from the location where the files are installed:
yum -y upgrade moogsoft-*7.3.0*.rpm
Merge the latest configuration file changes
Note
In Moogsoft AIOps v7.3.x, the Sigalisers (Cookbook and Tempus), and merge groups (default and custom) are imported into the database by default, enabling you to to access and configure them via the UI and API. The migration occurs once when Moogfarmd is restarted. A new flag has been added to the 7.3.x version of moog_farmd.conf that you can use to prevent the migration from taking place (file_only_config=true). If this flag is missing or is set to false, Moogfarmd attempts to perform the import when it starts.
Manually merge and compare .rpmsave versions of files with the new versions of those files. Add any new properties to the older versions of the files.
Note
If you have already completed this step previously (as part of this upgrade process) on the current host, you can skip this step.
The config and bot files from the previous version should not be copied on top of (replace) the new version of those files in 7.3.x, as they are not always forwards-compatible, and some config/bot lines need to be added for the new version to work.
To find files that have been changed, moved or deleted, run these commands:
find $MOOGSOFT_HOME -name '*.rpmsave' find /etc/init.d/ -name '*.rpmsave'
For example, the following command displays the differences in the new version of the system.conf file:
diff -u $MOOGSOFT_HOME/config/system.conf $MOOGSOFT_HOME/config/system.conf.rpmsave
Follow this process to merge the file differences:
Rename the new versions of the files, without the
.rpmsaveextension, to end with.bak.Merge the
.rpmsavefile with the new.bakfile by adding new properties/configuration where needed (from the new version of the file into the old version), so the structure matches the new version of the file.Rename the
.rpmsavefile to remove the.rpmsaveextension.
Update JVM to use Java 11
Note
Only perform this step if you are upgrading from Moogsoft AIOps v7.0.x or v7.1.x.
Note
If you have already completed this step previously (as part of this upgrade process) on the current host, you can skip this step.
On each server with a Moogsoft AIOps RPM package installed, run the following command to replace the /usr/java/latest symlink so it points at the new JDK11 JAVA_HOME directory:
source $MOOGSOFT_HOME/bin/utils/moog_init_functions.sh
If there are non-Moogsoft AIOps packages on this server that do not support JDK11, you must update those applications to use a different JAVA_HOME symlink (not /usr/java/latest).
To confirm this has worked, run the following command:
$JAVA_HOME/bin/java -version
It should return:
openjdk version "11.0.2" 2019-01-15 LTS
You can also use the 'alternatives' command to point the system 'java' shortcut to the new version:
alternatives --config java
Remove references to the old MySQL connector
Note
Only perform this step if you are upgrading from Moogsoft AIOps v7.0.x or v7.1.x.
Note
If you have already completed this step previously (as part of this upgrade process) on the current host, you can skip this step.
The MySQL connector is upgraded in this release.
The original connector may be used by the External Database module in the current deployment, configured in $MOOGSOFT_HOME/config/moog_external_db_details.conf.
If this file is configured in the current deployment, update it to reference the new mariadb connector here: $MOOGSOFT_HOME/lib/cots/mariadb-java-client-2.4.0.jar.
Change ownership of Apache Tomcat folders
Apache Tomcat is now run as the 'moogsoft' system user, which requires a change in ownership for the folders previously owned by Apache Tomcat.
Run the following commands to change ownership:
chown -R moogsoft:moogsoft /var/lib/moogsoft chown -R moogsoft:moogsoft /var/run/apache-tomcat chown -R moogsoft:moogsoft $MOOGSOFT_HOME/etc/saml
If SAML SSO is in use in the deployment, the IDP metadata file specified in $MOOGSOFT_HOME/config/security.conf needs to be readable by the 'moogsoft' system user. Use an appropriate chmod/chown command to ensure the readability and ownership is correct.
Upgrade Apache Tomcat and Nginx
Moogsoft AIOps v7.3.0 ships with Apache Tomcat version 9.0.22.
Note
Moogsoft AIOps v7.3.0 no longer runs Apache Tomcat as the 'tomcat' UNIX user. When you follow the instructions below, the new version of Apache Tomcat is deployed to run as the 'moogsoft' user instead. As more threads and processes are now used by the moogsoft UNIX system user, you may need to increase ulimits for this user.
Run the following commands in this section on the server with the moogsoft-ui RPM package installed on it.
Stop Apache Tomcat on any servers where it is running:
service apache-tomcat stop; ps -ef | grep java | grep tomcat | awk '{print $2}' | xargs kill -9 2>/dev/nullRemove the existing Apache Tomcat:
rm -rf /etc/init.d/apache-tomcat; rm -rf $APPSERVER_HOME rm -rf /usr/share/apache-tomcat
Back up the Nginx configuration files and any certificates. Copy the files in the following location to another location before continuing:
/etc/nginx/. This folder is based on the default Nginx installation location.Deploy the new version of Apache Tomcat and Nginx:
$MOOGSOFT_HOME/bin/utils/moog_init_ui.sh -tfn
If you made any changes to the original Apache Tomcat service script, apply the same changes to the new version.
Update
/etc/nginx/conf.d/moog-ssl.confwith the locations of any certificates used and then restart Nginx:service nginx restart
To continue with the upgrade, see RPM v7.3.x - Upgrade Core components.