We do not support or recommend the upgrade of a production AIOps deployment to an Edge release.
Backup Existing System
- Backup $MOOGSOFT_HOME.
- Take a snapshot (for VMs).
- Backup MySQL.
Stop Services and Processes
LAMs. This can be done by stopping their service scripts:
Alternatively, stop their running processes using:
Ensure all the above processes have been stopped before continuing:
Disable the events_analyser from running during the upgrade process:
Run the following command to comment out the relevant lines in crontab:
Run the following command to stop any running events_analyser processes:
Important: Stop all RabbitMQ nodes before performing this step:
If you are on RHEL6/CentOS6, run the following commands to switch the current version of Erlang to v20.1.7. For offline upgrades see the information panel at the bottom of this section:
If you are on RHEL7/CentOS7 please run the following commands to switch the current version of erlang to v20.1.7. For offline upgrades see the information panel at the bottom of this section:
If the upgrade is being done using the offline tarballs, replace the URL to the erlang RPM in the erlang.sh script with 'erlang-20.1.7'
Target the AIOps Edge yum Repository
To upgrade to an AIOps Edge Release, you must configure the baseurl AIOps yum repository to point to the edges endpoint:
- Configure the
Contact email@example.com with questions rearding regarding access to
Single host from a remote repository:
Single host from same machine (assumes the RPMs have been downloaded locally and are in the current folder):
- Distributed installs: select the relevant update commands to run depending on which package(s) are installed on the current host. For example:
Merge Latest Config File Changes
'.rpmsave' versions of files will need to be manually merged/compared with the new versions of those files (any new properties should be added to the older versions of the files).
An example of the process to do this:
- Rename the new versions of the files (without the .rpmsave extension) to end with '.bak'.
- Merge the .rpmsave file with the new '.bak' file by adding new properties/config where needed so the structure matches the new version of the file.
- Rename the .rpmsave file to delete the '.rpmsave' extension.
[Optional - but recommended] Update /etc/my.cnf with new guidance on innodb-buffer-pool performance tuning:
Upgrade the AIOps schema
Scripted (suitable for single host upgrade):
OR Manual (replace 'moogdb' and 'moog_reference' database names as appropriate):
Upgrade the Apache-Tomcat Webapps
Scripted (recommended for standard/single-host installs):
- If tomcat startup/shutdown failures are reported (such as: Web apps are not rebuilding, please check catalina.out) when running the above command then tomcat should be forcibly shutdown (using kill -9 on the apache-tomcat java PID) then the step repeated
Alternative manual steps:
Check the apache-tomcat java process is no longer running:
Delete the existing extracted webapps:
Copy in the new webapp WAR files (and replace the old ones):
The following command needs to be run to extract Integrations and SolutionPaks:
OPTIONAL: If the images for some Integrations are not visible post-upgrade, the following script can be run to replace the broken image links in the database:
Restart moogfarmd and Verify Services
Check all required services are running and start moogfarmd by entering these commands:
Re-enable the events_analyser cronjobs using the following command:
Any previously running integrations can be restarted using the following command:
Manual Verification of Upgrade
Carry out the following basic steps to ensure the upgrade to 6.3.1 was successful:
- Ensure the $MOOGSOFT_HOME/etc/VERSION.txt file contains just "6.3.1"
- Check that the last entry in the moogdb.schema_upgrades table in MySQL is the 6.3.1 upgrade entry
- Check that the UI login page displays "Version 6.3.1" at the top
- Check that the UI "Support Information" window correctly indicates the current version as 6.3.1 and shows the correct schema upgrade history
- Integrations: If any of the Integrations images are not displaying, please see the 'Integrations' section above and run the final optional step/command.
Automated Verification of Upgrade
Confirm all Moog files have been deployed correctly within $MOOGSOFT_HOME using this utility:
Confirm all apache-tomcat files have been deployed correctly within $MOOGSOFT_HOME using this utility:
Confirm that all the steps are successful. If there are some webapp differences then this can be resolved using moog_init_ui.sh -w which will extract the webapps with the right files.
If there are only hostname-related differences in the apache-tomcat server.xml file then these can be safely ignored.
Confirm the schema has been upgraded successfully:
Confirm that all the steps are successful.
If there are some schema differences, they may be valid (e.g.: custom_info related) but if there are more substantial differences then it would be worth investigating further to check if all the pre-requisite upgrade scripts have been applied in the right order: