RPM - Upgrade database components

Follow these steps to perform an RPM upgrade on the Moogsoft Enterprise database components to v8.0.x from v7.1.x, v7.2.x, or 7.3.x:

Moogsoft Enterprise databases
MySQL/Percona

Refer to Upgrade Moogsoft Enterprise for general information and upgrade instructions for other components and versions.

Upgrade Moogsoft Enterprise

To upgrade Moogsoft Enterprise, run the upgrade command below that corresponds to your chosen upgrade mechanism.

If you have already run this step on the current host as part of this upgrade (for single-host upgrade for example), you can skip this step.

If you are using a remote or offline Yum repository, run the following command on every host where a Moogsoft Enterprise RPM package is installed:
```
yum -y upgrade $(rpm -qa --qf '%{NAME}\n' | grep moogsoft | sed 's/$/-8.0.0.1/')
```
If you are using downloaded RPM files on a host, run the following command from the location where the files are installed:
```
yum -y upgrade moogsoft-*8.0.0.1*.rpm
```

Update my.cnf for MySQL

You must update the /etc/my.cnf file to reflect the new variables for MySQL. Run the following commands to update this file. The following command assumes the Database my.cnf file is under the /etc/ directory.

sed -i '/\[mysqld\]/a log_bin_trust_function_creators = 1\nthread_stack = 524288\n' /etc/my.cnf

Restart MySQL:

service mysqld restart

Upgrade Percona to v5.7.28

Note

Use the following instructions if the database deployed is Percona instead of MySQL Community. Instructions for MySQL Community follow this section.

You should upgrade Percona to v5.7.28 to address a number of bugs and security vulnerabilities.

You should back up your database before upgrading.

Check if Percona is configured to run with --gtid-mode=ON using the following command in the MySQL CLI:
```
show variables like 'gtid_mode';
```
Remember what the value is because you will need it later.

Download and install the Percona packages using the appropriate step below:

Stop Percona:
```
service mysqld stop
```

If the server running Percona has internet access, download and install the packages from the Percona Yum repository:

yum -y upgrade Percona-XtraDB-Cluster-shared-compat-57-5.7.28 \
Percona-XtraDB-Cluster-client-57-5.7.28 \
Percona-XtraDB-Cluster-server-57-5.7.28 \
Percona-XtraDB-Cluster-shared-57-5.7.28

If the server running Percona does not have internet access, download the packages on a server that does have internet access and copy the files to the server running Percona:

curl -L -O http://repo.percona.com/percona/yum/release/7/RPMS/x86_64/Percona-XtraDB-Cluster-shared-57-5.7.28-31.41.1.el7.x86_64.rpm
curl -L -O http://repo.percona.com/percona/yum/release/7/RPMS/x86_64/Percona-XtraDB-Cluster-client-57-5.7.28-31.41.1.el7.x86_64.rpm
curl -L -O http://repo.percona.com/percona/yum/release/7/RPMS/x86_64/Percona-XtraDB-Cluster-server-57-5.7.28-31.41.1.el7.x86_64.rpm
curl -L -O http://repo.percona.com/percona/yum/release/7/RPMS/x86_64/Percona-XtraDB-Cluster-shared-compat-57-5.7.28-31.41.1.el7.x86_64.rpm
curl -L -O http://repo.percona.com/percona/yum/release/7/RPMS/x86_64/percona-xtrabackup-24-2.4.19-1.el7.x86_64.rpm

Then manually install the packages:

yum -y upgrade Percona-*5.7.28*.rpm

Restart Percona:
```
service mysqld restart
```
If the gtid-mode was OFF (based on the command run earlier in the upgrade), run the MySQL upgrade utility. Provide the MySQL root password when prompted, or press Enter if you have not set a password.
```
mysql_upgrade -u root -p
```
if the gtid-mode was ON, you do not need to do this step.
See the MySQL documentation on restrictions on replication with GTIDs for more information.
Restart Percona to save any changes to system tables:
```
service mysqld restart
```

See the Upgrading MySQL documentation for more information.

Upgrade MySQL Community to v5.7.28

Note

Use the following instructions if the database deployed is MySQL Community instead of Percona.

You should upgrade MySQL to 5.7.28 to address a number of bugs and security vulnerabilities.

Before you start the upgrade process, you should back up your database.

Check if MySQL is configured to run with --gtid-mode=ON using the following command in the MySQL CLI:
```
show variables like 'gtid_mode';
```
Remember what the value is because you will need it later.
Stop mysql:
```
service mysqld stop
```

Download and install the MySQL packages using the appropriate step below:

If the server running MySQL has internet access, download and install the packages from the MySQL Yum repository:

yum -y upgrade mysql-community-libs-5.7.28 \
		mysql-community-libs-compat-5.7.28 \
		mysql-community-server-5.7.28 \
		mysql-community-common-5.7.28 \
		mysql-community-client-5.7.28

If the server running MySQL does not have internet access, download the packages on a server that does have internet access and copy the files to the server running mySQL:

curl -L -O https://repo.mysql.com/yum/mysql-5.7-community/el/7/x86_64/mysql-community-libs-5.7.28-1.el7.x86_64.rpm;
curl -L -O https://repo.mysql.com/yum/mysql-5.7-community/el/7/x86_64/mysql-community-libs-compat-5.7.28-1.el7.x86_64.rpm;
curl -L -O https://repo.mysql.com/yum/mysql-5.7-community/el/7/x86_64/mysql-community-server-5.7.28-1.el7.x86_64.rpm;
curl -L -O https://repo.mysql.com/yum/mysql-5.7-community/el/7/x86_64/mysql-community-common-5.7.28-1.el7.x86_64.rpm;
curl -L -O https://repo.mysql.com/yum/mysql-5.7-community/el/7/x86_64/mysql-community-client-5.7.28-1.el7.x86_64.rpm;

Then manually install the packages:

yum -y upgrade mysql-*5.7.28*.rpm

Restart mysql:
```
service mysqld restart
```
If the gtid-mode was OFF (based on the command run earlier in the upgrade), run the MySQL upgrade utility. Provide mysql root password when prompted or just press Enter if no password set.
```
mysql_upgrade -u root -p
```
if the gtid-mode was ON, you do not need to do this step.
See the MySQL documentation on restrictions on replication with GTIDs for more information.
Restart MySQL to save any changes to system tables:
```
service mysqld restart
```

See the Upgrading MySQL documentation for more information.

Merge the latest configuration file changes

Note

In Moogsoft Enterprise v7.3.x and 8.0.x, the Cookbooks, Tempus, and merge groups (default and custom) are imported into the database by default, enabling you to to access and configure them via the UI and API. The migration occurs once when Moogfarmd is restarted.

A file_only_config=true flag has been added to the 7.3.x and 8.0.x versions of moog_farmd.conf that you can use to prevent the migration from taking place. If this flag is missing or is set to false, Moogfarmd attempts to perform the import when it starts.

Note

If the file_only_config flag is set to true, UI-based Cookbooks will not run.

The following moolets are no longer supported in v8.0.x and should be removed from the moog_farmd.conf file as part of the upgrade:

Sigaliser Classic
Nexus
Speedbird
AlertRootCause

Version specific config file differences:

v7.1.x-v7.2.x
- $MOOGSOFT_HOME/config/system.conf
  - message_persistence is now enabled by default
- $MOOGSOFT_HOME/config/security.conf
  - The 'Google' realm has been deprecated and removed
- $MOOGSOFT_HOME/config/servlets.conf
  - The toolrunner servlet now optionally supports ssh key authentication as well as username/password-based authentication
- $MOOGSOFT_HOME/config/moog_farmd.conf
  - alert_workflows.conf moolet has been added
  - enrichment_workflows.conf moolet has been added
  - event_workflows.conf moolet has been added
  - situations_workflows.conf moolet has been added
v7.2.x-v7.3.x
- $MOOGSOFT_HOME/config/system.conf
  - New integration database property has been added: intdb_database_name
- $MOOGSOFT_HOME/config/moog_farmd.conf
  - The entire sig_resolution block containing merge groups, retention_period etc has been removed but is still supported as long as file_only_config is true
  - alert_root_cause.conf moolet has been removed and is no longer supported
  - nexus.conf moolet has been removed and is no longer supported
  - speedbird.conf moolet has been removed and is no longer supported
  - sigaliser.conf moolet has been removed and is no longer supported
  - cookbook.conf has been removed but is still supported as long as file_only_config is set to true
  - tempus.conf has been removed but is still supported as long as file_only_config is set to true
v7.3.x-v8.0.x
- $MOOGSOFT_HOME/config/system.conf
  - ElasticSearch now supports basic authentication
- $MOOGSOFT_HOME/config/security.conf
  - There is a new global_settings block which allows control of the CSRF protection feature
- $MOOGSOFT_HOME/config/servlets.conf
  - It is now possible to configure the toolrunner to run on a port other than 22 using toolrunnerport property
- $MOOGSOFT_HOME/config/moog_farmd.conf
  - alert_inform_workflows.conf moolet has been added
  - situation_inform_workflows.conf moolet has been added
  - The 'modules' block has been removed as it only contained Topology-related functionality and this has been deprecated in the v8.0.x release - the Topology feature works differently.

Manually merge and compare .rpmsave versions of files with the new versions of those files. Add any new properties to the older versions of the files. You can skip this step if you have already completed this step as part of this upgrade process on the current host.

The config and bot files from the previous version should not be copied on top of (replace) the new version of those files in 7.3.x, as they are not always forwards-compatible, and some config/bot lines need to be added for the new version to work.

To find files that have been changed, moved or deleted, run these commands:

find $MOOGSOFT_HOME -name '*.rpmsave'
find /etc/init.d/ -name '*.rpmsave'

For example, the following command displays the differences in the new version of the system.conf file:

diff -u $MOOGSOFT_HOME/config/system.conf $MOOGSOFT_HOME/config/system.conf.rpmsave

Follow this process to merge the file differences:

Rename the new versions of the files, without the .rpmsave extension, to end with .bak.
Merge the .rpmsave file with the new .bak file by adding new properties/configuration where needed (from the new version of the file into the old version), so the structure matches the new version of the file.
Rename the .rpmsave file to remove the .rpmsave extension.

Update JVM to use Java 11

Install the latest Java packages using the command below. It's possible that the yum upgrade of the moogsoft packages will have done this already and even possibly installed a newer version of Java than the one below. In this case, yum will report the packages are already installed and are the latest version. Continue with the subsequent steps even if this is the case.

yum -y install java-11-openjdk-11.0.7.10 \
               java-11-openjdk-devel-11.0.7.10 \
               java-11-openjdk-headless-11.0.7.10

On each server with a Moogsoft Enterprise RPM package installed, run the following command to replace the /usr/java/latest symlink so it points at the JDK11 JAVA_HOME directory:

source $MOOGSOFT_HOME/bin/utils/moog_init_functions.sh

If there are non-Moogsoft Enterprise packages on this server that do not support JDK11, you must update those applications to use a different JAVA_HOME symlink (not /usr/java/latest).

You can skip this step if you have already completed this step as part of this upgrade process on the current host.

To confirm this has worked, run the following command:

$JAVA_HOME/bin/java -version

It should return (as a minimum version):

openjdk version "11.0.7" 2020-04-14 LTS

You can also use the 'alternatives' command to point the system 'java' shortcut to the new version:

alternatives --config java

Remove references to the old MySQL connector

Upgrade the Moogsoft Enterprise database schema

Before upgrading the schema, check if the MySQL log_bin_trust_function_creators variable is enabled:

mysql -e "show variables like '%log_bin_trust_function_creators%';"

The value must be ON or 1. If the value is OFF or 0, you must enable this variable and restart the database:

Open the /etc/my.cnf file on the host running Percona MySQL.
If the variable is not in this file, run the following to add the variable to the file:
```
sed -i 's/$innodb_autoinc_lock_mode.*$/\1\nlog_bin_trust_function_creators = 1\n/' /etc/my.cnf
```
If the variable is already in the /etc/my.cnf file, set the variable to 1.
Once you have enabled the variable, restart the database:
```
service mysqld restart
```

To upgrade the Moogsoft Enterprise database, provide the Auto Upgrader utility with the credentials of a database user with super privileges. For single-host installations where MySQL was installed as part of the Moogsoft Enterprise deployment, you can use the default 'root' user.

Run the following command, replacing <MySQL-SuperUsername> with the username of your super user:
Note
Run this command on the server where the database is installed (on an RPM deployment, this is where the moogsoft-db package is deployed).
```
bash $MOOGSOFT_HOME/bin/utils/moog_db_auto_upgrader -t 8.0.0 -u <MySQL-SuperUsername>
```
Enter the password for the user.
Note
You can provide the password to the utility with the -p flag but Moogsoft does not recommend this in non-test deployments for security reasons.

Drop deprecated historic database tables

Run the following commands to drop two tables from the historic database that are no longer used. If you do not drop the tables at this stage, the Database Validator utility that you run during the validation step reports their presence as a delta.

Note

Run these commands on the server where the database or, on RPM deployments, the moogsoft-db package is installed.

These commands may fail if DBSplit has never been enabled. You can ignore these errors.

bash $MOOGSOFT_HOME/bin/utils/moog_mysql_client -i -e "drop table room_post_sigs" 2>/dev/null;
bash $MOOGSOFT_HOME/bin/utils/moog_mysql_client -i -e "drop table room_posts" 2>/dev/null;

To continue with the upgrade, see RPM - Upgrade data ingestion components.

In this section: