Skip to main content

Patch Moogsoft Onprem RPM v9.2.0 to v9.2.0.1

This topic describes how to patch an RPM-based distribution of Moogsoft Onprem to v9.2.0.1 from v9.2.0

Important

This process requires the presence of a third Moogsoft Onprem server to act a redundancy server as per : Fully Distributed HA Installation

In the process below, Server 1 and Server 2 are servers running MoogFarmD, Apache-Tomcat, RabbitMQ etc

In the process below, Server 3 is the Redundancy Server running just RabbitMQ and possibly OpenSearch too

  1. OPTIONAL (but recommended) : On any server - perform cluster health checks

    1. Confirm the RabbitMQ cluster is healthy:

      rabbitmqctl cluster_status

      This output should show three 'Running nodes' in the cluster

    2. Confirm the Percona cluster is healthy:

      moog_mysql_client -e "SHOW STATUS LIKE 'wsrep_cluster_size';"

      The above command should return 3 for a 3-node cluster etc.

      Now check all nodes are synced:

      for CLUSTER_NODE in $(moog_mysql_client -sNe "SELECT VARIABLE_VALUE FROM performance_schema.global_status WHERE VARIABLE_NAME='wsrep_incoming_addresses'" 2>/dev/null | tr ',' '\n' | cut -d: -f1); do
        echo -n "${CLUSTER_NODE}: "; curl -s http://${CLUSTER_NODE}:9198
      done

      This command show show 'Percona XtraDB Cluster Node is synced' for every node

    3. Confirm the OpenSearch cluster is healthy:

      curl -XGET 'https://localhost:9200/_cluster/health?pretty' -u moog_opensearchuser:$(${MOOGSOFT_HOME}/bin/utils/moog_config_reader -k search.password)

      This should report: "number_of_nodes" : 3

  2. One-time multi-server upgrade step

    Update all Percona nodes to the latest version using the instructions here: Percona Cluster 8.4 RPM Minor Version Upgrade

  3. One-time multi-server upgrade step

    Important

    RabbitMQ v4.3.4 can only be upgraded to from v4.2.0 onwards, so any clusters running RabbitMQ running v4.1.x need to be upgraded using this process

    For the purposes of this document, Server3 is the redundancy node, and Server1/Server2 are application nodes also running RabbitMQ

    1. Check the version of RabbitMQ (run this command on any node in the cluster):

      rabbitmqctl version

      If this command returns v4.1.x, proceed with these instructions. If the command returns v4.2.x, you can skip this process and continue the rest of the upgrade

    2. Enable all feature flags for RabbitMQ v4.1.x (this command only needs to be run once, and on any node):

      rabbitmqctl enable_feature_flag all
    3. Starting with the redundancy RabbitMQ node (Server3):

      1. Drain the current node:

        rabbitmq-upgrade drain
      2. Stop the RabbitMQ process:

        service rabbitmq-server stop
      3. Upgrade RabbitMQ on the current node:

        • Online deployments:

          yum -y upgrade rabbitmq-server-4.2.0
        • Offline deployments:

          Download the RPM on a server with internet access:

          curl -L -O https://github.com/rabbitmq/rabbitmq-server/releases/download/v4.2.0/rabbitmq-server-4.2.0-1.el8.noarch.rpm

          Copy the RPM to the RabbitMQ node in question, and upgrade it:

          yum -y upgrade rabbitmq-server-4.2.0-1.el8.noarch.rpm
      4. Restart RabbitMQ:

        service rabbitmq-server restart
      5. Revive the node:

        rabbitmq-upgrade revive
      6. Confirm that all three 'Running nodes' are still reporting as present and running:

        rabbitmqctl cluster_status
    4. Repeat the previous step on Server1

    5. Repeat the previous step on Server2

    6. After all three nodes are upgraded to RabbitMQ v4.2.0, enable all new feature flags (this command only needs to be run once, and on any node):

      rabbitmqctl enable_feature_flag all
  4. On Server 1

    1. Stop MoogFarmd to pause event processing (events will buffer in RabbitMQ)

      service moogfarmd stop
  5. On Server 2

    1. Stop MoogFarmD to pause event processing (events will buffer in RabbitMQ)

      service moogfarmd stop
  6. On Server 3 (Redundancy Server)

    1. Stop all services

      systemctl stop opensearch
      rabbitmq-upgrade drain
      systemctl stop rabbitmq-server
      
    2. Upgrade erlang

      • Online deployments:

        yum -y upgrade https://github.com/rabbitmq/erlang-rpm/releases/download/v27.3.4.13/erlang-27.3.4.13-1.el8.x86_64.rpm
      • Offline deployments (assumes offline yum repository is present)::

        yum -y upgrade erlang-27.3.4.13
    3. Upgrade the packages on the server

      This topic describes how to patch an RPM-based distribution of Moogsoft Onprem to v9.2.0.1 from v9.2.0

      1. Ensure the patch RPMs are available to each server being patched:

        • For internet-connected hosts, ensure there is a repo file under the /etc/yum.repos.d/ directory pointing to the 'speedy esr' yum repo.

          An example file is below:

          [moogsoft-aiops-9]
          name=moogsoft-aiops-9
          baseurl=https://<username>:<password>@speedy.moogsoft.com/v9/repo/
          enabled=1
          gpgcheck=0
          sslverify=false
        • For offline-hosts:

          1. Download the two offline yum repository files (requires 'speedy' yum credentials):

            https://speedy.moogsoft.com/v9/offline/2026-07-29-1785381987-MoogsoftESR_offline_repo.tar.gz
            https://speedy.moogsoft.com/v9/offline/2026-07-29-1785381987-MoogsoftESR_9.2.0.1_offline_repo.tar.gz
          2. Move the two offline installer bundle files to each server being upgraded as needed

          3. Create two directories to house the repositories. For example:

            sudo mkdir -p /media/localRPM/BASE/;
            sudo mkdir -p /media/localRPM/ESR/;
          4. Extract the two Tarball files into separate directories. For example:

            tar xzf *-MoogsoftBASE8_offline_repo.tar.gz -C /media/localRPM/BASE/;
            tar xzf *-MoogsoftESR_9.2.0.1_offline_repo.tar.gz -C /media/localRPM/ESR/;
          5. Back up the existing /etc/yum.repos.d directory. For example:

            mv /etc/yum.repos.d /etc/yum.repos.d-backup;
          6. Create an empty /etc/yum.repos.d directory. For example:

            mkdir /etc/yum.repos.d;
          7. Create a local.repo file in the /etc/yum.repos.d/ folder ready to contain the local repository details for example:

            [BASE]
            name=MoogRHE-$releasever - MoogRPM
            baseurl=file:///media/localRPM/BASE/RHEL
            gpgcheck=0
            enabled=1
            
            [ESR]
            name=MoogRHEL-$releasever - MoogRPM
            baseurl=file:///media/localRPM/ESR/RHEL
            gpgcheck=0
            enabled=1
          8. Clean the Yum cache:

            yum clean all
      2. FOR ALL VERSIONS

        For online-deployments only: Configure the latest RabbitMQ yum repository:

        curl -s https://packagecloud.io/install/repositories/cloudamqp/rabbitmq/script.rpm.sh | sudo bash
      3. Remove any 'rpmsave' files present. An rpmsave file will exist if a configuration/bot file was customised post-install/upgrade, and the product was later upgraded to a version which contained a version of the same file with new changes.

        If an rpmsave file is present when the product is upgraded again, and the new version contains additional changes to that file, a new rpmsave file will not be created, so any changes made between the new 'out of the box' version of the file, and the current rpmsave file, will be lost.

        A command to search for rpmsave files is below:

        find ${MOOGSOFT_HOME} -name '*.rpmsave'

        Either move these files to a backup directory, or remove them

      4. FOR ALL VERSIONS

        On each host where moogsoft packages are installed, install the patch RPMs:

        • For internet-connected hosts run the following command:

          yum -y upgrade $(rpm -qa --qf '%{NAME}\n' | grep moogsoft | sed 's/$/-9.2.0.1/')
        • For offline hosts, run the following command in the directory containing the patch RPMs:

          yum -y upgrade $(rpm -qa --qf '%{NAME}\n' | grep moogsoft | sed 's/$/-9.2.0.1*.rpm/')
        • For ALL RPM-based deployments, ensure the Java JDK folder permissions are correct by running the following command as root (or a user with sudo permissions):

          chmod -R 755 /usr/java /usr/lib/jvm
    4. Restart the RabbitMQ service and rejoin the cluster:

      systemctl restart rabbitmq-server
      rabbitmq-upgrade revive
      rabbitmq-queues rebalance quorum
      
    5. Confirm the RabbitMQ node has rejoined the existing cluster:

      rabbitmqctl cluster_status

      Three nodes should be listed

  7. On Server 1

    1. Stop all services (update the commands below as needed)

      service apache-tomcat stop
      service opensearch stop
      service restlamd stop
      service socketlamd stop
      kill -9 $(ps -ef | grep java | grep Broker | awk '{print $2}') 2>/dev/null;
      rabbitmq-upgrade drain
      service rabbitmq-server stop
      service nginx stop
    2. Upgrade Erlang:

      • Online deployments:

        yum -y upgrade https://github.com/rabbitmq/erlang-rpm/releases/download/v27.3.4.13/erlang-27.3.4.13-1.el8.x86_64.rpm
      • Offline deployments (assumes offline yum repository is present):

        yum -y upgrade erlang-27.3.4.13
    3. Upgrade the packages on the server

      This topic describes how to patch an RPM-based distribution of Moogsoft Onprem to v9.2.0.1 from v9.2.0

      1. Ensure the patch RPMs are available to each server being patched:

        • For internet-connected hosts, ensure there is a repo file under the /etc/yum.repos.d/ directory pointing to the 'speedy esr' yum repo.

          An example file is below:

          [moogsoft-aiops-9]
          name=moogsoft-aiops-9
          baseurl=https://<username>:<password>@speedy.moogsoft.com/v9/repo/
          enabled=1
          gpgcheck=0
          sslverify=false
        • For offline-hosts:

          1. Download the two offline yum repository files (requires 'speedy' yum credentials):

            https://speedy.moogsoft.com/v9/offline/2026-07-29-1785381987-MoogsoftESR_offline_repo.tar.gz
            https://speedy.moogsoft.com/v9/offline/2026-07-29-1785381987-MoogsoftESR_9.2.0.1_offline_repo.tar.gz
          2. Move the two offline installer bundle files to each server being upgraded as needed

          3. Create two directories to house the repositories. For example:

            sudo mkdir -p /media/localRPM/BASE/;
            sudo mkdir -p /media/localRPM/ESR/;
          4. Extract the two Tarball files into separate directories. For example:

            tar xzf *-MoogsoftBASE8_offline_repo.tar.gz -C /media/localRPM/BASE/;
            tar xzf *-MoogsoftESR_9.2.0.1_offline_repo.tar.gz -C /media/localRPM/ESR/;
          5. Back up the existing /etc/yum.repos.d directory. For example:

            mv /etc/yum.repos.d /etc/yum.repos.d-backup;
          6. Create an empty /etc/yum.repos.d directory. For example:

            mkdir /etc/yum.repos.d;
          7. Create a local.repo file in the /etc/yum.repos.d/ folder ready to contain the local repository details for example:

            [BASE]
            name=MoogRHE-$releasever - MoogRPM
            baseurl=file:///media/localRPM/BASE/RHEL
            gpgcheck=0
            enabled=1
            
            [ESR]
            name=MoogRHEL-$releasever - MoogRPM
            baseurl=file:///media/localRPM/ESR/RHEL
            gpgcheck=0
            enabled=1
          8. Clean the Yum cache:

            yum clean all
      2. FOR ALL VERSIONS

        For online-deployments only: Configure the latest RabbitMQ yum repository:

        curl -s https://packagecloud.io/install/repositories/cloudamqp/rabbitmq/script.rpm.sh | sudo bash
      3. Remove any 'rpmsave' files present. An rpmsave file will exist if a configuration/bot file was customised post-install/upgrade, and the product was later upgraded to a version which contained a version of the same file with new changes.

        If an rpmsave file is present when the product is upgraded again, and the new version contains additional changes to that file, a new rpmsave file will not be created, so any changes made between the new 'out of the box' version of the file, and the current rpmsave file, will be lost.

        A command to search for rpmsave files is below:

        find ${MOOGSOFT_HOME} -name '*.rpmsave'

        Either move these files to a backup directory, or remove them

      4. FOR ALL VERSIONS

        On each host where moogsoft packages are installed, install the patch RPMs:

        • For internet-connected hosts run the following command:

          yum -y upgrade $(rpm -qa --qf '%{NAME}\n' | grep moogsoft | sed 's/$/-9.2.0.1/')
        • For offline hosts, run the following command in the directory containing the patch RPMs:

          yum -y upgrade $(rpm -qa --qf '%{NAME}\n' | grep moogsoft | sed 's/$/-9.2.0.1*.rpm/')
        • For ALL RPM-based deployments, ensure the Java JDK folder permissions are correct by running the following command as root (or a user with sudo permissions):

          chmod -R 755 /usr/java /usr/lib/jvm
    4. If the Enrichment Database (Create the Enrichment API Data Store) is being used, ensure the schema is updated using the schema update command on that page.

    5. Restart RabbitMQ to rejoin the cluster:

      service rabbitmq-server restart
      rabbitmq-upgrade revive
      rabbitmq-queues rebalance quorum
    6. Confirm the RabbitMQ node has rejoined the existing cluster:

      rabbitmqctl cluster_status

      Three nodes should be listed

    7. Upgrade Apache-Tomcat

      Warning

      This process will completely remove and re-deploy Apache-Tomcat. If the Apache-Tomcat logs need to be kept, a copy should be made before proceeding.

      Deploy the new version of Apache Tomcat and the latest webapps:

      $MOOGSOFT_HOME/bin/utils/moog_init_ui.sh -twf
      

      If you made any changes to the original Apache Tomcat service script (such as the Xmx value), apply the same changes to the new version. This will require a restart of the apache-tomcat service

    8. Perform final checks

      1. Validate the patch:

        $MOOGSOFT_HOME/bin/utils/moog_install_validator.sh
        $MOOGSOFT_HOME/bin/utils/tomcat_install_validator.sh
        $MOOGSOFT_HOME/bin/utils/moog_db_validator.sh

        If there are any errors from the validators, contact Moogsoft Support

      2. Important

        Re-install the latest 'Addons' pack: Install Moogsoft Add-ons

    9. Start the LAMs (update the commands below as needed)

      service restlamd restart
      service socketlamd restart
    10. Restart Nginx:

      service nginx restart
  8. On Server 2

    1. Stop all services (update the commands below as needed)

      service apache-tomcat stop
      service restlamd stop
      service socketlamd stop
      kill -9 $(ps -ef | grep java | grep Broker | awk '{print $2}') 2>/dev/null;
      rabbitmq-upgrade drain
      service rabbitmq-server stop
      service nginx stop
    2. Upgrade erlang:

      • Online deployments:

        yum -y upgrade https://github.com/rabbitmq/erlang-rpm/releases/download/v27.3.4.13/erlang-27.3.4.13-1.el8.x86_64.rpm
      • Offline deployments (assumes offline yum repository is present):

        yum -y upgrade erlang-27.3.4.13
    3. Upgrade the packages on the server

      This topic describes how to patch an RPM-based distribution of Moogsoft Onprem to v9.2.0.1 from v9.2.0

      1. Ensure the patch RPMs are available to each server being patched:

        • For internet-connected hosts, ensure there is a repo file under the /etc/yum.repos.d/ directory pointing to the 'speedy esr' yum repo.

          An example file is below:

          [moogsoft-aiops-9]
          name=moogsoft-aiops-9
          baseurl=https://<username>:<password>@speedy.moogsoft.com/v9/repo/
          enabled=1
          gpgcheck=0
          sslverify=false
        • For offline-hosts:

          1. Download the two offline yum repository files (requires 'speedy' yum credentials):

            https://speedy.moogsoft.com/v9/offline/2026-07-29-1785381987-MoogsoftESR_offline_repo.tar.gz
            https://speedy.moogsoft.com/v9/offline/2026-07-29-1785381987-MoogsoftESR_9.2.0.1_offline_repo.tar.gz
          2. Move the two offline installer bundle files to each server being upgraded as needed

          3. Create two directories to house the repositories. For example:

            sudo mkdir -p /media/localRPM/BASE/;
            sudo mkdir -p /media/localRPM/ESR/;
          4. Extract the two Tarball files into separate directories. For example:

            tar xzf *-MoogsoftBASE8_offline_repo.tar.gz -C /media/localRPM/BASE/;
            tar xzf *-MoogsoftESR_9.2.0.1_offline_repo.tar.gz -C /media/localRPM/ESR/;
          5. Back up the existing /etc/yum.repos.d directory. For example:

            mv /etc/yum.repos.d /etc/yum.repos.d-backup;
          6. Create an empty /etc/yum.repos.d directory. For example:

            mkdir /etc/yum.repos.d;
          7. Create a local.repo file in the /etc/yum.repos.d/ folder ready to contain the local repository details for example:

            [BASE]
            name=MoogRHE-$releasever - MoogRPM
            baseurl=file:///media/localRPM/BASE/RHEL
            gpgcheck=0
            enabled=1
            
            [ESR]
            name=MoogRHEL-$releasever - MoogRPM
            baseurl=file:///media/localRPM/ESR/RHEL
            gpgcheck=0
            enabled=1
          8. Clean the Yum cache:

            yum clean all
      2. FOR ALL VERSIONS

        For online-deployments only: Configure the latest RabbitMQ yum repository:

        curl -s https://packagecloud.io/install/repositories/cloudamqp/rabbitmq/script.rpm.sh | sudo bash
      3. Remove any 'rpmsave' files present. An rpmsave file will exist if a configuration/bot file was customised post-install/upgrade, and the product was later upgraded to a version which contained a version of the same file with new changes.

        If an rpmsave file is present when the product is upgraded again, and the new version contains additional changes to that file, a new rpmsave file will not be created, so any changes made between the new 'out of the box' version of the file, and the current rpmsave file, will be lost.

        A command to search for rpmsave files is below:

        find ${MOOGSOFT_HOME} -name '*.rpmsave'

        Either move these files to a backup directory, or remove them

      4. FOR ALL VERSIONS

        On each host where moogsoft packages are installed, install the patch RPMs:

        • For internet-connected hosts run the following command:

          yum -y upgrade $(rpm -qa --qf '%{NAME}\n' | grep moogsoft | sed 's/$/-9.2.0.1/')
        • For offline hosts, run the following command in the directory containing the patch RPMs:

          yum -y upgrade $(rpm -qa --qf '%{NAME}\n' | grep moogsoft | sed 's/$/-9.2.0.1*.rpm/')
        • For ALL RPM-based deployments, ensure the Java JDK folder permissions are correct by running the following command as root (or a user with sudo permissions):

          chmod -R 755 /usr/java /usr/lib/jvm
    4. Restart RabbitMQ to rejoin the cluster:

      service rabbitmq-server restart
      rabbitmq-upgrade revive
      rabbitmq-queues rebalance quorum
    5. Confirm the RabbitMQ node has rejoined the existing cluster:

      rabbitmqctl cluster_status

      Three nodes should be listed

    6. Upgrade Apache-Tomcat

      Warning

      This process will completely remove and re-deploy Apache-Tomcat. If the Apache-Tomcat logs need to be kept, a copy should be made before proceeding.

      Deploy the new version of Apache Tomcat and the latest webapps:

      $MOOGSOFT_HOME/bin/utils/moog_init_ui.sh -twf
      

      If you made any changes to the original Apache Tomcat service script (such as the Xmx value), apply the same changes to the new version. This will require a restart of the apache-tomcat service

    7. Perform final checks

      1. Validate the patch:

        $MOOGSOFT_HOME/bin/utils/moog_install_validator.sh
        $MOOGSOFT_HOME/bin/utils/tomcat_install_validator.sh
        $MOOGSOFT_HOME/bin/utils/moog_db_validator.sh

        If there are any errors from the validators, contact Moogsoft Support

      2. Important

        Re-install the latest 'Addons' pack: Install Moogsoft Add-ons

    8. Enable all RabbitMQ feature flags:

      rabbitmqctl enable_feature_flag all
    9. Start the LAMs (update the commands below as needed)

      service restlamd restart
      service socketlamd restart
    10. Start Nginx:

      service nginx restart
  9. The OpenSearch Cluster now needs to be recreated: Opensearch Clustering Guide - RPM

  10. OPTIONAL (but recommended) : On any server - perform cluster health checks

    1. Confirm the RabbitMQ cluster is healthy:

      rabbitmqctl cluster_status

      This output should show three 'Running nodes' in the cluster

    2. Confirm the Percona cluster is healthy:

      moog_mysql_client -e "SHOW STATUS LIKE 'wsrep_cluster_size';"

      The above command should return 3 for a 3-node cluster etc.

      Now check all nodes are synced:

      for CLUSTER_NODE in $(moog_mysql_client -sNe "SELECT VARIABLE_VALUE FROM performance_schema.global_status WHERE VARIABLE_NAME='wsrep_incoming_addresses'" 2>/dev/null | tr ',' '\n' | cut -d: -f1); do
        echo -n "${CLUSTER_NODE}: "; curl -s http://${CLUSTER_NODE}:9198
      done

      This command show show 'Percona XtraDB Cluster Node is synced' for every node

    3. Confirm the OpenSearch cluster is healthy:

      curl -XGET 'https://localhost:9200/_cluster/health?pretty' -u moog_opensearchuser:$(${MOOGSOFT_HOME}/bin/utils/moog_config_reader -k search.password)

      This should report: "number_of_nodes" : 3

  11. On Server 1

    1. Start MoogFarmD

      service moogfarmd restart
  12. On Server 2

    1. Start MoogFarmD

      service moogfarmd restart
  13. Restart any event feeds if they were stopped

  14. Upgrade the Moogsoft Bridge Servers