Updates for Moogsoft Enterprise v8.x

Moogsoft has issued the following updates to Moogsoft Enterprise v8.x. All updates are cumulative and include the fixes from previous releases.

Contact Moogsoft support for instructions for applying an update. Refer to the following for information on upgrading a single-host deployment (such as for POC, POV, UAT and other similar deployments):

Moogsoft Enterprise v8.1.0


Due to incompatibilities between the new/upgraded version of RabbitMQ required by/bundled with this release and the old version (3.7.4), 'rolling upgrades' are not supported, and therefore the entire RabbitMQ cluster requires a 'full stop' upgrade. Moogsoft EnterpriseSAAS deployments where a local 'on-prem LAM-only' environment is also present can stop the local RabbitMQ LAM during the SAAS upgrade to buffer events in the local Rabbit, then restart the LAM when the SAAS upgrade is complete, so no events are lost.

This version coincides with the release of an extensive set of new Workflow Actions. See ???.

The update includes the following improvements and fixes:

  • MOOG-17113 : The minimum version supported of NGiNX has been increased to 1.20.1

  • MOOG-17222 : The versions of Percona XtraDB Cluster and XtraBackup (as deployed by the install_percona_nodes* scripts) have been upgraded to 5.7.33 and 2.4.23 respectively. Note that the version of XtraBackup deployed by install_percona_nodes.sh is now explicitly set in the script (rather than pulling the latest available). This is to keep versions in sync with the tarball variant install_percona_nodes_tarball.sh

  • MOOG-17177 : The minimum JDK version supported and the version shipped is now

  • MOOG-17174 : The list of ciphers accepted by NGiNX has been updated to increase the security of the web server.

  • MOOG-17161 : Apache-Tomcat has been upgraded to 9.0.43

  • MOOG-17088 : The supported version of RabbitMQ has been upgraded to v3.8.16 and Erlang to v23.3.

  • MOOG-17226 : It is now possible to expand or collapse list of Impacted Services, Team Rooms, or Situation/Alert Views in the Left Navigation panel.

  • MOOG-17171 : A 'Situation Client tool' can now be set as a default tool, for easy access in the Situation Room

  • MOOG-17170 : The maximum number of headings configurable for the Situation Room has been increased.

  • MOOG-17209 : The moog_db_split_configurer utility now has an option for setting event batch size : -e,--events_batch_size <value> Number of events/snapshots per batch during deletion (default : 100000) This option sets the batch size used for event/snapshot deletion by the Splitter which, if lowered, may help improve Splitter performance and reduce Database load, particularly in environments where alerts have high event counts.

  • MOOG-17208 : The process_log entries made by the Splitter have been improved to give more detailed breakdown of tasks, counts and timings as a run progresses.

  • MOOG-17082 : Updated the health logging for the moogpoller servlet to include active web socket subscription counts and internal queue build up count.

  • MOOG-17125 : An issue causing duplicate MoogFarmD moolet messages to be persisted in Hazelcast has been fixed.

  • MOOG-17245 : An issue preventing some Alert or Situation List exports from downloading has been fixed.

  • MOOG-17233 : The install_percona_nodes scripts (for rpm and tarball) have been update to ensure that the service-startup-timeout is disabled for Percona Cluster nodes built using them. This ensures that a long SST activity (e.g a node joining the cluster) is not timed out and aborted by the service script that starts the node.

  • MOOG-17230 : An issue preventing Moogsoft Enterprise URLs from being followed during login via SAML has been fixed.

  • MOOG-17168 : The Graze 'Authorization' header is no longer case-sensitive.

  • MOOG-17154 : An issue with improperly merged moogpoller servlet updates has been fixed.

  • MOOG-17123 : An issue that could lead to the splitter leaving behind old merged Situations, has been fixed.

  • MOOG-17062 : It is now possible to use the /topologies/{topologyName}/replace API endpoint on topologies which have been used in a Recipe. After the API call is made, new matching events sent into this Recipe will result in new Situations, (they won't be added to existing Situations as the old clusters in memory are purged).

  • MOOG-17223 : An issue impacting the functionality of the Situation Room after PRC feedback is submitted has been fixed.

  • MOOG-17215 : HTML element sanitization is now performed on Thread Entries added via the MoogDB moobot module

  • MOOG-17211 : It is no longer possible to 'Resolve' Alerts if they are in the 'Closed' state.

  • MOOG-17210 : The correct user ID is now stored in the SigAction message when any Situation Team(s) changes

  • MOOG-17188 : Fixed an issue where incorrect columns would be displayed for a saved Situation view after navigating back from a Situation Room in the browser

  • MOOG-17181 : The moog_historic_post_migration.sh script will now only attempt to use a 'socket' file if it detects a non-root tarball deployment type, and a ~/.my.cnf file is present, and that file contains the path to a socket file.

  • MOOG-17166 : An issue causing an exception to be spread across multiple log lines instead of a single line has been fixed.

  • MOOG-17164 : Fixed an issue wherein the 'PRC top three alerts' pane on the 'Next steps' tab in the Situation Room was not visible for split Situations

  • MOOG-17117 : The graze updateUser endpoint now supports a new parameter 'default_timeout' which if set to 'on' and the session_expiry parameter is set to null, will reset the user's current session timeout value to instead be inherited from their Role or Team.

  • MOOG-17138 : The linking of Situations to Services and Processes is now more resilient in high throughput environments.

  • MOOG-17128 : Fixes an issue where updates arriving in the UI would erroneously mark an opened Situation filter as 'edited/dirty'.

  • MOOG-16274 : The Maintenance Window UI will now show the most recent windows by default

  • MOOG-13057 : The Situation Room Visualize tab for a historic Situation now has a more relevant error message informing the user that visualize data is not available for such situations

  • INT-6277 : A typo in one of the custom_info mappings in the Nagios LAM 'bot' file has been fixed.

  • INT-6271 : Fixed an issue polling for batches of emails in the Outlook365 LAM

  • MOOG-17218 : The 'How can we help you' page (the right hand UI panel) is now working, regardless of whether the 'Show help and enablement' Settings option is enabled or not.

Moogsoft Enterprise v8.0.0.7

This update includes the following fixes:

  • MOOG-17094: Adds support for an encrypted systemUser password to LDAP; it is enabled by adding or setting the encryptedSystemPassword property in $MOOGSOFT_HOME/config/system.conf, inside the LDAP block.

  • MOOG-17104: Fixes an issue which prevented the export of Alert and Situation lists under certain circumstances.

  • MOOG-17124: Adds the ability to control which files can be uploaded to Situation Rooms and Team Rooms based on their content type; it is configured under System Settings > Customization> System Settings (all file types are allowed by default).

  • MOOG-17126: Adds a new endpoint option to the failover.hazelcast.man_center block of system.conf for connection to Hazelcast Management Center instances running with non-default settings.

  • MOOG-17131: Fixes an issue so the moobot function Bot.getActiveMoolets() now correctly reports moolet status if moolets were stopped or started using the farmd_cntl utility.

  • MOOG-17137: Fixes an issue that could lead to memory accumulation in Tomcat as a result of using custom_info merging in the AlertBuilder.

  • MOOG-17141: Updates a log message about a failed database query retry loop so that it now reports the correct number of retries.

  • MOOG-17153: Fixes an issue which caused a second moog_farmd instance to interfere with the Hazelcast cluster.

  • MOOG-17157: Removes repeated logging around the database configuration during runtime; these lines are now printed out only when Apache Tomcat or moog_farmd starts.

  • MOOG-17163: Fixes an issue that could lead to a session leak and memory accumulation in Tomcat after accessing the Situation Room.

  • MOOG-17167: Removes some verbose TRACE logging from Apache Tomcat logs.

  • INT-6275: Fixes an issue with the moogsoft_bridge_watchdog script which prevented control of the moogsoft_bridge using its logging JSON file.

  • INT-6278: Updates the on-screen instructions for installing the UI-based HP OMi plugin integration.

Moogsoft Enterprise v8.0.0.6

This update includes the following fixes:

  • MOOG-17093: Updates the version of Percona XtraDB Cluster (installed by the install_percona_nodes.sh and install_percona_nodes_tarball.sh scripts) to v5.7.31 and the version of Percona XtraBackup (installed by the install_percona_nodes_tarball script) to v2.4.21.

  • MOOG-17114: Introduces an optional password policy for DB Realm users, including configurable complexity via regular expressions.

  • MOOG-17115: Updates the version of Apache Tomcat to 9.0.40.

  • MOOG-17116: Updates the moog_init_ui.sh script so that it uses the correct command when starting and stopping apache-tomcat in non-root deployments.

  • MOOG-17118: Ensures that the Name field is always accessible during the Cookbook Recipe creation process.

  • MOOG-17120: Fixes an issue where some Situation filters did not show the correct count when custom_info fields changed.

  • MOOG-17122: Updates the Moogsoft Enterprise RPMs so that they force an upgrade of the JDK to if an earlier version is installed.

Moogsoft Enterprise v8.0.0.5

This update includes the following fixes:

  • MOOG-13781: Improves single sign-on (SSO) configuration so users don't need to re-enter their credentials as often when used with certain IDPs - the 'forceAuth' property is now configurable in any defined 'SAML' realm in the $MOOGSOFT_HOME/config/security.conf file.

  • MOOG-16346: Updates Alert Builder to fix a race condition resulting in an incorrect severity for some Situations.

  • MOOG-16982: Upgrades the version of Apache Tomcat to 9.0.37.

  • MOOG-17031: Fixes an issue where the Percona installation script continued running in the background after completing successfully.

  • MOOG-17034: Fixes an issue with the Percona installation scripts so they correctly validate arguments for hostnames to support both single-node and three-node deployments.

  • MOOG-17035: Fixes a real-time browser updating issue for alert and Situation counts when using customized filters.

  • MOOG-17040: Fixes an issue where Cookbook recipes changed exclusion filters on alert fields from "is null" to "=null" when saved.

  • MOOG-17044: Fixes an issue where teams created programmatically using Graze with duplicate Services in the request were not visible in the UI on the Teams page.

  • MOOG-17047: The moog_db_validator_utility tool now reports the lack of compression on the historic snapshots table as an INFO-level discrepancy instead of an ERROR.

  • MOOG-17053: Fixes an issue where the wrong version of Percona XtraBackup was specified in the script install_percona_nodes_tarball.sh.

  • MOOG-17054: Fixes an issue with HazelcastSerializationException which could cause a failure to serialize com.moogsoft.mooms.CMooMsg.

  • MOOG-17064: Fixes an issue with the workflow engine (updated in the Add-ons 2.2 release) to allow updates to PagerDuty incident statuses and adding notes to PagerDuty incidents.

  • MOOG-17065: Fixes an issue with filter views so that filter results update correctly when filters using custom_info fields change.

  • MOOG-17066: Updates the "database splitter" so it deletes events and snapshots in batches to avoid issues with large alerts and Percona replication writeset limits.

  • MOOG-17068: Ensures the DB Splitter process retries after an error occurs.

  • MOOG-17072: Fixes an issue where the forking thread would unnecessarily query the database repeatedly.

  • MOOG-17075: Prevents system health issue reports in clean installations by commenting out the default list of reserved processes.

  • MOOG-17087: Updates OpenJDK to v11.0.9.11.

  • MOOG-17089: Fixes an issue so the Alert and Situation lists now display tooltips for columns based on custom_info fields.

  • MOOG-17092: Fixes an issue preventing non-superusers from using the Right Click Tool with integrations (the sendToWorkflow function now only requires the moolet_inform permission).

  • MOOG-17096: Ensures the correct encoding of Situation room comments (for low-risk markup tags etc).

Moogsoft Enterprise v8.0.0.4

This update includes the following fixes:

  • MOOG:16964: Fixes an issue so that alerts are not assigned or acknowledged when they are in Situations with no user assigned and propagation enabled.

  • MOOG-16990: Prevents logging multiple entries in catalina.out when a Situation has more than one primary team.

  • MOOG-17006: Fixes an issue where logging in to the base server URL on a new tab resulted in a blank page.

  • MOOG-17008: Upgrades the Percona database to version 5.7.29.

  • MOOG-17009: Updates the moog_snapshots_online_table_change.sh and moog_historic_post_migration.sh scripts so they use the correct socket file when connecting to the database in tarball deployments.

  • MOOG-17010: Ensures users are presented with a relevant validation error message when attempting to perform getSituationWithFlag or setSituationWithFlag Graze API operations on Situations in the historic database.

  • MOOG-17011: Improves the moog_db_validator utility so it identifies valid differences (such as custom_info columns) and those indicating potentially serious issues in the schema (such as missing tables).

  • MOOG-17012: Increases the reliability of downloading dependency packages using the install_percona_nodes_tarball.sh script.

  • MOOG-17013: Fixes an issue with the install_percona_nodes_tarball.sh so that it now logs all expected output to its log file.

  • MOOG-17014: Updates moog_db_validator.sh so that it does not run when the script cannot connect to the MySQL database.

  • MOOG-17015: Updates moog_init_mooms.sh to account for a longer RabbitMQ start time.

  • MOOG-17016: Ensures that the install_percona_nodes_tarball.sh script checks that the required 'nc' package is available before continuing.

  • MOOG-17017: Adds the primary_team_id field to all Situation objects as they are processed by Moobots for consistency and reporting purposes.

  • MOOG-17018: Adds the option to clear the historic database using moog_init_db.sh , giving it clearing functionality similar to MoogDb and MoogReference. The new parameter ${MOOGSOFT_HOME}/bin/utils/moog_init_db.sh -s facilitates this new option. Use this feature with extreme caution. This option must not be used in production environments.

  • MOOG-17019: Ensures that the moog-proxies.conf file is no longer copied into the Nginx config directory when moog_init_ui.sh runs, eliminating potential confusion caused by two conf files in the same directory (for non-RPM deployments only).

  • MOOG-17020: Removes warnings about stale messages from the Apache-Tomcat log file.

  • MOOG-17021: Updates the moog_init.sh script so it now adds the MOOGSOFT_HOME environment variable when the process_keepalive.sh non-root cronjob is configured.

  • MOOG-17022: Adds timestamps to echo lines in the Historic Database Post Migration Utility (moog_historic_post_migration.sh) output.

  • MOOG-17024: Fixes an issue where a user with the Grazer role was able to log in using SAML authentication.

  • MOOG-17025: Fixes an issue so the Workflow Engine persists and restores any Context data during failover.

  • MOOG-17026: Changes the default Apache Tomcat maximum memory pool size (Xmx) from 2 Gb to 4 Gb.

  • MOOG-17027: Fixes an issue where it was possible to select an invalid timezone for a maintenance window, causing an "unknown time-zone exception" error in the Moogfarmd and Apache-Tomcat logs.

  • MOOG-17028: Fixes an issue where it was possible to update the default Cookbook through the UI but not through the Graze API updateCookbook call.

  • MOOG-17029: Enhances error handling by logging the thread/database pool dump when all resources in the thread/database pool are busy.

  • MOOG-17030: Updates the Alert Entropy UI to only allow incremental runs, and includes a new default schedule (a daily incremental run at 03:00 with a one day "readage" and a three week "keepage").

  • INT-6261: Ensures all LAMs support the loadMib("") method.

  • INT-6262: Updates the Nagios LAM UI instructions and documentation around using special characters in text fields.

Moogsoft Enterprise v8.0.0.3

This update includes the following fixes:

  • MOOG-16697: Fixes an issue where custom Situation columns were blank in exported files.

  • MOOG-16731: Fixes an issue where it was possible to assign, acknowledge, and resolve Superseded and Dormant Situations.

  • MOOG-16750: Adds additional default logging reporting for Tomcat current memory usage, viewable incatalina.out.

  • MOOG-16762: Reduces moog_archiver memory and CPU usage during export and removal, and shortens the time required for exports.

  • MOOG-16851: Fixes an issue ensuring that SigAction.details messages and calls to the moogdb.getSituation() method return the right custom info after calls to the Graze endpoint addSituationCustomInfo.

  • MOOG-16852: Fixes an issue where topologies were blocked from filling the entire viewable area in full screen mode.

  • MOOG-16879: Fixes an issue where Situation Manager failed to label some Situations due to SigAction message objects changing type prior to Workflow Engine processing.

  • INT-6208: Fixes an issue with Cherwell bidirectional workflows caused by the name change from "AIOps" to "Enterprise."

  • INT-6239: Removes an unnecessary text file from aiops-puppet-module.zip.

Moogsoft Enterprise v8.0.0.2

The update includes the following fixes:

  • MOOG-16568: Added primary_team_id in graze response of getSituationsDetails.

  • MOOG-16348: The UI timezones used by the UI and server-side components are now consistent.

  • MOOG-16512: Displays the correct alert count for custom filters in the side menu when you make changes that affect the alert count in the alert grid. For example closing alerts.

  • INT-6120: Fixes an issue where events remained in the remote RabbbitMQ after failover of the RabbitMQ LAM.

  • INT-6183: Fixes the concurrency issue when using constants.get() and constant.reload() in Moobots.

  • MOOG-16364: You can configure the splitter to remove process logs that are older than configured log retention period.

  • MOOG-16683: Dropped stored procedures and tables created in that are no longer used.

  • MOOG-16685: Client tool prompts no longer list internal variables.

  • MOOG-16686: Fixes an issue in the 7.3.0 upgrade script affecting cloud customers.

  • MOOG-16705: Upgrades Nginx version to 1.18.

Moogsoft Enterprise v8.0.0.1

The update includes the following fixes:

  • MOOG-16149: Updates Apache Tomcat to v9.0.35.

  • MOOG-16283: Adds an event code for API alert modification. A new event code is now added for any API alert modification using updateAlert and updateClosedAlert that does not impact the severity of an alert. Events with this code are filtered out of the UI timeline.

  • MOOG-16286: Removing all services and teams from a Situation now updates the Situation Room correctly.

  • MOOG-16354: A new MoogDb V2 method resolveAlerts can be used to resolve alerts. It accepts an array of alerts IDs and optional thread entry comment and returns true for a successful request and false for an unsuccessful request. See resolveAlerts for more information.

  • MOOG-16432: Resolves an issue with the exit handler in the remove_aged_snapshots database splitter stored procedure.

  • MOOG-16462: In SigAction driven workflows, blended objects containing attributes from two source objects now work correctly with the delay action.

  • INT-5766: The AWS SNS integration no longer generates different signatures for the same alerts.

  • INT-6068: Resolves an issue that caused Broker JWT tokens to be invalid for several hours if created by a Controller with a system timezone before UTC, for example EDT or UTC -4.

  • INT-6076: Resolves an error that prevented the RabbitMQ LAM from consuming events.

Workflow Engine updates

For information on releases of the Workflow Engine, see Moogsoft Enterprise Add-ons.