This section shows the "focus" of the release (via radar/spider chart) in terms of ticket distribution across 5 focus areas:
- Customer Request
- Speed of Deployment
- General Enhancement
SPECIAL CONSIDERATIONS FOR THIS RELEASE
- A new moogsoft-common RPM has been created:
- This RPM gathers together common libraries and packages and hence becomes an install dependency of all other moogsoft RPMs
- These common libraries and packages are no longer part of the other RPMS hence their file size is reduced
- $MOOGSOFT/bin/utils/moog_init_ui.sh flags have been changed:
- A new flag '-f' or '--configure_tomcat' was added (for handling tomcat configuration files)
- The flag '-o' or '--openssl' now will generate nginx certificates only
- $MOOGSOFT/bin/utils/moog_init_db.sh flags have been added to support resetting the two schemas to a default state:
Flag '-o' or '--clean_moogdb_schema' will clear all the data in the moogdb schema
Flag '-p' or '--clean_reference_schema' will clear all the data in the reference schema
- Tomcat access log is now called localhost_access_log<XXX> instead of <hostname>_access_log<XXX>
- Only affects clean installs
[MOOG-6367 / MOOG-6427] [BETA] - Support for SAML2.0-based authentication
Phase 1 of SAML2 Authentication functionality has been added to AIOps 6.1.0.
SAML2 can be configured via $MOOGSOFT_HOME/config/security.conf. Please see the commented out example/template within that file.
Please keep in mind that if you enable SAML2 Authentication that you will no longer be given the standard Moogsoft AIOps Login page where you can enter the username and password for Database Users. To re-enable this method of authentication simply comment out all "SAML2" realmType configurations.
For full documentation please visit this link: Authentication
There is a known issue with this feature in that, if a user has signed in to IdP via SAML2 and has logged in to AIOps, if they then log out of AIOPs, they will automatically logged back in. This will be fixed in an upcoming release
[MOOG-6589 / MOOG-6600] - Bulk Addition / Management of Service Filters for a Team
Improvements have been made in the UI on the 'System Settings' -> 'Teams' page -> 'General' Tab as follows:
- Allow addition or removal of all Services (or a selected subset) to/from a team at once
- The Service list is now searchable as seen in the screenshot below, and all matching Services can then be added in a single operation
- Fixed a bug where previously only 6 lines of Services could be seen
Adding more than 200 Services to a team could seriously impact UI login performance for members of that team
[MOOG-4764] - Allow the REST.V2 module to communicate via a proxy and authentication
The REST.V2 moobot module now can establish a connection where the outbound is via a proxy. See REST.V2 for more and examples.
Example moobot code below (additional examples on the main docs page linked above):
[MOOG-6209] - [moog_init] Tomcat server.xml localhost → hostname substitution no longer needed
As nginx fronts Tomcat (as of v6.0.0) and only nginx provides the SSL certificates, there is no need to make any localhost->hostname substitutions in $APPSERVER_HOME/conf/server.xml.
This change means that, in new install scenarios, the Tomcat access log is now called localhost_access_log<XXX> rather than <hostname>_access_log<XXX>.
[MOOG-6245] - moog_init_ui.sh should only enable nginx IPv6 support if system supports IPv6
If a system does not have IPv6 support (or IPv6 has been specifically disabled), the nginx configuration files under /etc/nginx/conf.d/ will not try and use an IPv6 address.
[MOOG-6608] - Service Scripts: Add a kill -3 (thread dump) to the restart option to capture useful debug
The moogsoft service scripts (i.e. moogfarmd and all LAMs) have been updated to invoke a Java thread dump using 'kill -3' if the process takes too long to shut down. This output should be sent to Moogsoft Support to assist in debugging any shutdown or hang issues with the relevant process. The output from the thread dump is written to the relevant service log file. e.g. moog_farmd default log file is /var/log/moogsoft/moogfarmd.log
[MOOG-6644] - Addition of moogsoft-common package
A new moogsoft-common package has been created which is required in any Moogsoft AIOPs v6.1.0 installation onwards. All of the existing Moogsoft packages require this package and it will be downloaded when they are installed as a dependency. The advantage of this new package is that the total size of all the RPMs is now half that of v6.0.0.
[MOOG-6642] - LAM and Integration improvements
- [LAMS-1562] HP OMi (Operations Manager i) LAM Added
- HP Operations Manager i is an automated IT operations management software. HP OMi provides automated monitoring, root cause identification and prioritization, with automated remedial action
- The HP OMi LAM fetches events from the HP OMi and forwards it to Moogsoft AIOps
- [LAMS-1650] - Fix for SSL functionality in Kafka LAM
- [LAMS-1839] - Zabbix LAM improvement for all events and alert data
- [LAMS-1012] - Modify REST Client LAM requests and responses
- REST Client LAM now allows you to modify the REST request sent externally, via a LAMBot function. The response received can also be manipulated in a separate LAMBot function
- The new functionality allows you to share values between requests and responses via the Constants module
- The LAMBot's Constants module can be saved and loaded from a .state file
- [LAMS-1757] - Dynatrace APM LAM now correctly puts all alert details into event overflow
[MOOG-5855] - Graze: Provide basic authentication mechanism
Added standard basic authentication mechanisms to allow URL / Header embedding of credentials in order to streamline the effort of integrating 3rd parties into Moogsoft AIOps, and also be consistent with the ingestion which does support basic authentication.
Example with user:password as a parameter:
or in the request URI:
[MOOG-6108] - [Graze] Provide additional options for the management of users / teams
New graze endpoints have been added for creating new users, creating new teams, and updating existing teams. See official Graze API documentation for all the parameters and examples.
createUser curl example:
createTeam curl example:
updateTeam curl example:
[MOOG-6101 / MOOG-6111] - Add deltas to the alerts and situation messages
Behind the scenes work has been done to allow future version of AIOps to use deltas on important system messages (for performance improvements).
As part of these changes, there is now a new way to reset the moogdb and moog_reference databases to their 'clean' states. The $MOOGSOFT_HOME/bin/utils/moog_init_db.sh script now has two new flags to support this functionality:
- '-o' to clean the moogdb schema
- '-p' to clean the moog_reference schema
Both of these flags will be run automatically when the 'full initialisation' $MOOGSOFT_HOME/bin/utils/moog_init.sh -I command is run on a clean install.
[MOOG-6207] - moog_init_ui.sh should deploy tomcat files separately from certificate config steps
The $MOOGSOFT_HOME/bin/utils/moog_init_ui.sh script has been modified and the functionality of the '-o' flag has been split into two separate flags:
- The '-o' or '--openssl' flag will generate nginx certificates and update nginx configuration files only
- A new '-f' or '--configure_tomcat' flag was added to configure apache-tomcat (which applies the AIOPs versions of the following files: $APPSERVER_HOME/conf/server.xml and $APPSERVER_HOME/conf/context.xml)
Both of these flags will be run automatically when the 'full initialisation' $MOOGSOFT_HOME/bin/utils/moog_init.sh -I command is run on a clean install.
[MOOG-6230] - UI sending duplicate moogsvr requests
Some duplicated UI requests have been fixed - 'getUserInfo', 'getTeamOverviewStats' and 'getStory' are now sent out only once.
[MOOG-6239] - Elasticsearch is only listening on localhost - potentially an issue for distributed/HA setups
Elasticsearch can now be configured to listen on all interfaces rather than just localhost.
To enable this, run
which will add the line:
/etc/elasticsearch/elasticsearch.yml and will restart the elasticsearch service.
[MOOG-6275] - Database memory optimisations
A number of small memory leaks (for example around PRC functionality and getUserRoles()) have been fixed which will increase stability of the product.
[MOOG-6281] - Data export on very large data set causes apache-tomcat instability
Exporting a large amount of data via the UI is now more reliable due to improvements in how the data is extracted from the database and sent back to the user.
[MOOG-6357] - Migration utility did not remove external_priority from situation_column_names
The 600_data_migrator utility in $MOOGSOFT_HOME/etc/moog/schema_upgrades/5.2.3_ESR3-6.0.0/ now removes the 'external_priority' column from 'situation_column_names' table.
This utility is also needed for users upgrading from 5.1.3_ESR1, 5.2.1_ESR2, or 5.2.3_ESR3 to 6.1.0_ESR5.
[MOOG-6395] - Integration extraction utility fails on second run
An issue that prevented the integration installation utility ($MOOGSOFT_HOME/bin/utils/moog_init_ui.sh -x) from being run more than once has been fixed.
[MOOG-6426] - mid_server_presence tool returns incorrect exit code when ran by toolrunner
The ServiceNow integration has been updated to improve the reliability of the MID server presence check.
[MOOG-6431] - nginx URL rewrite is incorrect on the login page when using port forwarding
This is to support the case of running AIOps in a Docker container with a host port mapped to 443.
[MOOG-6479] - AutoClose mechanism reports two close events in sig_journal table
An issue where two 'Situation Close' events were registered when a situation was closed by the AutoClose Housekeeper mechanism has been fixed.
[MOOG-6376] - moog_archiver reliability improvements and output fix
An issue which meant that an exception would be printed out when the moog_archiver was run at DEBUG loglevel has been fixed.
Additionally, if a connection is dropped during the archival process, the connection will be re-attempted between each retry period.
[MOOG-6411] - Service Startup Script - LOG_FILE Name Error - underscore prevents $SERVICE_NAME from being used
An issue which meant that the log file name used by AIOPs service scripts was not correctly expanded has been fixed.
[MOOG-6618] - Maintenance Window Manager: Does not pass the updated event on the bus to downstream moolets
An issue where the MaintenanceWindowManager moolet would not forward an updated alert object (with the correct custom_info) to the next moolet in the chain has been fixed.
[MOOG-6259] - Logging changes for integrations
An issue which reported a full stack trace to the end user in the apache-tomcat log when an integration failed to install has been fixed. A simplified one line WARN message is now printed instead.
[MOOG-6276] - [UI] ServiceNow Column is not getting added or removed via the UI column View button
An issue which prevented custom_info columns from being added and removed from a Situation View or Alert View under certain circumstances has been fixed.
[MOOG-6358] - [UI] internal_priority and queue variables do not get resolved in situation client tools
An issue that prevented the 'internal_priority' and 'queue' Situation variables from being correctly substituted in Situation Client Tools has been fixed. This affected the ServiceNow URL tool in the previous release.
[MOOG-6401] - [UI] System Information dialog does not display Database Schema Upgrade History
An issue where the list of applied SQL schema upgrades would not display in the UI has been fixed.
[MOOG-6651] - SituationMgr: Enable the default moobot to forward situations downstream
The SituationMgr.js and SituationMgrLabeller.js moobots have situation.forward(this) lines added to their functions to ensure that any downstream moolets (for example the TeamsMgr) pick up any changes they make to situations.
Additionally, commented out code has been added to all the out-of-the-box SituationMgr moobots to remind users to ensure the situation object is updated before it is forwarded on under certain circumstances.
[MOOG-6653] - Archiving with only the export option will mean lots of repeat data in the export
An issue that caused the moog_archiver to put duplicate entries into the generated files when it was set to perform an export operation (no remove) has been fixed.
[MOOG-6665] - [UI] When a team becomes inactive, the UI does not remove the services from the Summary Page
An issue whereby the UI did not correctly remove 'Inactive' Services from the Summary page has been fixed.
[MOOG-6668] - create_remote_user stored procedure wrong number of arguments to user table insert command
The create_remote_user stored procedure has been updated and fixed to reflect the removal of the Twitter user field in v6.0.0.
[MOOG-6671] - Team room attachments not working correctly
Issues that affected the uploading and download of attachments within a Team room (causing duplicate entries to appear) have been fixed.
[MOOG-6673] - Variable substitution doesn't work for Alert Server Tools
An issue whereby Alert variables substitutions would not correctly translate when used in Alert Client Tools has been fixed.
[MOOG-6682] - Modification of custom info columns requires refresh
An issue whereby the UI would not correctly acknowledge an addition/change/removal of new Alert/Situation custom_info columns via the UI has been fixed. The 'Save' button now correctly reflects whether the users chose to save the changes.
[MOOG-6681] - Propagate Situation Actions not working in v6
An issue which prevented the 'Situation Action Propogations' from working (enabled in System Administration → Customisation) has been fixed.
[MOOG-6679] - Using createAlert(event, true) on alert with no custom_info and event which has custom_info causes exception
An issue whereby an exception would be generated within moogfarmd and was seen in the moogfarmd.log has been fixed. In the previous release, when an event was sent into Moogsoft AIOps which had custom_info and the custom_info was merged with a matching alert which did not have custom_info, an exception would occur.
[MOOG-6686] - Memory leak on adding alerts to situation
A small memory leak which occurred when Alerts were added to a Situation has been fixed.
[MOOG-6680] - moog_farmd alerts_added_to_situations health logging counter always 0
An issue whereby the moogfarmd health logging would not report the alerts_added_to_situations metric correctly has been fixed.
[MOOG-6698] - [UI] Low upload limit for nginx and no UI message warning that the upload if above limit
An issue whereby the UI would silently reject attachments larger than 2MB has been fixed. The maximum size supported in clean installs is now 10MB. This setting can be changed by editing the following file: /etc/nginx/conf.d/moog-ssl.conf
and adding the following line to the location /moogsvr JSON segment:
- client_max_body_size 10m;
It should now look like this:
Restart nginx using: service nginx restart
[MOOG-6705] - Situation Room collaboration tab refreshes unnecessarily when updates occur
An issue whereby any update to a situation would trigger a re-draw of the entire Situation Room has been fixed.
[MOOG-6710] - Adding or modifying Alert Column is causing a refresh on all opened Alert Filters with empty filter
An issue whereby a change to Alert columns via System Administration would trigger any open Alert list to be refreshed with an empty filter has been fixed.
[MOOG-6721] - moog_init_ui.sh should not create tomcat certificates
The moog_init_ui.sh script no longer creates apache-tomcat certificates and restarts apache-tomcat when the -o (reset nginx certificates step) is run.
[MOOG-6714] - migration utility does not remove MaintenanceWindowManager from list of moolets
An issue whereby the v6.0.0 migration utility did not remove references to MaintenanceWindowManagers linked to deprecated Integrations has been fixed.
Known issues with functionality in this release:
- [MOOG-4616] - [SERVER] After rotation of the target logfile, the logfile_lam stops reading the target
- [MOOG-5552] - [UI] Self monitoring tree-view occasional render issue
- [MOOG-5665] - [UI] Self Monitoring -> Processing Metrics: Cannot distinguish between multiple components of same process type
- [MOOG-5672] - [SERVER] With Hazelcast enabled, moobot constants cannot use objects as values
- [MOOG-5741] - [SERVER] Depending on mooms config, moog component "Control" queues spread across Rabbit Cluster on startup - leads to problems if rabbit node (or server) lost
- [MOOG-5796] - [HA] Not possible to change moogfarmd log level at runtime on passive instance
- [MOOG-5891] - [UI] In MSEdge, Situation and Alert view context menu option to Copy Cell(s) and Row(s) does not work
- Workaround: Use the normal keyboard shortcuts for copying text
- [MOOG-5979] - [Graze] Inconsistency with sitn_id return parameters (for successful requests)
- [MOOG-4549] - [Graze] Error messages should refer to the correct field name (for unsuccessful requests)
- [MOOG-6027] - [HA] The asynchronous Persistence Store operations should guarantee that the passed Object will not be concurrently modified