Moogsoft AIOps employs two databases, an active database and a historic database to enhance performance of the UI and extend data retention capabilities.

See Historic Database Benefits for information on the performance and scalability advantages of separating the active and historic databases.

You can use various closing strategies to help maintain your active database at an optimal size:

• Manually close alerts.
• Programmatically close alerts.
• Use the Auto Close feature.

The historic database can grow without affecting performance. When it is time to retire data from the historic database, you can archive selected Situations and alerts.

Control Historic Data Retention

Historic data retention requires the Housekeeper Moolet to work. Verify you have configured the Housekeeper Moolet and that it is enabled within Moogfarmd. The Housekeeper periodically identifies eligible closed alerts and Situations in the active database and moves them into the historic database. 

You can control historic data retention using the database split enabler at $MOOGSOFT_HOME/bin/utils/moog_db_split_enabler.

See the Historic Data Utility Command Reference for a full list of available arguments.

Access Historic Data

By default, the database split enabler creates a database called historic_moogdb within the same MySQL instance as the active database: moogdb.

The historic database contains alert and Situation related tables that have the same structure as their equivalents in the active database.

You can access this historic data in the UI in read-only format:

• Search results (when "include close" is selected)
• Direct URL (for Situation Room, Alert Timeline etc.)
• Situation Room (including Situation Alert View and Situation Timeline)
• Similar Situations (historical closed Situations will feature in the Similar Situations list for a Situation)
• PRC feedback can be set for closed alerts from the historic database (as accessed from the Alerts Tab of a Situation Room)

Historic data is also accessible from various Graze endpoints and their equivalent MoogDb methods:

• getAlertIds
• getAlertDetails
• getSituationAlertIds
• getSituationDetails
• getSituationHosts
• getSituationProcesses
• getSituationServices
• getSituationActions

Moogsoft AIOps rejects Graze endpoints and MoogDb methods that attempt to modify historic alerts and Situations.

Disable Historic Data Retention

When you disable the historic database, the data already in the historic database remains there. The Housekeeper Moolet does not move the data back to the active database. The old historic database is only accessible via SQL queries.

To disable the retention of historic data in a separate database, run the following command:

moog_db_split_enabler -d

Enable the Historic Database

If you are upgrading from Moogsoft AIOps v. 6.4 or earlier you need to manually split the data into an active database and a separate historic database.

Before you split the database, ensure you have met the following requirements:

• The Housekeeper Moolet is configured and running within Moogfarmd.
• You have the username and password for a MySQL user with schema creation privileges.

To split historic data into a separate database, run this command:

moog_db_split_enabler -e

Alert and Situation data that meet the default criteria are moved. If you do not specify alternative criteria, all closed alerts and Situations that have not been updated within the past hour are moved into a historic database.

See the Historic Data Utility Command Reference for a full list of default and available arguments. Any changes to the default settings are picked up and applied immediately to a Moogfarmd with Housekeeper Moolet running.

If the process encounters closed alerts that are eligible to be moved but still have a relationship with an open Situation, it copies the alerts to the historic database instead of moving them. Later, after they're no longer related to an open Situation, the Housekeeper Moolet moves them to the historic database.

If your system contains a large amount of closed data, the first run may take a long time and require additional CPU and memory use for the Moogfarmd process.

When the splitting process is active, you can monitor its progress in the Moogfarmd log by running this command:

tail -f /var/log/moogsoft/moogfarmd.log|grep Splitter

After the database is split, UI based filters do not return closed alerts or Situations that have been moved to the historic database.

Database Split Examples

To split the database with a grace_period and a run_interval of 60 seconds and an alerts_batch_size and a sigs_batch_size of 1000, run this command:

moog_db_split_enabler -e -g 60 -r 60 -a 1000 -s 1000

During the splitting process you can see entries such as the following in the Moogfarmd log:

WARN : [0:House][20180315 15:58:44.207 +0000] [CSplitterService.java]:143 +|Data Splitter started|+
WARN : [0:House][20180315 15:58:44.503 +0000] [CSplitterTask.java]:205 +|Splitter will copy [17] alerts and will move [1983] alerts and [500] situations|+
WARN : [0:House][20180315 15:58:44.706 +0000] [CSplitterTask.java]:205 +|Splitter will copy [152] alerts and will move [1848] alerts and [0] situations|+
WARN : [0:House][20180315 15:58:46.434 +0000] [CSplitterTask.java]:205 +|Splitter will copy [78] alerts and will move [917] alerts and [0] situations|+
WARN : [0:House][20180315 15:58:47.280 +0000] [CSplitterTask.java]:201 +|Nothing more to split|+
WARN : [0:House][20180315 15:58:47.282 +0000] [CSplitterService.java]:145 +|Data Splitter completed|+

If there is no eligible data to move to the historic database, the following entry is logged:

WARN : [0:House][20180315 15:12:22.547 +0000] [CSplitterService.java]:143 +|Data Splitter started|+
WARN : [0:House][20180315 15:12:22.666 +0000] [CSplitterTask.java]:201 +|Nothing more to split|+
WARN : [0:House][20180315 15:12:22.667 +0000] [CSplitterService.java]:145 +|Data Splitter completed|+

Example Split Scenario

The following table illustrates how the Housekeeper Moolet splits an example set of Situations and alerts.

Notes on the above:

• The process copies closed alerts 5 and 6 to the historic database because they are related to open Situation 3. In the historic database they retain their relationship to closed Situation 2, but not open Situation 3 until it is closed and the Housekeeper Moolet moves it to the historic database.
• The process copies closed Alert 7 to the historic database, but within that database the relationship to open Situation 3 is removed, until Situation 3 is closed and the Housekeeper Moolet moves it to the historic database.