Historic Database Benefits
Separate active and historic databases are created as part of the Moogsoft Onprem installation process. The Housekeeper Moolet periodically migrates closed Situation and alert data from the active to the historic database. See Configure Historic Data Retention for further details.
The segmentation of open Situations and alerts from closed ones yields a number of performance and scalability improvements. The extent of the benefits depends on your system size and the size of your database.
Performance Improvement Metrics
Moogsoft engineering used the following baselines for testing performance differences between split-database and single-database systems:
Single Linux server with 64 x 2.3GHz cores
128 GB RAM
13 months of data from a very large system (100M events,14M alerts and 1M Situations)
3% open alerts and Situations.
Splitting the database improved all filtering and loading times. The bar chart below displays the loading times before and after the database split on 13 months of data. Note: most customers could not have operated in production for 13 months with such performance.
The percentage improvements to loading times for different areas of Moogsoft Onprem on the 13 months of data were as follows:
Other improvements included:
12% reduction in the time to run a full search re-index
12% reduction in the time taken for the Alert Builder to process 1M raw events from a Socket LAM to 600,000 events and 4,000 alerts in the database
the Cookbook Sigaliser algorithms improved performance.
In this test environment example, the open:closed ratio for alerts and Situations was 3:97. The database split only impacts closed alerts and situations, so you might see different results if your system has a higher proportion of open alerts and Situations.
Next Steps
Read the documentation: Configure Historic Data Retention. When you configure database splitting, be aware of the following impacts:
General system performance depends on the database splitting settings. Aggressive splitting can lead to an increased CPU load because both MySQL and moogfarmd are consuming CPU. You can run the Housekeeper Moolet in its own moogfarmd on a separate host to potentially mitigate any performance impacts.
Retaining more data means that your database will increase in size. Implement monitors to check your database growth.