You can run the command-line archiver tool included with Moogsoft AIOps to archive and delete Situations, alerts, and statistical data. The benefits of archiving data include improved system performance, faster backup and recovery, reduced maintenance, and lower storage costs.
How Archiving Works
The archiver tool archives and deletes a single day's worth of data at a time, to reduce the impact on the database. After you launch the archiver, it automatically processes data in batches which are configurable using the -b, -y and -z options in the Archiver Command Reference.
Both the moogsoft-db and moogsoft-utils packages include the archiver tool. You can find it at:
The archiver exports and deletes data from the historic database, unless you are deleting statistical data which resides only in the active database. If the historic database is disabled it performs all operations against the active database.
By default the archiver writes files to the
Launch the Archiver
To launch the archiver execute the moog_archiver command and pass either the -e argument to export or the -r option to delete.
Export all data older than 28 days to the default directory and retain the data in the database:
Delete all data older than 28 days:
See the Archiver Command Reference for a full list of available arguments.
Archive Loose Alerts
You can modify the selection criteria for loose alerts and Situations and their member alerts. You can choose to archive and delete loose alerts only using the last example below.
Export loose alerts that have not been modified in the past 28 days, and closed/dormant/superseded Situations and their member alerts that have not been modified in the past 4 days, and then delete the data from the database:
Export loose alerts that have not been modified in the past 2 days, and closed/dormant/superseded Situations and their member alerts that have not been modified in the past 7 days, and then delete the data from the database:
Export loose alerts that have not been modified in the past 28 days, and then delete the data from the database:
Archive Filtered Situations and Alerts
You can use global Situation and alert filters to limit the data that is eligible for archiving and deletion.
Export loose alerts that have not been modified in the past 28 days, and Situations and their member alerts that have not been modified in the past 7 days and match the global filter "My Global Alert Filter", and then delete the data from the database:
Delete all Situations that match the filter "My Global Situation Filter" and their member alerts, and delete all loose alerts that match the filter "My Global Alert Filter":
Use filters that extract data based on age with caution, as they can conflict with specified (or default) age constraints. If you use a filter that selects Situations created during the past day and apply an option to archive Situations older than 28 days, no data will be archived.
Delete Situations, Alerts and Statistical Data
You can use the archiver to delete Situations, alerts and statistical data that match specified criteria from the database.
Delete all Situation and alert data:
Delete statistical data older than 15 days:
Delete files older than 7 days from the default directory:
Archive File Names and Structure
Archive files are named and structured as follows:
- Archive files containing Situation data including alerts, events and snapshots have the filename format <table name>-<yyyymmdd>.<hhmmss>.csv.
For example alerts-20150410.143637.csv
- Archive files containing loose alert data have the filename format <table name>-loose<yyyymmdd>.<hhmmss>.csv.
For example alerts-loose-20150410.143637.csv
- Quotes are used within the files to handle occurrences of the delimiter. Quote characters in cells are enclosed in a second quote character. Null values from the database are written as \N.
The following tips can help you plan your archiving strategy:
- We recommend running the archiver tool outside core operational hours to minimize the impact to users. Users of the interface should refresh their sessions after the utility has been used to delete data.
- Archiving often in small quantities allows for fast execution and minimal impact.
- You can set up a cron job to run the archiver daily, outside core operational hours.
- You can use a specific alert or Situation filter to remove targeted events.
- Exporting and/or removing large amounts of data on a running system can be slow.
- Exporting from a remote machine is slower because of network latency.
- The archiver tool can export data from the prc_earliest_highest_severity_event table but it cannot delete this data.
- To run the archiver tool remotely from Elasticsearch, follow the instructions in the Distributed Installation section of the Implementor Guide to configure Elasticsearch to listen on the external interface.
- You do not need to re-run the indexer after using the archiver tool to delete data. The -r option deletes records from Elasticsearch to keep the search feature synchronized with the database.