Enrichment Overview
Situations in Moogsoft Enterprise are built from data ingested from your monitoring systems. You may have use cases for your Situations that require more information than is contained in the raw data. If this is the case, you can use a process called enrichment to add supplemental data to alerts or Situations. Enrichment can:
Improve accuracy for clustering alerts into Situations.
Improve readability of alerts for operators.
Aid operators in investigating Situations.
Provide critical reporting data.
The first step is to identify whether your existing data is sufficient. If it is lacking, identify the type of enrichment data that meets your requirements and the data source that can provide it. You can then choose the most effective and efficient method of enrichment for your specific needs.
Do you need to enrich?
The need to enrich depends on whether the data from your data source or monitoring system fulfils your requirements. Examine the use cases for your data to identify any omissions.
For example, an organization sets up Moogsoft Enterprise to ingest the following event data:
"node_name": "U0039-router01" "description": "Router down"
The data must fulfil these use cases:
Operators need the site name to understand where they need to take action to fix the problem.
Management needs the region for reporting requirements.
For this company, the node names are all based on the site name <site>-<component> so "U0039" reflects the site. There is no need to enrich for this use case.
The site name is not enough to determine the region, and the event data does not include region data. To satisfy the second use case, the company needs to enrich the event data.
Identify the enrichment purpose
The purpose of the enrichment indicates whether to enrich at alert or Situation level. Enrichment is expensive in terms of processing time and resource use. Inefficient enrichment can slow the processing of alerts, so it is important to enrich at the appropriate level.
Enrichment data can be broadly categorized to fulfil one of the following purposes:
Operational: Functionally modifies behavior within Moogsoft Enterprise to drive processes such as clustering. Ideally performed on alert creation.
Informational: Assists a consumer (operator or external system) to differentiate between Situations. Typically performed at Situation level. Includes updates to Situation description, services and processes.
Diagnostic: Assists operators to investigate Situations and can be performed at either alert or Situation level. Examples include updates to alert and Situation custom_info and updates to Situation discussion threads.
The region data in our example is informational.
Identify the enrichment source
If the required data exists externally, identify its type:
Static: Data that changes infrequently, for example a country code lookup to a country name.
Dynamic: Data that may be subject to change, for example a database query to match a hostname to a service.
In our example, the company database stores the site number and relates it to the site address and region. The data is static:
site address city state region U0039 1265 Battery St San Francisco CA US-WEST
Dynamic enrichment on every de-duplication has a greater performance impact than enrichment on alert creation. If the enrichment data is unlikely to change during the lifetime of an alert, enrich once on alert creation. See Enrich on Alert Creation for more details.
You can enrich from a static file in a LAMbot. All other enrichment is performed in a Moobot.
Processing for multiple enrichment sources
Enrichment should be limited and only used when necessary due to its consumption of processing time and resources. Parallel processing involves processing alerts in two or more enrichers simultaneously while serial processing hands the alert to one enricher at a time. The alert is passed onto the next enricher once it has been processed. If two enrichment sources are required for a single alert, serial processing should be used. Parallel processing could cause one enricher to disrupt the other enricher while processing the alert.
Select an enrichment method
Some enrichment methods are available in the UI:
Other methods are manually configured or accessed via the command line. The most common are:
REST.V2 module to retrieve data through HTTP.
ExternalDb module to retrieve data from supported SQL databases.
Graze API to update Situations and alerts statically.
Situation Manager Labeler to update Situations and alerts dynamically.
In our example, depending on the database specification, the company might use JDBC to add the region data into alert custom_info and the Situation Manager Labeler to add the region data to Situations.