Define a custom correlation definition

Watch a use case walkthrough: Configure a correlation engine ►

Moogsoft Cloud includes a default correlation definition called Similar Sources which correlates similar alerts into incidents. Observe the incidents that form using this default in the Incidents view. If the incidents created meet your needs, then the Similar Sources correlation definition is sufficient. However, if you require incidents to form using different criteria (which is likely), you must create a custom correlation definition.

NOTE: You can also use the Correlations API to create, retrieve, update, and delete correlation definitions.

Note

Before you begin, make sure you have first configured event ingestion.

If you want to correlate your alerts using additional data not present in the default payloads, set up alert enrichment to add this data.

Create a correlation definition

To create a custom correlation definition, complete the following steps. Use the links in the steps to jump to more information about the different settings.

  1. Navigate to Data Config > Correlation Engine.

  2. Click Add Correlation Definition.

  3. Complete Step 1: Construct Your Incident Description.

    1. In the Correlation Name field, enter a name for the correlation definition.

    2. In the Incident Description field, enter the description which displays for incidents created using this correlation definition.

  4. Complete Step 2: Definition.

    1. Under Scope, select whether to apply this correlation definition to all alerts or to only apply it to specific alerts that match a filter.

      If you select Filter alerts: Consider only alerts that match this filter, you are prompted to add a filter:

      1. Click Alert Scope.

      2. Compose your filter.

        Use the guidance provided to compose a filter using suggested terms, operators and values. You can also type your preferred filter, or paste an API filter in the box.

      3. Click Scope Preview to see a list of current alerts your filter includes. This preview shows the last 100 alerts in your data that match this filter.

        Use this information to verify that this is the correct scope for your correlation.

      4. Edit the filter if necessary, clicking Refresh after making your changes to update the data in the preview list.

      5. Click Apply to exit the preview window and apply the filter.

    2. Under Fields to Correlate, click Add Field and select the fields the correlation engine should evaluate to create incidents.

      For each field, enter the percentage that field values must match to be considered part of the same incident in the Similarity Threshold field.

  5. Optionally, complete Step 3: Advanced.

    This section includes advanced settings that you can change. It is recommended that you try the default settings first before making changes.

    1. Set the number of hours and minutes to use for the Correlation Time Window.

    2. In the Minimum Alerts Count section, select the minimum number of similar alerts required before an incident is created.

  6. Click Save.

Correlation definition field details

Refer to the following sections for assistance with completing the correlation definition fields.

Correlation Name

The Correlation Name identifies the correlation definition which resulted in the creation of an incident, so it is a best practice to select an easily identifiable and meaningful name. You cannot edit the name after the correlation definition is created.

The Correlation Name identifies the correlation definition responsible for the creation of each incident. The name displays in the Incidents view.

ViewCorrelationLinkIncidentsGrid.png

In the example shown above, the name of the correlation definition is

Incident description

The description to use for all incidents that get generated from this correlation definition. These descriptions appear in the Incidents view.

IncidentCEDefinitionDescription.png

You can either enter plain text for the description (example: Disk issues in AWS Virginia), or you can use a macro. Macros are dynamic, so the description will update with the information included in the incident.

For example, suppose you are defining a correlation based on the Service alert field. You can then specify a label string such as:

Service Incident: cited(service) in classes unique(class,3) for cited(check,2) checks

Given this string, the resulting descriptions include the three most-cited services and the number of times each service is cited by a member alert:

Service Incident: ShoppingCart, Online Store in classes Storage, Compute, Network for Disk, CPU checks

Incident macros

You can use the following macros to generate incident descriptions:

  • Count (alert-field) — Return the count of alert-field citations, including duplicates.

  • Unique Count (alert-field) — Return the count of unique alert-field citations, excluding duplicates.

  • To List (alert-field) — Return a comma-separated string of all elements in a list, including duplicates.

  • Unique (alert-field, N) — Return a comma-separated string of N unique elements in a list, excluding duplicates.

  • Top (alert-field) — Return the top-cited item.

  • Cited (alert-field, N) — Return a list of the top-cited N items. If two or more items have the same number of citations, the items are sorted alphabetically.

Scope

The Scope allows you to define whether a correlation definition applies to all alerts or to specific matching alerts only.

Fields to Correlate

The Fields to Correlate section specifies the fields and tags in alerts that the correlation engine compares to determine if the alerts are part of the same or different incidents.

A definition can have multiple fields and tags. The Similarity Threshold column specifies how similar the field and tag values must be for an alert to be a match for another alert, and can be configured independently for different fields. Alerts with a similarity the same as or above the threshold are included in the same incident. When the similarity of two alerts falls below the Similarity Threshold, those alerts are not correlated into the same incident.

For detailed information on the methodology the correlation engine uses to determine field similarity, see Understand alert similarity.

Correlation Time Window

The Correlation Time Window defines how long an incident is a candidate for correlation using this definition. For more information on the Correlation Time Window, see Correlation time window.

Minimum Alerts Count

The minimum number of similar alerts required before creating an incident. This option is useful for reducing the total number of incidents by preventing the creation of incidents for one-off or intermittent alerts. The trade-off is that you might get "orphan" alerts that are not included in any incident. To find these alerts, go to the Alerts view, make sure that the Incidents column is included, and then sort on this column.

Fine-tune correlation definitions

After adding a new correlation definition, you can see if it is working as expected by viewing the resulting Incidents.

You can view the ID of the correlation definition responsible for creating any incident in the Correlation Definition column, in the incident summary information, or on the incident Details tab, under Correlation. The View Correlation link opens the correlation definition which caused alerts to combine into the selected incident.

CorrelationLinksForIncidents.png