Situation Merge - How It Works

Two types of merging occur when Moogsoft Enterprise clusters alerts into Situations. The following video explains the clustering stages and how Moogsoft Enterprise handles automatic merging.

Clustering Is Done at the Event Level

Clustering is done at the event level. The event is associated with an alert and this drives the alert’s situation membership. Each time an event passes through a Sigaliser it can potentially end up in a candidate cluster that will produce a new situation.  As a result of this, if configured in a certain manner, an alert that has 10 deduplicated events can potentially become a member in 10 distinct Situations. This is because each time the individual event was evaluated by the system, it had the opportunity to be added to a new or existing Situation.  Obviously this case is less likely to happen unless you specifically design for it; however, in order to avoid having multiple duplicate situations, the system has been designed to catch similarities through the Merge and Resolve process.

Clustering Stages

Cluster_Stages.png

There are a number of stages the initial cluster, as produced by Sigaliser, goes through until it emerges as a Situation. On a high level the stages are as following:

  1. Candidate Cluster

    • Generated by a Sigaliser such as Tempus or a Cookbook Recipe.

    • Must be triggered by the seed alert if configured by the Recipe.

    • Remains active in moogfarmd memory - regardless whether it has been promoted to a Candidate Situation or not - until its cook_for time expiry.

  2. Candidate Situation

    • A candidate cluster becomes a candidate situation when its Alert Threshold has been breached.

    • A candidate situation goes through a Merge and Resolve phase: the candidate situation is evaluated against all still opened Situations and identifies if the candidate Situation can be promoted into a new Situation or should be merged into an existing one.

    • Do not confuse the Merge and Resolve phase with the operational workflow action consisting in resolving the Situation once the incident has been addressed.

  3. Created or automatically merged Situation as a result of Merge and Resolve.

Merge Types

Merging can be one of the following types:

  • Automatic

    • Similarity

    • Superseding

  • Manual

The following diagram illustrates the merging logic:

merge-flow_diagram.jpeg

Superseding Merge

SupersedingMerge.png

Superseding merge happens if all the alerts in a candidate situation are a complete set, a subset or a superset of alerts contained in an already existing open situation. The result of this is that a reoccurring incident does not trigger a new situation if it can associate itself with a still-active one.

As another example, below is an excerpt from the Moogfarmd log describing the stage when the Candidate Cluster is created and its corresponding Candidate Situation is merged into an existing Situation. Note that the Candidate Situation contains 3 alerts while the already Created Situation has 2 alerts. Although the logs at first may seem confusing here is how they should be interpreted. The Candidate Situation (in the log as SIG 1) is actually a superset of the already existing situation (in the log as SIG 20). In the UI this will be seen as the additional alert from the candidate situation - in this case, it is the alert with id [61] and signature [SIGNATURE::3] - added into already existing Situation. This could sometimes explain for example why an alert is added into an already existing Situation beyond its cook_for time expiry.

DEBUG: [8:Default Cookbook][20190503 16:34:59.895 +0100] [CAbstractRecipe.java:574] +|An event for alert is a NEW cluster [59]:[SIGNATURE::1]|+
DEBUG: [9:Default Cookbook][20190503 16:35:00.022 +0100] [CAbstractRecipe.java:609] +|An event for alert JOINS existing cluster [60]:[SIGNATURE::2]|+
DEBUG: [0:Default Cookbook][20190503 16:35:00.045 +0100] [CAbstractRecipe.java:609] +|An event for alert JOINS existing cluster [61]:[SIGNATURE::3]|+
 
DEBUG: [Default Cookbook-SituationProcessing][20190503 16:35:00.054 +0100] [CDetectedSitnDb.java:1256] +|SIG 20 is a superset of SIG 1: 2 vs 3|+
DEBUG: [Default Cookbook-SituationProcessing][20190503 16:35:00.054 +0100] [CDetectedSitnDb.java:1294] +|Completed Merge of new & active SIGs...|+
DEBUG: [Default Cookbook-SituationProcessing][20190503 16:35:00.054 +0100] [CDetectedSitnDb.java:650] +|Starting self-resolution of active SIGs... |+
DEBUG: [Default Cookbook-SituationProcessing][20190503 16:35:00.054 +0100] [CDetectedSitnDb.java:929] +|Self Resoltion updated 0 SIGs|+

Note that you might hear this type of merging interchangeably referred to as Superset, Subset or Superseding merge. Under the Visualize Tab in the UI the merge type will be set to Superseding Merge. Also not to be confused with the Superseded Category which all situations take - regardless of the type of merge - when they get merged into the parent situation.This type of merging cannot be disabled.

Similarity Merge

Similarity - also knows as Overlap - merge between two or more situations occurs when they share a number of alerts that satisfy the configured similarity level. This type of merge is configurable via merge groups. Only Situations produced by the Sigalisers listed within the same group can merge into each other.  To note that same is applicable for Situations produced by the same Cookbook regardless of the merge_groups configured, i.e. all situations created by Recipes within the same cookbook are eligible for merging into each other.

Similarity_Merge.png