Moogsoft Docs

Cookbook Configuration Changes

If you change the configuration of a Cookbook or Cookbook Recipe, Moogsoft AIOps may re-evaluate any Cookbook clusters, depending on your persistence setting and the severity of the configuration change. This applies to clusters of alerts that Moogsoft AIOps holds in memory and has not yet formed into Situations. It does not affect clusters that have already become Situations that users can see and have been saved in the database.

Your persistence setting affects whether Cookbook re-evaluates clusters as follows:

  • If persistence is turned off, Cookbook resets every cluster and all new incoming alerts will form new Situations.

  • If persistence is turned on, Cookbook updates existing clusters depending on the configuration parameters that have been changed, as described below. Cookbook may remove clusters or create new Situations or it may persist the clusters so that new alerts are added after you have changed the configuration.

Configuration categories

Moogsoft AIOps groups Cookbook and Recipe configuration changes into three categories:

  • Cosmetic changes: These configuration properties are non-functional and do not affect how Cookbook creates and maintains clusters. When you change these properties, there are no functional changes to existing clusters.

  • Property changes: These configuration properties affect how Cookbook maintains clusters and generates Situations.

  • Core changes: These configuration properties fundamentally govern how Cookbook creates clusters and groups alerts into Situations.

Cookbook and Recipe configuration properties are grouped into the following categories:

Category

Cookbook Property

Recipe Property

Cosmetic changes

Name

Name

Description

Situation Description

Property changes

 

Alert Threshold

Cook For

Cook For

Cook For Extension

Cook For Extension

Max Cook For

Max Cook For

Core changes

 

Trigger Filter

 

Exclusion Filter

 

Seed Alert Filter

 

Rate Filter

Cluster By

Cluster By

 

Hop Limit

Entropy Threshold

Scale by Severity

Recipe Matching

Recipe Matching

Clustering

Cosmetic changes

Cosmetic changes to Cookbook and Recipe properties have the following effects on clusters:

Name

If you change the name of a Cookbook or a Recipe, Cookbook makes no operational changes to clustering.

Description

If you change the Situation Description for a Recipe, Cookbook applies the new description to all Situations created after the change. Cookbook maintains the old description for all Situations created before the change, regardless of whether new alerts are added to the Situation.

Property changes

Property changes to Cookbook and Recipe properties have the following effects on clusters:

Alert Threshold

Changes to the Alert Threshold

Reducing the Alert Threshold

For changes to the Alert Threshold, the main behavioral difference occurs when the Alert Threshold is reduced. In the example below, before the configuration change, the Alert Threshold was set to 3 and two alerts had arrived and formed a cluster in memory. If you change the Alert Threshold configuration from 3 to 1, the cluster satisfies the new configuration so Cookbook automatically creates a Situation in the database containing the these two alerts. New alerts coming into the system can continue to be added to this cluster.

Cookbook_-_Reduce_Alert_Threshold.png
Increasing the Alert Threshold

If you increase the Alert Threshold configuration, the cluster will persist in memory until the higher Alert Threshold is reached.

Cook For/Cook For Extension/Max Cook For

Cookbook adopts a similar logic for changes to all three of these attributes because they all affect the cluster's expiry time.

Extending Cook For/Cook For Extension/Max Cook For

If you extend any of these properties, the cluster expiry time is extended from the first event time. In the example below, before the configuration change, the Cook For time is 30 seconds and alerts 1 and 2 arrive at 0 and 25 seconds, respectively. After the Cook For time changes from 30 seconds to 40 seconds, alert 3 arrives at 35 seconds. Cookbook clusters this alert with the persisted cluster from the previous configuration. When alert 4 arrives at 45 seconds, Cookbook creates a new cluster because it satisfies the newly defined Cook For time. Cookbook behaves similarly if Cook For Extension or Max Cook For properties are extended.

Cookbook_-_Extend_Cook_For.png
Reducing Cook For/Cook For Extension/Max Cook For

If you reduce any of these properties, Cookbook relies on the first event time to establish whether clusters are still valid. In the example below, before the configuration change, the Cook For time is 30 seconds and alerts 1 and 2 arrive at 0 and 25 seconds, respectively. If you reduce the Cook For time from 30 seconds to 20 seconds, the arrival time of the most recent (second) alert exceeds the new Cook For time so Cookbook expires and removes the cluster.

Cookbook_-_Reduce_Cook_For_1.png

In the second example below, before the configuration change, the Cook For time is 60 seconds and alerts 1 and 2 arrive at 0 seconds and 25 seconds, respectively. If you reduce the Cook For time from 60 seconds to 40 seconds, the cluster still persists because it is still within the new Cook For time so that, when alert 3 arrives at 35 seconds, it joins the existing cluster. Alert 4 arrives at 45 seconds which exceeds the new Cook For time so Cookbook places it in a new cluster.

Cookbook_-_Reduce_Cook_For_2.png

In the same example, if alerts 3 and 4 had arrived (at 35 and 45 seconds) and the configuration change occurred at 50 seconds, Cookbook would close the cluster immediately with the four alerts in it, as shown below.

Cookbook_-_Reduce_Cook_For_3.png
Core changes

For all properties that are in the core changes group, any changes cause Cookbook to remove the associated clusters. This is because the fundamental rules on which Cookbook clusters alerts have changed, and it is no longer meaningful to cluster new alerts with old ones.

As an example, consider the example below in which you change the similarity of a Recipe. Initially, the recipe uses a 50% similarity on source ID. Alerts 1 and 2 arrive and Cookbook clusters them together. If you increase the similarity from 50% to 100%, Cookbook removes the cluster from memory. The diagram below shows how confusing it would be if the cluster persisted, visibly seeing a cluster containing alerts which clearly contradict a 100% match on source ID.

Cookbook_-_Core_Change.png

This behavior, to remove any old clusters and start new clusters when new alerts arrive, is consistent across all core configuration changes.