Moogsoft Docs

Cookbook Reference

Cookbook is a deterministic clustering algorithm that creates Situations in Moogsoft AIOps defined by alert relationships .

You can configure Cookbook to group together alerts with specific characteristics such as temporal or topological proximity and cluster them int o Situations . Each Cookbook is a collection of R ecipes; sets of configurable filters, triggers and other calculations such as priority ordering and entropy threshold. You can see some example Recipes here .

Configure Cookbook

You can configure Cookbook using parameters in $MOOGSOFT_HOME/config/moog_farmd.conf .

run_on_startup : Determines whether Cookbook runs when Moogsoft AIOps starts . If enabled, Cookbook captures all alerts from the moment the system starts, without you having to configure or start it manually.

Type : Boolean
Default : false

persist_state : Enables Cookbook to save its state for High Availability systems so if a failover occurs, the second moogfarmd can continue from the same point.

Type : Boolean
Default : false

metric_path_moolet : Determines whether Cookbook factors into the Event Processing metric for Self Monitoring .

Type : Boolean
Default : false

moobot : Specifies the associated Moobot the Cookbook Moolet loads at startup.

Type : String
Default : Cookbook.js

process_output_of : Defines the source of the alerts that Cookbook processes.

Type : List
One of: AlertBuilder, AlertRulesEngine, MaintenanceWindowManager, EmptyMoolet
Default : AlertBuilder

The default Cookbook parameters are as follows:

name 				: "Cookbook",
classname 			: "CCookbook",
run_on_startup  	: false,
persist_state       : false,
metric_path_moolet  : true,
moobot              : "Cookbook.js",
#process_output_of  : "AlertRulesEngine",
process_output_of   : "MaintenanceWindowManager",

Note

Do not change name and classname . These are hardcoded.

You can also create and configure Cookbooks under Settings in the Moogsoft AIOps UI. These run alongside Cookbooks created in moog_farmd. For more information see Cookbooks .

Configure the Algorithm

You can configure t he Cookbook algorithm in $MOOGSOFT_HOME/config/moog_farmd.conf using the following parameters:

membership_limit : The number of Situations an alert can be part of. This does not impact alerts in merged Situations. Decreasing this limit decreases the number of Situations with many alerts but increases the number of Situations with a smaller number of associated alerts and vice versa. The optimal value is between 1 and 5.

Type : Integer
Default : 3

scale_by_severity : Treat alerts with a higher severity like alerts with a higher entropy value. Moogsoft AIOps divides the severity number by the maximum severity (5) to calculate the scale. For example, for an alert with minor severity, the entropy would be 3/5.

Type : Boolean
Default : False

entropy_threshold : The minimum entropy value for an alert to be clustered into a Situation. Cookbook does not include any alerts with an entropy value below the threshold in Situations. Set to a value between 0.0 and 1.0 . The default of 0.0 means all alerts are processed.

Type : Decimal
Default : 0.0

Configure Alert Cluster Matching

You can enable additional parameters to impose a priority order on your recipes and configure alert cluster matching.

single_recipe_matching : Enabling single_recipe_matching causes Cookbook to treat Recipes as being in an order of priority, based on the order of configuration in this file, highest priority first.

Individual alerts may only appear in a single Situation generated by a particular Recipe. Subsequently the same alerts can also reappear in a Situation generated by a higher priority Recipe.

Type : Boolean
Default : false

cluster_match_type : Defines how Cookbook matches clusters. You can select the first_match in order so alerts are added to the first cluster over the similarity threshold value. This is the default behaviour for Cookbook.

Alternatively, select closest_match to add alerts to the cluster with the highest similarity greater than the similarity threshold value. This option may be less efficient because Cookbook has to compare alerts against each cluster in a Recipe. The cluster_match_type parameter is overwritten by the definition in the Recipes.

Type : List
Default : "first_match",

Configure Recipes

You can create Recipes to determine the alerts relationships to detect using Cookbook.

You can configure Recipes with different event filters, triggers and similarity comparisons using these parameters :

chef : The recipe type: CValueRecipe or CBotRecipe. A CValueRecipe clusters according to the recipe definitions whereas a CBotRecipe follows custom clustering logic defined by a Moobot. This is mandatory.

Type : String
One of: CValueRecipe, CBotRecipe
Default : "CValueRecipe",

name : Name of the Recipe. Use a unique or descriptive name . This is mandatory.

Type : String
Default : "SplitBySourceAndDescription",

description : Description of the Recipe.

Type : String
Default : "Value Recipe outage",

recipe_alert_threshold : The maximum number of alerts to cluster before creating a Situation . If left as the default '0', there is no limit so a single alert can generate a new Situation.

Type : Integer
Default : 0,

exclusion : Defines a filter determining the alerts to exclude from Situation creation. By default Cookbook excludes all alerts with a severity less than critical. For details on creating a filter, see Advanced Filter Query Syntax .

Type : String
Default : "severity < 5",

trigger : Defines a filter determining the alerts that Cookbook considers for Situation creation. Cookbook ignores alerts that match the exclusion filter.

Type : String
Default : "null",

seed_alert : Defines a filter determining whether to create a Situation from a seed alert if it meets both trigger and seed_alert filter criteria. Cookbook considers subsequent alerts for clustering if they meet the trigger filter criteria. Alerts that arrived prior to the seed alert that met the trigger filter criteria do not form Situations.

The seed_alert filter is a mechanism to ensure that only specific events create Situations . For example, if you create a seed_alert filter if the description matches 'Switch failure', alerts are only eligible for clustering after a seed alert with the matching description arrives to create a Situation.

The default of " null " means Cookbook ignores the seed_alert filter.

Type : String
Default : "null",
Example : 'Description' MATCHES "Switch failure",

rate : Defines a filter determining the minimum event rate per minute required for Cookbook to create a Situation. Cookbook only calculates the rate after the cluster meets the threshold defined by min_sample_size or max_sample_size .

Type : Integer (Number of events per minute).
Default : "0",

min_sample_size : The minimum number of events contained in a cluster before Moogsoft AIOps calculates the rate.

Type : Integer
Default : "5",

max_sample_size : The maximum number of events contained in a cluster before Moogsoft AIOps calculates the rate.

Type : Integer
Default : "10",

cook_for : The length of time in seconds a Recipe retains cluster information before discarding it, at which point Cookbook resumes analyzing the event stream . Different cook_for times per Recipe are useful for monitoring systems with different fail rates, to ensure the Recipe clusters all the relevant Events relating to a failure. For example:

  • A Recipe monitoring for network link failures, which have a fast fail rate and many e vents in a short time, should have a short cook_for time
  • A Recipe monitoring for disc or CPU issues, which have a slower fail rate as the issue builds, should have a longer cook_fo r time.

You can define cook_for values at the Recipe level but this is optional because Recipes inherit the Cookbook's cook_for value.

Type : Integer
Default : "5000",

cluster_match_type : Defines how Cookbook matches alerts to clusters. The first_match default option adds alerts to the first cluster above the similarity threshold value. The alternative is closest_match to add alerts to the cluster with the highest similarity greater than the similarity threshold value. The latter option may be less efficient since it has to compare alerts against each cluster in a Recipe.

Type : String
Default : "first_match",

Configure Matcher for a CValueRecipe

You can add additional similarity values under matcher that cause Cookbook to cluster alerts . For a CValueRecipe there are two parameters: components and hop_limit .

components : The component values that alerts must match for Cookbook to include them in a Situation. You can provide multiple values such as source, description, service etc. using custom_info fields.

To make Cookbook match each value in the list individually, set "treat_as : list" in the component configuration. For example, components: [ { name: "custom_info.source_id", similarity: 1.0, treat_as: "list" } ] . If this configuration is not used, Cookbook treats the components value as a string.

Type : String.
Default : { name: "source_id", similarity: 1.0 }, { name: "description", similarity: 0.5 }

hop_limit : The maximum number of hops between the alert source nodes in order for the alerts to qualify for clustering. This is measured from the first alert that formed the Situation and always follows the shortest possible route in the network.

A hop is the jump between two directly connected nodes in a network. For example, if you have three nodes ; node A, node B and node C, the number of hops between node A and node C is two. If the number of hops between the nodes exceeds the hop_ limit , no cluster is formed. For more information on hops, see Vertex Entropy .

Type : Integer
Default : "2",

Configure Matcher for a CBotRecipe

You can add similarity values under the matcher block that cause Cookbook to cluster alerts. For a CBotRecipe there is a calculated similarity based on a defined MooBot function call.

This example of a CBotRecipe that moog_farmd includes by default:

matcher: {
          initialise_function : "initBuckets",
          member_function     : "checkBucket",
          similarity          : 0.8
         }

Example Recipes

There are a number of example recipes included in $MOOGSOFT_HOME/config/moog_farmd.conf by default.

The example CValueRecipe below shows a recipe that splits alerts into clusters with either an identical source_id (hostname) or a description that is 50% similar. It also only creates a Situation from a seed alert with a Vertex Entropy value of 0.75 which indicates a node of high topological importance. See Vertex Entropy for more information.

{
    chef: "CValueRecipe",
    name: "SplitBySourceAndDescription",
    description: "Value Recipe outage",
    recipe_alert_threshold: 0,
    exclusion: "severity < 5",
    trigger: null,
    seed_alert: "vertex_entropy = 0.75",
    rate: 0,
    #Given in events per minute
    min_sample_size: 5,
    max_sample_size: 10,
    cook_for: 5000,
    cluster_match_type : "first_match",
    matcher: {
        components: [{
                name: "source_id",
                similarity: 1.0
            },
            {
                name: "description",
                similarity: 0.5
            }
        ]
    }
}

The example CValueRecipe below shows a recipe that can be used alongside a New Relic integration.

This recipe clusters alerts that have an identical source_id (hostname) every fifteen minutes:

{
    chef: "CValueRecipe",
    name: "New Relic Hostname Recipe",
    description: "Recipe to create situations based on 100% similarity of the hostname received from New Relic",
    recipe_alert_threshold: 1,
    exclusion: null,
    trigger: null,
    seed_alert: null,
    rate: 0,
    #Given in events per minute
    min_sample_size: 5,
    max_sample_size: 10,
    cook_for: 900,
    matcher: {
        components: [{
            name: "source_id",
            similarity: 1.0
}

The example CBotRecipe below shows a recipe that uses methods in the Cookbook.js Moobot to cluster by topological similarity.

It excludes alerts that have a severity of less than minor and clusters alerts that are 80% similar.

{
    chef: "CBotRecipe",
    name: "MaxwellDaemon",
    description: "Maxwell Recipe outage",
    recipe_alert_threshold: 0,
    trigger: null,
    exclusion: "severity < 3",
    rate: 1,
    #Given in events per minute
    min_sample_size: 5,
    max_sample_size: 10,
    cluster_match_type : "first_match",
    matcher: {
        initialise_function: "initBuckets",
        member_function: "checkBucket",
        similarity: 0.8
    }
	cook for: 2000,
}