Configure Events Analyser

You can configure the Events Analyser to analyze all the event data received by Moogsoft AIOps together or to analyze event data by partitions or streams. See the Events Analyser for more information on these options.

Edit the configuration file at $MOOGSOFT_HOME/config/events_analyser.conf to control the behavior of the Events Analyser.

See the Events Analyser Reference for a full description of all properties. Some properties in the file are commented out by default. Uncomment the properties to enable them.

To configure the Events Analyser:

  1. The default configuration uses the EntropyV2 calculation method. Moogsoft recommends using the default EntropyV2 calculation method for calculating entropy values because it has improved modelling of alert probabilities. However, if you want to, you can change the setting to use the EntropyClassic calculation method. Entropy data for EntropyClassic and EntropyV2 calculation methods are not compatible. If you switch between the two calculation methods, you must execute a full priming run of the Events Analyser after you have changed the setting to ensure that all the entropy data matches the same configuration. See Run Events Analyser for further details on executing a full priming run of the Events Analyser.

  2. Use the default values for the priming_source_data.

  3. Configure whether or not the Events Analyser partitions the entropy data. See the ??? and the ??? for further details.

  4. Configure the "default" Events Analyser behavior. See the ??? for further details.

  5. If using partitioned data, configure the Events Analyser for any partitions that you want to behave differently. If you do not add a separate configuration for a partition, the Events Analyser uses the "default" configuration for that partition. The Events Analyser also uses the "default" configuration for any properties that are not defined in a partition configuration. See the ??? for further details.

Example of non-partitioned data

The default configuration file at $MOOGSOFT_HOME/config/events_analyser.conf, similar to the example shown below, contains a non-partitioned configuration. The "partition_by" property has been set to null to show that the entropy data is not to be partitioned. The "default" settings have been configured for all entropy values. See the Events Analyser Reference for further information on these properties.

{
    "entropy_calc": "EntropyV2",
    "priming_source_data" :
            {
                "alerts_table" : "alerts",
                "events_table" : "events",
                "snapshots_table" : "snapshots",
                "timestamp_column" : "last_event_time"
              },
    "partition_by" : null,
    "default" :
        {
            "fields" :
                [
                    "description"
                 ],
            "mask" :
                {
                    "ip_address"    : false,
                    "mac_address"   : false,
                    "oid"           : false,
                    "date_time"     : true,
                    "number"        : true,
                    "path"          : false,
                    "guid"          : false,
                    "hex"           : false,
                    "url"           : false,
                    "email"         : false,
                    "word"          : false,
                    "stop_word"     : false
                },
            "casefold" : true,
            "stop_words" : true,
            "stop_word_length" : 0,
            "stop_word_file" : "stopwords",
            "priority_words" : false,
            "priority_word_file" : "prioritywords",
            "stemming" : false,
            "stemming_language" : "english"
        }

Example of partitioned data

The example below shows additional configuration of the Events Analyser for two partitions "san_francisco" and "new_york". These settings override the "default" configuration in the example of non-partitioned data above.

In this example, the source field is used to partition the entropy data:

"partition_by" : "source",

The configuration for the "san_francisco" partition uses the description, agent and source fields for calculating entropy values and does not use stop words. The "new_york" partition uses different masking properties to the "default" configuration: date_time is not masked but ip_address, email, and url are masked. This partition also uses stemming for calculating entropy values. Since the language is not specified, the default of English is used. All other properties that have not been configured in these partitions will use the properties in the "default" configuration.

If there are any other partitions, for example, "los_angeles", that do not have any properties specified in the configuration file, they will use the "default" configuration.

See the Events Analyser Reference for further information on these properties.

, "partition_overrides" :
  {
    "san_francisco": {
      "fields": [
        "description"
      ],
      "casefold": true,
      "stop_words": false,
      "priority_words": false,
      "stop_word_length": 5
    },
    "new_york": {
      "mask": {
        "date_time": false,
        "ip_address": true
      },
      "stemming": true,
      "stop_words": true,
      "priority_words": true,
      "stop_words_list": [
        "france",
        "germany",
        "italy",
        "peru",
        "india",
        "japan",
        "korea"
      ],
      "stop_word_length": 1,
      "priority_words_list": [
        "reboot",
        "shutdown"
      ]
    }
  }

Disabling entropy calculations

Moogsoft recommends that you configure the clustering algorithms to use entropy thresholds so that they exclude 'noisy' alerts which contain low levels of important information. This allows operators to concentrate on Situations containing important alerts. See the Clustering Algorithm Guide for more information. However, if you do not intend to use entropy calculations, you should:

  • Set the 'entropy_calc' property to 'EntropyClassic'.

  • Set the 'properties_from_db' property to 'false' for all running Alert Builder Moolets.