Moogsoft Docs

Situation Design Maintenance

Your monitored environment and business practices are constantly evolving, so you need to change your alert clustering settings over time. This topic discusses what you can do to detect changes and update your clustering logic.

Ensure all important alerts become Situations

PeriodicExamining.png
LooseAlerts.gif

Your teams can examine the content of these Situations periodically to ensure nothing important has been missed. If an operator identifies a loose alert that is of low entropy but should be part of a Situation, they can take the following actions:

  • Create a specific Cookbook Recipe just for that alert type.

  • Investigate lowering the entropy threshold.

  • Create a list of priority words that bias the entropy of corresponding alerts. Events containing a word in the priority words are always given an entropy value of 1. Choose priority words with care. They should be very uncommon; a widely used word bumps up the entropy for all alerts containing that word. See Events Analyser for more information on priority words.

These methods also provide a good way to validate your Situation design after initial configuration. During the tuning phase, check to see if any of the loose alerts (alerts that are not clustered into any Situations) should be creating Situations on their own or be part of existing Situations created at the time.

Avoid old Situations crowding the workspace

Autoclose.png
Autoclose.gif

If you do not action Situations, but leave them open, your system may become overrun with old open Situations. This is not only distracting for the operators, but also has performance implications as active Situations are retained in memory for merge and resolve activities. Keeping alerts open prevents them from receiving renewed enrichment data.

You can set up an Auto Close action on unworked alerts and Situations of a specified age to avoid such problems. See Auto Close for more information. See Archive Situations and Alerts for more information.

Handle noisy alerts

Alerts with high numbers of events impact performance when data is moved from the active to historic database. You can implement an Auto Close action in the Alert Rules Engine, so that when an overly noisy alert goes beyond a certain event count limit it is closed automatically.

See below for a sample configuration in the Alert Rules Engine. In this case, alerts with an event count higher than 10000 are transitioned to the CloseAlert state. Once the alert enters the state it is processed by the closeAlert function in ARE.

The benefit of using the Alert Rules Engine instead of the usual Auto Close rule is because Auto Close requires the alert to be older than a certain period (minimum 5 minutes given the task run) while the Alert Rules Engine closes the alert instantly once it reaches the count limit.

Note

This function only closes alerts that are not part of any active Situations.

transition__1_.png
action_state__1_.png

In AlertRulesEngine.js:

// function in ARE to close an alert that is not part of any active Situations
function closeAlert(alert,associated){
     var alert_id=alert.value("alert_id");
     if (alert.value("active_sig_list").length === 0 ) {
         var closed_flag=moogdb.closeAlert(alert_id);
         if (closed_flag) {
                 logger.debug("Alert closed by ARE. Alert_id: " + alert_id);
         } else {
                 logger.warning("closeAlert: Alert failed to close by ARE. Alert_id: " + alert_id);
         }
     } else {
         alert.forward(this);
     }
}