Choosing Threshold Values

You can think of information entropy as a proxy for importance. It's possible that the most common alerts with the lowest entropy are only contributing to noise in your system, and that you don't want them pulling your operators' attention away from what is really important. Each system is different, though, so the Alert Analyzer lets you customize Cookbook entropy thresholds to the patterns in your data, including setting different thresholds for different managers.

Overview

Set global and subset entropy thresholds.

  1. Experiment with setting different entropy thresholds by clicking on the entropy distribution graph or entering values in the threshold entropy value box. Examine the excluded and included alerts at each threshold, and use your best judgment to choose a global default threshold that excludes the maximum number of unimportant alerts while retaining important alerts. Note that you can use a percentage threshold if you want to exclude a fixed share of alerts, or use a fixed threshold if you want to exclude alerts below a specific entropy threshold.

  2. Using the left navigation menu, go to the subset threshold display and review the entropy distributions for each manager. Use your judgment to set subset thresholds that are different from the global default threshold if appropriate. For some managers, you might decide not to set a subset threshold.

Step-by-Step Instructions
  1. Look at the entropy distribution chart. You'll see that the distribution has gaps, but there is a very tall spike at about .28. You'll also see that in this data the number of alerts generally decreases as entropy values increase.

  2. If you have not already done so, click on 'Included Alerts' in the review window to see the alerts, which are sorted in order of entropy. Examine the descriptions and managers.

  3. It seems that all the alerts with entropies of .28 have the description 'Backup Files Present'. These alerts seem unimportant, so set a higher entropy threshold by clicking on the graph to the right of the tall spike on the left. Notice in the threshold statistics above that excluding this spike excludes more than a quarter of the alerts in the system--a substantial noise reduction.

  4. Examine the included and excluded alerts at the threshold and above. Do you think you should set a higher threshold to exclude more alerts? In real life, you would want to consult with your operators about which alerts they feel are actionable and should be further processed by clustering in Situations.

  5. Note that the padlock icon next to the 'Percentage of System Excluded' box is locked, and the 'Threshold Entropy Value' padlock is unlocked. The percentage entropy threshold excludes exclude a fixed share of alerts, which allows for some drift in the entropy values of your live data.

    For this lab we will use a fixed global entropy threshold. Click on the 'Threshold Entropy Value' padlock to change from a percentage threshold to the equivalent fixed value, and notice that the value padlock is now locked.

  6. Click 'Save' to save your global default entropy threshold.

  7. It's likely that alerts from different sources have different entropy distributions, so you may want to set different thresholds for different managers. Click on the '>' on the upper right to expand the menu and click on 'Create Subset Threshold'. Refresh your browser if necessary to activate the subset threshold button.

  8. Examine the visualization pane for the subset thresholds.

    1. There is a dropdown menu at the top which shows the managers for which you can configure thresholds.

    2. There are now two summary statistics bars, one for all alerts (gray), and one for subsets (blue).

    3. The chart shows two sets of graphs: subset alerts are in blue and all alerts are in faint gray.

    4. You can remove the 'all alerts' graph from the display by clicking on the legend button at the bottom.

    5. The suggested default subset threshold is 10%, but this is freely configurable.

  9. Click on 'Included Alerts' and examine the alert descriptions for the first manager, BNT. All of the 'Backup Files Present' alerts belong to BNT, and they will be excluded by the global default threshold. The next spike of alerts for BNT is at about .38, and they are Network Time Protocol server down messages. Depending on where you set the global threshold, if you decide you want to include the NTP alerts the global threshold may work well for BNT.

  10. Switch to the next manager, GS. Click to the left of the first blue data spike in the graph to reset the subset threshold to 0.0% and then click on 'Included Alerts.'

  11. Examine the alert descriptions and entropies. The lowest entropy value for GS is higher than for other managers. The lowest two spikes for GS are for 'DSL Modem Status Update' and 'GNK Status Update' alerts. If you set a separate, higher subset threshold for GS, you can exclude these status update alerts from clustering. Clear the value in the threshold entropy box and enter '.46'.

  12. Click on the percentage padlock to change to the equivalent dynamic threshold.

  13. Examine the included and excluded alerts at this threshold and then click 'Save'.

  14. Go back to the left navigation menu. You will see that you now have a subset threshold defined for the GS manager.

  15. Click on 'Create Subset Threshold' again and examine the distributions for the other managers. Use your best judgment to decide whether or not to set and save subset thresholds for the other managers.

This concludes the lab section.