Entropy
Entropy is defined as the degree of disorder or randomness in a system. In Moogsoft Enterprise, entropy is a measure of how unexpected or unpredictable an event or an alert is. According to information theory, the more unpredictable or unexpected an event is, the more information it is deemed to carry. Therefore, entropy is a measure of the amount of information contained in an event.
The Alert Analyzer Utility utility is a standalone process that assigns an entropy value to an event token based on its uniqueness. The Configure Alert Builder assigns an entropy value to each alert based on the token entropies. The entropy value is a numeric value between 0 and 1 (accurate to 16 decimal places). It provides an indication of how important an alert is. An entropy value of 0 means that the alert is just ‘noise’ and a value of 1 means that the alert is significant. You can configure the clustering algorithms to ignore common alerts with a low entropy value; this reduces ‘noise’ in Moogsoft Enterprise. See the Clustering Algorithm Guide for more information.
How Moogsoft Enterprise evaluates entropy
The Alert Analyzer analyzes the text attributes of events to assign a semantic entropy value. In the default Moogsoft Enterprise implementation, the Alert Analyzer uses the description field but you can configure it to use other text fields. The Alert Analyzer divides the text in between spaces into tokens. For example, the following description has five tokens:
Link down on port 2/32
The Alert Analyzer calculates the entropy of each token and stores the token in the Moogsoft Enterprise reference database with its associated entropy value. Initially, a new token has a value of 1. The Alert Analyzer reduces this entropy value as more events occur which contain the same token.
You can configure the Alert Analyzer to mask volatile token types, such as dates, times, numbers, URLs or IP addresses, so that they are not included in the tokens. See the Alert Analyzer Utility for further details of the analysis it performs.
The Alert Builder uses the entropy value of the tokens within an alert to calculate the entropy of that alert.
The Alert Analyzer calculates entropy values in real-time based on any tokens it has encountered before. The Alert Builder assigns the entropy of an alert based on the entropy value of the tokens within the alert rather than the entire database. Tokens within an alert which occur frequently contribute negatively to the entropy of an alert, indicating that the alert may not be as significant as an alert with tokens that are seen less frequently.
If the Alert Builder receives an event with a token that it has encountered before, from a previous run of the Alert Analyzer, it sets the alert entropy to match the value saved in the reference database. If the Alert Builder receives an event with a token that it has not encountered before, it calculates the entropy value in real-time and applies this value to the alert. The Alert Builder also saves the entropy value in the reference database for future retrieval.
Setting entropy thresholds
You can set a global entropy threshold that Moogsoft Enterprise uses for all alerts, or you can set entropy thresholds for different managers within Moogsoft Enterprise. You can set these entropy thresholds using the Moogsoft Enterprise UI or the Graze API.
See Configure Entropy Thresholds with Alert Analyzer for more information.
Configuring and running the Alert Analyzer
You can configure the Alert Analyzer to include or exclude information in the entropy calculations. See Configure Entropy Generation Details for more information.
The Alert Analyzer is preset to perform an incremental run of the Alert Analyzer at 3 am every day, with a keep age of two weeks. Moogsoft recommends this as the optimum entropy generation schedule. However, if you want to change this schedule, see Configure Entropy Generation Schedule.
Vertex Entropy
Vertex Entropy uses a different form of entropy, topological entropy, to establish how critical the nodes are in your network topology. You can use Vertex Entropy calculations within Cookbook to create Situations which cluster alerts from important nodes. See Vertex Entropy for more information.