Run Events Analyser
The Events Analyser is responsible for analysing the tokens within alerts and calculating their entropy values. The Events Analyser updates the alerts with the calculated entropy value and also updates the reference database with all the tokens and their associated entropy values.
-
Command Line Options
-
Run Events Analyser
-
Daily Run
-
Hourly Run
-
Run Events Analyser Manually
-
-
Multiple Streams and Partitions
-
Usage Examples
Command line options
The events_analyser
command line executable accepts the following
options:
Option |
Input |
Description |
---|---|---|
|
String: <file path/name> |
Name and path of the configuration file specific to running the Events Analyser. The
default is Example: |
-l, --loglevel <arg> |
One of: WARN, INFO, DEBUG, TRACE, OFF |
Specifies the amount of logging information. Defaults to WARN, which is the recommended level in all production implementations. |
--incremental |
- |
Analyzes only new event data that was received since the last time the Events Analyser was run. |
|
Number, followed by one of:
|
Amount of data to analyse, in seconds, minutes, hours, days or weeks. Example: |
|
Number, followed by one of:
|
Amount of data to keep, in seconds, minutes, hours, days or weeks. Example: |
|
String: <alert stream name> |
Stream name to be given to the current analysis. Example: |
|
String: <partition value> |
Name of the partition to be analyzed. It must be a valid value of the Example: |
Run Events Analyser
Moogsoft recommends that you run the Events Analyser regularly as follows:
-
Daily: analyzes the last two weeks of data.
-
Hourly: in incremental mode which analyses all new event data since the last time the Events Analyser was run.
These default settings are specified in moog_init_server.sh
.
You can also run the Events Analyser manually on an ad hoc basis.
Daily run
To initiate a daily run, that is, where all entropy values are calculated for the last two weeks of event data, you should specify the Events Analyser to run with the following command line options:
./events_analyser --readage 2w
In this case, the Events Analyser:
-
Uses the default configuration file
$MOOGSOFT_HOME/etc/events_analyser.conf
. -
Analyzes all data received in the last two weeks, based on the
timestamp_column
property in theevents_analyser.conf
file. -
Adds all analyzed data to the reference database for the default stream.
-
Leaves any data for other, named streams unchanged.
Hourly run
The Events Analyser utility provides the ability for incremental priming. When the Events
Analyser utility is run repeatedly with the --incremental
option, each
subsequent run of the utility analyses the event data starting from the last analyzed event. For
example, if the first run analyzes data up to event ID = 666, the next incremental run of the
utility analyzes data from 667 to say 999, the third incremental run reads in data from event ID
1000, and so on.
To initiate an hourly run, that is, where all entropy values are calculated since the last analyzed event, you should specify the Events Analyser to run with the following command line options:
./events_analyser --incremental
In this case, the Events Analyser:
-
Uses the default configuration file
$MOOGSOFT_HOME/etc/events_analyser.conf
. -
Analyzes all data since the last incremental run, based on the
timestamp_column
property in theevents_analyser.conf
file. -
Adds all analyzed data to the reference database for the default stream.
-
Leaves any data for other, named, event streams unchanged.
Run Events Analyser manually
To run the Events Analyser manually, you can run it without any command line options. This command runs the Events Analyser for all new event data received in the last two weeks or since the last analysis, whichever is most recent.
./events_analyser
In this case, the Events Analyser:
-
Uses the default configuration file
$MOOGSOFT_HOME/etc/events_analyser.conf
. -
Analyzes all event data received in the last two weeks or since the last time the Events Analyser was run, whichever is most recent, based on the
timestamp_column
property in theevent_analyser.conf
file. -
Adds all analyzed data to the reference database for the default stream.
-
Leaves any data for other, named, event streams unchanged.
To run the Events Analyser to analyze event data over a longer period, you should include the
--readage
option. In this example, the --readage
option is set to 13 weeks:
./events_analyser --readage 13w
In this case, the Events Analyser:
-
Uses the default configuration file
$MOOGSOFT_HOME/etc/events_analyser.conf
. -
Analyzes all event data received in the last 13 weeks.
-
Adds all analyzed data to the reference database for the default stream.
-
Leaves any data for other, named, event streams unchanged.
Note
If you use a large value in the--readage
option, you may find that the
Events Analyser fails to complete the analysis. If this occurs, rerun it using a shorter period
of time.
Multiple streams and partitions
You can run the Events Analyser for specific streams or partitions. In this example, the --stream
option is specified to add the analyzed data to the "SECONDARY"
event stream. The --readage
option restricts the data analyzed to the
last eight weeks of event data.
./events_analyser --stream “SECONDARY” --readage 8w
In this case, the Events Analyser:
-
Uses the default config file
$MOOGSOFT_HOME/etc/events_analyser.conf
. -
Analyzes all event data received in the last eight weeks, based on the
timestamp_column
property in theevent_analyser.conf
file. -
Adds all analyzed data to the reference database for the “SECONDARY” event stream.
-
Leaves data for all other, named, event streams unchanged.
You can use the --partition
option to limit the data that is analysed
to a specified partition. In this example, the --readage
option
restricts the data analyzed to the last four weeks of event data:
./events_analyser --stream “SECONDARY” --partition “SanFrancisco” --readage 4w
In this case, the Events Analyser:
-
Uses the default config file
$MOOGSOFT_HOME/etc/events_analyser.conf
. -
Analyzes all event data received in the last four weeks for the “SanFrancisco” partition only.
-
Adds all analyzed data to the reference database for the “SanFrancisco” partition in the “SECONDARY” event stream.
-
Leaves data for all other event streams and partitions unchanged.
Note
Moogsoft recommends that you always use the --readage
option when analyzing streams or partitions to ensure that the
Events Analyser processes the required amount of data. If the --readage
option is not specified, the Events Analyser only analyzes new
event data received in the last two weeks or since the last analysis, whichever is the most
recent, regardless of whether this was for a different stream or partition.
Usage examples
There are many combinations of command line options. Some common usage scenarios include:
Command Line Options |
Typical Use Case |
|
|
|
|
|
|
|
|
|
|