Self Monitoring

The table below describes the possible states for a package:

You can set processes as reserved or unreserved in the system.conf file ($MOOGSOFT_HOME/config/system.conf. If a package's 'reserved' setting is 'true', the self monitoring reports a warning if the package is not running. Stopped unreserved processes do not generate warnings.

The Self Monitoring screen is divided into a number of tabs. Each section displays the states of the various processes, indicating which are running or which have issues:

This tab, which is open by default when Self Monitoring is launched, displays event processing times and other metrics.

The icon in the top left corner indicates the overall state of event processing. This is determined by the Current Maximum Event Processing Time in seconds. This time is indicated by the position of the gray bar on the colored bullet graph shown below. The Current Maximum Event Processing Time is 1.917s in this example:

The default bullet chart color values are as follows:

The numeric value itself may not be an absolute measurement of health, so as a general rule, look for unusual or sudden changes in the values or behavior. See the examples below:

All of these can result in the bullet chart showing an increasing Current Maximum Event Processing Time, from green to yellow to red.

To use the Processing Metrics tab, open the LAMs and moog_farmd folders and look for deviations from normal values.

The numeric value itself may not be an absolute measurement of health, so as a general rule, look for unusual or sudden changes in the values or behavior. See the examples below:

Both of these result in the bullet chart (at the top) showing increasing Current Maximum Event Processing Time, from green to yellow to red.

This tab contains a process group including Moogfarmd (the core Moogsoft Enterprise application) and the Moolets, such as AlertBuilder, Alert Rules Engine, Sigalisers.

The icon in the top left corner indicates the overall state (running normally in the example above). The group and cluster names are displayed in the top right corner. The time and date of the last heartbeat is displayed above the list of Moolet processes.

This tab contains all processes related to Tomcat web applications: moogsvr, moogpoller, toolrunner and Graze.

Each row displays the following information:

This tab displays information about the state of all processes relating to the LAMs and the individual processes which process raw data and create events:

The controls in the far right column can be used to stop and restart active LAM processes or to start inactive LAMs.

The final tab provides a link to the Message Bus Console, also known as the MooMs (Moogsoft Messaging System). This is hosted by message-queueing software RabbitMQ.

Click the link to proceed to the RabbitMQ management console.

The username and password to log in are specified and can be configured in $MOOGSOFT_HOME/config/system.conf (under mooms.username and mooms.password in the JSON) and correspondingly in RabbitMQ. See Configure the Message Bus for more information.

Once logged in, RabbitMQ displays information about message rates, connections, channels, queued messages, etc.

The 'Restart/Stop/Start' feature uses the moogfarmd/LAM service scripts under /etc/init.d, for example, /etc/init.d/moogfarmd and /etc/init.d/logfilelamd, in combination with the Apache Tomcat 'toolrunner'.

You need Super User role permissions to configure this feature. Create a user in the 'moogsoft' group. This user must be used by the toolrunner and the services in order to start/stop services via the UI. For example:

Moogsoft recommends that you do not use the default 'moogsoft' user because that is a system user and does not allow you to log in using a password. Update the /etc/init.d/ service scripts to have the correct:

Add the name of the service script into the 'service_name' field in $MOOGSOFT_HOME/config/system.conf for that Moogsoft Enterprise process. To ensure the service appears in the right Self Monitoring tab, the process_type field must be set. See the default system.conf file for examples.

If a Moogfarmd service or LAM service is run that does not match a configuration block in system.conf/'processes', then it still appears within the UI 'Self Monitoring' dialog, but it is not possible to start/stop/restart the service.

The 'toolrunner' is used to control the services (requires configuring $APPSERVER_HOME/webapps/toolrunner/WEB-INF/web.xml):

It is more likely that an existing LAM/Moogfarmd service will have been run already in upgrade scenarios. If the service is one which needs to be controlled via the UI, then the service log file and PID (if present) need to be 'chowned' to the new service script PROCESS_OWNER/toolrunner user before it will work. For example:

chown toolrunneruser /var/log/moogsoft/moogfarmd.log

See the example of a $MOOGSOFT_HOME/config/system.conf file below:

{
"group"         : "moog_farmd",
"instance"      : "",
"service_name"  : "moogfarmd",
"process_type"  : "moog_farmd",
"reserved"      : true,
"subcomponents" : 
    [
        "Event Workflows",
        "AlertBuilder",
        "Default Cookbook",
        "Journaller",
        "TeamsMgr"
        #"Alert Workflows",
        #"SituationMgr",
        #"Notifier"
    ]
},