Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

High Availability (HA) deployments of Moogsoft AIOps comprise multiple instances of Moogsoft AIOps, moogfarmdMoogfarmd, and the associated LAMs to minimize downtime and data loss. Component redundancy protects against single points of failure. It also provides reliable mechanisms to enable failover from one component to another to avoid performance degradation and data loss.

...

Instances are individual components that run on a single machine. Process Groups and Clusters, however, can span multiple machines. Their configuration allows the flexibility to define architectural groupings for failover actions as long as they are within the same MooMS Zone:

ComponentDescription
Active / Passive modeInstances are configured to operate in Active or Passive mode. Instances that are operational are set to Active mode. Instances that are backup (redundant components) operate in Passive mode
Leader

Defines which Instance in a Process Group becomes Active when the whole Process Group switches from Passive to Active state. Normally, only one Instance per Process Group should be defined as a Leader. Leader definition availability is as follows:

ComponentLeader definition availability
moog_farmdMandatory for a Process Group with more than one moog_farmd
Socket LAM, Logfile LAM,TrapdLAMOptional
REST LAM, UI servletsNot applicable

Leadership status is a property of Process Groups. There are two states of group leadership status (as seen in the output of  haha_cntl --view command. See below) as follows:

StateDescription
"only leader should be active"This is the default setting for components where Leader definition is supported (in moog_farmd, Socket LAM, Logfile LAM, Trapd LAM)
"no leader - all can be active"This is the default setting for components where Leader definition is not supported (REST LAM and the UI servlets), OR if there is one or more Instances in the Process Group configured with "only_leader_active = false,". The behavior is dynamic, i.e. terminating such an Instance will change back the Process Group's status to "only leader should be active"

...


The above creates an Instance of  moogmoog_farmd and defines it as a member of the Surbiton Cluster, with the Instance name MASTER. It also defines it as the Leader Instance in its Process Group and configures it to operate in Active mode. No Process Group (--group) is defined, so the default name (from the component configuration file) moog_farmd is used.

...

When the UI is in Passive mode, the moogsvr servlet will reject all requests with an HTTP status of 503  (server unavailable) and the moogpoller servlet will not accept incoming websocket upgrade requests. When switching from active to passive the moogsvr servlet will start rejecting requests and the moogpoller servlet will disconnect any existing websocket sessions.

A Load balancer can therefore determine whether a UI is running in Active or Passive mode by sending a GET request to https https://<server>Li<server>:<port>/moogsvr/hastatus. A  204 204 response indicates that the UI is Active, a 503 response indicates Passive mode.

The following example curl command can be sent from the command line to check servlet status:

...

Info

All Instances of moog_farmd within the same Process Group must have identical configuration.

LAMs

LAMs operating in Passive mode do not send Events to the MooMS bus. The REST LAM in Passive mode will reject POST requests with an HTTP status of 503 (server unavailable).

...

and as previously mentioned, ensure that the messagethe message_persistence property in the MooMS section of the systemthe system.conf file is set to trueto true:

Code Block
languagetext
"mooms": 
{
	"zone": "MOOG",
	"brokers": [
      {
		"host": "localhost",
		"port": 5672
	  }
    ],
	"username": "moogsoft",
	"password": "m00gs0ft",
	"message_persistence": true,
	"max_retries": 100,
	"retry_interval": 200,
	"cache_on_failure": false,
	"cache_ttl": 900
}

...