Page tree
Skip to end of metadata
Go to start of metadata

Overview

Zenoss is an Infrastructure monitoring application that monitors network, server, storage, power devices etc. It is distributed software that monitors the IT environment and generates events based on data which has been exposed with rest APIs or from third parties. This document describes the configurations required to establish a connection between the Zenoss application and the Zenoss LAM.

Process workflow

  1. The LAM reads configuration from given zenoss_conf file.
  2. The LAM connects with the host and request for the event console data.
  3. The event data of all the devices in Zenoss is received by the LAM as a response. The format of received events is JSON.
  4. If uid_filter variable in monitor section is set, then the LAM filters the events based on the uid_filter value.
  5. The LAM parses the events and convert it into map messages and then publish it to MooMS bus.
  6. After parsing, the event data is filtered and the device data along with unmapped event data is sent to the overflow variable.
  7. The LAM creates events that are submitted to MooMS bus.
  8. The events are published to the subject “/Events”.
     

HTTP request with basic user authentication is used in the Zenoss LAM

Lam uses http://"IP name":8080/zport/dmd/evconsole_router” as end point for event request

A user can configure how the Zenoss LAM processes events received from a Zenoss server by accessing the zenoss_lam.conf file, at the following path  /usr/share/moogsoft/config

Zenoss configuration

The Zenoss server does not need any configuration to connect to the Zenoss LAM, only the following information is required which can be entered in the zenoss_lam.conf file.

  • The URL of the Zenoss server along with the port
  • Username and  password of the Zenoss console
     

The Zenoss LAM works with the Zenoss server version 4.2.5

Zenoss LAM Configuration

The events received from the Zenoss server are processed according to the configurations in the zenoss_lam.conf file. The processed alarms are published to Moogsoft AIOps

The configuration file contains a JSON object. At the first layer of the object, LAM has a parameter called config, and the object that follows config has all the necessary information to control the LAM.

The following sections are available for configuration in the Zenoss LAM configuration file:

Monitor

The details of the connection with the Zenoss server are defined in the monitor section of the configuration file. The user can configure the parameters here to establish the connection with Zenoss server.

 monitor:

        {

           name                           : "ZENOSS Lam Monitor",

           class                   		  : "CZenossMonitor",  
         
		   url                         	  : "http://localhost:8080",

           user_name            		  : "username",

           password            			  : "password",

           encrypted_password  			  :"ieytOFRUdLpZx53nijEw0rOh07VEr8w9lBxdCc7229o=",

           uid_filter          			  :"/zport/dmd/Devices/Server/Windows",
		   events_date_format			  : "",
           polling_interval    			  : 10,

           max_retries     			      : 10,
  
           retry_interval      			  : 60,


		   timeout						  : 120,

           event_severity        		  : ["5", "4", "3", "2", "1" ],

		   use_ssl                        : false,

           disable_certificate_validation : true,

           ssl_keystore_file_path 		  : "KeyStore.jks",

           ssl_keystore_password		  : "password"
            
        }

The above example specifies:

  • url: It is the IP address and the port of the Zenoss server
  • user_name: It is the username of the Zenoss console login
  • password: It is the password of the Zenoss console login
  • encrypted_password: If encrypted password is used then enter the encrypted password in this field and comment the password field

    At a time either password or the encrypted_password field is used. If both the fields are not commented then the field encrypted_password will be used by the Zenoss LAM

  • uid_filter: If a user needs to filter events based on device UID, the user can provide the uid_filter parameter. Multiple UIDs can be provided by separating the UIDs with a comma ‘,’

    If the uid_filter field is commented or left blank then the events received by the Zenoss LAM are not filtered. The Zenoss LAM receives all the events generated by all the devices monitored by Zenoss

  • events_date_format: The date/time format of the event received in response . The possible value would be like "yyyy-MM-dd HH:mm:ss", "yyyy-MM-dd'T'HH:mm:ss'Z'", etc. If this value is set to blank then event date/time will be epoch time.

  • polling_interval: The polling time interval between the requests after which the event data is fetched from Zenoss. The polling interval is entered in seconds

    The default value is set to 10 seconds, if 0 is entered in this field then the time interval is by default set to 10 seconds

  • max_retries: The maximum number of retry attempts to reconnect with the Zenoss server in case of a connection failure

    The default value is set to 10, if 0 is entered in this field then the LAM by default takes the value 10 and will try at least 10 times to reconnect

    If all the number of retries are exhausted, then an alarm is sent to Moogsoft AIOps about the connection failure. For re-establishing the connection the LAM has to be restarted

  • retry_interval: The time interval between two successive retry attempts

    The default value is set to 60 seconds, if 0 is entered in this field then the time interval is by default set to 60 seconds

  • timeout: If for any reason the response is not received from the Server against a request, then the LAM discards the request after waiting for some time. The time that the LAM waits before discarding is given here in the timeout field. For example, If the timeout field has 120 entered in it, then the LAM will wait for 120 seconds for a response from the server, against a request. If no response is received for 120 seconds, then the LAM discards the request and sends a new request

    The entry in the fields polling_interval , max_retries, retry_interval and timeout should be an integer, therefore enter the values in these fields without quotation marks

  • event_severity: This field helps in filtering events based on severity, Zenoss sends events with severity codes, the event with a severity code matching with the severity code in this field are only processed by the LAM. The severity codes of Zenoss are as follows:

    •  0 is Clear
    •  1 is Debug
    •  2 is Info
    •  3 is Warning
    •  4 is Error
    •  5 is Critical
    If the user does not want the events with severity Clear or Debug, then enter ["5", "4", "3", "2" ] in the event_severity field
  • use_ssl: Set it to true to enable SSL communication. If set to true then use https instead of http in the url field
  • disable_certificate_validation: If SSL certificate is not to be used and https communication is to be bypassed then set it to true

  • ssl_keystore_file_path: Enter the keystore certificate along with the fully qualified path. E.g. /usr/local/zenoss_ssl/keystore.jks

  • ssl_keystore_password: Enter the keystore password

Agent configuration

Agent allows the user to define 2 parameters:

agent:
{
        name    : "Zenoss",
        #log    : "/var/log/moogsoft/zenoss_lam.log"
},

The above example specifies: 

  • name: This is the agent name, the events sent to MooMs by the Zenoss LAM are identified by the agent name in the log. In this example the agent name is Zenoss
  • log: In this instance, the Zenoss LAM will write its ingress contents to zenoss_lam.log located at /var/log/moogsoft/

HA configuration

Refer the document HA Configuration of LAM

Variables

For the Zenoss LAM the JSON parsing is used, hence the variable section is not used. The variable section is only used for text message. For JSON parsing, the mapping section is used. In mapping there is a value called rules, which is a list of assignments. 

{
            catchAll: "overflow",
            rules:
            [
				{ name: "signature", rule:      "$agent::$eventClass.uid" },
                { name: "source_id", rule:      "$device.uid" },
                { name: "external_id", rule:    "$id" },
                { name: "manager", rule:         "Zenoss" },
                { name: "source", rule:         "$device.uid" },
                { name: "class", rule:          "$eventClass.uid"},
                { name: "agent", rule:          "$LamInstanceName"},
                { name: "agent_location", rule: "$LamInstanceName" },
                { name: "type", rule:           "$device.uid" },
                { name: "severity", rule:       "$severity" },
                { name: "description", rule:    "$summary" },
                { name: "agent_time", rule:     "$moog_now"}
            ]
        },

		filter:
        {
            presend: "ZenossLam.js"
        }

In the example above, the signature field is used by the LAM to identify the correlated alarms, it is mapped to a combination of agent and eventClass.uid. However, user can change it as per the requirement.

User defines a number of these rules covering the base attributes of an event. For reference, the system expects a minimum set of attributes in an event that are shown in the above example.

To map the sub-field values of a field in the Zenoss alarm the “. “operator is used e.g. "$device.uid”. Here “uid” is the subfield of the field device. So to map the subfield the “.” operator is used


Example of Zenoss events:

Constants and conversions

There are rules in mapping section for which conversions are to be defined. The conversions convert the received input from one format to another. E.g. in the above example of mapping, for the mapped field agent_time, the time is converted by calling the conversion function timeConverter. The example of calling a conversion is as follows:


{ name: "agent_time", rule:     "$lastTime", conversion: "timeConverter"}


The example of constants and conversions sections is as follows:


constants:
        {
            severity:
            {
                "CLEAR"         : 0,
                "INDETERMINATE" : 1,
                "WARNING"       : 2,
                "MINOR"         : 3,
                "MAJOR"         : 4,
                "CRITICAL"      : 5
            }
        },
        conversions:
        {
            sevConverter:
            {
                lookup: "severity",
                input:  "STRING",
                output: "INTEGER"
            },
             
            stringToInt:
            {
                input:      "STRING",
                output:     "INTEGER"
            }
          
        },


The above example specifies: 

  • Severity and sevConverter: The severity field has a conversion defined as sevConverter in the Conversions section, this looks up the value of severity defined in the severity section of constants and returns back the mapped integer corresponding to the severity
  • stringToInt: It is used in a conversion, which forces the system to turn a string token into an integer value

Custom Info

The alarms/events are displayed in the Moogsoft AIOps, the data in the fields of the alarm or event mapped in the mapping section are shown in the respective columns of Moogsoft AIOps columns. The fields of alarms and events which are not mapped in the mapping section are displayed in the Custom Info field of the alarm. An example of Custom Info:

catchALL

The attribute that is never referenced in a rule is collected and placed as a JSON object in a variable called overflow defined here and passed as part of the event.

{
            catchAll: "overflow",
            rules:
            [
				{ name: "signature", rule:      "$agent::$eventClass.uid" },
                { name: "source_id", rule:      "$device.uid" },
                { name: "external_id", rule:    "$id" },
                { name: "manager", rule:         "Zenoss" },
                { name: "source", rule:         "$device.uid" },
                { name: "class", rule:          "$eventClass.uid"},
                { name: "agent", rule:          "$LamInstanceName"},
                { name: "agent_location", rule: "$LamInstanceName" },
                { name: "type", rule:           "$device.uid" },
                { name: "severity", rule:       "$severity" },
                { name: "description", rule:    "$summary" },
                { name: "agent_time", rule:     "$moog_now"}
            ]
        },

The Zenoss field “DeviceClass” is sent to the Zenoss LAM. Since it is not mapped to a field in the zenoss_lam.conf  file, it is placed in the overflow JSON object. The fields that are placed in the overflow variable can be viewed in the Zenoss LAM log file.

Example of an overflow JSON object created in the Zenoss LAM log file

INFO : [0:Event][20161026
12:18:25.400 +0530] [CMooMsg.java]:1099 +|Encoded size [3592]
json[{"_MOOTADATA_":{"creation_time":1477464505398},"agent":"localhost","agent_location":"$agent_location","agent_time":1477050262,"class":{"text":"/Status/Snmp"},"description":"SNMP
agent down - no response received","device_name":"undefined","device_uid":"undefined","external_id":"$external_id","manager":"/zport/dmd/Devices/Server/Linux/devices/localhost","overflow":"{\"DeviceClass\":[{\"uid\":\"/zport/dmd/Devices/Server/Linux\",\"name\":\"/Server/Linux\"}],\"DeviceGroups\":[],\"prodState\":\"Production\",\"totalCount\":2,\"ntevid\":null,\"eventState\":\"New\",\"details\":{\"manager\":[\"zenoss\"],\"zenoss.device.device_class\":[\"/Server/Linux\"],\"zenoss.device.priority\":[\"3\"],\"zenoss.device.ip_address\":[\"::1\"],\"zenoss.device.production_state\":[\"1000\"]},\"eventGroup\":\"SnmpTest\",\"LamInstanceName\":\"ZENOSSLAM\",\"evid\":\"08002710-8661-8f24-11e6-920af5e877fa\",\"eventClassMapping\":\"\",\"devices\":[{\"serialNumber\":\"\",\"osModel\":{\"uid\":\"/zport/dmd/Manufacturers/Unknown/products/Linux
2.6.32-573.el6.x86_64\",\"name\":\"Linux
2.6.32-573.el6.x86_64\"},\"ipAddress\":2130706433,\"productionState\":1000,\"groups\":[],\"hwManufacturer\":{\"uid\":\"/zport/dmd/Manufacturers/Generic\",\"name\":\"Generic\"},\"priority\":3,\"collector\":\"localhost\",\"pythonClass\":\"Products.ZenModel.Device\",\"tagNumber\":\"\",\"uid\":\"/zport/dmd/Devices/Server/Linux/devices/127.0.0.1\",\"hwModel\":{\"uid\":\"/zport/dmd/Manufacturers/Generic/products/Net-SNMP
Agent\",\"name\":\"Net-SNMP Agent\"},\"systems\":[],\"ipAddressString\":\"127.0.0.1\",\"name\":\"localhost\",\"osManufacturer\":{\"uid\":\"/zport/dmd/Manufacturers/Unknown\",\"name\":\"Unknown\"},\"location\":null,\"events\":{\"debug\":{\"acknowledged_count\":0,\"count\":0},\"critical\":{\"acknowledged_count\":0,\"count\":1},\"clear\":{\"acknowledged_count\":0,\"count\":0},\"warning\":{\"acknowledged_count\":0,\"count\":0},\"error\":{\"acknowledged_count\":0,\"count\":0},\"info\":{\"acknowledged_count\":0,\"count\":0}}},{\"serialNumber\":\"\",\"osModel\":null,\"ipAddress\":1,\"productionState\":1000,\"groups\":[],\"hwManufacturer\":null,\"priority\":3,\"collector\":\"localhost\",\"pythonClass\":\"Products.ZenModel.Device\",\"tagNumber\":\"\",\"uid\":\"/zport/dmd/Devices/Server/Linux/devices/localhost\",\"hwModel\":null,\"systems\":[],\"ipAddressString\":\"::1\",\"name\":\"localhost\",\"osManufacturer\":null,\"location\":null,\"events\":{\"debug\":{\"acknowledged_count\":0,\"count\":0},\"critical\":{\"acknowledged_count\":0,\"count\":0},\"clear\":{\"acknowledged_count\":0,\"count\":0},\"warning\":{\"acknowledged_count\":0,\"count\":0},\"error\":{\"acknowledged_count\":0,\"count\":1},\"info\":{\"acknowledged_count\":0,\"count\":0}}}],\"count\":194,\"ipAddress\":[\"::1\"],\"stateChange\":\"2016-10-14
18:07:23\",\"ownerid\":null,\"priority\":null,\"message\":\"SNMP
agent down - no response
received\",\"DevicePriority\":\"Normal\",\"Systems\":[],\"firstTime\":\"2016-10-14
18:07:23\",\"component\":{\"uid\":null,\"text\":null,\"uuid\":null,\"url\":null},\"eventKey\":\"agent_down\",\"success\":true,\"dedupid\":\"localhost||/Status/Snmp|agent_down|4\",\"clearid\":null,\"facility\":null,\"device\":{\"text\":\"localhost\",\"uuid\":\"34287108-f697-4163-acaa-be1e1f1bef0f\",\"url\":\"/zport/dmd/goto?guid=34287108-f697-4163-acaa-be1e1f1bef0f\"},\"eventClassKey\":null,\"hash\":\"2\",\"Location\":[]}","severity":4,"signature":"zenperfsnmp","source":"zenperfsnmp","source_id":"08002710-8661-8f24-11e6-920af5e877fa","type":"/zport/dmd/Events/Status/Snmp"}]|+

Starting the Zenoss LAM

To start the Zenoss LAM enter the following command:

service zenosslamd start

To stop the Zenoss LAM enter the following command:

service zenosslamd stop

To view the status of the Zenoss LAM enter the following command:

service zenosslamd status

Quotes

In some instances, the attribute strings are quoted. Our JSON parser ignores it, but the standard requires quoting for all strings, so Moogsoft recommends that user quote all strings.

Comments

A user can comment out lines by appending them with a hash.

Command Line Attributes

The zenoss_lam is a command line executable that can be run as a service daemon, and takes 4 attributes, which can be viewed by typing: 

zenoss_lam --help

Option

Description

--config

Points to a pathname to find the configuration file for the LAM. This is where the entire configuration for the LAM is specified

--help

Displays all the command line options

--version

Displays the component’s version number

--log level

Specifies the level of debug. By default, you get everything. In common with all executables in MOOG, having it set at that level can result in a lot of output (many messages per event message processed). In all production implementations, it is recommended that log level be set to WARN, which only informs you of matters of importance

Version Information

LAM Version

Tool Version

Tested?

Expected to Work

1.0

Zenoss 4.2.5 

Yes

Yes

1.1

Zenoss 4.2.5 

Yes

Yes

1.2Zenoss 4.2.5YesYes
1.3Zenoss 4.2.5YesYes
1.4Zenoss 4.2.5YesYes

System Information

This LAM was tested on a system with the following configurations:

CPU2 core
RAM4 GB
Operating SystemCentOS Linux release 6.7

The system must at least have the above mentioned system requirements to run the LAM.

  • No labels