Moogsoft Docs

XClarity Integration Reference


image2017-7-5_17-44-23.png

XClarity is an Infrastructure Monitoring Application which monitors network, server, storage, power devices, etc. It is a distributed software that monitors the IT environment and generate events based on data which has been exposed with rest APIs or from third parties. This document describes the configurations required to establish a connection between the XClarity application and the XClarity LAM.

The workflow for gathering events from an XClarity server, and publishing it to Moogsoft AIOps is as follows:

  1. The LAM reads configuration from given xclarity_lam.conf file.
  2. The LAM connects with the host and sends the GET REST request based on uri provided in LAM Config.
  3. The event data of all the devices monitored in XClarity is received by the LAM as a response. The format of received events is JSON.
  4. If filter variable in monitor section is set, then LAM will filter the events based on the filter criteria set in the config file.
  5. The LAM parses the events and then publishes it to the MooMS bus.
  6. The unmapped event data is sent to the overflow variable.
  7. The events are published to the subject “Events”.

Note

HTTP/HTTPS request with basic user authentication is used in the XClarity LAM.

XClarity LAM Configuration

The events received from the XClarity server are processed according to the configurations in the xcl arity_lam.conf file. The processed events are published to Moogsoft AIOps.

The configuration file contains a JSON object. At the first layer of the object, LAM has a parameter called config , and the object that follows config has all the necessary information to control the LAM.

The following configuration sections are available in the XClarity LAM configuration file:

Monitor

Agent

HA Configuration

Mapping

Constants and Conversions

Monitor

The XClarity LAM fetches the events from the XClarity Rest Server. You can configure parameters here to establish a connection with XClarity:

General

Field Type Description Example
name and class String Reserved fields: do not change. Default values are Xclarity Lam Monitor and CXclarityMonitor.
host_name String Enter the hostname/IP address/FQDN address of xclarity server. Its default value is localhost.
uri String
This is the XClarity URI that will be used to fetch events. The following two URI's are available:
  • events : Enter this to fetch events available in the event log.
  • events/audit : Enter this to fetch events available in the audit log.

Note

To fetch events from both the logs, i.e. event log and audit log of the XClarity Server, enter both the URI in the uri field.

Only events will be fetched by default. If you want to audit logs, then you have to mention the uri for that as well.


user_name String Enter the username of XClarity users who have the privileges to fetch events.
password String Enter the password of XClarity users who have the privileges to fetch events.
encrypted_password String If the password is encrypted, then enter the encrypted password in this field and comment out the password field. At a time, either password or the encrypted_password field is used. If both the fields are not commented, then the field encrypted_password will be used by the XClarity LAM.
polling_interval Integer

The polling time interval, in seconds, between the requests after which the event data is fetched from XClarity LAM.

Default = 10 seconds. If specified value is less than 1, the time interval will set to 10 seconds.


max_retries Integer

The maximum number of retry attempts to reconnect with XClarity Server in case of a connection failure.

Default = -1, if no value is specified, then there will be infinite retry attempts.

If the specified value is greater than 0, then the LAM will try that many times to reconnect; in case of 0 or any other value less than 0, max retries will set to default.


retry_interval Integer The time interval between two successive retry attempts.
Default = 60 seconds. If specified value is less than 1, the retry_interval will set to default.

timeout Integer

This is the timeout value in seconds, which will be used to timeout a connection, socket and request. If no value is specified, then the timeout will set to 120 seconds.


events_date_format String This is the date/time format of the event received in response . The possible value would be like "yyyy-MM-dd HH:mm:ss", "yyyy-MM-dd'T'HH:mm:ss'Z'", etc. If this value is set to blank, then event date/time will be epoch time.

Filter

Field Sub Field Type Description
filter filter_type Object
Enter true here to enable filters. The following filters are used in combination to filter out the received events:
filter_type is the type of filter that has to be used for XClarity events. Here the events will be filtered on the basis of comparisons done on the event fields. The filter criteria is defined by using the filter_comparison_operator, filter_field_names and filter_field_values fields. The filter_type field defines the type of a filter used. Either a regular expression filter or a non regular expression filter (operators) is used for filtering.

Regular Expression filter types

    • FIELDREGEXAND : Regular expression filter of type AND. This will filter only those events which meets the condition of filter criteria.
    • FIELDREGEXOR : Regular expression of type OR. This will filter those events which meets any one of the condition of filter criteria.
    • FIELDREGEXNOT : Regular expression of type NOT. This will filter those events which do not meet the filter criteria.

Non-Regular Expression filter types

    • FIELDNOTREGEXAND : Non-Regular Expression filter of type AND. This will filter only those events which meets the condition of filter criteria.
    • FIELDNOTREGEXOR : Non-Regular Expression filter of type OR. This will filter those events which meets any one of the condition of filter criteria.
    • FIELDNOTREGEXNOT : Non-Regular Expression filter of type NOT. This will filter those events which do not meet the filter criteria.

filter_comparison_operators Integer

When you are using non-Regular Expression filter type, then this field will be used. The operators that will be used for comparing event field values are:

    • EQ: Equal To operator
    • GT: Greater Than operator
    • GTE: Greater Than Equal To operator
    • LT: Less Than operator
    • LTE: Less Than Equal To operator
    • NOT: NOT operator

filter_field_names String

Enter the event fields on the basis of which events are filtered. You can also enter multiple fields here.



filter_field_values String

Enter the value by which event fields value will be compared.The sequence of field names in filter_field_names , and the corresponding values with which the field values will be compared in filter_field_values , should be same. If field names and field values are not defined in the correct sequence, then events will be filtered incorrectly.

Secure Sockets Layer

Field Type Description
ssl Boolean
Enter true here, to enable SSL Communication:
  • ssl_keystore_file_path : Enter the path of the XClarity keystore certificate. This is the path where the generated keystore file is copied in Moogsoft AIOps, e.g. "usr/share/moogsoft/xclarity/Keystore/KeyStore.jks".

  • ssl_keystore_password : Enter the password of XClarity keystore certificate. It is the same password that was entered when the keystore was generated.


The following table lists the fields which are included in an event and the operators which can be used on a field:

CN EQ GT GTE LT LTE NOT
userID X



X
eventClass X X X X X X
severity X X X X X X
timeStamp X X X X X X
sourceID X X X X X X
sourceLogSequence X X X X X X
localLogID X X X X X X
localLogSequence X X X X X X
eventID X



X
eventDate X X X X X X
args X



X
msgID X



X
msg X



X
serialnum X



X
mtm X



X
service X X X X X X
action X X X X X X
location X



X
failSNs X



X
failFRUs X



X
componentID X X X X X X
search X X X X X X

Note

In the above table, "X" indicates that this operator can be used with this field.

Example

Config File
monitor:
        {
            
            name                    		: "Xclarity Lam Monitor",

            class                   		: "CXclarityMonitor",

			host_name                   	: "localhost",

            uri                         	: [
												"events",
										  	  ],
			user_name                   	: "username",
            
            password                    	: "password",

			#encrypted_password         	: "ieytOFRUdLpZx53nijEw0rOh07VEr8w9lBxdCc7229o=",

			ssl 							: false,
            
            ssl_keystore_file_path      	: "KeyStore.jks",
            
            ssl_keystore_password       	: "password",
            
   			polling_interval            	: 10,

            max_retries                 	: -1,
  
            retry_interval              	: 60,

			timeout							: 120,
 			events_date_format          	: "yyyy-MM-dd'T'HH:mm:ss'Z'",

            filter                      	: "false",

            filter_type                 	: "FIELDREGEXAND",    
          
            filter_comparison_operators 	: [
                                           		"EQ"                                    
                                          	  ],

            filter_field_names          	: [
                                           		"cn"
                                          	  ],

			filter_field_values         	: [
                                            	 ""
                                          	  ]
		   
		    
         },
The entry in the fields polling_interval, max_retries, retry_interval and timeout should be an integer, and hence, enter values in these fields without quotation marks.

Examples of filter criteria:

Example 1 : Non-regular Expression filtering with all event fields.

			filter                      : "true",

            filter_type                 : "FIELDNOTREGEXOR",    
          
            filter_comparison_operators : "EQ",                                    
            
            filter_field_names          : "cn",

            filter_field_values         : "1",


Here the FIELDNOTREGEXOR filter type is used. Since "cn" is entered in the field filter_field_names ,  all the fields of an event will be compared with the value "1" in the field filter_field_values . If any of the field has value equal to "1", then events will be filtered and processed by the LAM.

Example 2 : Non-regular Expression filtering with selected event fields.

			filter                      : "true",

            filter_type                 : "FIELDNOTREGEXOR",    
          
            filter_comparison_operators : [
                                           "LT",
                                           "GT"                                    
                                          ],

		    filter_field_names          : [
										   "severity",
										   "timeStamp"
										  ],

            filter_field_values         : [
										   "200",
                                           "2016-11-07T16:01:03Z"
                                          ],

Here the FIELDNOTREGEXOR filter type is used. Only two fields of an event are entered in the filter_field_names , and the filter_comparison_operators field also has two operators defined for each event field in filter_field_names. Therefore, the events with severity less than "200" or time stamp greater than "2016-11-07T16:01:03Z" will be filtered and processed by the LAM.

Note

There should be an operator present for each field of event entered in filter_field_names , else it will throw an error.

Note

The operators entered for a field in filter_comparison_operators should be compatible with the field for which it will be used.

Note

The entries in the fields filter_comparison_operators, filter_field_names and filter_field_values must correspond to the positions of entries. For example, the event field at position 1 of filter_field_names will be compared with the value at position 1 of filter_field_values by using the operator at position 1 of filter_comparison_operators.

Example 3 : Regular Expression filtering with selected event fields.

			filter                      : "true",

            filter_type                 : "FIELDREGEXOR",                                        
            
            filter_field_names          : [
										   "severity",
										   "timeStamp"
										  ],

            filter_field_values         : [
										   "200",
                                           "2016-11-07T16:01:03Z"
                                          ],

Here the FIELDREGEXOR filter type is used. The operators do not work with a regular expression filter. The events with severity equal to "200" or time stamp equal to "2016-11-07T16:01:03Z" will be filtered and processed by the LAM.

Agent

Agent allows you to define two parameters:

Field
name This is the agent name, the events sent to MooMS by the XClarity LAM are identified by the agent name in the log. In this LAM, the agent name is XClarity.
log XClarity LAM will write its ingress contents in the file xclarity_lam.log located at /var/log/moogsoft/.

HA Configuration

Refer to the document Integrations HA Configuration

Mapping

Variables section is not required in the XClarity LAM, you can directly map the event's field of XClarity with moogsoft fields. The parameters of the received events are displayed in the Moogsoft AIOps according to the mapping done here:

mapping :
        {
            catchAll: "overflow",
            rules:
            [
                { name: "signature", rule:      "$eventID" },
                { name: "source_id", rule:      "$sourceID" },
                { name: "external_id", rule:    "$componentID" },
                { name: "manager", rule:        "XClarity" },
                { name: "source", rule:         "$systemName" },
                { name: "class", rule:          "$typeText" },
                { name: "agent", rule:          "$LamInstanceName" },
                { name: "agent_location", rule: "$LamInstanceName" },
                { name: "type", rule:           "$typeText" },
                { name: "severity", rule:       "$severityText", conversion: "sevConverter" },
                { name: "description", rule:    "$msg" },
                { name: "agent_time", rule:     "$eventDate", conversion: "timeConverter"}
            ]
        },
        filter:
        {
            presend: "XclarityLam.js"
        }

In the above example, the signature field is used by the LAM to identify the correlated events, it is mapped to eventID here. However, you can also change it as per the requirement. Data not mapped to Moogsoft AIOps Fields goes into "Custom Info".

You can define number of these rules covering the base attribute of an event.

Constants and Conversions

Field Description Example
Severity and sevConverter has a conversion defined as sevConverter in the Conversions section, this looks up the value of severity defined in the severity section of constants and returns back the mapped integer corresponding to the severity
severity:
{
 "OK"            : 0,
"Informational" : 1,
"Warning" : 2,
"Minor" : 3,
"Major" : 4,
"Critical" : 5,
"Fatal" : 5,
"Error" : 5
}        
sevConverter:
 {
     lookup  : "severity",
     input   : "STRING",
     output  : "INTEGER"
  },

stringToInt used in a conversion, which forces the system to turn a string token into an integer value
stringToInt:
{
    input  : "STRING",
    output : "INTEGER"
},
timeConverter used in conversion which forces the system to convert time. If epoch time is to be used, then timeFormat mentioned in timeConverter should be commented. Otherwise, the user should provide the timeFormat
timeConverter:
{
    timeFormat : "yyyy-MM-dd'T'HH:mm:ss.SSS",
    input      : "STRING",
    output     : "INTEGER"
}

Example

Example Constants and Conversions
constants:
        {
            severity:
            {
                "OK"            : 0,
                "Informational" : 1,
                "Warning"       : 2,
                "Minor"         : 3,
                "Major"         : 4,
                "Critical"      : 5,
                "Fatal"         : 5,
                "Error"         : 5
            }
        },
        conversions:
        {
            sevConverter:
            {
                lookup: "severity",
                input:  "STRING",
                output: "INTEGER"
            },
	    	            
            stringToInt:
            {
                input:      "STRING",
                output:     "INTEGER"
            },     
 
			timeConverter:
            {
                timeFormat: "yyyy-MM-dd'T'HH:mm:ss",
                input:      "STRING",
                output:     "INTEGER"
            }  
            
        },

Severity Reference

Moogsoft Severity Levels
severity:
        {
            "CLEAR" 		: 0,
            "INDETERMINATE" : 1,
            "WARNING" 		: 2,
            "MINOR" 		: 3,
            "MAJOR" 		: 4,
            "CRITICAL" 		: 5,
            
        }
Level Description
0 Clear
1 Indeterminate
2 Warning
3 Minor
4 Major
5 Critical

Service Operation Reference

Process Name Service Name
xclarity_lam xclaritylamd

Start the LAM Service:

service xclaritylamd start

Stop the LAM Service:

service xclaritylamd stop

Check the LAM Service status:

service xclaritylamd status

Command Line Reference

To see the available optional attributes of the XClarity_lam, run the following command:


xclarity_lam --help

The xclarity_lam is a command line executable, and has the following optional attributes:

Option Description

--config

Points to a pathname to find the configuration file for the LAM. This is where the entire configuration for the LAM is specified.
--help Displays all the command line options.
--version

Displays the component’s version number.

--loglevel

Specifies the level of debugging. By default, user gets everything. In common with all executables in MOOG, having it set at that level can result in a lot of output (many messages per event message processed).

In all production implementations, it is recommended that log level is set to WARN. This ensures only warning, error and fatal messages are recorded.

Performance Information

Minimum requirement
Component Value
CPU 2 core
RAM 4 GB
Operating System CentOS Linux release 6.7

Version

LAM Version Tool Version Verified By
1.0

Xclarity Server  1.2.0

Moogsoft
1.1

Xclarity Server  1.2.0

Moogsoft
1.2

Xclarity Server  1.2.0

Moogsoft
1.3

Xclarity Server  1.2.0

Moogsoft