XClarity is an Infrastructure monitoring application that monitors network, server, storage, power devices etc. It is a distributed software application that monitors the IT environment and generates events based on data which has been exposed with rest APIs or from third parties. This document describes the configuration required to establish a communication between the XClarity application and the XClarity LAM.
The workflow of gathering alarms and events from an XClarity server and publishing it to Moogsoft AIOps is:
- The LAM reads configuration from given xclarity_conf file.
- The LAM connects with the host and sends the GET REST request based on uri provided in LAM Config.
- The event data of all the devices monitored in XClarity is received by the LAM as a response. The format of received events is JSON.
- If filter variable in monitor section is set, then the LAM filters the events based on the filter criteria set in the config file.
- The LAM parses the events and then publish it to MooMS bus.
- The unmapped event data is sent to the overflow variable.
The events are published to the subject “/Events”.
HTTP/HTTPS request with basic user authentication is used in the XClarity LAM
You can configure how the XClarity LAM processes events received from an XClarity server by accessing the xclarity_lam.conf file, at the following path /usr/share/moogsoft/config
The XClarity server does not need any configuration to connect to the XClarity LAM. The following information is required in the xclarity_lam.conf file:
- The IP address or the hostname of the XClarity server
- Username and password of the users in XClarity who have the permission to send events.
The LAM supports XClarity version 1.2.0
XClarity LAM Configuration
The events received from the XClarity server are processed according to the configurations in the xclarity_lam.conf file. The processed alarms are published to Moogsoft AIOps.
The configuration file contains a JSON object. At the first layer of the object, LAM has a parameter called config, and the object that follows config has all the necessary information to control the LAM.
The following sections are available for configuration in the XClarity LAM configuration file:
The details of the connection with the XClarity server are defined in the monitor section of the configuration file. The user can configure the parameters here to establish the communication with XClarity server.
The above example specifies:
- name and class: These fields are reserved and should not be changed. The default values are Xclarity Lam Monitor and CXclarityMonitor respectively
- host name: It is the IP address or hostname of the XClarity server. By default it is set to localhost
- uri: The URI of the XClarity application from which the events are fetched. The following two uri's are available:
- events: If this is entered then events in the events log is fetched
- events/audit: If this is entered then events in the audit log are fetched. This is not mentioned in the config file but can be used while configuring it.
To fetch events from both the events log and the audit log of the XClarity Server, enter both the uri in the uri field
- user_name: It is the user name of the user in XCLarity who have the permission to send events
- password: It is the password corresponding to the user name given in the field user_name
- encrypted_password: If encrypted password is used then enter the encrypted password in this field and comment the password field. At a time either password or the encrypted_password field is used. If both the fields are not commented then the field encrypted_password will be used by the XClarity LAM
ssl: If SSL communication is enabled in XClarity Server then select "true" here
ssl_keystore_file_path: Enter the keystore path of the XClarity keystore certificate in this field. E.g. usr/share/moogsoft/xclarity/Keystore/KeyStore.jks
ssl_keystore_password: Enter the password of the XClarity keystore certificate, the same password should be given which was used during generating the keystore certificate
ssl_truststore_file_path: Enter the truststore path of the XClarity truststore certificate in this field. E.g. usr/share/moogsoft/xclarity/Keystore/KeyStore.jks
ssl_truststore_password: Enter the password of the XClarity truststore certificate, the same password should be given which was used during generating the truststore certificate
The entries in the field ssl_keystore_file_path, ssl_keystore_password , ssl_truststore_file_path and ssl_truststore_password fields should only be made if "true" is entered in the ssl field, otherwise these fields should be commented.
For Windows, use "\\" instead of "\" in the path given in ssl_keystore_file_path and truststore_file_path. E.g. C:\\Users\\User1\\Documents\\Keystore\\KeyStore.jks
For XClarity KeyStore.jks certificate is also used as a TrustStore certificate, therefore in the config file for ssl_truststore_file_path field enter the path of the KeyStore.jks certificate.
- timeout: If for any reason the response is not received from the Server against a request, then the LAM discards the request after waiting for some time. The time that the LAM waits before discarding is given here in the timeout field. For example, If the timeout field has 120 entered in it, then the LAM will wait for 120 seconds for a response from the server, against a request. If no response is received for 120 seconds, then the LAM discards the request and sends a new request
- events_date_format: The date/time format in event response. The possible values could be like "yyyy-MM-dd HH:mm:ss" or "yyyy-MM-dd'T'HH:mm:ss'Z'". If this value is set to blank then event date/time will be in epoch time.
- filter: It is set to true to filter events received by the XClarity LAM. The XClarity LAM filters the events based on comparisons done in between the values of fields in an event with predefined values in filter_field_values field. The comparisons can be done by either using a regular expression or using operators. By default it is set to false. The following fields are part of an event:
- filter_type: The type of the filter that has to be used for XClarity events. The filter filters events based on comparisons done on the event fields. The filter criteria is defined by using the filter_comparison_operator, filter_field_names and filter_field_values fields. The filter_type field defines the type of a filter used. Either a regular expression filter or a non regular expression filter (operators) is used for filtering
- Regular Expression filter types
FIELDREGEXAND: Regular expression filter of type AND, filters the events when all the conditions of filter criteria meet
FIELDREGEXOR: Regular expression of type OR, filters the events when any one of the conditions of filter criteria meet
FIELDREGEXNOT: Regular expression of type NOT, filters the events which do not meet the filter criteria
- Non-Regular Expression filter types
FIELDNOTREGEXAND: Non-Regular Expression filter of type AND, filters the events when all the conditions of filter criteria meet
FIELDNOTREGEXOR: Non-Regular Expression filter of type OR, filters the events when any one of the conditions of filter criteria meet
FIELDNOTREGEXNOT: Non-Regular Expression filter of type NOT, filters the events which do not meet the filter criteria
- Regular Expression filter types
- filter_comparison_operator: This field is used if non-Regular Expression filter type is used. The operators that will be used for comparison of values of event fields is given here. The following operators can be entered in this field:
- EQ: Equal To operator
- GT: Greater Than operator
- GTE: Greater Than Equal To operator
- LT: Less Than operator
- LTE: Less Than Equal To operator
- NOT: NOT operator
The following table lists the fields that are included in an event and the operators that can be used on a field.
CN EQ GT GTE LT LTE NOT userID X X eventClass X X X X X X severity X X X X X X timeStamp X X X X X X sourceID X X X X X X sourceLogSequence X X X X X X localLogID X X X X X X localLogSequence X X X X X X eventID X X eventDate X X X X X X args X X msgID X X msg X X serialnum X X mtm X X service X X X X X X action X X X X X X location X X failSNs X X failFRUs X X componentID X X X X X X search X X X X X X
In the above table "X" in the operator column against the field indicates that this operator can be used with this field
filter_field_names: The fields of event based on which the events are filtered, are entered in this field. Multiple fields can be entered here as shown in the above example
To include all the fields of an event for a filter criteria, enter "cn" in this field
- filter_field_values: The value with which the value of the field of event is compared is entered here, the values should be entered as shown in the above example. The sequence of field names in filter_field_names, and the corresponding values with which the field values will be compared in filter_field_values, should be same. The events will be wrongly filtered if the field names and field values are not defined in the correct sequence.
polling_interval: The polling time interval between the requests after which the event data is fetched from XClarity server. The polling interval is entered in seconds
The default value is set to 10 seconds, if 0 is entered in this field then the time interval is by default set to 10 seconds
max_retries: The maximum number of retry attempts to reconnect with the XClarity server in case of a communication failure
The default value is set to 10, if 0 is entered in this field then the LAM by default takes the value 10 and will try at least 10 times to reconnect
If all the number of retries are exhausted, then an alarm is sent to Moogsoft AIOps about the connection failure. For re-establishing the connection the LAM has to be restarted
retry_interval: The time interval between two successive retry attempts
The default value is set to 60 seconds, if 0 is entered in this field then the time interval is by default set to 60 seconds
The entry in the fields polling_interval , max_retries and retry_interval should be an integer, therefore enter the values in these fields without quotation marks
Examples of filter criteria:
Example1:Non-regular Expression filtering with all the event fields.
In the above example the FIELDNOTREGEXOR filter type is used. Since "cn" is entered in the field filter_field_names, all the fields of an event will be compared with the value "1" in the field filter_field_values. If any of the field has value equal to "1" then the event is filtered and processed by the LAM.
Example2:Non-regular Expression filtering with selected event fields.
In the above example the FIELDNOTREGEXOR filter type is used. Since only two fields of an event are entered in filter_field_names, the filter_comparison_operators field also has two operators defined for each event field in filter_field_names. The events with severity less than "200" or time stamp greater than "2016-11-07T16:01:03Z" are filtered and then processed by the LAM.
There should be an operator present for each field of event entered in filter_field_names, otherwise it will throw an error
The operators entered for a field in filter_comparison_operators should be compatible with the field for which it will be used. The compatibility of the operator with a field can be checked in the above table
The entries in the fields filter_comparison_operators, filter_field_names and filter_field_values corresponds to positions of entries. For example, the event field at position 1 of filter_field_names will be compared to the value at position 1 of filter_field_values using the operator at position 1 of filter_comparison_operators
Example3:Regular Expression filtering with selected event fields.
In the above example the FIELDREGEXOR filter type is used. The operators do not work with a regular expression filter. The events with severity equal to "200" or time stamp equal to "2016-11-07T16:01:03Z" are filtered and then processed by the LAM.
Agent allows the user to define 2 parameters:
The above example specifies:
- name: This is the agent name, the events sent to MooMs by the XClarity LAM are identified by the agent name in the log. In this example the agent name is XClarity
- log: In this instance the XClarity LAM will write its ingress contents to xclarity_lam.log located at /var/log/moogsoft
Refer the document HA Configuration of LAM
For the XClarity LAM the JSON parsing is used, hence the variable section is not used. The variable section is only used for text message. For JSON, the mapping section is used. In mapping there is a value called rules, which is a list of assignments, that maps the tokens received from XClarity to the fields in Moogsoft AIOps.
In the example above, the signature field is used by the LAM to identify the correlated alarms, it is mapped to eventID here. However, user can change it as per the requirement.
User defines a number of these rules covering the base attributes of an event. For reference, the system expects a minimum set of attributes in an event that are shown in the above example.
Example of XClarity events:
Constants and conversions
There are rules in the mapping section for which conversions are to be defined. The conversions convert the received input from one format to another. E.g. in the above example of mapping, for the mapped field agent_time, the time is converted by calling the conversion function timeConverter. The example of calling a conversion is as follows:
The example of constants and conversions sections is as follows:
The above example specifies:
Severity and sevConverter: The severity field has a conversion defined as sevConverter in the Conversions section, this looks up the value of severity defined in the severity section of constants and returns back the mapped integer corresponding to the severity
An XClarity alarm/event contains severity in the form of severity field and severity_text. In the above example we have used severity text. To obtain severity from the severity field the severity section in the above example has to be changed according to the severity field
- stringToInt: It is used in a conversion, which forces the system to turn a string token into an integer value
- timeConverter: It is used in conversion which forces the system to convert time. If epoc time is to be used, then timeFormat mentioned in timeConverter should be commented. Otherwise, user should provide the timeFormat. For the XClarity LAM the time format "yyyy-MM-dd'T'HH:mm:ss" is used. The time format "%D %T" is not used in XClarity so it is commented
The alarms/events are displayed in the Moogsoft AIOps, the data in the fields of the alarm or event mapped in the mapping section are shown in the respective columns of Moogsoft AIOps columns. The fields of alarms and events which are not mapped in the mapping section are displayed in the Custom Info field of the alarm. An example of Custom Info:
The attribute that is never referenced in a rule is collected and placed as a JSON object in a variable called overflow defined here and passed as part of the event.
The XClarity field “args” is sent to the XClarity LAM. Since it is not mapped with a field in the xclarity_lam.conf file, it is placed in the overflow JSON object. The fields that are placed in the overflow variable can be viewed in the XClarity LAM log file.
Example of an overflow JSON object created in the XClarity LAM log file
Starting the XClarity LAM
To start the XClarity LAM enter the following command:
To stop the XClarity LAM enter the following command:
To view the status of the XClarity LAM enter the following command:
In some instances, the attribute strings are quoted. Our JSON parser ignores it, but the standard requires quoting for all strings, so Moogsoft recommends that user quote all strings.
A user can comment out lines by appending them with a hash.
Command Line Attributes
The xclarity_lam is a command line executable that can be run as a service daemon, and takes 4 attributes, which can be viewed by typing:
Points to a pathname to find the configuration file for the LAM. This is where the entire configuration for the LAM is specified
Displays all the command line options
Displays the component’s version number
Specifies the level of debug. By default, you get everything. In common with all executables in MOOG, having it set at that level can result in a lot of output (many messages per event message processed). In all production implementations it is recommended that log level be set to WARN, which only informs you of matters of importance
Expected to Work
Xclarity Server 1.2.0
Xclarity Server 1.2.0
Xclarity Server 1.2.0
Xclarity Server 1.2.0
This LAM was tested on a system with the following configurations:
|Operating System||CentOS Linux release 6.7|
The system must at least have the above mentioned system requirements to run the LAM.