The Rest Client LAM is an HTTP client LAM and makes use of a REST API to request event data and ingest it into Moogsoft AIOps. It sends HTTP requests to the REST server at configurable intervals and parses the JSON responses received from the server and processes events from the responses.
The REST Client LAM initiates a "pull" request to a REST API and in response it receives events in JSON format (e.g. Solarwinds, Zabbix, etc.)
The Rest Client LAM ingests event data from RESTful Services. The following information explains the configuration required to do this.
The ingestion of event data from RESTful Services requires Moogsoft AIOps to be running a Rest Client LAM, configured to parse JSON responses from RESTful Services. It also requires account credentials. These credentials are specified in the RestClient LAM configuration file
The REST Client LAM processes the events received from the REST API based on the configurations done in the following 2 files:
|$MOOGSOFT/config/rest_client_lam.conf||Rest Client LAM configuration file|
|$MOOGSOFT/bots/lambots/RestClientLam.js||LAMBot file; processes the event data received from the event source|
The user has to configure the above 2 files as per their requirements based on the event format received from the REST API.
The performance of Moogsoft AIOps depends on the number of events received per second and the specifications of the Moogsoft AIOps system on which the Rest Client LAM is running
Rest Client LAM configuration
The configuration file contains a JSON object. At the first layer of the object, the LAM has a parameter called config, and the object that follows config has all the necessary information to control the LAM.
The following sections are available for configuration in the rest_client_lam.Conf file.
The REST Client LAM takes the connection information from the Monitor section of the config file. An example of Monitor section is as follows:
- name and class: These fields are reserved and should not be changed. The default values are REST Client Monitor and CRestClientMonitor respectively
- request_interval: This is the time interval between 2 successive http requests. For example, if 30 seconds is entered here, and the REST Client LAM is started, then after the first request it will wait for 30 seconds and then send the second request
max_intervals: This is the count that the REST Client LAM will try to send an http request, in the case of a failure of an http request
Example: If 3 is entered here and the request _interval is set to 120 seconds, then the LAM sends a request every two minutes (120 seconds), and the maximum number of intervals is set to 3. Therefore the maximum time to request data for if previous requests fail is (3 x 2) six minutes. So if the LAM starts at 09:00, requests are as follows:
09:02 - Request event data from 09:00 to 09:02... success
09:04 - Request event data from 09:02 to 09:04... fail
09:06 - Request event data from 09:02 to 09:06 (asks for data from the previously failed request)... fail
09:08 - Request event data from 09:02 to 09:08... fail
09:10 - Request event data from 09:04 to 09:10 (the maximum time to request data for is reached, so data from the last six minutes is requested)... fail
09:12 - Request event data from 09:06 to 09:12 (data from the last six minutes is requested)... success
09:14 - Request event data from 09:12 to 09:14 (previous request successful, reverts to requesting event data from the previous two minutes)
user: The username to access the REST server
- password: The password for the user accessing the REST server
encrypted_password: If the user wants to use an encrypted password, then encrypt the password and enter it in this field and comment the password field. At a time either password or the encrypted_password field is used. If both the fields are not commented then the field encrypted_password will be used by the REST Client LAM
- enable_proxy: If communication is to be established through a proxy, then enter true here
proxy_host: Enter the hostname or IP address of the proxy
proxy_port: Enter the port number of the proxy
proxy_user: Enter the user name of the proxy
encrypted_proxy_password: If the proxy password is not to be displayed in the config file, then encrypt the proxy password and enter here
proxy_password: Enter the password corresponding to the user name entered
use_ssl: Set it to true to enable SSL communication
disable_certificate_validation: Set to
falseif the SSL certificate for the event server is valid (default setting is
trueto disable SSL certificate validation for the event server)
path_to_ssl_files: The path where the SSL certificates is copied to on the machine where the REST Client LAM is running
server_cert_filename: Enter the server certificate name here. Use the certificate "server.crt" here. The cert file should be present in the directory given in path_to_ssl_files field
use_client_authentication: Enter true here if you want client authentication, otherwise set it to false. By default, it is set to false. If it is set to true, then the values are to be entered in the client_key_filename and the client_cert_filename fields.
client_key_filename: Enter the name of the key file here, e.g. "client.key". The key file should be present in the directory given in path_to_ssl_files field.
client_cert_filename: Enter the name of the certificate file here, e.g. "client.crt". The cert file should be present in the directory given in path_to_ssl_files field.
requests_overlap: How many seconds more than the
request_interval(see above) does the LAM wait to request data for. This is to ensure events are not missed
Example: If request interval is set to 120 seconds and request_overlap is set to 10, then the LAM sends a request every two minutes (120 seconds). The overlap is set to ten seconds, so each request asks for data from the last two minutes and ten seconds
url: A URL for the endpoint to pull events from (including hostname or IP Address). If IP address along with port is entered in the field, then append a "/" in the end, e.g. http://localhost:8896/
request_query_params: The request query parameters. All the members of this map will be added to the request URL as URL encoded, so the URL will look like url?key1=value1&key2=value2...
Example1: The following code block provides a sample request query
The URL for the above request query will be
Example2: The following code block provides a sample request query containing a to and from parameter
The default 'request_query_params block' includes optional 'to' and 'from' variables which are special internally calculated timestamps worked out based on the request_interval and the request_overlap compared to the current system time. These timestamps variables can be used in the request_query_params block by prefixing them with a dollar sign. The timestamp formats is governed by the field params_date_format.
params_date_format: The format of timestamps that are sent in the request URL is entered here. E.g. "%Y-%m-%dT%H:%M:%S"
enable_epoch_converter: If set to true, the time stamps in the request URL are converted to epoch time
results_as_list: Set it to true if the events will be received in a list, otherwise set it to false
results_path: If the events in the response JSON are not in a list, then this will identify where in the JSON blob to find the event JSON. E.g. "results.subObject.events"
overlap_identity_fields: If we set the requests_overlap value, we might get the same event twice, setting this parameter will tell the LAM how to identify the duplicate. Duplicate events will be ignored in the second interval. E.g.
num_threads: The number of threads the REST Client LAM will use
Agent allows the user to define two parameters:
The above example specifies:
name: This is the agent name, the events sent to MooMs by the REST Client LAM are identified by the agent name in the log. In this example the agent name is RestClientLam
log: In this instance, the REST Client LAM will write its ingress contents in the file rest_client_lam.log located at /var/log/moogsoft/
For events received in JSON format, a user can directly map the alarm/event fields of REST Client LAM with Moogsoft fields. In the case of an event received in text format, the event is first tokenized in the Variable section, and the tokenized event is then mapped here in the mapping section. The parameters of the received alarm/event are displayed in Moogsoft AIOps according to the mapping done here.
The above example specifies the mapping of the REST Server alarm fields with the Moogsoft AIOps fields.
The signature field is used by the LAM to identify correlated alarms
The above mapping section is an example, the user has to modify this section according to the format of the received event/alert
Conversions section allows the user to convert the format of the received data.
The above example specifies:
- Severity and sevConverter: The severity field has a conversion defined as sevConverter in the Conversions section, this looks up the value of severity defined in the severity section of constants and returns back the mapped integer corresponding to the severity
- timeConverter: It is used in the conversion of the time stamp of the received events in the required format
The attribute that is never referenced in a rule is collected and placed as a JSON object in a variable called overflow and passed as part of the event.
The REST Client LAM saves the last poll time of the request sent to the REST Server. This last poll time is saved in a state file. The REST Client LAM reads/writes the last poll time from/to this state file. The LAM reads/writes from the state file by using the RestClientLam.js file. To read and write the state file the RestClientLam.js has to be modified.
The state file is created at the same location where the rest_client_lam.conf file resides. The name of the state file will be same as the config file name. For example, if the name of the config file is rest_client_lam.config, then the corresponding state file name will be rest_client_lam.state
The following example shows the Lambot configuration for a POST request with a JSON payload in which the last poll time is being written and read from the state file.
The following example shows the Lambot configuration for a POST request with an XML payload in which the last poll time is being written and read from the state file.
The above examples have the following functions:
function onLoad(): This function is used to load data from state file. The state file has details of last poll time of the LAM. The state file is saved at the same location where the REST Client LAM config file is present. The statement constants.load() in this function loads the contents of the state file in the constants variable. The user has some more alternatives for playing with data of the state file. As of now, we are only saving the last poll time, but if the user wants to save some other data which is to be used after the restart of the LAM, then this data can be saved in the state file. The name of the state file will be same as the config file name. For example, if the name of the config file is rest_client_lam.config, then the corresponding state file name will be rest_client_lam.state
function preClientSend(): This function extracts the information about the URL from the parameters given in the field request_query_param of the config file, combines all the information and creates the URL. This function also appends the last poll time read from the state file to the URL. The created URL is then sent to the LAM which hits the REST Server for events and alarm data. The constants.save() saves the poll time to the state file which is then used in the next poll.
function modifyResponse(): This function makes changes to any response received from the REST Server. This function is used when the response data received from the REST Server needs any changes. If no changes are required then do not edit this function. If any modification is done then, after modification the event data is sent to the LAM for extraction, tokenization mapping. For Example, If we are receiving the event data in XML, then we have to convert it into JSON format for the LAM to process it further. This conversion can be carried out in the modifyResponse() function.
function presend(): This function makes any changes to the event data before sending it to MooMS.
The Lambot configuration is done to handle things that cannot be handled by the config file. Users can make changes to the file according to their requirements
It is advisable, that a user with a knowledge of java script makes changes to the Lambot file. Wrong changes done here may break the LAM or will not provide the desired results
In some instances, the attribute strings are quoted. Our JSON parser ignores it, but the standard requires quoting for all strings, so Moogsoft recommends that user quotes all strings.
A user can comment out lines by appending them with a hash.
Starting the REST Client LAM
To start the REST Client LAM enter the following command:
To stop the REST Client LAM enter the following command:
To view the status of REST Client LAM, enter the following command:
Command line attributes
Points to a pathname to find the configuration file for the LAM. This is where the entire configuration for the LAM is specified
Displays all the command line options
Displays the components version number
Specifies the level of debugging. By default, User gets everything. In common with all executables in MOOG, having it set at that level can result in a lot of output (many messages per event message processed).
In all production implementations, it is recommended that log level is set to WARN, which only informs user of matters of importance
Expected to Work
This LAM was tested on a system with the following configurations:
|Operating System||CentOS Linux release 6.7|
The system must at least have the above mentioned system requirements to run the LAM.