Page tree
Skip to end of metadata
Go to start of metadata

Introduction

Rest LAM is a REST endpoint allowing any tool to "push" events to the LAM.

The REST LAM is a link access module that provides a method of data ingestion through RESTful services. It enables Moogsoft AIOps to accept events via HTTP or HTTPS POST requests in a JSON, an XML and YAML format. In case of XML and YAML format, the REST LAM will by default convert the received events into JSON format. The REST LAM provide responses to the data sender in the form of a JSON object.

For the REST LAM to successfully ingest data, the incoming REST messages must conform to HTTP message format standard, which involves correct setting of the header and payload. The REST LAM should respond with standard HTTP response codes, and in the case of error, it should respond with JSON formatted response messages. You can even send multiple events to the LAM in a single request.

The REST LAM can be configured to listen to a specific port, and if authentication is configured, the LAM will reject those incoming requests which do not contain the expected secret auth_token. Additionally, you can configure the LAM to operate in SSL mode and accept requests only via HTTPS (TLS/SSL), which requires SSL certificates.

Configuring the REST LAM

The configuration file rest_lam.conf (default location /usr/share/moogsoft/config/) includes rules (data conversion), LAMBot and LAMBot module configuration. 

The configuration file contains a JSON object. At the first layer of the object, LAM has a parameter called config, and the object that follows config has all the necessary information to control the LAM.

The following sections are available for configuration in the rest_lam.conf file:

Monitor

Agent

HA Configuration

Mapping

Constants and Conversions

Monitor

The REST LAM takes the connection information from the Monitor section of the config file. You can configure the parameters here to establish a connection with REST.

General

Field
Type
Description
Example
name and classString

Reserved fields: do not change. Default values are REST Lam Monitor and CRestMonitor.


portIntegerThe port on the server that listens for REST messages. It is an optional value that defaults to  8888.
address IntegerThe address on the server that listens for REST messages. It is an optional value that defaults to all interfaces.0.0.0.0
use_sslBooleanSet to true if you want the server to use HTTPS. The default value is false.
use_client_certificatesBoolean

Set to true if you want client certification. By default it is set to false. If set to true, ssl must also be enabled.


client_ca_filename  String

Enter the name of the root ca file. This should be stored in the directory given in path_to_ssl_files field.

ca.crt
auth_tokenString

A shared secret token in the request body used to authenticate requests. If not defined or empty, then authentication via this means is disabled. If defined, then all requests must contain this token in the body otherwise the request will be rejected.

my_secret
encrypted_auth_tokenStringUse this option to specify the 'shared secret' body token in the encrypted.
header_auth_tokenString

A shared secret token in the request header used to authenticate requests. If not defined or empty, then authentication via this means is disabled. If defined, then all requests must contain this token in the header otherwise the request will be rejected.


encrypted_header_auth_tokenStringUse this option to specify the 'shared secret' header token in the encrypted.
authentication_typeString

You can set the authentication_type as follows:

basic: In this case the authentication will use the graze login. 

none: It's a default value, if no configuration is specified, then basic authentication will not occur.

jwt: In this case the authentication will use the JWT Token.

If you have set authentication_type as jwt, then you have to define the "claims" which was used inside a JWT claim set while creating a JWT token.

The following claims can be defined:

secretkey: It's a mandatory value and this will used to validate a token.

Issuer (iss): It's an optional claim. This identifies principal which has issued the JWT. If the iss does not  identify itself  in the Token , then Token will be rejected.

Subject (sub): It's an optional claim. This identifies Subject of the JWT. If the sub does not  identify itself  in the Token , then Token will be rejected.

Audience (aud): It's an optional claim. This claim identifies the recipients that the JWT is intended for. Each principal which is intended to process the JWT must identify itself with a value in the audience claim. If the principal processing the claim does not identify itself with a value in the aud claim when this claim is present, then the JWT must be rejected.

Expiration time (exp): It's an optional claim. This identifies the expiration time on or after which the JWT will not be accepted for processing.

Not before (nbf): It's an optional claim. This identifies the time on which the JWT will start to be accepted for processing.

Issued at (iat): It's an optional claim. This identifies the time at which the JWT was issued.

JWT ID (jti): It's an optional claim. This is a case sensitive unique identifier of the token among different issuers.



jwt: 
{     
secretKey : "secret",   
sub       : "moogsoft",   
iss       : "moogsoft",   
aud       : "moogsoft",
jti       : " "
}
authentication_cacheBoolean

Valid only when authentication_type is set to basic.

When set to true (default), a hashed version of user's password is kept in the internal cache for the duration of their session. This speeds request handling, but it can be a security issue for some users. If set to false, the basic authentication internal cache is disabled. Users are authenticated with each request, which slows request handling.


accept_all_jsonBoolean

If set to true, the REST LAM will expect and process incoming requests using any form of JSON. Moog REST LAM protocol will be used in case of default value false.

Incoming requests can be any valid form of JSON. The REST LAM configuration and LAMBot define the structure of the created AIOps Event.

When set to true:
 Use true when an event sender does not conform to the AIOps REST LAM protocol.

 

Together with list_contains_multiple_events set to true, an event sender (such as a third party commercial off-the-shelf product which defines its own rigid output of events) can be accepted into the REST LAM and be converted into AIOps Events that are sent out on to MooMS.

Incoming requests must use the AIOps REST LAM protocol. The default AIOps configuration for the REST LAM and LAMBot should accept, convert and send the incoming requests.


When set to false:
 Use false when you can determine how your event sender sends events, and you do not want to change the default rest lam config/lambot.







Example 1: Valid JSON for an incoming request, if accept_all_json is set to true:

curl http://<hostname>:8888 -H "Content-Type: application/json" -X POST -v --data '[
    {
        "signature":"SIGNATURE1",
        "source_id":"my_source_id",
        "external_id":"my_external_id",
        "manager":"TestMgr1",
        "source":"TestHost1",
        "class":"my_class",
        "agent":"TestAgent1",
        "agent_location":"my_agent_location",
        "type":"TestType1",
        "severity":3,
        "description":"TestDesc1",
        "first_occurred":1411134582,
        "agent_time":1411134582
    },
    {
        "signature":"SIGNATURE2",
        "source_id":"my_source_id2",
        "external_id":"my_external_id2",
        "manager":"TestMgr2",
        "source":"TestHost2",
        "class":"my_class",
        "agent":"TestAgent2",
        "agent_location":"my_agent_location",
        "type":"TestType2",
        "severity":5,
        "description":"TestDesc2",
        "first_occurred":1411134582,
        "agent_time":1411134582
    }
]'

Example 2: JSON using the AIOps REST LAM protocol, if accept_all_json is set to false:

{
  auth_token: <string>,
  events: [
            {event1},
            .
            .
            .
            {eventn}
          ]
}
lists_contain_multiple_eventsBoolean

When the Moogsoft Rest LAM protocol is not in use, then you must define whether a list should be interpreted as multiple events or as a single one.


num_threadsInteger

The number of worker threads that should be used to handle incoming connections. It is an optional value that defaults to the number of CPUs available (up to a maximum of 8).


rest_response_modeString

Defines when the REST response is sent during the event ingestion lifecycle.

Default mode: on_receipt. If not specified, the mode will set to on_receipt.

Possible Modes are:
on_receipt = Respond to REST once a valid event has been received.
event_forwarded = Respond to REST once the event is sent out to MooMS.
event_processed = Respond to REST once the event has been processed by a Farmd Instance's AlertBuilder Moolet.


rpc_response_timeoutInteger

Used when the rest_response_mode is in event_processed mode, it determines the number of seconds to wait for a response regarding an Event being processed by Farmd before timing out and sending a general error response to the REST connection. This configuration has no effect when rest_response_mode is in on_receipt or event_forwarded, as those modes do not require a response from Farmd or other bespoke processes.
Default: 20 seconds. If not specified, the default value will be used.


event_ack_modeString

This determines when an Event from this LAM should be acknowledged by the receiving process.
Default: queued_for_processing. If not specified, then default mode will be used.
Possible modes are:
queued_for_processing = Acknowledge Event once it has been added to the Moolet queue.
event_processed = Acknowledge Event once it has been
processed by a Moolet.



Secure Sockets Layer

Field
Type
Description

use_ssl

Boolean

 Set to true to enable SSL Communication:

  • path_to_ssl_files: Enter the path of the directory where all the certificates are stored. If the path begins with ‘.’ or ‘/’ then, the path will be used as specified. Otherwise, MOOGSOFT_HOME is prepended to the path. For example, if MOOGSOFT_HOME is /opt/moogsoft/ and path_to_ssl is set to config, then the location will be defined as /opt/moogsoft/config.
  • ssl_key_filename: Enter the ssl key filename here, e.g. "server.key".

  • ssl_cert_filename: Enter the ssl certificate filename here, e.g. "server.pem".
  • ssl_protocols: Only applicable if use_ssl = true. This configuration dictates which SSL protocols are enforced by the REST LAM; the following protocols are allowed to be specified:
    SSLv3
    TLSv1
    TLSv1.1
    TLSv1.2
    If SSL is in use and no value is specified for this configuration then only TLSv1.2 is allowed by default.

Example

Config File
config :
    {

        monitor:
        {
            name                    		  : "REST Lam Monitor",

            class               		      : "CRestMonitor",

			port                              : 8888,

            address                 		  : "0.0.0.0",

            use_ssl                      	  : false,

            path_to_ssl_files          		  : "config",

            ssl_key_filename        		  : "server.key",

	      	ssl_cert_filename         		  : "server.pem",

	      	use_client_certificates      	  : false, 
 
			client_ca_filename                : "ca.crt",
			
			auth_token						  : "my_secret",
            
			#encrypted_auth_token             : "dfJtTQMGiFHfiq7sCmxguBt6Jv+eytkoiKCquSB/7iWxpgGsG2aez3z2j7SuBtKj",

			header_auth_token           	  : "my_secret",

            #encrypted_header_auth_token 	  : "dfJtTQMGiFHfiq7sCmxguBt6Jv+eytkoiKCquSB/7iWxpgGsG2aez3z2j7SuBtKj",

            ssl_protocols                     : "TLSv1.2",

            authentication_type               : "none",

			authentication_cache			  : true,

            accept_all_json                   : true,

			lists_contain_multiple_events     : true,
			
			num_threads                       : 5,
			
            rest_response_mode                : "on_receipt",

   			rpc_response_timeout              : 20,

			event_ack_mode 					  : "queued_for_processing",
			
		
	  },

Agent

Agent allows you to define two parameters:

Field
nameThis is the agent name, the events sent to MooMS by the REST LAM are identified by the agent name in the log. In this LAM, the agent name is DATA_SOURCE.
logREST LAM will write its ingress contents in the file rest_lam.log located at /var/log/moogsoft/.

HA Configuration

Refer to the document Integrations HA Configuration

LamBot Configuration to extract sub fields from a nested event data structure

In the following example the received event has multiple nested fields. The field summary is to be extracted in the received format, but it is nested in env:envelope> env:Body> createRequest> Summary. The env: in the field name may create problem in the mapping section of the config file. So it's better to remove it from the received data in the  modifyResponse function of the LamBot.

Example of an event recieved by REST LAM in the XML format:

<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"><env:Header/><env:Body><createRequest
xmlns="http://moogsoft.com"
xmlns:emcf="http://xmlns.oracle.com/sysman/connector">

<Summary xmlns="http://moogsoft.com">CPUUtilization for 3 is 6.787%, crossed warning (5) or critical (10)threshold.</Summary>
<Description xmlns="http://moogsoft.com">
Event created in Enterprise Manager: 
Target information:
Target Type: host
Target Name: WIN-8U1RA5NAA6I
Target URL: http://win-8u1ra5naa6i:7788/em//em/redirect?pageType=TARGET_HOMEPAGE&amp;targetName=WIN-8U1RA5NAA6I&amp;targetType=host</Description>

<Severity xmlns="http://moogsoft.com">2</Severity>

<Priority xmlns="http://moogsoft.com">Medium</Priority>

</createRequest></env:Body></env:Envelope>

The corresponding configuration in the LamBot to remove env: form the received event fields and sending it to LAM for tokenizing and mapping is mentioned below :

Lambot:
function modifyResponse(inBoundEventData)
{               
// if you want to modify response data before injecting               
//the same in LAM for
tokenizing                           
/*inBoundEventData contain below field which can be manipulated as per
requirement                                              
1. responseData - the event data received from the rest server             
*/               
var inputString = inBoundEventData.value('responseData');                            
var obj = JSON.parse(inputString);                            
var modifiedinput = obj.(env:Envelope).(env:Body).createRequest;                          
logger.info(JSON.stringify(modifiedinput));               
inBoundEventData.set('responseData', JSON.stringify(modifiedinput));
return true;
}

The corresponding mapping in the configuration file is mentioned below. The mapping for class is done by $Description.content, if you have not done the configuration in the LamBot, then the mapping would be $env:envelope.env:Body.createRequest.Summary, which can throw an error.

Mapping:

                { name: "signature", rule:      "$signature" },
                { name: "source_id", rule:      "$source_id" },
                { name: "external_id", rule:    "$external_id" },
                { name: "manager", rule:        "$manager" },
                { name: "source", rule:         "$source" },
                { name: "class", rule:          "$class" },
                { name: "agent", rule:          "$LamInstanceName" },
                { name: "agent_location", rule: "$agent_location" },
                { name: "type", rule:           "$type" },
                { name: "severity", rule:       "$severity", conversion: "stringToInt" },
                { name: "description", rule:    "$description" },
                { name: "agent_time", rule:     "$agent_time", conversion: "stringToInt" }

Example configuration

  • rest_lam.conf configuration example without authorization enabled:


monitor:
  {
     name                : "Rest Lam Monitor",
     class               : "CRestMonitor",
     port                : 8888,
     address             : "localhost",
     use_ssl             : false,
     authentication_type : "none",
     accept_all_json     : false,
     num_threads         : 5,
     rest_response_mode  : "on_receipt",
     rpc_response_timeout: 20
  }


  • rest_lam.conf configuration example with basic authorization enabled:


monitor:
  {
     name                : "Rest Lam Monitor",
     class               : "CRestMonitor",
     port                : 8888,
     address             : "localhost",
     use_ssl             : true,
     path_to_ssl_files   : "/root/temp",
     ssl_key_filename    : "server.key",
     ssl_cert_filename   : "server.pem",
     auth_token          : "my_secret",
     authentication_type : “basic”,
     authentication_cache: true,
     accept_all_json     : false,
     num_threads         : 5,
     rest_response_mode  : "on_receipt",
     rpc_response_timeout: 20
  } 


  • rest_lam.conf configuration example with JWT authentication (non-ssl):


monitor:
  {
     name                : "Rest Lam Monitor",
     class               : "CRestMonitor",
     port                : 8888,
     address             : "localhost",
     use_ssl             : false,
     authentication_type : "jwt",
	 jwt				 : 
							{     
								secretKey : "secret",   
								sub       : "moogsoft",   
								iss       : "moogsoft",   
								aud       : "moogsoft",
								jti       : " "
							}						
     accept_all_json     : false,
     num_threads         : 5,
     rest_response_mode  : "on_receipt",
     rpc_response_timeout: 20
  }


JSON request structure

If possible, the JSON structure of incoming requests should be in standard AIOps format. The payload of the REST message is an array of events in the defined structure, for example:


{"auth_token" : "my_secret",
  "events" : [
    {
      "name1" 		: "1",
      "name2" 		: "false",
      "name3" 		: "MINOR",
      "signature" 	: "sig",
      "source_id" 	: "98",
      "external_id" : "ext",
      "manager" 	: "man",
      "source" 		: "db",
      "class" 		: "a class"
    },
    {
      "name1" : "2",
      "name2" : "false",
      "name3" : "MINOR",
    }
  ]
}

JSON response structure

REST LAM response messages are in the form of a JSON object, and are formatted to facilitate machine interpretation. 
In the response:

  • cached indicates whether the message content was cached (1 = yes, 0 = no).
  • processed indicates whether the message was successfully processed (1 = yes, 0 = no).
  • received indicates whether the message was successfully received (1 = yes, 0 = no).

For example: 

  • A successful response example:


    { 
       "message":"Processed 1 event(s)",
       "response":{ 
          "cached":0,
          "processed":1,
          "received":1
       },
       "success":true
    }
  • A failed response example without caching:


    { 
       "message":"General error. Processed 0 event(s)",
       "response":{ 
          "cached":0,
          "processed":0,
          "received":1
       },
       "statusCode":1000,
       "success":false
    }
  • A failed response example where caching has occurred:


    { 
       "message":"Content accepted but cached, processing not guaranteed. Processed 0 event(s)",
       "response":{ 
          "cached":1,
          "processed":0,
          "received":1
       },
       "statusCode":5003,
       "success":false
    }

    Testing and Troubleshooting

  • Use curl commands to test whether the REST LAM is correctly configured and accepting the REST messages or not.
  • Ensure that you have network access to both the host machine and the specified port, and also the port is open through the server firewall.

REST LAM Response Codes

The REST LAM provides the following standard HTTP return codes indicating the state of the received message:

HTTP
return
code


Description
200

OK
The REST LAM has received and accepted the payload, which is valid JSON. There is no response message. 

This is not a positive indication that the Event has been properly processed by the LAM or LAMbot.
Standard LAM log file response messages are issued for error conditions.

400Bad Request
The JSON in the payload is invalid, so the request is rejected. The error could be in any part of the JSON, so the details should be reviewed with a standard JSON parser or lint. The response message is a JSON object containing the statusCode and message.
401Unauthorised
The REST LAM configuration file has an auth_token defined but there is no auth_token in the header of the message, or the auth_token provided does not match. The request is rejected and will need to be resent with the correct auth_token. The response message is a JSON object containing the  statusCode and message.

405

Method not allowed
The HTTP request used an unsupported method. The only supported method is POST. The response message is a JSON object containing the  statusCode  and message.
406Method not acceptable
The HTTP request used a content type that is unsupported. The only supported content type is application/JSON. The response message is a JSON object containing the  statusCode and message.

In the event of an error, the REST LAM provides the following REST response codes indicating the status of the request:

REST
response
code 

Description
1000General error
The request could not proceed and failed in a way that was not expected
3000General Security Error
There was an error with authentication
3001Security Authentication Error
Authentication did not succeed
4000General Method Error
There was an error with the method used
4001Method Not Supported
The HTTP method used is not supported
5000General Content Error
There was an error with the content
5001Content Not Supported
The content type used is not supported
5002Content Decoding Error
The content could not be decoded

REST LAM configuration examples

The following REST LAM configuration file examples highlight authentication and SSL, along with the example of curl commands to generate Events. 

1. An example rest_lam.conf with no authentication or SSL (auth_token  commented out, use_ssl set to falsand related SSL properties commented out):

monitor:
{
      name : "Rest Lam Monitor",
      class : "CRestMonitor",
      port : 9876,
      use_ssl : false,
      #path_to_ssl_files : "/tmp",
      #ssl_key_filename : "server.key",
      #ssl_cert_filename : "server.pem",
      #auth_token : "abcd1234",
      num_threads : 5
    },
    agent:
    {
      name : "DATA_SOURCE"
      #log : "/tmp/rest_lam.log"
    },
    constants:
    {
    },
    conversions:
    {
      stringToInt:
      {
        input: "STRING",
        output: "INTEGER"
      }
    },
    mapping :
    {
      catchAll: "overflow",
      rules:
      [
        { name: "signature",      rule: "$signature" },
        { name: "source_id",      rule: "$source_id" },
        { name: "external_id",    rule: "$external_id" },
        { name: "manager",        rule: "$manager" },
        { name: "source",         rule: "$source" },
        { name: "class",          rule: "$class" },
        { name: "agent",          rule: "$agent" },
        { name: "agent_location", rule: "$agent_location" },
        { name: "type",           rule: "$type" },
        { name: "severity",       rule: "$severity",       conversion: "stringToInt" },
        { name: "description",    rule: "$description" },
        { name: "first_occurred", rule: "$first_occurred", conversion: "stringToInt" },
        { name: "agent_time",     rule: "$agent_time",     conversion: "stringToInt" }
      ]
    },
    filter:
    {
      presend: "RestLam.js"
}

The following is the curl command for the above configuration:


curl http://moogbox2:9876 -H "Content-Type: application/json" --insecure -X POST -v --data '
{ 
   "events":[ 
      { 
         "signature":"SIGNATURE1",
         "source_id":"my_source_id",
         "external_id":"my_external_id",
         "manager":"TestMgr1",
         "source":"TestHost1",
         "class":"my_class",
         "agent":"TestAgent1",
         "agent_location":"my_agent_location",
         "type":"TestType1",
         "severity":3,
         "description":"TestDesc1",
         "first_occurred":1411134582,
         "agent_time":1411134582
      }
   ]
}' 

The above curl command will generate the following Event from the REST LAM:


{ 
   "_MOOTADATA_":{ 
      "creation_time":1452090420708
   },
   "agent":"TestAgent1",
   "agent_location":"my_agent_location",
   "agent_time":1411134582,
   "class":"my_class",
   "description":"TestDesc1",
   "external_id":"my_external_id",
   "manager":"TestMgr1",
   "overflow":{ 
      "LamInstanceName":"DATA_SOURCE",
      "first_occurred":1411134582
   },
   "severity":3,
   "signature":"SIGNATURE1",
   "source":"TestHost1",
   "source_id":"my_source_id",
   "type":"TestType1"
}

The following is the curl command for the above configuration:


curl http://moogbox2:9876 -H "Content-Type: application/json" --insecure -X POST -v --data '
{ 
   "events":[ 
      { 
         "signature":"SIGNATURE1",
         "source_id":"my_source_id",
         "external_id":"my_external_id",
         "manager":"TestMgr1",
         "source":"TestHost1",
         "class":"my_class",
         "agent":"TestAgent1",
         "agent_location":"my_agent_location",
         "type":"TestType1",
         "severity":3,
         "description":"TestDesc1",
         "first_occurred":1411134582,
         "agent_time":1411134582
      },
      { 
         "signature":"SIGNATURE2",
         "source_id":"my_source_id",
         "external_id":"my_external_id",
         "manager":"TestMgr2",
         "source":"TestHost2",
         "class":"my_class",
         "agent":"TestAgent2",
         "agent_location":"my_agent_location",
         "type":"TestType2",
         "severity":3,
         "description":"TestDesc2",
         "first_occurred":1411134582,
         "agent_time":1411134582
      },
      { 
         "signature":"SIGNATURE3",
         "source_id":"my_source_id",
         "external_id":"my_external_id",
         "manager":"TestMgr3",
         "source":"TestHost3",
         "class":"my_class",
         "agent":"TestAgent3",
         "agent_location":"my_agent_location",
         "type":"TestType3",
         "severity":3,
         "description":"TestDesc3",
         "first_occurred":1411134582,
         "agent_time":1411134582
      }
   ]
}'

The above command will generate the following three separate Events from the REST LAM:

Event 1


{ 
   "_MOOTADATA_":{ 
      "creation_time":1452090542681
   },
   "agent":"TestAgent1",
   "agent_location":"my_agent_location",
   "agent_time":1411134582,
   "class":"my_class",
   "description":"TestDesc1",
   "external_id":"my_external_id",
   "manager":"TestMgr1",
   "overflow":{ 
      "LamInstanceName":"DATA_SOURCE",
      "first_occurred":1411134582
   },
   "severity":3,
   "signature":"SIGNATURE1",
   "source":"TestHost1",
   "source_id":"my_source_id",
   "type":"TestType1"
}

Event 2


{ 
   "_MOOTADATA_":{ 
      "creation_time":1452090542681
   },
   "agent":"TestAgent2",
   "agent_location":"my_agent_location",
   "agent_time":1411134582,
   "class":"my_class",
   "description":"TestDesc2",
   "external_id":"my_external_id",
   "manager":"TestMgr2",
   "overflow":{ 
      "LamInstanceName":"DATA_SOURCE",
      "first_occurred":1411134582
   },
   "severity":3,
   "signature":"SIGNATURE2",
   "source":"TestHost2",
   "source_id":"my_source_id",
   "type":"TestType2"
}

Event 3


{ 
   "_MOOTADATA_":{ 
      "creation_time":1452090542681
   },
   "agent":"TestAgent3",
   "agent_location":"my_agent_location",
   "agent_time":1411134582,
   "class":"my_class",
   "description":"TestDesc3",
   "external_id":"my_external_id",
   "manager":"TestMgr3",
   "overflow":{ 
      "LamInstanceName":"DATA_SOURCE",
      "first_occurred":1411134582
   },
   "severity":3,
   "signature":"SIGNATURE3",
   "source":"TestHost3",
   "source_id":"my_source_id",
   "type":"TestType3"
}

The -- insecure  option used in the curl commands, uses TSL but will not validate the certificate.

2. An example rest_lam.conf with authentication, but no SSL (auth_token  uncommented, use_ssl set to falsand related SSL properties commented out):

monitor:
{
      name : "Rest Lam Monitor",
      class : "CRestMonitor",
      port : 9876,
      use_ssl : false,
      #path_to_ssl_files : "/tmp",
      #ssl_key_filename : "server.key",
      #ssl_cert_filename : "server.pem",
      auth_token : "abcd1234",
      num_threads : 5
    },
    agent:
    {
      name : "DATA_SOURCE"
      #log : "/tmp/rest_lam.log"
    },
    constants:
    {
    },
    conversions:
    {
      stringToInt:
      {
        input: "STRING",
        output: "INTEGER"
      }
    },
    mapping :
    {
      catchAll: "overflow",
      rules:
      [
        { name: "signature",      rule: "$signature" },
        { name: "source_id",      rule: "$source_id" },
        { name: "external_id",    rule: "$external_id" },
        { name: "manager",        rule: "$manager" },
        { name: "source",         rule: "$source" },
        { name: "class",          rule: "$class" },
        { name: "agent",          rule: "$agent" },
        { name: "agent_location", rule: "$agent_location" },
        { name: "type",           rule: "$type" },
        { name: "severity",       rule: "$severity",       conversion: "stringToInt" },
        { name: "description",    rule: "$description" },
        { name: "first_occurred", rule: "$first_occurred", conversion: "stringToInt" },
        { name: "agent_time",     rule: "$agent_time",     conversion: "stringToInt" } 
      ]
    },
    filter:
    {
      presend: "RestLam.js"
}

The following is the curl command for the above configuration:

curl http://moogbox2:9876 -H "Content-Type: application/json" --insecure -X POST -v --data '
{ 
   "auth_token":"abcd1234",
   "events":[ 
      { 
         "signature":"SIGNATURE1",
         "source_id":"my_source_id",
         "external_id":"my_external_id",
         "manager":"TestMgr1",
         "source":"TestHost1",
         "class":"my_class",
         "agent":"TestAgent1",
         "agent_location":"my_agent_location",
         "type":"TestType1",
         "severity":3,
         "description":"TestDesc1",
         "first_occurred":1411134582,
         "agent_time":1411134582
      }
   ]
}'

The above command will generate the following Event from the REST LAM:


{ 
   "_MOOTADATA_":{ 
      "creation_time":1452090420708
   },
   "agent":"TestAgent1",
   "agent_location":"my_agent_location",
   "agent_time":1411134582,
   "class":"my_class",
   "description":"TestDesc1",
   "external_id":"my_external_id",
   "manager":"TestMgr1",
   "overflow":{ 
      "LamInstanceName":"DATA_SOURCE",
      "first_occurred":1411134582
   },
   "severity":3,
   "signature":"SIGNATURE1",
   "source":"TestHost1",
   "source_id":"my_source_id",
   "type":"TestType1"
}

3. An example rest_lam.conf with authentication and SSL (auth_token  uncommented and use_ssl set to true and related SSL properties uncommented):

A key and certificate file must be generated on the host running the rest_lam, for example:

openssl req -new -x509 -days 365 -nodes -subj "/C=''/ST=''/L=''/O='moogsoft'/OU=''/CN=moogbox2" -out /tmp/server.pem -keyout /tmp/server.key
monitor:
{
      name : "Rest Lam Monitor",
      class : "CRestMonitor",
      port : 9876,
      use_ssl : true,
      path_to_ssl_files : "/tmp",
      ssl_key_filename : "server.key",
      ssl_cert_filename : "server.pem",
      auth_token : "abcd1234",
      num_threads : 5
    },
    agent:
    {
      name : "DATA_SOURCE"
      #log : "/tmp/rest_lam.log"
    },
    constants:
    {
    },
    conversions:
    {
      stringToInt:
      {
        input: "STRING",
        output: "INTEGER"
      }
    },
    mapping :
    {
      catchAll: "overflow",
      rules:
      [
        { name: "signature",      rule: "$signature" },
        { name: "source_id",      rule: "$source_id" },
        { name: "external_id",    rule: "$external_id" },
        { name: "manager",        rule: "$manager" },
        { name: "source",         rule: "$source" },
        { name: "class",          rule: "$class" },
        { name: "agent",          rule: "$agent" },
        { name: "agent_location", rule: "$agent_location" },
        { name: "type",           rule: "$type" },
        { name: "severity",       rule: "$severity",       conversion: "stringToInt" },
        { name: "description",    rule: "$description" },
        { name: "first_occurred", rule: "$first_occurred", conversion: "stringToInt" },
        { name: "agent_time",     rule: "$agent_time",     conversion: "stringToInt" } 
      ]
    },
    filter:
    {
      presend: "RestLam.js"
}

The following is the curl command for the above configuration:


curl https://moogbox2:9876 -H "Content-Type: application/json" -X POST -v --data '
{ 
   "auth_token":"abcd1234",
   "events":[ 
      { 
         "signature":"SIGNATURE1",
         "source_id":"my_source_id",
         "external_id":"my_external_id",
         "manager":"TestMgr1",
         "source":"TestHost1",
         "class":"my_class",
         "agent":"TestAgent1",
         "agent_location":"my_agent_location",
         "type":"TestType1",
         "severity":3,
         "description":"TestDesc1",
         "first_occurred":1411134582,
         "agent_time":1411134582
      }
   ]
}'

The above command will generate the following Event from the REST LAM:


{ 
   "_MOOTADATA_":{ 
      "creation_time":1452090420708
   },
   "agent"  		:"TestAgent1",
   "agent_location"	:"my_agent_location",
   "agent_time"		:1411134582,
   "class"			:"my_class",
   "description"	:"TestDesc1",
   "external_id"	:"my_external_id",
   "manager"		:"TestMgr1",
   "overflow"		:{ 
      					"LamInstanceName":"DATA_SOURCE",
      					"first_occurred":1411134582
   					 },
   "severity"		:3,
   "signature"		:"SIGNATURE1",
   "source"			:"TestHost1",
   "source_id"		:"my_source_id",
   "type"			:"TestType1"
}

The remote host sending the curl command must have a copy of the certificate file in the correct location, for example:


/root/server.pem


Mapping 

For events received in JSON format, you can directly map the event fields of REST with Moogsoft fields. The parameters of the received events are displayed in Moogsoft AIOps according to the mapping done here:

mapping :
        {
            
       catchAll: "overflow",
            rules:
            [
                { name: "signature", rule:      "$signature" },
                { name: "source_id", rule:      "$source_id" },
                { name: "external_id", rule:    "$external_id" },
                { name: "manager", rule:        "$manager" },
                { name: "source", rule:         "$source" },
                { name: "class", rule:          "$class" },
                { name: "agent", rule:          "$LamInstanceName" },
                { name: "agent_location", rule: "$agent_location" },
                { name: "type", rule:           "$type" },
                { name: "severity", rule:       "$severity", conversion: "stringToInt" },
                { name: "description", rule:    "$description" },
                { name: "agent_time", rule:     "$agent_time", conversion: "stringToInt" }
            ]
        },
        filter:
        {           
            stream: "myStream",
            presend: "RestLam.js"
        }

Constants and Conversions

Constants and Conversions allows you to convert format of the received data.

Field
Description
Example
Severity and sevConverter
has a conversion defined as sevConverter in the Conversions section, this looks up the value of severity defined in the severity section of constants and returns back the mapped integer corresponding to the severity.
severity:
{

"CLEAR" : 0,
"INDETERMINATE" : 1,
"WARNING" : 2,
"MINOR" : 3,
"MAJOR" : 4
"CRITICAL" : 5

moog_lookup_default : 1

}, 

sevConverter:
{
lookup: "severity",
input : "STRING",
output: "INTEGER"
},

stringToIntUsed in a conversion which forces the system to turn a string token into an integer value.
stringToInt:
{
    input  : "STRING",
    output : "INTEGER"
},

Example

Example Constants and Conversions
constants:
        {
            severity:
            {
                "CLEAR"         : 0,
                "INDETERMINATE" : 1,
                "WARNING"       : 2,
                "MINOR"         : 3,
                "MAJOR"         : 4,
                "CRITICAL"      : 5,              
                moog_lookup_default: 1
            }
        },
        conversions:
        {
            sevConverter:
            {
                lookup: "severity",
                input:  "STRING",
                output: "INTEGER"
            },
           
            stringToInt:
            {
                input:      "STRING",
                output:     "INTEGER"
            }
        },     
           
      

Severity Reference

Moogsoft Severity Levels
severity:
        {
            "CLEAR" 		: 0,
            "INDETERMINATE" : 1,
            "WARNING" 		: 2,
            "MINOR" 		: 3,
            "MAJOR" 		: 4,
            "CRITICAL" 		: 5,
             moog_lookup_default: 1            
        }
LevelDescription
0Clear
1Indeterminate
2Warning
3Minor
4Major
5Critical

Service Operation Reference

Process NameService Name
rest_lamrestlamd

Start the LAM Service:

service restlamd start

Stop the LAM Service:

service restlamd stop

Check the LAM Service status:

service restlamd status

Command Line Reference 

To see the available optional attributes of the rest_lam, run the following command:

rest_lam --help

The rest_lam is a command line executable, and has the following optional attributes:

OptionDescription

--config

Points to a pathname to find the configuration file for the LAM. This is where the entire configuration for the LAM is specified
--helpDisplays all the command line options
--version

Displays the component’s version number

--log level

Specifies the level of debugging. By default, user gets everything. In common with all executables in MOOG, having it set at that level can result in a lot of output (many messages per event message processed).

In all production implementations, it is recommended that log level is set to WARN. This ensures only warning, error and fatal messages are recorded.

Performance Information

Minimum requirement
ComponentValue
CPU2 core
RAM4 GB
Operating SystemCentOS Linux release 6.7

Version

LAM VersionTool VersionVerified By

1.0

Generic

Yes

1.1

Generic

Yes

1.2

Generic

Yes

Introduction

Rest LAM is a REST endpoint allowing any tool to "push" events to the LAM.

The REST LAM is a link access module that provides a method of data ingestion through RESTful services. It enables Moogsoft AIOps to accept events via HTTP or HTTPS POST requests in a JSON, an XML and YAML format. In case of XML and YAML format, the REST LAM will by default convert the received events into JSON format. The REST LAM provide responses to the data sender in the form of a JSON object.

For the REST LAM to successfully ingest data, the incoming REST messages must conform to HTTP message format standard, which involves correct setting of the header and payload. The REST LAM should respond with standard HTTP response codes, and in the case of error, it should respond with JSON formatted response messages. You can even send multiple events to the LAM in a single request.

The REST LAM can be configured to listen to a specific port, and if authentication is configured, the LAM will reject those incoming requests which do not contain the expected secret auth_token. Additionally, you can configure the LAM to operate in SSL mode and accept requests only via HTTPS (TLS/SSL), which requires SSL certificates.

Configuring the REST LAM

The configuration file rest_lam.conf (default location /usr/share/moogsoft/config/) includes rules (data conversion), LAMBot and LAMBot module configuration. 

The configuration file contains a JSON object. At the first layer of the object, LAM has a parameter called config, and the object that follows config has all the necessary information to control the LAM.

The following sections are available for configuration in the rest_lam.conf file:

Monitor

Agent

HA Configuration

Mapping

Constants and Conversions

Monitor

The REST LAM takes the connection information from the Monitor section of the config file. You can configure the parameters here to establish a connection with REST.

General

Field
Type
Description
Example
name and classString

Reserved fields: do not change. Default values are REST Lam Monitor and CRestMonitor.


portIntegerThe port on the server that listens for REST messages. It is an optional value that defaults to  8888.
address IntegerThe address on the server that listens for REST messages. It is an optional value that defaults to all interfaces.0.0.0.0
use_sslBooleanSet to true if you want the server to use HTTPS. The default value is false.
use_client_certificatesBoolean

Set to true if you want client certification. By default it is set to false. If set to true, ssl must also be enabled.


client_ca_filename  String

Enter the name of the root ca file. This should be stored in the directory given in path_to_ssl_files field.

ca.crt
auth_tokenString

A shared secret token in the request body used to authenticate requests. If not defined or empty, then authentication via this means is disabled. If defined, then all requests must contain this token in the body otherwise the request will be rejected.

my_secret
encrypted_auth_tokenStringUse this option to specify the 'shared secret' body token in the encrypted.
header_auth_tokenString

A shared secret token in the request header used to authenticate requests. If not defined or empty, then authentication via this means is disabled. If defined, then all requests must contain this token in the header otherwise the request will be rejected.


encrypted_header_auth_tokenStringUse this option to specify the 'shared secret' header token in the encrypted.
authentication_typeString

You can set the authentication_type as follows:

basic: In this case the authentication will use the graze login. 

none: It's a default value, if no configuration is specified, then basic authentication will not occur.

jwt: In this case the authentication will use the JWT Token.

If you have set authentication_type as jwt, then you have to define the "claims" which was used inside a JWT claim set while creating a JWT token.

The following claims can be defined:

secretkey: It's a mandatory value and this will used to validate a token.

Issuer (iss): It's an optional claim. This identifies principal which has issued the JWT. If the iss does not  identify itself  in the Token , then Token will be rejected.

Subject (sub): It's an optional claim. This identifies Subject of the JWT. If the sub does not  identify itself  in the Token , then Token will be rejected.

Audience (aud): It's an optional claim. This claim identifies the recipients that the JWT is intended for. Each principal which is intended to process the JWT must identify itself with a value in the audience claim. If the principal processing the claim does not identify itself with a value in the aud claim when this claim is present, then the JWT must be rejected.

Expiration time (exp): It's an optional claim. This identifies the expiration time on or after which the JWT will not be accepted for processing.

Not before (nbf): It's an optional claim. This identifies the time on which the JWT will start to be accepted for processing.

Issued at (iat): It's an optional claim. This identifies the time at which the JWT was issued.

JWT ID (jti): It's an optional claim. This is a case sensitive unique identifier of the token among different issuers.



jwt: 
{     
secretKey : "secret",   
sub       : "moogsoft",   
iss       : "moogsoft",   
aud       : "moogsoft",
jti       : " "
}
authentication_cacheBoolean

Valid only when authentication_type is set to basic.

When set to true (default), a hashed version of user's password is kept in the internal cache for the duration of their session. This speeds request handling, but it can be a security issue for some users. If set to false, the basic authentication internal cache is disabled. Users are authenticated with each request, which slows request handling.


accept_all_jsonBoolean

If set to true, the REST LAM will expect and process incoming requests using any form of JSON. Moog REST LAM protocol will be used in case of default value false.

Incoming requests can be any valid form of JSON. The REST LAM configuration and LAMBot define the structure of the created AIOps Event.

When set to true:
 Use true when an event sender does not conform to the AIOps REST LAM protocol.

 

Together with list_contains_multiple_events set to true, an event sender (such as a third party commercial off-the-shelf product which defines its own rigid output of events) can be accepted into the REST LAM and be converted into AIOps Events that are sent out on to MooMS.

Incoming requests must use the AIOps REST LAM protocol. The default AIOps configuration for the REST LAM and LAMBot should accept, convert and send the incoming requests.


When set to false:
 Use false when you can determine how your event sender sends events, and you do not want to change the default rest lam config/lambot.







Example 1: Valid JSON for an incoming request, if accept_all_json is set to true:

curl http://<hostname>:8888 -H "Content-Type: application/json" -X POST -v --data '[
    {
        "signature":"SIGNATURE1",
        "source_id":"my_source_id",
        "external_id":"my_external_id",
        "manager":"TestMgr1",
        "source":"TestHost1",
        "class":"my_class",
        "agent":"TestAgent1",
        "agent_location":"my_agent_location",
        "type":"TestType1",
        "severity":3,
        "description":"TestDesc1",
        "first_occurred":1411134582,
        "agent_time":1411134582
    },
    {
        "signature":"SIGNATURE2",
        "source_id":"my_source_id2",
        "external_id":"my_external_id2",
        "manager":"TestMgr2",
        "source":"TestHost2",
        "class":"my_class",
        "agent":"TestAgent2",
        "agent_location":"my_agent_location",
        "type":"TestType2",
        "severity":5,
        "description":"TestDesc2",
        "first_occurred":1411134582,
        "agent_time":1411134582
    }
]'

Example 2: JSON using the AIOps REST LAM protocol, if accept_all_json is set to false:

{
  auth_token: <string>,
  events: [
            {event1},
            .
            .
            .
            {eventn}
          ]
}
lists_contain_multiple_eventsBoolean

When the Moogsoft Rest LAM protocol is not in use, then you must define whether a list should be interpreted as multiple events or as a single one.


num_threadsInteger

The number of worker threads that should be used to handle incoming connections. It is an optional value that defaults to the number of CPUs available (up to a maximum of 8).


rest_response_modeString

Defines when the REST response is sent during the event ingestion lifecycle.

Default mode: on_receipt. If not specified, the mode will set to on_receipt.

Possible Modes are:
on_receipt = Respond to REST once a valid event has been received.
event_forwarded = Respond to REST once the event is sent out to MooMS.
event_processed = Respond to REST once the event has been processed by a Farmd Instance's AlertBuilder Moolet.


rpc_response_timeoutInteger

Used when the rest_response_mode is in event_processed mode, it determines the number of seconds to wait for a response regarding an Event being processed by Farmd before timing out and sending a general error response to the REST connection. This configuration has no effect when rest_response_mode is in on_receipt or event_forwarded, as those modes do not require a response from Farmd or other bespoke processes.
Default: 20 seconds. If not specified, the default value will be used.


event_ack_modeString

This determines when an Event from this LAM should be acknowledged by the receiving process.
Default: queued_for_processing. If not specified, then default mode will be used.
Possible modes are:
queued_for_processing = Acknowledge Event once it has been added to the Moolet queue.
event_processed = Acknowledge Event once it has been
processed by a Moolet.



Secure Sockets Layer

Field
Type
Description

use_ssl

Boolean

 Set to true to enable SSL Communication:

  • path_to_ssl_files: Enter the path of the directory where all the certificates are stored. If the path begins with ‘.’ or ‘/’ then, the path will be used as specified. Otherwise, MOOGSOFT_HOME is prepended to the path. For example, if MOOGSOFT_HOME is /opt/moogsoft/ and path_to_ssl is set to config, then the location will be defined as /opt/moogsoft/config.
  • ssl_key_filename: Enter the ssl key filename here, e.g. "server.key".

  • ssl_cert_filename: Enter the ssl certificate filename here, e.g. "server.pem".
  • ssl_protocols: Only applicable if use_ssl = true. This configuration dictates which SSL protocols are enforced by the REST LAM; the following protocols are allowed to be specified:
    SSLv3
    TLSv1
    TLSv1.1
    TLSv1.2
    If SSL is in use and no value is specified for this configuration then only TLSv1.2 is allowed by default.

Example

Config File
config :
    {

        monitor:
        {
            name                    		  : "REST Lam Monitor",

            class               		      : "CRestMonitor",

			port                              : 8888,

            address                 		  : "0.0.0.0",

            use_ssl                      	  : false,

            path_to_ssl_files          		  : "config",

            ssl_key_filename        		  : "server.key",

	      	ssl_cert_filename         		  : "server.pem",

	      	use_client_certificates      	  : false, 
 
			client_ca_filename                : "ca.crt",
			
			auth_token						  : "my_secret",
            
			#encrypted_auth_token             : "dfJtTQMGiFHfiq7sCmxguBt6Jv+eytkoiKCquSB/7iWxpgGsG2aez3z2j7SuBtKj",

			header_auth_token           	  : "my_secret",

            #encrypted_header_auth_token 	  : "dfJtTQMGiFHfiq7sCmxguBt6Jv+eytkoiKCquSB/7iWxpgGsG2aez3z2j7SuBtKj",

            ssl_protocols                     : "TLSv1.2",

            authentication_type               : "none",

			authentication_cache			  : true,

            accept_all_json                   : true,

			lists_contain_multiple_events     : true,
			
			num_threads                       : 5,
			
            rest_response_mode                : "on_receipt",

   			rpc_response_timeout              : 20,

			event_ack_mode 					  : "queued_for_processing",
			
		
	  },

Agent

Agent allows you to define two parameters:

Field
nameThis is the agent name, the events sent to MooMS by the REST LAM are identified by the agent name in the log. In this LAM, the agent name is DATA_SOURCE.
logREST LAM will write its ingress contents in the file rest_lam.log located at /var/log/moogsoft/.

HA Configuration

Refer to the document Integrations HA Configuration

LamBot Configuration to extract sub fields from a nested event data structure

In the following example the received event has multiple nested fields. The field summary is to be extracted in the received format, but it is nested in env:envelope> env:Body> createRequest> Summary. The env: in the field name may create problem in the mapping section of the config file. So it's better to remove it from the received data in the  modifyResponse function of the LamBot.

Example of an event recieved by REST LAM in the XML format:

<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"><env:Header/><env:Body><createRequest
xmlns="http://moogsoft.com"
xmlns:emcf="http://xmlns.oracle.com/sysman/connector">

<Summary xmlns="http://moogsoft.com">CPUUtilization for 3 is 6.787%, crossed warning (5) or critical (10)threshold.</Summary>
<Description xmlns="http://moogsoft.com">
Event created in Enterprise Manager: 
Target information:
Target Type: host
Target Name: WIN-8U1RA5NAA6I
Target URL: http://win-8u1ra5naa6i:7788/em//em/redirect?pageType=TARGET_HOMEPAGE&amp;targetName=WIN-8U1RA5NAA6I&amp;targetType=host</Description>

<Severity xmlns="http://moogsoft.com">2</Severity>

<Priority xmlns="http://moogsoft.com">Medium</Priority>

</createRequest></env:Body></env:Envelope>

The corresponding configuration in the LamBot to remove env: form the received event fields and sending it to LAM for tokenizing and mapping is mentioned below :

Lambot:
function modifyResponse(inBoundEventData)
{               
// if you want to modify response data before injecting               
//the same in LAM for
tokenizing                           
/*inBoundEventData contain below field which can be manipulated as per
requirement                                              
1. responseData - the event data received from the rest server             
*/               
var inputString = inBoundEventData.value('responseData');                            
var obj = JSON.parse(inputString);                            
var modifiedinput = obj.(env:Envelope).(env:Body).createRequest;                          
logger.info(JSON.stringify(modifiedinput));               
inBoundEventData.set('responseData', JSON.stringify(modifiedinput));
return true;
}

The corresponding mapping in the configuration file is mentioned below. The mapping for class is done by $Description.content, if you have not done the configuration in the LamBot, then the mapping would be $env:envelope.env:Body.createRequest.Summary, which can throw an error.

Mapping:

                { name: "signature", rule:      "$signature" },
                { name: "source_id", rule:      "$source_id" },
                { name: "external_id", rule:    "$external_id" },
                { name: "manager", rule:        "$manager" },
                { name: "source", rule:         "$source" },
                { name: "class", rule:          "$class" },
                { name: "agent", rule:          "$LamInstanceName" },
                { name: "agent_location", rule: "$agent_location" },
                { name: "type", rule:           "$type" },
                { name: "severity", rule:       "$severity", conversion: "stringToInt" },
                { name: "description", rule:    "$description" },
                { name: "agent_time", rule:     "$agent_time", conversion: "stringToInt" }

Example configuration

  • rest_lam.conf configuration example without authorization enabled:


monitor:
  {
     name                : "Rest Lam Monitor",
     class               : "CRestMonitor",
     port                : 8888,
     address             : "localhost",
     use_ssl             : false,
     authentication_type : "none",
     accept_all_json     : false,
     num_threads         : 5,
     rest_response_mode  : "on_receipt",
     rpc_response_timeout: 20
  }


  • rest_lam.conf configuration example with basic authorization enabled:


monitor:
  {
     name                : "Rest Lam Monitor",
     class               : "CRestMonitor",
     port                : 8888,
     address             : "localhost",
     use_ssl             : true,
     path_to_ssl_files   : "/root/temp",
     ssl_key_filename    : "server.key",
     ssl_cert_filename   : "server.pem",
     auth_token          : "my_secret",
     authentication_type : “basic”,
     authentication_cache: true,
     accept_all_json     : false,
     num_threads         : 5,
     rest_response_mode  : "on_receipt",
     rpc_response_timeout: 20
  } 


  • rest_lam.conf configuration example with JWT authentication (non-ssl):


monitor:
  {
     name                : "Rest Lam Monitor",
     class               : "CRestMonitor",
     port                : 8888,
     address             : "localhost",
     use_ssl             : false,
     authentication_type : "jwt",
	 jwt				 : 
							{     
								secretKey : "secret",   
								sub       : "moogsoft",   
								iss       : "moogsoft",   
								aud       : "moogsoft",
								jti       : " "
							}						
     accept_all_json     : false,
     num_threads         : 5,
     rest_response_mode  : "on_receipt",
     rpc_response_timeout: 20
  }


JSON request structure

If possible, the JSON structure of incoming requests should be in standard AIOps format. The payload of the REST message is an array of events in the defined structure, for example:


{"auth_token" : "my_secret",
  "events" : [
    {
      "name1" 		: "1",
      "name2" 		: "false",
      "name3" 		: "MINOR",
      "signature" 	: "sig",
      "source_id" 	: "98",
      "external_id" : "ext",
      "manager" 	: "man",
      "source" 		: "db",
      "class" 		: "a class"
    },
    {
      "name1" : "2",
      "name2" : "false",
      "name3" : "MINOR",
    }
  ]
}

JSON response structure

REST LAM response messages are in the form of a JSON object, and are formatted to facilitate machine interpretation. 
In the response:

  • cached indicates whether the message content was cached (1 = yes, 0 = no).
  • processed indicates whether the message was successfully processed (1 = yes, 0 = no).
  • received indicates whether the message was successfully received (1 = yes, 0 = no).

For example: 

  • A successful response example:


    { 
       "message":"Processed 1 event(s)",
       "response":{ 
          "cached":0,
          "processed":1,
          "received":1
       },
       "success":true
    }
  • A failed response example without caching:


    { 
       "message":"General error. Processed 0 event(s)",
       "response":{ 
          "cached":0,
          "processed":0,
          "received":1
       },
       "statusCode":1000,
       "success":false
    }
  • A failed response example where caching has occurred:


    { 
       "message":"Content accepted but cached, processing not guaranteed. Processed 0 event(s)",
       "response":{ 
          "cached":1,
          "processed":0,
          "received":1
       },
       "statusCode":5003,
       "success":false
    }

    Testing and Troubleshooting

  • Use curl commands to test whether the REST LAM is correctly configured and accepting the REST messages or not.
  • Ensure that you have network access to both the host machine and the specified port, and also the port is open through the server firewall.

REST LAM Response Codes

The REST LAM provides the following standard HTTP return codes indicating the state of the received message:

HTTP
return
code


Description
200

OK
The REST LAM has received and accepted the payload, which is valid JSON. There is no response message. 

This is not a positive indication that the Event has been properly processed by the LAM or LAMbot.
Standard LAM log file response messages are issued for error conditions.

400Bad Request
The JSON in the payload is invalid, so the request is rejected. The error could be in any part of the JSON, so the details should be reviewed with a standard JSON parser or lint. The response message is a JSON object containing the statusCode and message.
401Unauthorised
The REST LAM configuration file has an auth_token defined but there is no auth_token in the header of the message, or the auth_token provided does not match. The request is rejected and will need to be resent with the correct auth_token. The response message is a JSON object containing the  statusCode and message.

405

Method not allowed
The HTTP request used an unsupported method. The only supported method is POST. The response message is a JSON object containing the  statusCode  and message.
406Method not acceptable
The HTTP request used a content type that is unsupported. The only supported content type is application/JSON. The response message is a JSON object containing the  statusCode and message.

In the event of an error, the REST LAM provides the following REST response codes indicating the status of the request:

REST
response
code 

Description
1000General error
The request could not proceed and failed in a way that was not expected
3000General Security Error
There was an error with authentication
3001Security Authentication Error
Authentication did not succeed
4000General Method Error
There was an error with the method used
4001Method Not Supported
The HTTP method used is not supported
5000General Content Error
There was an error with the content
5001Content Not Supported
The content type used is not supported
5002Content Decoding Error
The content could not be decoded

REST LAM configuration examples

The following REST LAM configuration file examples highlight authentication and SSL, along with the example of curl commands to generate Events. 

1. An example rest_lam.conf with no authentication or SSL (auth_token  commented out, use_ssl set to falsand related SSL properties commented out):

monitor:
{
      name : "Rest Lam Monitor",
      class : "CRestMonitor",
      port : 9876,
      use_ssl : false,
      #path_to_ssl_files : "/tmp",
      #ssl_key_filename : "server.key",
      #ssl_cert_filename : "server.pem",
      #auth_token : "abcd1234",
      num_threads : 5
    },
    agent:
    {
      name : "DATA_SOURCE"
      #log : "/tmp/rest_lam.log"
    },
    constants:
    {
    },
    conversions:
    {
      stringToInt:
      {
        input: "STRING",
        output: "INTEGER"
      }
    },
    mapping :
    {
      catchAll: "overflow",
      rules:
      [
        { name: "signature",      rule: "$signature" },
        { name: "source_id",      rule: "$source_id" },
        { name: "external_id",    rule: "$external_id" },
        { name: "manager",        rule: "$manager" },
        { name: "source",         rule: "$source" },
        { name: "class",          rule: "$class" },
        { name: "agent",          rule: "$agent" },
        { name: "agent_location", rule: "$agent_location" },
        { name: "type",           rule: "$type" },
        { name: "severity",       rule: "$severity",       conversion: "stringToInt" },
        { name: "description",    rule: "$description" },
        { name: "first_occurred", rule: "$first_occurred", conversion: "stringToInt" },
        { name: "agent_time",     rule: "$agent_time",     conversion: "stringToInt" }
      ]
    },
    filter:
    {
      presend: "RestLam.js"
}

The following is the curl command for the above configuration:


curl http://moogbox2:9876 -H "Content-Type: application/json" --insecure -X POST -v --data '
{ 
   "events":[ 
      { 
         "signature":"SIGNATURE1",
         "source_id":"my_source_id",
         "external_id":"my_external_id",
         "manager":"TestMgr1",
         "source":"TestHost1",
         "class":"my_class",
         "agent":"TestAgent1",
         "agent_location":"my_agent_location",
         "type":"TestType1",
         "severity":3,
         "description":"TestDesc1",
         "first_occurred":1411134582,
         "agent_time":1411134582
      }
   ]
}' 

The above curl command will generate the following Event from the REST LAM:


{ 
   "_MOOTADATA_":{ 
      "creation_time":1452090420708
   },
   "agent":"TestAgent1",
   "agent_location":"my_agent_location",
   "agent_time":1411134582,
   "class":"my_class",
   "description":"TestDesc1",
   "external_id":"my_external_id",
   "manager":"TestMgr1",
   "overflow":{ 
      "LamInstanceName":"DATA_SOURCE",
      "first_occurred":1411134582
   },
   "severity":3,
   "signature":"SIGNATURE1",
   "source":"TestHost1",
   "source_id":"my_source_id",
   "type":"TestType1"
}

The following is the curl command for the above configuration:


curl http://moogbox2:9876 -H "Content-Type: application/json" --insecure -X POST -v --data '
{ 
   "events":[ 
      { 
         "signature":"SIGNATURE1",
         "source_id":"my_source_id",
         "external_id":"my_external_id",
         "manager":"TestMgr1",
         "source":"TestHost1",
         "class":"my_class",
         "agent":"TestAgent1",
         "agent_location":"my_agent_location",
         "type":"TestType1",
         "severity":3,
         "description":"TestDesc1",
         "first_occurred":1411134582,
         "agent_time":1411134582
      },
      { 
         "signature":"SIGNATURE2",
         "source_id":"my_source_id",
         "external_id":"my_external_id",
         "manager":"TestMgr2",
         "source":"TestHost2",
         "class":"my_class",
         "agent":"TestAgent2",
         "agent_location":"my_agent_location",
         "type":"TestType2",
         "severity":3,
         "description":"TestDesc2",
         "first_occurred":1411134582,
         "agent_time":1411134582
      },
      { 
         "signature":"SIGNATURE3",
         "source_id":"my_source_id",
         "external_id":"my_external_id",
         "manager":"TestMgr3",
         "source":"TestHost3",
         "class":"my_class",
         "agent":"TestAgent3",
         "agent_location":"my_agent_location",
         "type":"TestType3",
         "severity":3,
         "description":"TestDesc3",
         "first_occurred":1411134582,
         "agent_time":1411134582
      }
   ]
}'

The above command will generate the following three separate Events from the REST LAM:

Event 1


{ 
   "_MOOTADATA_":{ 
      "creation_time":1452090542681
   },
   "agent":"TestAgent1",
   "agent_location":"my_agent_location",
   "agent_time":1411134582,
   "class":"my_class",
   "description":"TestDesc1",
   "external_id":"my_external_id",
   "manager":"TestMgr1",
   "overflow":{ 
      "LamInstanceName":"DATA_SOURCE",
      "first_occurred":1411134582
   },
   "severity":3,
   "signature":"SIGNATURE1",
   "source":"TestHost1",
   "source_id":"my_source_id",
   "type":"TestType1"
}

Event 2


{ 
   "_MOOTADATA_":{ 
      "creation_time":1452090542681
   },
   "agent":"TestAgent2",
   "agent_location":"my_agent_location",
   "agent_time":1411134582,
   "class":"my_class",
   "description":"TestDesc2",
   "external_id":"my_external_id",
   "manager":"TestMgr2",
   "overflow":{ 
      "LamInstanceName":"DATA_SOURCE",
      "first_occurred":1411134582
   },
   "severity":3,
   "signature":"SIGNATURE2",
   "source":"TestHost2",
   "source_id":"my_source_id",
   "type":"TestType2"
}

Event 3


{ 
   "_MOOTADATA_":{ 
      "creation_time":1452090542681
   },
   "agent":"TestAgent3",
   "agent_location":"my_agent_location",
   "agent_time":1411134582,
   "class":"my_class",
   "description":"TestDesc3",
   "external_id":"my_external_id",
   "manager":"TestMgr3",
   "overflow":{ 
      "LamInstanceName":"DATA_SOURCE",
      "first_occurred":1411134582
   },
   "severity":3,
   "signature":"SIGNATURE3",
   "source":"TestHost3",
   "source_id":"my_source_id",
   "type":"TestType3"
}

The -- insecure  option used in the curl commands, uses TSL but will not validate the certificate.

2. An example rest_lam.conf with authentication, but no SSL (auth_token  uncommented, use_ssl set to falsand related SSL properties commented out):

monitor:
{
      name : "Rest Lam Monitor",
      class : "CRestMonitor",
      port : 9876,
      use_ssl : false,
      #path_to_ssl_files : "/tmp",
      #ssl_key_filename : "server.key",
      #ssl_cert_filename : "server.pem",
      auth_token : "abcd1234",
      num_threads : 5
    },
    agent:
    {
      name : "DATA_SOURCE"
      #log : "/tmp/rest_lam.log"
    },
    constants:
    {
    },
    conversions:
    {
      stringToInt:
      {
        input: "STRING",
        output: "INTEGER"
      }
    },
    mapping :
    {
      catchAll: "overflow",
      rules:
      [
        { name: "signature",      rule: "$signature" },
        { name: "source_id",      rule: "$source_id" },
        { name: "external_id",    rule: "$external_id" },
        { name: "manager",        rule: "$manager" },
        { name: "source",         rule: "$source" },
        { name: "class",          rule: "$class" },
        { name: "agent",          rule: "$agent" },
        { name: "agent_location", rule: "$agent_location" },
        { name: "type",           rule: "$type" },
        { name: "severity",       rule: "$severity",       conversion: "stringToInt" },
        { name: "description",    rule: "$description" },
        { name: "first_occurred", rule: "$first_occurred", conversion: "stringToInt" },
        { name: "agent_time",     rule: "$agent_time",     conversion: "stringToInt" } 
      ]
    },
    filter:
    {
      presend: "RestLam.js"
}

The following is the curl command for the above configuration:

curl http://moogbox2:9876 -H "Content-Type: application/json" --insecure -X POST -v --data '
{ 
   "auth_token":"abcd1234",
   "events":[ 
      { 
         "signature":"SIGNATURE1",
         "source_id":"my_source_id",
         "external_id":"my_external_id",
         "manager":"TestMgr1",
         "source":"TestHost1",
         "class":"my_class",
         "agent":"TestAgent1",
         "agent_location":"my_agent_location",
         "type":"TestType1",
         "severity":3,
         "description":"TestDesc1",
         "first_occurred":1411134582,
         "agent_time":1411134582
      }
   ]
}'

The above command will generate the following Event from the REST LAM:


{ 
   "_MOOTADATA_":{ 
      "creation_time":1452090420708
   },
   "agent":"TestAgent1",
   "agent_location":"my_agent_location",
   "agent_time":1411134582,
   "class":"my_class",
   "description":"TestDesc1",
   "external_id":"my_external_id",
   "manager":"TestMgr1",
   "overflow":{ 
      "LamInstanceName":"DATA_SOURCE",
      "first_occurred":1411134582
   },
   "severity":3,
   "signature":"SIGNATURE1",
   "source":"TestHost1",
   "source_id":"my_source_id",
   "type":"TestType1"
}

3. An example rest_lam.conf with authentication and SSL (auth_token  uncommented and use_ssl set to true and related SSL properties uncommented):

A key and certificate file must be generated on the host running the rest_lam, for example:

openssl req -new -x509 -days 365 -nodes -subj "/C=''/ST=''/L=''/O='moogsoft'/OU=''/CN=moogbox2" -out /tmp/server.pem -keyout /tmp/server.key
monitor:
{
      name : "Rest Lam Monitor",
      class : "CRestMonitor",
      port : 9876,
      use_ssl : true,
      path_to_ssl_files : "/tmp",
      ssl_key_filename : "server.key",
      ssl_cert_filename : "server.pem",
      auth_token : "abcd1234",
      num_threads : 5
    },
    agent:
    {
      name : "DATA_SOURCE"
      #log : "/tmp/rest_lam.log"
    },
    constants:
    {
    },
    conversions:
    {
      stringToInt:
      {
        input: "STRING",
        output: "INTEGER"
      }
    },
    mapping :
    {
      catchAll: "overflow",
      rules:
      [
        { name: "signature",      rule: "$signature" },
        { name: "source_id",      rule: "$source_id" },
        { name: "external_id",    rule: "$external_id" },
        { name: "manager",        rule: "$manager" },
        { name: "source",         rule: "$source" },
        { name: "class",          rule: "$class" },
        { name: "agent",          rule: "$agent" },
        { name: "agent_location", rule: "$agent_location" },
        { name: "type",           rule: "$type" },
        { name: "severity",       rule: "$severity",       conversion: "stringToInt" },
        { name: "description",    rule: "$description" },
        { name: "first_occurred", rule: "$first_occurred", conversion: "stringToInt" },
        { name: "agent_time",     rule: "$agent_time",     conversion: "stringToInt" } 
      ]
    },
    filter:
    {
      presend: "RestLam.js"
}

The following is the curl command for the above configuration:


curl https://moogbox2:9876 -H "Content-Type: application/json" -X POST -v --data '
{ 
   "auth_token":"abcd1234",
   "events":[ 
      { 
         "signature":"SIGNATURE1",
         "source_id":"my_source_id",
         "external_id":"my_external_id",
         "manager":"TestMgr1",
         "source":"TestHost1",
         "class":"my_class",
         "agent":"TestAgent1",
         "agent_location":"my_agent_location",
         "type":"TestType1",
         "severity":3,
         "description":"TestDesc1",
         "first_occurred":1411134582,
         "agent_time":1411134582
      }
   ]
}'

The above command will generate the following Event from the REST LAM:


{ 
   "_MOOTADATA_":{ 
      "creation_time":1452090420708
   },
   "agent"  		:"TestAgent1",
   "agent_location"	:"my_agent_location",
   "agent_time"		:1411134582,
   "class"			:"my_class",
   "description"	:"TestDesc1",
   "external_id"	:"my_external_id",
   "manager"		:"TestMgr1",
   "overflow"		:{ 
      					"LamInstanceName":"DATA_SOURCE",
      					"first_occurred":1411134582
   					 },
   "severity"		:3,
   "signature"		:"SIGNATURE1",
   "source"			:"TestHost1",
   "source_id"		:"my_source_id",
   "type"			:"TestType1"
}

The remote host sending the curl command must have a copy of the certificate file in the correct location, for example:


/root/server.pem


Mapping 

For events received in JSON format, you can directly map the event fields of REST with Moogsoft fields. The parameters of the received events are displayed in Moogsoft AIOps according to the mapping done here:

mapping :
        {
            
       catchAll: "overflow",
            rules:
            [
                { name: "signature", rule:      "$signature" },
                { name: "source_id", rule:      "$source_id" },
                { name: "external_id", rule:    "$external_id" },
                { name: "manager", rule:        "$manager" },
                { name: "source", rule:         "$source" },
                { name: "class", rule:          "$class" },
                { name: "agent", rule:          "$LamInstanceName" },
                { name: "agent_location", rule: "$agent_location" },
                { name: "type", rule:           "$type" },
                { name: "severity", rule:       "$severity", conversion: "stringToInt" },
                { name: "description", rule:    "$description" },
                { name: "agent_time", rule:     "$agent_time", conversion: "stringToInt" }
            ]
        },
        filter:
        {           
            stream: "myStream",
            presend: "RestLam.js"
        }

Constants and Conversions

Constants and Conversions allows you to convert format of the received data.

Field
Description
Example
Severity and sevConverter
has a conversion defined as sevConverter in the Conversions section, this looks up the value of severity defined in the severity section of constants and returns back the mapped integer corresponding to the severity.
severity:
{

"CLEAR" : 0,
"INDETERMINATE" : 1,
"WARNING" : 2,
"MINOR" : 3,
"MAJOR" : 4
"CRITICAL" : 5

moog_lookup_default : 1

}, 

sevConverter:
{
lookup: "severity",
input : "STRING",
output: "INTEGER"
},

stringToIntUsed in a conversion which forces the system to turn a string token into an integer value.
stringToInt:
{
    input  : "STRING",
    output : "INTEGER"
},

Example

Example Constants and Conversions
constants:
        {
            severity:
            {
                "CLEAR"         : 0,
                "INDETERMINATE" : 1,
                "WARNING"       : 2,
                "MINOR"         : 3,
                "MAJOR"         : 4,
                "CRITICAL"      : 5,              
                moog_lookup_default: 1
            }
        },
        conversions:
        {
            sevConverter:
            {
                lookup: "severity",
                input:  "STRING",
                output: "INTEGER"
            },
           
            stringToInt:
            {
                input:      "STRING",
                output:     "INTEGER"
            }
        },     
           
      

Severity Reference

Moogsoft Severity Levels
severity:
        {
            "CLEAR" 		: 0,
            "INDETERMINATE" : 1,
            "WARNING" 		: 2,
            "MINOR" 		: 3,
            "MAJOR" 		: 4,
            "CRITICAL" 		: 5,
             moog_lookup_default: 1            
        }
LevelDescription
0Clear
1Indeterminate
2Warning
3Minor
4Major
5Critical

Service Operation Reference

Process NameService Name
rest_lamrestlamd

Start the LAM Service:

service restlamd start

Stop the LAM Service:

service restlamd stop

Check the LAM Service status:

service restlamd status

Command Line Reference 

To see the available optional attributes of the rest_lam, run the following command:

rest_lam --help

The rest_lam is a command line executable, and has the following optional attributes:

OptionDescription

--config

Points to a pathname to find the configuration file for the LAM. This is where the entire configuration for the LAM is specified
--helpDisplays all the command line options
--version

Displays the component’s version number

--log level

Specifies the level of debugging. By default, user gets everything. In common with all executables in MOOG, having it set at that level can result in a lot of output (many messages per event message processed).

In all production implementations, it is recommended that log level is set to WARN. This ensures only warning, error and fatal messages are recorded.

Performance Information

Minimum requirement
ComponentValue
CPU2 core
RAM4 GB
Operating SystemCentOS Linux release 6.7

Version

LAM VersionTool VersionVerified By

1.0

Generic

Yes

1.1

Generic

Yes

1.2

Generic

Yes



  • No labels