Page tree
Skip to end of metadata
Go to start of metadata


Overview

vRealize Log lnsight delivers heterogeneous and highly scalable log management. It provides deep operational visibility and faster troubleshooting across physical, virtual and cloud environments. The vRealize Log Insight LAM connects with the vRealize Log Insight server and fetches events from it. The LAM after fetching the events, forwards it to Moogsoft AIOps.

Process Overview

  1. LAM reads the configuration from the vrealize_loginsight_lam.conf file.
  2. LAM will connect with the vRealize Log Insight Server using the given host name or IP Address.
  3. The response is received with event data in JSON format.
  4. The events are parsed and converted into normalized Moogsoft AIOps events.
  5. The normalized events are then published to MooMS bus.

vRealize Log Insight LAM Configuration

The alarms received from vRealize Log Insight are processed according to the configurations in the vrealize_loginsight_lam.conf file. The processed alarms are published to Moogsoft AIOps.

The configuration file contains a JSON object. At the first layer of the object, the LAM has a parameter called config, and the object that follows config has all the necessary information to control the LAM.

The following sections are available for configuration in the vRealize Log Insight LAM configuration file.

Monitor

The vRealize Log Insight LAM takes the incidents from vRealize Log Insight. The user can configure the parameters here to establish a connection with vRealize Log Insight.

   config :
    {
        monitor:
        {

            name                      : "LogInsight Lam Monitor",

            class                     : "CVrealizeLogInsighMonitor",

            host_name                 : "localhost",

            user_name                 : "username",
            
            password                  : "password",

            # encrypted_password      : "ieytOFRUdLpZx53nijEw0rOh07VEr8w9lBxdCc7229o=",

            use_ssl                   : false,
            
            path_to_ssl_files         : "config",

            server_cert_filename      : "server.crt",

            use_client_authentication : false,

            client_key_filename       : "client.key",

            client_cert_filename      : "client.crt",
         
            polling_interval 		  : 10,
             
            max_retries 			  : 10, 
            
            retry_interval 			  : 60,

			timeout					  : 120,

            filter 					 :	{           
           								hostnames :  [],
           								sources   :  []
           								}             
		}, 
  • name and class: These fields are reserved and should not be changed. The default values are LogInsight Lam Monitor and CVrealizeLogInsighMonitor respectively
  • host_name: Enter the hostname or the IP address of the vRealize LogInsight  server. E.g. 10.12.12.15
  • user_name and Password: Enter the username and password of the vRealize LogInsight console
  • encrypted_password: If the encrypted password is to be used then enter the encrypted password in this field and comment the password field. At a time either password or the encrypted_password field is used. If both the fields are not commented then the field encrypted_password will be used by the vRealize Log Insight LAM
  • use_ssl: Enter true here, to enable SSL Communication. By default, it is set to false

  • path_to_ssl_files: Enter the path of the directory where all the certificates are stored, e.g. "/usr/local/ssl"

  • server_cert_filename: Enter the server certificate name here. Use the certificate "server.crt" here. The cert file should be present in the directory given in path_to_ssl_files field

  • use_client_authentication: Enter true here if you want client authentication, otherwise set it to false. By default, it is set to false. If it is set to true, then the values are to be entered in the client_key_filename and the client_cert_filename fields

  • client_key_filename: Enter the name of the key file here, e.g. "client.key".  The key file should be present in the directory given in path_to_ssl_files field

  • client_cert_filename: Enter the name of the certificate file here, e.g. "client.crt". The cert file should be present in the directory given in path_to_ssl_files field

  • polling_interval: The polling time interval between the requests after which the event data is fetched from vRealize Log Insight. The polling interval is entered in seconds

    The default value is set to 10 seconds, if 0 is entered in this field then the time interval is by default set to 10 seconds

  • max_retries: The maximum number of retry attempts to reconnect with the vRealize Log Insight server in case of a connection failure

    The default value is set to 10, if 0 is entered in this field then the LAM by default takes the value 10 and will try at least 10 times to reconnect

    If all the number of retries are exhausted, then an alarm is sent to Moogsoft AIOps about the connection failure. For re-establishing the connection the LAM has to be restarted

  • retry_interval: The time interval between two successive retry attempts

    The default value is set to 60 seconds, if 0 is entered in this field then the time interval is by default set to 60 seconds

  • timeout: If for any reason the response is not received from the Server against a request, then the LAM discards the request after waiting for some time. The time that the LAM waits before discarding is given here in the timeout field. For example, If the timeout field has 120 entered in it, then the LAM will wait for 120 seconds for a response from the server, against a request. If no response is received for 120 seconds, then the LAM discards the request and sends a new request

    The entry in the fields polling_interval, max_retries, retry_interval, max_events and timeout should be an integer, therefore enter the values in these fields without quotation marks

  • filter: The following filters can be used to fetch the events form vRealize Log Insight based on the applied filter

    • hostnames: Enter the hostname of the machine, this filter criteria, when applied fetches events containing the listed hostnames e.g.:

      hostnames  :  ["localhost","dellserver","moogsoftserver"]
    • Sources: Enter the source of the machine, this filter criteria, when applied fetches events containing the listed sources e.g.:

      sources   :  ["10.24.56.78", "10.54.87.35"]

      If all the filters are used i.e. every filter having a value, then the events having all the values listed in all the filters will be fetched.

      The hostname and sources are joined using the "and" condition while the fields within the filters are joined using the "or" condition. This means that if we mentioned only the following filter, hostnames : ["localhost","dellserver","moogsoftserver"], then all the events having the hostname "localhost" or "dellserver" or "moogsoftserver will be fetched. The same is the case if the filter sources : ["10.24.56.78", "10.54.87.35"] is applied, then all the events having the source "10.24.56.78" or "10.54.87.35" are fetched.

      In the case where both the filters hostnames and sources are applied, the events are fetched which have both the hostname and the source as given in the filters. For example, if we have the filters applied hostnames : ["localhost","dellserver","moogsoftserver"] and sources : ["10.24.56.78", "10.54.87.35"], then the events which have both the hostname and source from any of the entered filtered values will be fetched. The event coming from the dellserver source 10.24.56.78 will be fetched, but from any other source say 10.24.58.96 will not be fetched.

      The following table provides the hostname and their respective sources information, and the whether the events will be fetched or not for the filter hostnames : ["localhost","dellserver","moogsoftserver"] and sources : ["10.24.56.78", "10.54.87.35"]:

      hostnamesourceEvents fetched
      localhost10.24.56.78Y
      10.24.59.96N
      dellserver10.54.87.35Y
      10.58.64.28N
      moogsoftserver10.57.64.87N
      10.24.56.78Y

The LAM starts fetching the events from the current time. After that it saves the last poll time (in epoch format) in the state file. The state file is generated in the same folder where the config file is present e.g. $MOOGSOFT_HOME/config, and has the same name as the config file.

It is recommended not to make any changes to the state file as this may lead to loss of alarms or events

Agent

Agent allows the user to define two parameters:

agent:
        {
		       name    : "vRealize Log Insight"
               #log    : "/var/log/moogsoft/vrealize_loginsight_lam.log"
        },


The above example specifies:  

name: This is the agent name, the events sent to MooMS by the vRealize Log Insight LAM are identified by the agent name in the log. In this example the agent name is vRealize Log Insight

log: In this instance, the vRealize Log Insight LAM will write its ingress contents in the file vRealize_Log_Insight_lam.log located at /var/log/moogsoft/

HA Configuration

Refer the document HA Configuration of LAM

Mapping 

For events received in JSON format, a user can directly map the alarm/event fields of vRealize Log Insight with moogsoft fields. In the case of an event received in text format, the event is first tokenised in the Variable section, and the tokenised event is then mapped here in the mapping section. The parameters of the received alarm/event are displayed in Moogsoft AIOps according to the mapping done here.

 mapping :
        {
            catchAll: "overflow",
			rules:
            [
                { name: "signature", rule:      "$hostname::$event_type" },   
                { name: "source_id", rule:      "$source" },
                { name: "external_id", rule:    "$appname" },
                { name: "manager", rule:        "vRealize Log_Insight Lam" },
                { name: "source", rule:         "$hostname" },
                { name: "class", rule:          "$event_type" },
                { name: "agent", rule:          "$LamInstanceName" },
                { name: "agent_location", rule: "$LamInstanceName" }, 
                { name: "type", rule:           "$event_type" },
                { name: "severity", rule:       "0",conversion: "stringToInt" },
                { name: "description", rule:    "$text" },
                { name: "agent_time", rule:     "$time_changed"} 
            ] 
        },
        filter:
       {
        	modules: [	
					"SeverityUtil.js",
					"LamUtility.js"
					 ],
            presend:"vrealizeloginsightLam.js"
        }


The above example specifies the mapping of the vRealize Log Insight alarm fields with the Moogsoft AIOps fields.

The signature field is used by the LAM to identify correlated alarms


An example of vRealize Log Insight events:


Constants and Conversions

Constants and Conversions allow the user to convert formats of the received data defined users.

constants:
        {
            severity:
            {
            	"clear"			: 0,
                "info"          : 1,
                "warning"       : 2,
                "minor"         : 3,
                "major"         : 4,
                "critical"      : 5
            }
           
        },
        conversions:
        {
            sevConverter:
            {
                lookup: "severity",
                input:  "STRING",
                output: "INTEGER"
            },

            stringToInt:
            {
                input:      "STRING",
                output:     "INTEGER"
            },
         
            timeConverter:
            {
                timeFormat: "yyyy-MM-dd'T'HH:mm:ss.SSS",
                input:      "STRING",
                output:     "INTEGER"
            }
        },


The above example specifies:

  • Severity and sevConverter: The severity field has a conversion defined as sevConverter in the Conversions section, this looks up the value of severity defined in the severity section of constants and returns back the mapped integer corresponding to the severity

  • stringToInt: It is used in a conversion, which forces the system to turn a string token into an integer value
  • timeConverter: It is used in conversion which forces the system to convert time. If epoc time is to be used, then timeFormat mentioned in timeConverter should be commented. Otherwise, the user should provide the timeFormat

Custom Info

The alarms/events are displayed in the Moogsoft AIOps, the data in the fields of the alarm or event mapped in the mapping section are shown in the respective columns of Moogsoft AIOps columns. The fields of alarms and events which are not mapped in the mapping section are displayed in the Custom Info field of the alarm. An example of Custom Info:

catchALL 

The attribute that is never referenced in a rule is collected and placed as a JSON object in a variable called overflow defined here and passed as part of the event.

mapping :
        {
            catchAll: "overflow",
			rules:
            [
                { name: "signature", rule:      "$hostname::$event_type" },   
                { name: "source_id", rule:      "$source" },
                { name: "external_id", rule:    "$appname" },
                { name: "manager", rule:        "vRealize Log Insight" },
                { name: "source", rule:         "$hostname" },
                { name: "class", rule:          "$event_type" },
                { name: "agent", rule:          "$LamInstanceName" },
                { name: "agent_location", rule: "$LamInstanceName" }, 
                { name: "type", rule:           "$event_type" },
                { name: "severity", rule:       "0",conversion: "stringToInt" },
                { name: "description", rule:    "$description" },
                { name: "agent_time", rule:     "$time_changed"} 
            ]
        },

The vRealize Log Insight event field vc_event_type is sent to vRealize Log Insight LAM. Since it is not mapped to a field in the vrealize_loginsight_lam.conf file it is placed in the overflow JSON object. The fields that are placed in the overflow variable can be viewed in the vRealize Log Insight LAM log file or the custom info field of the event in Moogsoft AIOps GUI .

An example of an overflow JSON object created in the vRealize Log Insight LAM log file:

{"overflow":{"vc_event_type":"com.vmware.vim25.UserLoginSessionEvent","vc_username":"VSPHERE.LOCAL\\Administrator","text":"2017-06-28
09:09:58.243 10.142.24.23 vcenter-server: User VSPHERE.LOCAL\\Administrator@10.142.24.61 logged in as JAX-WS RI 2.2.9-b130926.1035
svn-revision#5f6196f2b90e9460065a4c2f4e30e065b245e51e"}}

Starting the vRealize Log Insight LAM

To start the vRealize Log Insight LAM enter the following command:

service vrealizeloginsightlamd start

To stop the vRealize Log Insight LAM enter the following command:

service vrealizeloginsightlamd stop


To view the status of vRealize Log Insight LAM, enter the following command:

service vrealizeloginsightlamd status

Version Information

LAM Version

Tool Version

Tested?

Expected to Work

1.0

4.3.0

Yes

Yes

1.1

4.3.0

Yes

Yes

System Information

This LAM was tested on a system with the following configurations:

CPU2 core
RAM4 GB
Operating SystemCentOS Linux release 6.7

The system must at least have the above mentioned system requirements to run the LAM.


  • No labels