Moogsoft Docs

Configure the vRealize Log Insight LAM

vRealize Log lnsight delivers heterogeneous and highly scalable log management. It provides deep operational visibility and faster troubleshooting across physical, virtual and cloud environments. The vRealize Log Insight LAM connects with the vRealize Log Insight server and fetches events from it. The LAM after fetching the events, forwards it to Moogsoft AIOps.

See VMware vRealize Log Insight for UI configuration instructions.

29961805.png
  1. LAM reads the configuration from the vrealize_loginsight_lam.conf file.

  2. LAM will connect with the vRealize Log Insight Server using the given host name or IP Address.

  3. The response is received with event data in JSON format.

  4. The events are parsed and converted into normalized Moogsoft AIOps events.

  5. The normalized events are then published to MooMS bus.

vRealize Log Insight LAM Configuration

The events received from vRealize Log Insight are processed according to the configurations in thevrealize_loginsight_lam.conffile. The processed alarms are published to Moogsoft AIOps.

The configuration file contains a JSON object. At the first layer of the object, LAM has a parameter called config, and the object that follows config has all the necessary information to control the LAM.

Monitor

The vRealize Log Insight LAM takes the connection information from the Monitor section of the config file. You can configure the parameters here to establish a connection with vRealize Log Insight Client.

General

Field

Type

Description

name and class

String

Reserved fields: do not change. Default values are LogInsight Lam Monitorand CVrealizeLogInsightMonitor.

target

JSON Object

A top-level container for which you can define one or more target vRealize sources. You can specify the configuration for each target. If you don't specify a request_interval the target uses the globally defined interval.

host_name

Integer

The host name or IP address of the vRealize Log Insight server. Default address is localhost.

user_name and password

String

Enter the username and password for vRealize Log Insight console login.

encrypted_password

String

If the password is encrypted, then enter the encrypted password in this field and comment out the password field. At a time, either password or the encrypted_password field is used. If both the fields are not commented, then the field encrypted_password will be used by the vRealize Log Insight LAM.

server_cert_filename

String

Enter the server certificate name here. Use the certificate "server.crt" here. The cert file should be present in the directory given in path_to_ssl_filesfield.

use_client_authentication

Boolean

If you want client authentication, set it to true, else you can set it to false. By default, it is set to false. If it is set to true, then the values will be entered in the client_key_filename and the client_cert_filename fields.

client_key_filename

String

Enter the name of the key file here. The key file should be present in the directory given in path_to_ssl_filesfield. For example: "client.key"

client_cert_filename

String

Enter the name of the certificate file here. The cert file should be present in the directory given in path_to_ssl_filesfield. For example: "client.crt"

polling_interval

Integer

The polling time interval, in seconds, between the requests after which the event data is fetched from vRealize Log Insight LAM.

Default = 10 seconds. If 0 is entered, the time interval will set to 10 seconds.

max_retries

Integer

The maximum number of retry attempts to reconnect with vRealize Log Insight Server in case of a connection failure.

Default = -1, if no value is specified, then there will be infinite retry attempts.

If the specified value is greater than 0, then the LAM will try that many times to reconnect; in case of 0 or any other value less than 0, max retries will set to default.

retry_interval

Integer

The time interval between two successive retry attempts.

Default = 60 seconds, if 0 is entered, the time interval will set to default.

request_interval

Integer

Length of time to wait between requests, in seconds. Can be overridden by request_interval in individual targets. Defaults to 60.

retry_recovery

Object

Specifies the behavior of the LAM when it re-establishes a connection after a failure.

- recovery_interval: Length of time to wait between recovery requests in seconds. Must be less than the request_interval set for each target. Defaults to 20.

- max_lookback: The period of time for which to recover missed events in seconds. Defaults to -1 (recover all events since the last successful poll).

timeout

Integer

This is the timeout value in seconds, which will be used to timeout a connection, socket and request. If no value is specified, then the time interval will set to to 120 seconds.

Default: 120 seconds, if no value is specified, then timeout will set to default.

Filter

Field

Type

Description

filter

Object

The following filters can be used to fetch events form the vRealize Log Insight LAM:

  • hostnames: Enter the hostname of the machine, this filter criteria will fetch events containing the listed hostnames e.g.:

    hostnames  :  ["localhost","dellserver","moogsoftserver"]
  • Sources: Enter the source of the machine, this filter criteria will fetch events containing the listed sources e.g.:

    Note

    sources   :  ["10.24.56.78", "10.54.87.35"]

    Note

    If you are using all the filter, then events having all the values listed in all the filters will be fetched.

    Note

    The hostname and sources are joined using the "AND" condition while the fields within the filters are joined using the "OR" condition. If you have mentioned the following filter, hostnames : ["localhost","dellserver","moogsoftserver"], then all the events having the hostname "localhost" or "dellserver" or "moogsoftserver will be fetched. Same is the case with filter sources, if you have applied the filter sources : ["10.24.56.78", "10.54.87.35"], then all the events having the source "10.24.56.78" or "10.54.87.35" will be fetched.

    In case where you have applied both the filters i.e. hostnames and sources, then those events which have both the hostname and the source as given in the filters will be fetched. For example, if you have applied the filters hostnames : ["localhost","dellserver","moogsoftserver"] AND sources : ["10.24.56.78", "10.54.87.35"], then the events which have both the hostname and source from any of the entered filtered values will be fetched. The event coming from the dellserver source 10.24.56.78 will be fetched, but from any other source say 10.24.58.96 will not be fetched.

    The following table provides the hostname and their respective sources information, and the whether the events will be fetched or not for the filter hostnames : ["localhost","dellserver","moogsoftserver"] andsources : ["10.24.56.78", "10.54.87.35"] :

    hostname

    source

    Events fetched

    localhost

    10.24.56.78

    Y

    10.24.59.96

    N

    dellserver

    10.54.87.35

    Y

    10.58.64.28

    N

    moogsoftserver

    10.57.64.87

    N

    10.24.56.78

    Y

Secure Sockets Layer

Field

Type

Description

use_ssl

Boolean

Set to true, to enable SSL Communication:

  • path_to_ssl_files: Enter the path of the directory where all the certificates are stored. If the path begins with ‘.’ or ‘/’ then, the path will be used as specified. Otherwise, MOOGSOFT_HOME is prepended to the path. For example, if MOOGSOFT_HOME is /opt/moogsoft/ and path_to_ssl is set to config, then the location will be defined as /opt/moogsoft/config.

  • ssl_protocols: Only applicable if use_ssl = true. This configuration dictates which SSL protocols are enforced by the vRealize Log Insight LAM; the following protocols are allowed to be specified:

    SSLv3

    TLSv1

    TLSv1.1

    TLSv1.2

    If SSL is in use and no value is specified for this configuration then only TLSv1.2 is allowed by default.

Example

You can configure the vRealize LAM to retrieve events from one or more sources. The following example demonstrates a configuration that targets two vRealize sources. For a single source comment out the target2 section. If you have more than two sources, add a target section for each one and uncomment properties to enable them.

monitor:
    {
        name                                    : "LogInsight Lam Monitor",
        class                                   : "CVrealizeLogInsightMonitor",
        request_interval                        : 60,
        max_retries                             : -1,
        retry_interval                          : 60,
        targets:
        {
            target1:
            {
                                url                                                             : "https://examplevrealize1",
                                user_name                       : "vrealize_user1",
                #password                       : "password",
                encrypted_password              : "qJAFVXpNDTk6ANq65pEfVGNCu2vFdcoj70AF5BIebEc=",
                                disable_certificate_validation  : false,
                path_to_ssl_files               : "config",
                server_cert_filename            : "server1.crt",
                client_key_filename             : "client1.key",
                client_cert_filename            : "client1.crt",
                                request_interval                : 60,
                                timeout                                                 : 120,
                max_retries                     : -1,
                retry_interval                  : 60,
                                filter                                                  :
                                                                                                {
                                                                                                        hostnames: [],
                                                                                                        sources: [],
                                                                                                }
                        target2:
                        {
                                url                                                             : "https://examplevrealize2",
                                user_name                       : "vrealize_user2",
                #password                       : "password",
                encrypted_password              : "bDGFSClSHBn8DSw43nGwSPLSv2dGwdsj50WD4BHdfVa&",
                                disable_certificate_validation  : false,
                path_to_ssl_files               : "config",
                server_cert_filename            : "server2.crt",
                client_key_filename             : "client2.key",
                client_cert_filename            : "client2.crt",
                                request_interval                : 60,
                                timeout                                                 : 120,
                max_retries                     : -1,
                retry_interval                  : 60,
                                filter                                                  :
                                                                                                {
                                                                                                        hostnames: [],
                                                                                                        sources: [],
                                                                                                }
                        }
        }
        }
Agent and Process Log

Agent and Process Log allow you to define the following properties:

  • name: Identifies events the LAM sends to the Message Bus.

  • capture_log: Name and location of the LAM's capture log file.

  • configuration_file: Name and location of the LAM's process log configuration file.

Mapping

For events received in JSON format, you can directly map the event fields of vRealize Log Insight LAM with Moogsoft fields. The parameters of the received events are displayed in Moogsoft AIOps according to the mapping done here:

 mapping :
        {
            catchAll: "overflow",
            rules:
            [
                { name: "signature", rule:      "$hostname::$event_type" },   
                { name: "source_id", rule:      "$source" },
                { name: "external_id", rule:    "$appname" },
                { name: "manager", rule:        "vRealize Log Insight" },
                { name: "source", rule:         "$hostname" },
                { name: "class", rule:          "$event_type" },
                { name: "agent", rule:          "$LamInstanceName" },
                { name: "agent_location", rule: "$LamInstanceName" }, 
                { name: "type", rule:           "$event_type" },
                { name: "severity", rule:       "0",conversion: "stringToInt" },
                { name: "description", rule:    "$description" },
                { name: "agent_time", rule:     "$time_changed"} 
            ]
        },
        filter:
        {
             modules: [     
                         "SeverityUtil.js",
                         "LamUtility.js"
               ],
            presend:"VrealizeLogInsightLam.js"
        }

The above example specifies the mapping of the vRealize Log Insight event fields with the Moogsoft AIOps fields. Data not mapped to Moogsoft AIOps Fields goes into "Custom Info".

Note

The signature field is used by the LAM to identify correlated events.

Constants and Conversions

Constants and Conversions allows you to convert format of the received data.

Field

Description

Example

Severity and sevConverter

has a conversion defined as sevConverter in the Conversions section, this looks up the value of severity defined in the severity section of constants and returns back the mapped integer corresponding to the severity.

severity:
{

"Clear" : 0,

"Info" : 1,

"Warning" : 2,

"Minor" : 3,

"Major" : 4

"Critical" : 5

}, 

sevConverter:

{

lookup: "severity",

input : "STRING",

output: "INTEGER"

},

stringToInt

used in a conversion, which forces the system to turn a string token into an integer value

stringToInt:
{
    input  : "STRING",
    output : "INTEGER"
},

timeConverter

Used in conversion which forces the system to convert time. If epoc time is to be used, then timeFormat mentioned in timeConverter should be commented. Otherwise, the user should provide the timeFormat.

timeConverter:
{
    timeFormat : "%Y-%m-%dT%H:%M:%S",
    input      : "STRING",
    output     : "INTEGER"
}
Example

Example Constants and Conversions

constants:
        {
            severity:
            {
                "clear"                       : 0,
                "info"          : 1,
                "warning"       : 2,
                "minor"         : 3,
                "major"         : 4,
                "critical"      : 5
            }
           
        },
        conversions:
        {
            sevConverter:
            {
                lookup: "severity",
                input:  "STRING",
                output: "INTEGER"
            },

            stringToInt:
            {
                input:      "STRING",
                output:     "INTEGER"
            },
         
            timeConverter:
            {
                timeFormat: "yyyy-MM-dd'T'HH:mm:ss.SSS",
                input:      "STRING",
                output:     "INTEGER"
            }
        },
Severity Reference

Moogsoft AIOps Severity Levels

severity:
        {
           "clear"                    : 0,
           "info"           : 1,
           "warning"        : 2,
           "minor"          : 3,
           "major"          : 4,
           "critical"       : 5
            
        }

Level

Description

0

Clear

1

Info

2

Warning

3

Minor

4

Major

5

Critical

Service Operation Reference

Process Name

Service Name

vrealizeloginsight_lam
vrealizeloginsightlamd

Start the LAM Service:

service vrealizeloginsightlamd start

Stop the LAM Service:

service vrealizeloginsightlamd stop

Check the LAM Service status:

service vrealizeloginsightlamd status

If the LAM fails to connect to one or more vRealize Log Insight sources, Moogsoft AIOps creates an alert and writes the details to the process log. Refer to the logging details for LAMs and integrations for more information.

Command Line Reference

To see the available optional attributes of the vrealizeloginsight_lam, run the following command:

vrealizeloginsight_lam --help

The vrealizeloginsight_lam is a command line executable, and has the following optional attributes:

Option

Description

--config

Points to a pathname to find the configuration file for the LAM. This is where the entire configuration for the LAM is specified.

--help

Displays all the command line options.

--version

Displays the component’s version number.

--loglevel

Specifies the level of debugging. By default, user gets everything. In common with all executables in Moogsoft AIOps, having it set at that level can result in a lot of output (many messages per event message processed).In all production implementations, it is recommended that log level is set to WARN. This ensures only warning, error and fatal messages are recorded.