Page tree
Skip to end of metadata
Go to start of metadata

Overview

The JMS LAM is a link access module that communicates with the following Application Servers and message brokers, and takes its input from Java Messaging Services.

  • JBoss: JBoss now known as WildFly is an Application Server which takes messages from Java Messaging Services in a queue or topic and forward them to application that are connected or subscribed to it
  • ActiveMQ: Apache ActiveMQ is an open source message broker written in Java together with a full Java Message Service (JMS) client
  • WebLogic: WebLogic is a server software application that runs on a middle tier, between back-end databases and related applications and browser-based thin clients. WebLogic server include connectors that make it possible for any legacy application on any client to inter-operate with server applications, Enterprise Java Bean (EJB) components, resource pooling, and connection sharing that make applications very scalable

This documentation explains the basic configuration for enabling the above mentioned 3 Java based Application Servers and the configuration of the JMS LAM config file (jms_lam.conf).

Process Workflow

The workflow of gathering alarms from the Application Server and publishing it to Moogsoft AIOps is:

  1. JMS LAM monitors message data being written to a Queue/Topic in the JMS provider.
  2. JMS LAM parses this message data according to the configuration file.
  3. Events are constructed from the monitored message data and then are passed to the MOOMs bus.
  4. Events are then published to the subject Events.

JMS LAM Configuration

The alarms received from the any of the 3 servers are processed according to the configuration in the jms_lam.conf file. The processed alarms are published to Moogsoft AIOps.

The configuration file contains a JSON object. At the first layer of the object, LAM has a parameter called config, and the object that follows config has all the necessary information to control the LAM.

The following sections are available for configuration in the JMS LAM configuration file.

Monitor

monitor defines the object to be monitored:

monitor:
        {
           
            name                    	      : "JMS Lam Monitor",
          
            class                   	      : "CJMSMonitor",
       
            initial_context_factory           : "x.x.x.x",
           
            provider_url                	  : "tcp://localhost:61616",
            
            provider_user_name                : "user_name",
            
            provider_password                 : "password",
            
            encrypted_provider_password 	  : "ieytOFRUdLpZx53nijEw0rOh07VEr8w9lBxdCc7229o=",
            
            connection_factory_name           : "ConnectionFactory",
            
            entity_name                       : "queuename",
            
            user_name                         : "username",
            
            password                          : "password",
            
            encrypted_password  			  : "ieytOFRUdLpZx53nijEw0rOh07VEr8w9lBxdCc7229o=",

            max_retries 					  : 10,

 			retry_interval 					  : 60,

		    message_type             		  : "TextMessage",
                        
            ssl_conn_activemq                 : false,

            ssl_connection                    : false,
            
            ssl_keystore_filename             : "",

            ssl_truststore_filename           : "",

            ssl_keystore_password             : "",    

            ssl_truststore_password           : "",
            
            response_require                  : true,
            
            feedback_queue                    : "feedback_queue_name"
           
        },

The above example specifies:

  • name and class: These fields are reserved and should not be changed the default values are JMS Lam Monitor and CJMSMonitor respectively
  • initial_context_factory: The LAM identifies the JMS server provider by this field. The value entered in this field is the JNDI name of the context factory of the provider. The values entered for the 3 server providers are as follows:

    JMS Server Providerintial_context_factory
    ActiveMQorg.apache.activemq.jndi.ActiveMQInitialContextFactory
    JBossorg.jboss.naming.remote.client.InitialContextFactory
    WebLogicweblogic.jndi.WLInitialContextFactory
  • provider_url: This field contains the URL of the provider to establish connection with the JMS Server provider

    JMS Server Providerprovider_url
    ActiveMQtcp:// IP address of ActiveMQ server:61616
    JBosshttp-remoting://IP address of JBoss server :8080
    WebLogic

    t3:// IP address of the WebLogic server:7001

    For SSL the following URLs are used

    JMS Server Providerprovider_url
    ActiveMQssl:// IP address of ActiveMQ server:61616
    JBosshttps-remoting://IP address of JBoss server :8443
    WebLogic

    t3s:// IP address of the WebLogic server:7002

    The above given ports are the default ports that the providers use. The port number in the LAM config file should be given as per the configurations in the provider server. E.g. if JBOSS is running on port 8081, then in the config file also it should be 8081

  • provider_user_name and provider_password: The provider user name and password which is required for the connection to be established between the JMS server provider and the JMS LAM. If there is no password configured then leave it blank. For JBoss it is the user name and password of the user which is both a management and an application user, created in JBoss. For Active MQ the user name is admin and password is also admin. For WebLogic it is the user name and password of the Administration Console, created during its installation

  • encrypted_provider_password: If the provider password is encrypted then enter the encrypted password in this field and comment the provider_password field. At a time either provider_password or the encrypted_provider_password field is used. If both the fields are not commented then the field encrypted_provider_password will be used by the JMS LAM
  • connection_factory_name: The connection factory name of the JMS server provider is entered here. The connection factory names of the 3 JMS server providers are as follows:

    JMS Server Providerconnection_factory_name
    ActiveMQConnectionFactory
    JBoss jms/RemoteConnectionFactory
    WebLogic

    It is the name of the connection factory that is created in the WebLogic administration console

  • entity_name: The name of the queue or topic is entered in this field. The format in which the entity name is to be entered is as follows:

    JMS Server Providerentity_name
    ActiveMQdynamicQueues/name of the queue or topic
    JBossjms/queue/name of the queue or topic
    WebLogic

    JNDI name of the queue or topic.e.g. jms/queue/queue1

  • user_name and password: The queue or topic username and password is entered in these fields. If there is no user name and password configured for the queue or topic then leave it blank. For JBoss it is the user name and password of the user which is both a management and an application user, created in JBoss. For Active MQ the user name is admin and password is also admin. For WebLogic it is the user name and password of the Administration Console, created during its installation
  • encrypted_password: If queue or topic password is encrypted then enter the encrypted password in this field and comment the password field. At a time either password or the encrypted_password field is used. If both the fields are not commented then the field encrypted_password will be used by the JMS LAM
  • max_retries: The maximum number of retry attempts to reconnect with the JMS provider in case of a connection failure

    The default value is set to 10, if 0 is entered in this field then the LAM by default takes the value 10 and will try at least 10 times to reconnect

    If all the number of retries are exhausted, then an alarm is sent to Moogsoft AIOps about the connection failure. For re-establishing the connection the LAM has to be restarted

  • retry_interval: The time interval between two successive retry attempts

    The default value is set to 60 seconds, if 0 is entered in this field then the time interval is by default set to 60 seconds

    The entry in the fields max_retries and retry_interval should be an integer, therefore enter the values in these fields without quotation marks

  • message_type: The message type of the messages received from the application can be set here. The following 3 message types are supported:
        • TextMessage
        • MapMessage
        • ObjectMessage
  • ssl_conn_activemq: To enable an SSL connection for ActiveMQ server provider enter true in this field
  • ssl_connection: To enable an SSL connection for JBoss or WebLogic enter true in this field

  • ssl_keystore_filename: If an SSL connection is enabled then enter the ssl keystore filename along with its path in Moogsoft AIOps, E.g. if the ssl keystore is in the directory usr/share/moogsoft /ssl, then enter usr/share/moogsoft/ssl/client.ks

  • ssl_truststore_filename: If an SSL connection is enabled then enter the ssl truststore filename along with its path in Moogsoft AIOps, E.g. if the ssl keystore is in the directory usr/share/moogsoft/ssl, then enter usr/share/moogsoft/ssl/client.ts

  • ssl_keystore_password: If an SSL connection is enabled then enter the ssl keystore password in this field

  • ssl_truststore_password: If an SSL connection is enabled then enter the ssl truststore password in this field

  • response_require: If response is to be sent back to queue or topic for received messages, enter true in this field

  • feedback_queue: Enter the queue name in which the response feedback is sent at the JMS server provider

Agent configuration

Agent allows the user to define 2 parameters:

        agent:
        {
                name    : "JMS",
                #log     : "/var/log/moogsoft/jms_lam.log"
        },

The above example specifies:

  • name: This is the agent name, the events sent to MooMs by the JMS LAM are identified by the agent name in the log. In this example the agent name is JMS
  • log: In this instance the JMS LAM will write its ingress contents to jms_lam.log file located at /var/log/moogsoft

HA configuration

Refer the document HA Configuration of LAM

Data parsing

Any received data needs to be broken up into tokens. When you have the tokens, you can start assembling an event. There are a number of parameters that allow you to control how this will work. The first two are start and end character. You can have multiple start and end characters. The system generates an event after all the tokens between a start and an end character is assembled.

Parsing:
{
            type: "",
            start_and_end:
            {
                start:      [],
                end:        ["\n"],
                delimiters:
                {
                    ignoreQuotes: true,
                    stripQuotes: true,
                    ignores:    "",
                    delimiter:  [",","\r"]
                }
            }
        },

        # Parsing block with regular expressions, using delimter
        # based tokenising:
        #
        # parsing:
        # {
        #     type: "regexp",
        #     regexp:
        #     {
        #         pattern : "(?mU)^(.*)$",
        #         capture_group: 1,
        #         tokeniser_type: "delimiters",
        #         delimiters:
        #         {
        #                 ignoreQuotes: true,
        #                 stripQuotes: false,
        #                 ignores:    "",
        #                 delimiter:  ["\r"]
        #         }
        #     }
        # },
        #


The above example specifies the following 3 types of parsing:

  • JSON parsing: To enable this parsing the type is set to blank
  • Text Messgae: To enable this parsing the type is set to Start_and_End
  • Regular Expression: To enable this the type is set to regexp

In the above example only one parsing method is used at a time. Either regexp or Text Message/JSON.

JSON Parsing

Any received data needs to be broken up into tokens. Once the tokens are received, the assembling of an event starts. There are a number of parameters that allow the user to control how this will work. The first 2 are a start and end character. The square brackets [] are the JSON notation for a list. You can have multiple start and end characters. The system considers an event as all of the tokens between any start and end character.


start: [], 
end: ["\n"],


The above example specifies:

  • There is nothing defined in start; however, a carriage return (new line) is defined as the end character

In the example above, the LAM is expecting a whole line to be written followed by a return, and it will process the whole line as one event.

If set up carefully, user can accept multi-line events.

Text Message Parsing

The Type should be set start_and_end and as shown in the below example.

type: start_and_end:
{
    start:      [JMS_MSG],
    end:        ["\n"],
...

The parsing in above example the parsing will start when it gets JMS_MSG and end when it gets new line.

Regular Expression Parsing

In regular expression the parser searches for strings as per the expression defined in pattern. The extracted string is then delimited as per the defined delimiters. In the above example the parser searches for the expression "(?mU)^(.*)$".

Delimiters

Delimiters define how a line is split into tokens. For example, if you have a line of text data, it needs to be split up into a sequence of sub strings that are referenced by position from the start. So if you were processing a comma-separated file, where a comma separates each value, it would make sense to have the delimiter defined as a comma. Then the system would take all the text between start and end and break it up into tokens between the commas. The tokens could then be referenced by position number in the string starting from one, not zero.

For example if the input string was the,cat,sat,on,the,mat and comma was used as a separator, token 1 would be the, token 2 cat and so on.

Be aware, there are complications when you come to tokenisation and parsing. For example, if you say comma is the delimiter, and the token contains a comma, you will end up with that token containing a comma to be split into 2 tokens. To avoid this it is recommended that you quote strings. You must then allow the system to know whether it should strip or ignore quotes, hence the stripQuotes and ignoreQuotes parameters.

 ignoreQuotes: true, 
 stripQuotes: false, 
 ignores: "", 
 delimiter: [",","\r"]

The above example specifies:

  • If you have strings that are quoted between delimiters, ignoreQuotes set to true will look for delimiters inside the quote. For example, <delimiter>hello inside quote goodbye<delimiter> gives a token [hello inside quote goodbye]
  • Setting stripQuotes to true removes start and end quotes from tokens. For example, hello world gives a token [hello world]
  • ignores is a list of characters to ignore. Ignored characters are never included in tokens
  • Delimiter is the list of valid delimiters used to split strings into tokens

Variables

For each event in the file, there is a positioned collection of tokens. Moog enables a user to name these positions. Naming of the positions helps the user to identify the tokens.In the below given example token at position number 6 is a Manager name, so the user names the token as "Manager".
This section is used for text message.

variables:
        [
            #
            # Note that positions start at 1, and go up
            # rather than array index style counting from zero
            #
            { name: "signature",   position: 1 },
            { name: "source_id",   position: 4 },
            { name: "external_id", position: 3 },
            { name: "Manager",     position: 6 },
            { name: "AlertGroup",  position: 7 },
            { name: "Class",       position: 8 },
            { name: "Agent",       position: 9 },
            { name: "severity",    position: 5 },
            { name: "description", position: 10 },
            { name: "agent_time",  position: 1 }
        ],

The above example specifies:

position 1 is assigned to signature; position 4 is assigned to source_id and so on. Positions start at 1, and go up.

Note: The variable section is used when the message type is TextMessage

The variable section is only used for text message. For JSON, MapMessage and ObjectMessage the mapping section is used. In mapping there is a value called rules, which is a list of assignments.

mapping :
        {
            #
            # All unused variables live as a JSON object
            # referenced by this variable (if defined)
            #
             builtInMapper:  "CJsonDecoder",
            # Input is restricted to Json so the builtInMapper option is not
            # used for this LAM
            #
            catchAll: "overflow",
            rules:
            [
                { name: "signature", rule:      "$signature" },
                { name: "source_id", rule:      "$source_id" },
                { name: "external_id", rule:    "$external_id" },
                { name: "manager", rule:        "JMS },
                { name: "source", rule:         "$source" },
                { name: "class", rule:          "$class" },
                { name: "agent", rule:          "$LamInstanceName" },
                { name: "agent_location", rule: "$agent_location" },
                { name: "type", rule:           "$type" },
                { name: "severity", rule:       "$severity", conversion: "sevConverter" },
                { name: "description", rule:    "$description" },
                { name: "agent_time", rule:     "$agent_time", conversion: "stringToInt" }
            ]
        },
        filter:
        {
            presend: "JmsLam.js"
        }
    }
}

In the example above, the first assignment name: "signature, rule:"$signature ($signature is a string with $ syntax) means for signature field take the tokens called signature.

User defines a number of these rules covering the base attributes of an event. For reference, the system expects a minimum set of attributes in an event that are shown in the above example.

For JSON, MapMessage and ObjectMessage enable the builtInMapper: "CJsonDecoder" by uncommenting it. The variable section is ignored if builtInMapper is uncommented
For TextMessage builtInMapper option should be commented
In JMS the event fields depends on the events that are fed in the queue or topic. The above mapping is just an example and has to be changed according to the alarms/events received from the queue/topic


Constants and conversions

There are rules in mapping section for which conversions are to be defined. The conversions convert the received input from one format to another. E.g. in the above example of mapping, for the mapped field severity, an integer is received which is converted to text and displayed on the Moogsoft AIOps UI. The lookup for conversions is kept in the constants section. The available conversions are kept in the conversions section and called during mapping. The example of calling a conversion is as follows:

{ name: "severity", rule:       "$severity", conversion: "sevConverter" }

The example of constants and conversions sections are as follows:

constants:
        {
            severity:
            {
                "CLEAR"         : 0,
                "INDETERMINATE" : 1,
                "WARNING"       : 2,
                "MINOR"         : 3,
                "MAJOR"         : 4,
                "CRITICAL"      : 5
            }
        },
        conversions:
        {
            sevConverter:
            {
                lookup: "severity",
                input:  "STRING",
                output: "INTEGER"
            },
            
            stringToInt:
            {
                input:      "STRING",
                output:     "INTEGER"
            },
         
            timeConverter:
            {
                timeFormat: "%D %T",
                input:      "STRING",
                output:     "INTEGER"
            }
        },

The above example specifies:

  • Severity and sevConverter: The severity field has a conversion defined as sevConverter in the Conversions section, this looks up the value of severity defined in the severity section of constants and returns back the mapped integer corresponding to the severity
  • stringToInt: It is used in a conversion, which forces the system to turn a string token into an integer value
  • timeConverter: It is used in conversion which forces the system to convert to time. If time is epoc time, then timeFormat mentioned in timeConverter should be commented. Otherwise, user should provide the timeFormat i.e. (%D %T) and it should be uncommented

JSON Events

The capability of JMS LAM is the ability to consume JSON events. JSON is a sequence of attribute/value, and the attribute is used as a name. Under mapping, you must define the following attribute builtInMapper: "CJsonDecoder". It automatically populates all of the values contained in the JSON object, prior to the rules being run.

For example, if the JSON object to be parsed was:

{"signature": "11898.9","source_id": "Server1", "severity": "MINOR" so on}

The attributes available to the rules in the mapping section would be $signature=11898.9, $Severity= Minor and so on. Similarly, user can map ObjectMessage and MapMessage.

For TextMessage user should use variable section.

Below are few samples of TextMessage, MapMessage and ObjectMessage.

TextMessage
JMS_MSG:3600de30-92f8-71e2-0408-97bfd0490000||1||26||13639605v53||1364210285||1363960556||NEO||delta-server-loggingAdaptor||default.log||2013-03-25 11:17:19.560 ERROR [Response--6 - BlockingEntitlementsCheckHandle] Unauthorised access detected, throwing UnauthorisedAccessException||xstm3022xpap.stm.swissbank.com||NEO-PROD


MapMessage

hostname=10.112.70.125

port=8080

destination-jndi=jms/topic/test

username=administrator

password=India@123

signature=8.9

source_id=server1

external_id=123.1345

manager=JMS

source=server1

class=server

agent=test

agent_location=test

type=test

severity=MAJOR

description=Test server1

agent_time=07/24/12 18:06:01


ObjectMessage

hostname=10.112.70.123

port=8080

connection-factory-jndi=jms/RemoteConnectionFactory

destination-jndi=jms/queue/test

#connection-factory-jndi=ConnectionFactory

#destination-jndi=dynamicQueues/final

username=administrator

password=India@123

server-name=Jboss

# Properties related to Websphere mq

message-type=object

# Properties related to object message

signature=1234.8.9

source_id=server2

external_id=hmoscsysd2

manager=JMS

source=server2

class=test

agent=test

agent_location=test

type=server

severity=MAJOR

description=test server2

agent_time=07/24/12 18:06:01

catchAll

The attribute that is never referenced in a rule is collected and placed as a JSON object in a variable called overflow defined here and passed as part of the event.

 catchAll: "overflow",
            rules:
            [
                { name: "signature", rule:      "$signature" },
                { name: "source_id", rule:      "$source_id" },
                { name: "external_id", rule:    "$external_id" },
                { name: "manager", rule:        "JMS" },
                { name: "source", rule:         "$source" },
                { name: "class", rule:          "$class" },
                { name: "agent", rule:          "$LamInstanceName" },
                { name: "agent_location", rule: "$agent_location" },
                { name: "type", rule:           "$type" },
                { name: "severity", rule:       "$severity", conversion: "sevConverter" },
                { name: "description", rule:    "$description" },
                { name: "agent_time", rule:     "$agent_time", conversion: "stringToInt" }
            ]

The above example specifies the mapping of tokens and the variable overflow for catchAll.

The attribute test1 and test2 is not mapped with a field in the jms_lam.conf file, it is placed in the overflow JSON object. The fields that are placed in the overflow variable can be viewed in the JMS LAM log file.

Example of a message sent through a JMS queue.

Message
[{"signature":"0.1.8","source_id":"xvsdfgdg","external_id":"dduncan9","manager":"Sonsitng","source":"Indonesian","class":180,"agent_location":"Liangwa","type":"Violet","severity":"WARNING","description":"Yuan
Renminbi","agent_time":"07/27/12 19:06:01","test1":"1","test":"2"}]

Example of an overflow JSON object containing the unmapped test1 and test2 tokens, created in the JMS LAM log file:

INFO
: [EventFa][20161027 17:04:38.701 +0530] [CMooMsg.java]:1099 +|Encoded size
[376]
json[{"_MOOTADATA_":{"creation_time":1477568078591},"agent":"JMSLAM","agent_location":"Liangwa","agent_time":0,"class":180,"description":"Yuan
Renminbi","external_id":"dduncan9","manager":"Sonsitng","overflow":"{\"test\":\"2\",\"test1\":\"1\"}","severity":2,"signature":"0.1

Starting the JMS LAM

To start the JMS LAM enter the following command:


service jmslamd start

To stop the JMS LAM enter the following command:

service jmslamd stop

To view the status of JMS LAM enter the following command:

service jmslamd status

Quotes

In some instances, the attribute strings are quoted. Our JSON parser ignores it, but the standard requires quoting for all strings, so Moogsoft recommends that user quote all strings.

Comments

A user can comment out lines by appending them with a hash.

Command Line Attributes

The jms_lam is a command line executable that can be run as a service daemon, and takes 4 attributes, which can be viewed by typing:

jms_lam--help

Option

Description

--config

Points to a pathname to find the configuration file for the LAM. This is where the entire configuration for the LAM is specified

--help

Displays all the command line options

--version

Displays the components version number

-- loglevel

Specifies the level of debug. By default, you get everything. In common with all executables in MOOG, having it set at that level can result in a lot of output (many messages per event message processed). In all production implementations it is recommended that log level be set to WARN, which only informs you of matters of importance

Version Information

LAM Version

Tool Version

Tested?

Expected to Work

1.0

Active MQ 5.13.4YesYes
Jboss 10YesYes
WebLogic12.0YesYes

1.1

Active MQ 5.13.4YesYes
Jboss 10YesYes
WebLogic12.0YesYes


1.2

Active MQ 5.13.4YesYes
Jboss 10YesYes
WebLogic12.0YesYes

System Information

This LAM was tested on a system with the following configurations:

CPU2 core
RAM4 GB
Operating SystemCentOS Linux release 6.7

The system must at least have the above mentioned system requirements to run the LAM.

  • No labels