WebSphere MQ

Install the IBM JDK on the Moogsoft Onprem Server and set the java class path in the wmq_lam to the installed IBM JDK class path. To set the classpath proceed as follows:

Monitor section defines the object to be monitored:

monitor:
        {
           
            name                                :"WMQ Lam Monitor",
            
            class                       :"CWmqMonitor",
         
            manager_name                :"QueueManagerName",
            
            host_name                   :"localhost",

            port_number                 : 1414,

            channel_name                :"MyRemoteChannel",
 
            destination_name            :"TestQueue",

            user_name                   :"username",

            password                    :"password",

            #encrypted_password             :"ieytOFRUdLpZx53nijEw0rOh07VEr8w9lBxdCc7229o=",

            destination_type            :"queue",     

            max_retries                 : 10,
 
            retry_interval              : 60,

            message_type                :"TextMessage",

            response_require            :false,

            feedback_queue                      :"TempQueue",

            ssl_connection                              :false,

            ssl_truststore_filename     :"",

            ssl_truststore_password     : "",

            ssl_truststore_type         :"",

           ssl_ciphersuite_value        :"SSL_RSA_WITH_3DES_EDE_CBC_SHA",

           ssl_fipsRequired             :false,

           ssl_protocol                 :""
     
        },

The above example specifies:

The Agent and Process Log sections allow you to configure the following properties:

Any received data needs to be broken up into tokens. Once user have the tokens, user can start assembling an event. There are a number of parameters that allow user to control how this will work. The first 2 are start and end character. User can have multiple start and end characters. The system generates an event after all the tokens between a start and an end character are assembled.

Parsing:
{
            type: "",
            start_and_end:
            {
                start:      [],
                end:        ["\n"],
                delimiters:
                {
                    ignoreQuotes: true,
                    stripQuotes: true,
                    ignores:    "",
                    delimiter:  [",","\r"]
                }
            }
        },
 
        # Parsing block with regular expressions, using delimter
        # based tokenising:
        #
        # parsing:
        # {
        #     type: "regexp",
        #     regexp:
        #     {
        #         pattern : "(?mU)^(.*)$",
        #         capture_group: 1,
        #         tokeniser_type: "delimiters",
        #         delimiters:
        #         {
        #                 ignoreQuotes: true,
        #                 stripQuotes: false,
        #                 ignores:    "",
        #                 delimiter:  ["\r"]
        #         }
        #     }
        # },
        #

The above example specifies the following 3 types of parsing:

In the above example only one parsing method is used at a time. Either regexp or Text Message/JSON.

JSON Parsing

Any received data needs to be broken up into tokens. Once the tokens are received, the assembling of an event starts. There are a number of parameters that allow the user to control how this will work. The first 2 are a start and end character. The square brackets [] are the JSON notation for a list. You can have multiple start and end characters. The system considers an event as all of the tokens between any start and end character.

The above example specifies:

In the example above, the LAM is expecting a whole line to be written followed by a return, and it will process the whole line as one event.

If set up carefully, user can accept multi-line events.

Text Message Parsing

The Type should be set start_and_end and as shown in the below example.

type: start_and_end:
            {
                start:      [WMQ_MSG],
                end:        ["\n"],

The parsing in above example the parsing will start when it gets WMQ_MSG and end when it gets new line.

Regular Expression Parsing

In regular expression the parser searches for strings as per the expression defined in pattern. The extracted string is then delimited as per the defined delimiters. In the above example the parser searches for the expression "(?mU)^(.*)$".

Delimiters

Delimiters define how a line is split into tokens – “tokenising”. For example, if you have a line of text data, it needs to be split up into a sequence of sub strings that are referenced by position from the start. So if you were processing a comma-separated file, where a comma separates each value, it would make sense to have the delimiter defined as a comma. Then the system would take all the text between start and end and break it up into tokens between the commas. The tokens could then be referenced by position number in the string starting from one, not zero.

For example if the input string was “the,cat,sat,on,the,mat” and comma was used as a separator, token 1 would be “the”, token 2 “cat” and so on.

Be aware, there are complications when you come to tokenisation and parsing. For example, if you say comma is the delimiter, and the token contains a comma, you will end up with that token containing a comma to be split into 2 tokens. To avoid this it is recommended that you quote strings. You must then allow the system to know whether it should strip or ignore quotes, hence the stripQuotes and ignoreQuotes parameters.

ignoreQuotes: true, 
 stripQuotes: false, 
 ignores: "", 
 delimiter: [",","\r"]

The above example specifies:

For each event in the file, there is a positioned collection of tokens. Moogsoft Onpremenables a user to name these positions. Naming of the positions helps the user to identify the tokens.In the below given example token at position number 6 is a Manager name, so the user names the token as "Manager".

This section is used for text message.

variables:
        [
            #
            # Note that positions start at 1, and go up
            # rather than array index style counting from zero
            #
            { name: "signature",   position: 1 },
            { name: "source_id",   position: 4 },
            { name: "external_id", position: 3 },
            { name: "Manager",     position: 6 },
            { name: "AlertGroup",  position: 7 },
            { name: "Class",       position: 8 },
            { name: "Agent",       position: 9 },
            { name: "severity",    position: 5 },
            { name: "description", position: 10 },
            { name: "agent_time",  position: 1 }
        ],

The above example specifies:

Token at position 1 is assigned to signature; Token at position 4 is assigned to source_id and so on. Token positions starts from 1, and go up.

The variable section is used when the message type is TextMessage

The variable section is only used for text message. For JSON, MapMessage and ObjectMessage the mapping section is used. In mapping there is a value called rules, which is a list of assignments.

mapping :
        {
            #
            # All unused variables live as a JSON object
            # referenced by this variable (if defined)
            #
             builtInMapper:  "CJsonDecoder",
            # Input is restricted to Json so the builtInMapper option is not
            # used for this LAM
            #
            catchAll: "overflow",
            rules:
            [
                { name: "signature", rule:      "$signature" },
                { name: "source_id", rule:      "$source_id" },
                { name: "external_id", rule:    "$external_id" },
                { name: "manager", rule:        "WMQ" },
                { name: "source", rule:         "$source" },
                { name: "class", rule:          "$class" },
                { name: "agent", rule:          "$LamInstanceName" },
                { name: "agent_location", rule: "$agent_location" },
                { name: "type", rule:           "$type" },
                { name: "severity", rule:       "$severity", conversion: "sevConverter" },
                { name: "description", rule:    "$description" },
                { name: "agent_time", rule:     "$agent_time", conversion: "stringToInt" }
            ]
        },
        filter:
        {
            presend: "WmqLam.js"
        }
    }
}

In the example above mapping assignments are shown, the first assignment name: "signature”, rule:"$signature” (“$signature is a string with $ syntax) means for signature field take the tokens called signature. User defines a number of these rules covering the base attributes of an event. For reference, the system expects a minimum set of attributes in an event that are shown in the above example. For JSON, MapMessage and ObjectMessage, enable the builtInMapper: "CJsonDecoder" by uncommenting it. The variable section is ignored if builtInMapper is uncommented. For TextMessage builtInMapper option should be commented.

There are rules in mapping section for which conversions are to be defined. The conversions convert the received input from one format to another. E.g. in the above example of mapping, for the mapped field severity, an integer is received which is converted to text and displayed on the Moogsoft Onprem UI. The lookup for conversions is kept in the constants section. The available conversions are kept in the conversions section and called during mapping. The example of calling a conversion is as follows:

{ name: "severity", rule:       "$severity", conversion: "sevConverter" }

The example of constants and conversions sections are as follows:

constants:
        {
            severity:
            {
                "CLEAR"         : 0,
                "INDETERMINATE" : 1,
                "WARNING"       : 2,
                "MINOR"         : 3,
                "MAJOR"         : 4,
                "CRITICAL"      : 5
            }
        },
        conversions:
        {
            sevConverter:
            {
                lookup: "severity",
                input:  "STRING",
                output: "INTEGER"
            },
             
            stringToInt:
            {
                input:      "STRING",
                output:     "INTEGER"
            },
          
            timeConverter:
            {
                timeFormat: "%D %T",
                input:      "STRING",
                output:     "INTEGER"
            }
        },

The above example specifies:

The WMQ LAM has the ability to consume JSON events. JSON is a sequence of attribute/value, and the attribute is used as a name. Under mapping, user must define the following attribute builtInMapper: "CJsonDecoder". It automatically populates all of the values contained in the JSON object, prior to the rules being run.

For example, if the JSON object to be parsed was:

{"signature": "11898.9","source_id": "Server1", "severity": "MINOR"……so on}

The attributes available to the rules in the mapping section would be $signature=”11898.9”, $Severity=” Minor” and so on. Similarly, user can map ObjectMessage and MapMessage.

For TextMessage user should use variable section.

Below are few samples of TextMessage, MapMessage and ObjectMessage.

TextMessage

WMQ_MSG:3600de30-92f8-71e2-0408-97bfd0490000||1||26||13639605v53||1364210285||1363960556||NEO||delta-server-loggingAdaptor||default.log||2013-03-25 11:17:19.560 ERROR [Response--6 - BlockingEntitlementsCheckHandle] Unauthorised access detected, throwing UnauthorisedAccessException||xstm3022xpap.stm.swissbank.com||NEO-PROD

MapMessage

hostname=10.112.70.125

port=8080

destination-jndi=wmq/topic/test

username=administrator

password=India@123

signature=8.9

source_id=server1

external_id=123.1345

manager=WMQ

source=server1

class=server

agent=test

agent_location=test

type=test

severity=MAJOR

description=Test server1

agent_time=07/24/12 18:06:01

ObjectMessage

hostname=10.112.70.123

port=8080

connection-factory-jndi=wmq/RemoteConnectionFactory

destination-jndi=wmq/queue/test

#connection-factory-jndi=ConnectionFactory

#destination-jndi=dynamicQueues/final

username=administrator

password=India@123

server-name=Jboss

# Properties related to Websphere mq

message-type=object

# Properties related to object message

signature=1234.8.9

source_id=server2

external_id=hmoscsysd2

manager=WMQ

source=server2

class=test

agent=test

agent_location=test

type=server

severity=MAJOR

description=test server2

agent_time=07/24/12 18:06:01

The attribute that is never referenced in a rule is collected and placed as a JSON object in a variable called overflow defined here and passed as part of the event.

catchAll: "overflow",
           rules:
           [
                { name: "signature", rule:      "$signature" },
                { name: "source_id", rule:      "$source_id" },
                { name: "external_id", rule:    "$external_id" },
                { name: "manager", rule:        "WMQ" },
                { name: "source", rule:         "$source" },
                { name: "class", rule:          "$class" },
                { name: "agent", rule:          "$LamInstanceName" },
                { name: "agent_location", rule: "$agent_location" },
                { name: "type", rule:           "$type" },
                { name: "severity", rule:       "$severity", conversion: "sevConverter" },
                { name: "description", rule:    "$description" },
                { name: "agent_time", rule:     "$agent_time", conversion: "stringToInt" }
            ]

The above example specifies the mapping of tokens and the variable overflow for catchAll.

The attribute test1 and test2 is not mapped with a field in the wmq_lam.conf file, it is placed in the overflow JSON object. The fields that are placed in the overflow variable can be viewed in the WMQ LAM log file.

Example of a message sent through a WebSphere MQ queue.

Message
[{"signature":"0.1.8","source_id":"xvsdfgdg","external_id":"dduncan9","manager":"Sonsitng","source":"Indonesian","class":180,"agent_location":"Liangwa","type":"Violet","severity":"WARNING","description":"Yuan
Renminbi","agent_time":"07/27/12 19:06:01","test1":"1","test":"2"}]

Example of an overflow JSON object containing the unmapped test1 and test2 tokens, created in the WMQ LAM log file:

 NFO: [EventFa][20161027 17:04:38.701 +0530] [CMooMsg.java]:1099 +|Encoded size
[376]
json[{"_MOOTADATA_":{"creation_time":1477568078591},"agent":"WMQLAM","agent_location":"Liangwa","agent_time":0,"class":180,"description":"Yuan
Renminbi","external_id":"dduncan9","manager":"Sonsitng","overflow":"{\"test\":\"2\",\"test1\":\"1\"}","severity":2,"signature":"0.1