Page tree
Skip to end of metadata
Go to start of metadata

LAMs and LAMBots

LAMs (Link Access Modules) listen for and ingest raw data. LAMs contain LAMBots; Javascript files which describe how to harvest the data, and map it to the Moogsoft AIOps Event object before publishing on the MooMS bus.

As part of the LAMs system, there is a parallel set of embeddable LAMBot modules which consist of JavaScript files that are read by the LAMs and executed into a private context. LAMBots can also load external JavaScript modules. The functionality defined in the JavaScript files, which is accessed in a configurable way, is:

  1. To determine whether a detected Event is passed on to the system by means of a filter
  2. To inspect the event object which is published to the /Events bus and make modifications to it*

*Please note:  This is useful when a LAMBot is dealing with semi-structured input. For example, in Syslog where a standard Syslog message the bulk of the interesting information is in a freeform text field.

For other types of source systems, such as Cisco routers, you need to process the description field and undertake textual processing using regular expressions. This allows you to determine the type of the message and to extract useful information from a message, for example, physical card, port, destination IP address, which you would want to package up as part of the Event, and eventually the Alert 

The LAMBots have a number of prebuilt modules, logger and constants that allow you to both log output to the standard logging output of the LAM. As they are run in a multi-threaded environment, you can share the state between the various LAMBots to allow you to undertake complex processing. 

In parallel to the Moobots, there is a standard file called LamBot.js  that resides in the LAMBots directory, which is loaded into the LAMBots and forms the base context for the execution of the JavaScript function. You define in the LAM configuration under the filter section a pre-send module, which is where you must also define a pre-send function used by the LAMBot to determine whether or not to pass the Event object onto the Event bus. You can also write a number of modules to allow you to structure the LAMBot code into shared code, which can be used across a number of different LAMs. This is particularly useful when you are building configurations that are re-used. For example, with Cisco Syslog messages, the different standard Syslog messages that a Cisco device sends can be captured and processed in a similar way, whether written to a file, come in as a Trap, or, picked up using the Socket LAM from the network. 

Time conversion

Time conversion in Moogsoft AIOps LAMs supports the Java platform standard API specification: 

https://docs.oracle.com/javase/8/docs/api/java/text/SimpleDateFormat.html 

Some Unix time formats are indirectly supported and LAM logging will indicate any automatic conversion that took place at startup.

The only PCRE/Perl modifier automatically converted is the lone 'U' ungreedy modifier, PCRE's '-U' is not supported.

If a pattern contains a -U it should be removed manually

Standard configuration for a LAMBot


Every LAMBot has to define two functions, firstly, onLoad(), which takes no parameters and is run by every threaded instance of the LAMBot at start-up.

When this file is loaded in and a new LAMBot thread is created the function is called. By default there are 10 threads for every LAM. Moogsoft recommends that you setup shared values or lookup tables in the onLoad() function.


function onLoad()
{
    if (constants.contains("key1") == false)
    {
        constants.put("key1",2);
    }
    logger.info("Key: key2, store status: " + constants.contains("key2"));
}


  • Examine the constants module to see if it contains a value at "key1" 
  • If not store the value 2 at "key1"
  • Then output using the logger module whether "key2"contains a value (result should be false) 

You must then define a presend function which is called every time the LAMBot assembles an object that it will publish on the /Events bus for processing by farmd, and subsequently turning into alerts and Situations. 


function presend(event)
{
    //
    // For example, test the value of agent variable
    // 
    var agent=event.valueOf("agent");
    logger.info("Hello world with event: " + agent);
    logger.info("Setting value test " + agent);
    if(agent!="Mickey Mouse")
    {
        logger.info("Setting value");
 
        //
        // Everything is, pi, 2 and Mickey mouse
        //
        event.setValue("agent","Mickey Mouse");
        event.setValue("agent_location",3.1412);
        event.setValue("first_occurred",2.0);
    }
}


The 'presend' function takes as an argument, 'event' which is a built in utility object CEvent. It will return either a Boolean or a special type of object that will allow you to define both the true and false as to whether or not to pass the object out to the events bus.

It also allows you to partition Event streams into substreams for differential processing in potentially a distributed farmd environment. The system will tolerate you sending the Boolean if you have a simple configuration where all the Event objects will be published on /Events.

Alternatively, you can pass back a JSON object literal that contains two members: 'passed,' which is either true or false, and 'stream', which defines the substream that the event will be sent upon:


return (true);

See second example below:


return {{passed: true, stream: "foo"}};

Inside of the presend function, there are no limitations on how you configure the Event, for example you can: change values; access lookup tables; add or remove key value binding access regular expressions, and, extract tokens. To instruct the LAM to use the presend function you require the following line, which calls the 'filterFunction ()' function in the global LAMBot variable: 


LamBot.filterFunction("presend");


'filterFunction()' takes a string, which is the name of the function to use as presend in the LAMBot. These JavaScript files are executed at start-up, so the line above gets executed, as does the definition of the function, at which point, V8 compiles it into near-byte code so it can all run at high speed and this is activated when calling the presend function. In the configuration file for the LAM, if you do not define a presend function, the LAMBot functionality is not used and every event is passed out on the bus.

Currently there are two modules available: Logger and Constants.

 

Logger

The Logger module can be loaded in the script, and allows you to write log messages to the common log file for the Moolet. For example, as you write the Moobots, you can use the Logger for debug. Writing a log message to a log file is an IO operation and comes with execution time cost. When developing the Moobot it can be helpful to have a number of logging statements; however, once the Moobot is operational, you should keep log messaging to a minimum. 

The Logger module exposes a number of functions that allow you to send a log message at the INFO, WARN and FATAL level.

Debug level
 Explanation
 INFO                 Lowest level of debug for perennial messages
 WARN Highlights genuine warnings
 FATAL Causes farmd to exit

Format strings (i.e. those which use %d for an integer, %f for a float or a %s for a string) are not permitted, instead simple concatenation should be used.

Constants

The Moobots run in their own thread, and the instances of the Moobots are independent of each other. So, if you want to share logic, states, or, flags between Moobots, you will need to use the Constants module. The Constants module enables you to build a key value dictionary mapping that is shared across the Moobot instances. 

The format for how you load the Constants module is similar to the Logger and Process modules.

Worked Example

The following example in SocketLam.js shows the usage of the Constants module with two methods that allow you to post and retrieve values from a shared scratchpad.


var count=0
constants.put("counter",count);


  • The variable count is set to 0 and stored using the label "counter" 

  • put() takes the name of the variable you want to store, counter, and the variable, count

You can later retrieve a value by calling the method get() and passing the name of the shared attribute you want, which is returned as a JavaScript local variable. 


var count_val=constants.get("counter");
count_val++;
constants.put("counter",count_val);


  • get takes the name of the shared attribute, counter
  • counter_val is incremented
  • put takes the name of the variable you want to store, counter, and the incremented value, count_val

If you do not have anything stored under counter, you will get a JavaScript null object back from the LAMbot.

  • No labels