Page tree
Skip to end of metadata
Go to start of metadata

Purpose 

Test LAM is a specially designed LAM that produces dummy data for profiling, prototyping, building scripts, and, integration testing, all undertaken in a controllable way. The general principle behind test_lam.conf is that you can configure a number of event templates and source templates. The Test LAM will fire off, in a semi random way, an event onto the events bus according to the event template, one for each of the source definitions referenced in the event template. The semi-random aspect means that you can configure an interval between events including the degree of variance in that interval. The events will be sent off and the average period between events will form a distribution as defined in the configuration. 

You can also configure event templates to be chained. For example, you can test link up/link down style functionality where you would never want the link up to arrive before the corresponding link down; therefore, you would chain a link up template to follow a link down. 

test_lam.conf

In common with all other LAM’s, there is a configuration file, test_lam.conf  with the associated LamBot, test_lam.js

The config file contains a JSON object. All config in the MOOG system is presented as a JSON object. At the first layer of the object, you have a parameter called config, and the object that follows config has all the necessary information to control the LAM. 

Quoting

In some instances the attribute strings are quoted. Our JSON parser is forgiving, but the standard requires quoting for all strings, so Moogsoft recommends that you quote all strings. 

Hash

You can comment out lines by prepending them with a hash. 

test_lam.conf walk through

Monitor object

In the monitor section, all the templates and the control for the generation of the events are defined. The first two entries are documentation values for the code: 

     monitor: 
     { 
            name : "Test File Monitor", 
            class : "CTestMonitor", 

data_rate

data_rate  is where you instruct the test LAM to generate a given quantity of output in a minute. The output is defined as the number of events per minute, and, the test LAM compresses time to adjust the period you define for each of the templates to achieve on average the rate you supply. 

    data_rate : 500, 

The above example specifies:

  • A data rate of 500 events per minute 

For example, if you have one template with one source host defined, that sends an event every 20 seconds, you would have an actual rate of three a minute. If you wanted to have 30 a minute, the system would compress the period from 20 to two seconds to achieve 30 events a minute. 

You can have an arbitrarily high number, but the system minimum time between one event is a millisecond, a 1000th of a second. You would not want to try and send out that many events in a single LAM (for just one template and source this would equate to 60,000 events per minute), as you would overload the relevant systems. 

total_events

The default for total_events total_events is -1, but anything above 0 will cause the LAM to produce that many events and then exit. Less than 0, or, not set will cause the LAM to continue indefinitely. 

    total_events : -1, 

The above example specifies: 

  • -1 tells the LAM to continue forever

For example if total_events  is set to 100, the LAM will generate 100 events and then complete. 

resolution

resolution controls the execution of the templates and is an internal field. resolution controls the number of the slices that a minute is split into. The test LAM attempts to distribute the generation of events evenly across a minute and uses these subdivisions of a minute to fill with individual template/source combinations. The internal execution of the LAM causes all event templates in a given time slice to be evaluated for execution. 

    resolution : 1, 

The above example specifies: 

  • In all normal implementations resolution is set to 1, which causes no slicing of time and every template is checked every second 

resolution is effectively the granularity of time to use in the LAM. For example, if you have 20 seconds as a period, resolution controls whether it is 20 seconds exactly (set to 1), or, plus or minus five seconds (set to 12). For example, if you divide time slices into a second, you can have periods that are resolved up to a second. If you divide time slices into 5, if you then have a period of 18, it would either execute on 15 seconds or 20 seconds. 

The reason you configure this is that the wider the resolution, the less compute the LAM uses to generate events with the trade-off that the greater the error in the randomness. 

resolution is significant because if you define a template for every 20 seconds and you have 10,000 hosts, what you must avoid is on the 0 second sending 10,000 events, and then, sending nothing until the next 20 seconds before sending 10,000 more. The system spreads out in time the load, so, you end up with a consistent event rate, which averages to the value set in data_rate; thus, avoiding over loading the system and returning an unrealistic event rate exhibiting high kurtosis. Therefore, resolution is used to determine how closely packed you can put exemplar’s for an event template.

sources

sources defines the imaginary sources or hosts that the events will come from. You can define these in one of two ways: you can give a simple list of source names as shown below: 

    sources: { 
            "switches" : ["switchA","switchB","switchC","switchD" ], 

For every event template that uses the “switches” sources, it will generate an event for Switch A, an event for Switch B, an event for Switch C and so on. This is a time consuming way to generate a number of events from different sources. 

The second way is to use an autohost, which allows for the autogeneration of a long list of similar hosts, where you define the template hostname; in the following example it is xldn%dpap . You must also define a range with a start and a count as shown below: 

    "autohost" : { 

            "template" : "xldn%dpap", 
            "range" : { 
                        "start" : 12, 
                        "count" : 10 
                      } 
                 } 

It substitutes for the %d a number that starts at 12 and then produces 10 instances. So you would have 12, 13, 14, 15, 16 all the way up to 21 in this instance. This is a good way to generate many events from many different sources. For example, you can have a start of 1 and have a count of 10,000, which results in 10,000 events from 10,000 different sources. 

sources is an object that contains entries that take a list, or, in the case of an autohost, take an object that takes a template and a range, which is an object. 

events is a list of event templates, which are JSON objects. Each event template starts with a name, where you should give the name of the event template: 

    events: [ 
                { 
                    "name" : "DBRestart
  • This is important because the signature for the event that is generated is by default composed of “$name:$source” 

So you de-duplicate alerts from a given event template by template name and source. To do this you specify which of the source definitions to combine with the template to generate the events. So sources must match a definition of sources: 

    #"sources" : "hosts", 
    "sources" : "autohost", 

timing

You can either send the events with a fixed period, or, if you specify a time period, for example every 20 seconds, you can randomise the period between events. 

    "timing" : { 
                    "randomised" : true, 

With randomised set to true the exact period between events is randomised, and if the core period you specify is 20 seconds, that will be the average period over time but you will get individual events spaced with a random period i.e., 18 seconds, 21 seconds… The way it averages up to the 20 is defined in distribution. distribution  defines a tuned probabilistic distribution for the periods 

You can also specify the distribution of periods by specifying a distribution class, which controls the sequence of intervals between events from a given template according to a class which implements a given probability density function. In this case CUniformDist  randomly selects periods up to max_period  in increments of increment, but ensures that over sample_norm, chunks of max_period  that each period (in this case 0 (we actually use 1 second rather than 0),10,20,30,40,50,60) occur equally likely, and the period will average to 30.143 (a period of 0 seconds is not allowable, which is why the answer is not exactly 30!). 

    "distribution" : 
                    { 
                        class: "CUniformDist", 
                        max_period : 60, 
                        increment : 10, 
                        sample_norm : 2 
                    } 

The above example specifies: 

  • class is an internal class used to achieve the distribution. A uniform distribution is a type of probability distribution where each of the possible outcomes has equal probability. In the example above you would want the average to be 30, but you want the probability of having 40 seconds versus 20 seconds to be identical. This is sometimes referred to as a flat distribution. Using this you will end up with an equal number of 10 seconds, an equal number of 20 seconds, an equal number of 30 seconds and so on over a very long period of time, which will present an average period of 30 seconds

Moogsoft currently only supports uniform distribution, which splits max_period into increment chunks and then randomly chooses from amongst that event set with equal probability. The sample_norm is the time over which the engine guarantees that the mean is max_period/2. 

In future releases Moogsoft will support normal distribution, and be able to define the kurtosis, the standard deviation and variance. 

fields

fields defines all the various components of events that are sent on the events bus. For example in your test environment you can specify: the class, a description, an external identifier etc., as shown below: 

    "fields" : { 
                    "agent_location" : "localhost", 
                    "class" : "AppMon", 
                    "description" : "Database instance has restarted",
                    "external_id" : "12345", 
                    "manager" : "PATROL", 
                    "severity" : 2, 
                    "type" : "DBFail", 
                    "entropy" : 0.8 
                } 

MOOG will fill in the first occurred, the last occurred, the agent time etc. The whole of the file is a sequence of these templates. There are different examples of timing parameters and distribution. 

LinkDown

For example in the LinkDown, in timing you have an interval of 60, randomised  set to true, and a variance of 20. 

    { 
        "name" : "LinkDown", 
        "sources" : "switches", 
        "timing" : { 
                        "interval" : 60, 
                        "randomised" : true, 
                        "variance" : 20 
                    }, 

The above example specifies: 

  • On average the period is going to be 60 seconds but the variance specifies that it can range from 40 to 80 seconds, and take a random value between these two values, which averages to 60 

LinkUp

The LinkUp template has the precedent field, which references the LinkDown template, which means you can only get a Link Up once there has been a Link Down for the given defined source (switches). 

    "name" : "LinkUp", 
    "sources" : "switches", 
    "precedent" : "LinkDown", 
    "timing" : { 
                   "interval" : 60, 
                   "randomised" : true, 
                   "variance" : 20 
                }, 

The above example specifies: 

  • On average the period is going to be 60 seconds after the Link Down, with a variant ranging from 40 to 80 seconds. 

Web Server Down Trap

In the Web Server Trap Down, you only have two fields: randomised and variance

    "name" : "Web Server Down Trap", 
    "sources" : "hosts", 
    "timing" : { 
                       "randomised" : true, 
                       "variance" : 180 
               }, 
  • With randomised set to true and variance set to 180 you will get a period between occurrences that is a random number between one and 180 seconds. On average you will get an event every 90 seconds

agent and filter

The standard LAM configuration values, agent and filter define the LamBot for test LAM. The agent name appears as a $LamInstanceName . 

The filter defines whether Moog uses a LamBot. If you comment out the presend value, no filter is applied to the events produced and all events are sent unchanged to the MooMs bus. If a LamBot is defined, for every event the system assembles using this configuration, the event is passed to the LamBot (via the presend function defined in the LamBot). 

    filter: 
            { 
                presend: "TestLam.js" 
            } 

Therefore you must define a presend function in your JavaScript file. 

The return value of the presend function will determine whether the event is sent on to the MooMs bus. The presend function can also define sub-streams that events are sent out on, so events can be sent to different farmd’s. 

You can provide an additional parameter to presend called modules that takes a JSON list. The JSON list is a list of optional JavaScript files that are loaded into the context of the LamBot and executed; thus, you can share modules between LAMs. For example, you can write a generic Syslog processing module that is used in both the Socket LAM and the log file LAM. You therefore do not need to needlessly replicate code in the Moobot for each of the LAMs.


  • No labels