Configure the Kafka LAM

The parsing section is used for parsing the received event and tokenizing it.

The Kafka LAM receives data in two formats:

A received event is a positioned collection of tokens. The variables section enables you to name these positions. The naming of the positions helps you to identify the tokens.

Positions start at 1 and increase. In the example below, the token in position number 6 is a Manager name, so the user names the token as "Manager". The naming helps in mapping the tokens to the Moogsoft Onprem fields in the mapping section.

variables:
        [
            { name: "signature",   position: 1 },
            { name: "source_id",   position: 4 },
            { name: "external_id", position: 3 },
            { name: "Manager",     position: 6 },
            { name: "AlertGroup",  position: 7 },
            { name: "Class",       position: 8 },
            { name: "Agent",       position: 9 },
            { name: "severity",    position: 5 },
            { name: "description", position: 10 },
            { name: "agent_time",  position: 2 }
        ],

An example Kafka LAM configuration is as follows:

monitor:
{
        name: "Kafka Lam Monitor",
        class: "CKafkaMonitor",
        kafka_listener: "localhost:9092",
        topic_name: [
                     "topic1",
                     "topic2"
                    ],                       
        groupid: "consumer-group",
        ssl_connection: false
}

parsing:
        {
            type: "start_and_end",
            start_and_end:
            {
                start: [],
                end: ["\n"],

                delimiters:
                {

                    ignoreQuotes: true,
                    stripQuotes: true,
                    ignores: "",
                    delimiter: [",","\r"]
                }
            }
        },

         #parsing:
         #{
             #type: "regexp",
             #regexp:
             #{
                 #pattern: "(?mU)^(.*)$",
                 #capture_group: 1,
                 #tokeniser_type: "delimiters",
                 #delimiters:
                 #{
                         #ignoreQuotes: true,
                         #stripQuotes: false,
                         #ignores: "",
                         #delimiter: ["\r"]
                 #}
             #}
         #},

        variables:
        [
            { name: "signature", position: 1 },
            { name: "source_id", position: 4 },
            { name: "external_id", position: 3 },
            { name: "Manager", position: 6 },
            { name: "AlertGroup", position: 7 },
            { name: "Class", position: 8 },
            { name: "Agent", position: 9 },
            { name: "severity", position: 5 },
            { name: "description", position: 10 },
            { name: "agent_time", position: 2 }
        ],

        mapping:
        {

            builtInMapper: "CJsonDecoder",
            catchAll: "overflow",
            rules:
            [
                { name: "signature", rule: "$signature" },
                { name: "source_id", rule: "$source_id" },
                { name: "external_id", rule: "$external_id" },
                { name: "manager", rule: "Kafka" },
                { name: "source", rule: "$source" },
                { name: "class", rule: "$class" },
                { name: "agent", rule: "$LamInstanceName" },
                { name: "agent_location", rule: "$agent_location" },
                { name: "type", rule: "$type" },
                { name: "severity", rule: "$severity", conversion: "sevConverter" },
                { name: "description", rule: "$description" },
                { name: "agent_time", rule: "$agent_time" }
            ]
        },

    }
    ,log_config:
    {
       configuration_file: "$MOOGSOFT_HOME/config/logging/custom.log.json"
    }
}

Configure the Kafka LAM for high availability if required. See High Availability Overview for details.

The Kafka LAMbot processes and filters events before sending them to the Message Bus. You can customize or bypass this processing if required. You can also load JavaScript files into the LAMbot and execute them.

See LAMbot Configuration for more information. An example Kafka LAM filter configuration is shown below.

filter:
{
    presend: "KafkaLam.js",
}

Restart the Kafka LAM to activate any changes you make to the configuration file or LAMbot.

The LAM service name is kafkalamd.

See Control Moogsoft Onprem Processes for the commands to start, stop and restart the LAM.