Skip to main content

Netcool Legacy LAM

The Netcool Legacy LAM enables Moogsoft Onprem to receive data from IBM Tivoli Netcool/OMNIbus.

In its default configuration, the LAM ingests data from a default Tivoli Netcool/OMNIbus setup. The LAM can also ingest data from non-default setups through its mapping utilities.

Workflow

The workflow of gathering events from IBM Tivoli Netcool/OMNIbus and publishing it to Moogsoft Onprem is as follows:

  1. Data received from Tivoli Netcool/OMNIbus has fields checked to identify event type (eg. Problem, Resolution), state (eg. INSERT, UPDATE, DELETE) and severity.

  2. The Legacy LAMbot creates events (based on the LAMbot configuration), containing all the mandatory Netcool event fields, and any additional optional fields.

  3. The LAMbot passes events to the Netcool Alert Builder for de-duplication and alert creation.

  4. The Alert Builder passes alerts to the Netcool Alert Rules Engine for filtering, which then passes them on to Sigalisers for further processing.

  5. The Legacy LAM processes ITNM (IBM Tivoli Network Manager) root cause and symptom events, using Configure Deterministic Alert Clustering with Cookbook to create Situations.

Before You Begin

Before you start to set up the LAM, ensure you have met the following requirements:

  • You have system administrator privileges in IBM Tivoli Netcool/OMNIbus.

  • Moogsoft Onprem is configured to allow inbound communication from IBM Tivoli Netcool/OMNIbus.

You also require the following files, all of which Moogsoft Onprem installs by default:

  • $MOOGSOFT/config/netcool_lam.conf

  • $MOOGSOFT/bots/lambots/NetcoolLam.js

  • $MOOGSOFT/bots/lambots/NetcoolUtility.js

  • $MOOGSOFT_HOME/bin/utils/netcool_lam.conf.template

In addition, Moogsoft Onprem also provides the following Moobots which you will require:

  • AlertBuilderNetcool.js: Allows special handling for the 'repeatDetection' and 'stopDeduplication' processes.

  • AlertRulesEngineNetcool.js: Used to process DELETE and REPEAT type events.

  • SituationMgrNetcool.js: Creates description labels for Situations based around root cause and symptom detection, based on IBM Tivoli Network Manager parameters.

Configure IBM Tivoli Netcool/OMNIbus

Configure the socket gateway and socket map files to send data to Moogsoft Onprem.

Socket Gateway

Tivoli Netcool/OMNIbus includes a socket gateway which is able to pass data to a third party system. This permits integration with Moogsoft Onprem.

In the IBM Netcool/OMNIbus gateway configuration file, NCO_GATE.props, configure the following:

Field

Value

Gate.Socket.Host

Hostname or IP address of the Moogsoft Onprem system.

Gate.Socket.Port

Port number defined in the Legacy LAM. For example 8411.

Gate.Socket.Separator

The delimiter used in the data sent to Moogsoft Onprem. Set this to double pipe - ||

Gate.Socket.InsertTrailer

NC_EVENT_END;\n

Gate.Socket.UpdateTrailer

NC_EVENT_END;\n

Gate.Socket.DeleteTrailer

NC_EVENT_END;\n

Gate.Socket.InsertHeader

INSERT||

Gate.Socket.UpdateHeader

UPDATE||

Gate.Socket.DeleteHeader

DELETE||

Socket Map Files

In socket.map, configure the fields you want to send to Moogsoft Onprem, ensuring that no fields are set to ON INSERT ONLY. Gateway mapping should include all the mandatory fields, and any additional optional ones (such as @NodeAlias).

An example mapping configuration is as follows:

CREATE MAPPING StatusMap
 (
        '' = '@Identifier',
        '' = '@Serial',
        '' = '@Node',
        '' = '@LocalNodeAlias',
        '' = '@Manager',
        '' = '@Agent',
        '' = '@AlertGroup',
        '' = '@AlertKey',
        '' = '@Severity',
        '' = '@Summary',
        '' = '@FirstOccurrence',        
        '' = '@LastOccurrence',
        '' = '@Class',        
        '' = '@OwnerUID',         
        '' = '@Acknowledged',
        '' = '@ExpireTime',
        '' = '@SuppressEscl',
        '' = '@TaskList',
        '' = '@LocalRootObj',       
        '' = '@RemoteNodeAlias',
        '' = '@RemoteRootObj',        
        '' = '@ServerName',
        '' = '@ServerSerial',
        '' = '@StateChange',        
        '' = '@InternalLast',
        '' = '@Tally',                                                          
        '' = '@Type',
        '' = '@EventId',
        '' = '@NodeAlias'
 );

Configure the LAM

Mapping Utility

You define mapping in netcool_lam.conf. By default, the LAM maps to the data fields in a default setup of Tivoli Netcool/OMNIbus.

If ingesting data from a non-default Tivoli Netcool/OMNIbus setup, you can use the moog_netcool_lam_mapper utility to map between Tivoli Netcool/OMNIbus data fields and Moogsoft Onprem fields.

The mapping utility can use either a Tivoli Netcool/OMNIbus map file or a log file containing event data from Tivoli Netcool/OMNIbus. Before running it, ensure you back up the existing netcool_lam.conf file.

To use a map file with the mapping utility, enter the command and arguments as follows (the map file is socket.map):

sh $MOOGSOFT_HOME/bin/utils/moog_netcool_lam_mapper -f socket.map -t map

To use a log file with the mapping utility, enter the command and arguments as follows (the log file is event-data.txt):

sh $MOOGSOFT_HOME/bin/utils/moog_netcool_lam_mapper -f event-data.txt -t logfile

When running the mapping utility (using either a map file or a log file), you can also optionally set the address (using the -a argument in the command line) and port number (using the -p argument in the command line), as follows:

sh $MOOGSOFT_HOME/bin/utils/moog_netcool_lam_mapper -a remote_host -f socket.map -p 8455 -t map

The above example sets the address to remote_host and the port number to 8455.

Once the mapping utility has successfully completed, a new Legacy LAM configuration file netcool_lam.conf is automatically generated and placed in $MOOGSOFT/config, overwriting the existing netcool_lam.conf.

To verify the utility's successful completion, check the Legacy LAM configuration file to ensure that the following fields are set correctly:

  • port: The port number on which the LAM receives data from Netcool.

  • address: The hostname of the system running Moogsoft Onprem. If running on-premise, the default address is 0.0.0.0. For an on-demand service, such as Amazon Web Services, the address is likely similar to ew2.234.234.compute.amazonaws.com.

  • mode: The operation mode in which the socket LAM runs. Ensure this is set to SERVER

Configure for High Availability

Configure the Netcool Legacy LAM for high availability if required. See High Availability Overview for details.

Configure LAMbot Processing

The LAMbot performs the core processing of the data received from Tivoli Netcool/OMNIbus. You can edit the configuration in NetcoolLam.js.

Configure ITNM

You can enable the processing of ITNM (IBM Tivoli Network Manager) Route Cause and Symptom events in the LAMbot. In the NetcoolLam.js file, set the usingITNM value to true and ensure the @NmosCauseType and @NmosSerial fields are included in the event. You can then proceed with the configuration steps below.

To enable the processing of ITNM fields, apply the following configurations to $MOOGSOFT/config/moog_farmd.conf:

  1. In the sig_resolution section, uncomment the following merge group, making it available as an additional merge group:

    merge_groups:
    [
         {
              name: "ITNM Route Causes & Symptoms",
              moolets: ["ITNM"],
              alert_threshold      : 2,
              sig_similarity_limit : 0.65
          }
    ],

  2. In the moolets section, uncomment the ITNM Cookbook and recipe:

    {
    # Moolet
    name                : "ITNM",
    classname           : "CCookbook",
    run_on_startup      : true,
    metric_path_moolet  : true,
    moobot              : "Cookbook.js",
    #process_output_of  : "AlertRulesEngine",
    process_output_of   : "AlertBuilder",


    # Algorithm
    membership_limit       : 1,
    scale_by_severity      : false,
    entropy_threshold      : 0,
    single_recipe_matching : false,
    
         recipes : [
         {
              chef                   : "CValueRecipe",
              name                   : "ITNM Route Cause & Symptom Detection",
              description            : "Root cause and Symptom alerts detected based on ITNM",
              recipe_alert_threshold : 1,
              exclusion              : "custom_info.nmosCauseType = 0",
              trigger                : "custom_info.suppressedSerial > 0",
              rate                   : 0,
              min_sample_size        : 5,
              max_sample_size        : 10,
              cook_for               : 1200,
              matcher                :    {
                                          components:[
                                                     { name: "custom_info.suppressedSerial", similarity: 1.0 }
                                                     ]
                                          }
         }
                   ],
    cook_for  :    1200
}

  3. Enable the Netcool Situation Manager Moolet:

    name               : "SituationMgr",
classname          : "CSituationMgr",
run_on_startup     : true,
metric_path_moolet : false,
moobot            : "SituationMgr.js",
moobot             : "SituationMgrNetcool.js",
process_output_of  : [
                      "ITNM",
                      "Sigaliser",
                      "TemplateMatcher",
                      "Speedbird"
                     ]

  4. Save the Moogfarmd configuration file.

Configure Moobot Processing

Alert Builder Moobot

The Netcool Alert Builder detects whether a received event is repeating and whether to create an alert from it. This is in addition to the standard functionality of the Alert Builder.

In the Alert Builder Moobot ($MOOGSOFT/bots/moobots/AlertBuilderNetcool.js), there are three configurable settings:

repeatDetection

If an incoming Event is an UPDATE type, repeat detection is enabled (DELETE and INSERT types cannot be repeating events). The event signature is then matched to existing alerts. If no match is found, or a match is found to a closed alert, a new alert is created. If an open alert match is found, de-duplication is carried out (see below).

  • Set to true to enable the repeat detection process, where the Netcool Alert Builder Moobot determines if newly created alerts are similar to existing alerts. To enable the repeat detection functionality, in $MOOGSOFT/config/moog_farmd.conf, in the AlertBuilder moolet section, apply the follow configuration:

    name              : "AlertBuilder",
classname         : "CAlertBuilder",
run_on_startup    : true,
#moobot           : "AlertBuilder.js",
moobot            : "AlertBuilderNetcool.js",

stopDeduplication

This determines whether the repeat detection process is carried out before or after an alert is created.

  • Set to true to prevent the de-duplication process from occurring. The repeat detection process is then performed before an alert is created.

  • Set to false to create a new alert based on the event that has been received. Then the repeat detection process is performed, based on the newly created alert and existing alerts. This also allows you to forward the alert to a different Moolet chain.

overwriteCustomInfo

This defines whether the custom_info is updated when updating (de-duplicating) alerts.

  • Set to true to update alerts custom_info from new event data.

  • Set to false to leave alerts custom_info unchanged when an alert is updated.

If Tivoli Netcool/OMNIbus is sending Route Cause and Symptom events from ITNM (IBM Tivoli Network Manager), and using ITNM is set to to true in the LAMbot configuration file (see LAMbot configuration above), then the alerts custom_info field must be updated when de-duplicating: set overwriteCustomInfo to true.

AlertRulesEngine Moobot

Enable the AlertRulesEngineNetcool.js Moobot along with the associated action states and transitions. It is used to process DELETE and REPEAT type alerts, determining whether they are discarded or passed onto the Sigalisers for further processing.

To do this, open the Moogfarmd configuration file and uncomment the line containing the AlertRulesEngineNetcool.js Moobot within the AlertRulesEngine moolet section, as shown below:

name                   : "AlertRulesEngine",
classname              : "CAlertRulesEngine",
run_on_startup         : true,
metric_path_moolet     : true,
#moobot                : "AlertRulesEngine.js",
moobot                 : "AlertRulesEngineNetcool.js",
#standalone            : true
process_output_of      : "AlertBuilder"

Ensure that run_on_startup is set to true

Save the changes.

The rules (action states and transitions) to process DELETE and INSERT type Alerts should be added to the Moogsoft Onprem instance by entering the following command:

sh $MOOGSOFT_HOME/bin/utils/moog_netcool_are_installer

Default Field Mapping

The following table shows the default Tivoli Netcool/OMNIbus field mappings to Moogsoft Onprem fields. These mappings are defined either in the Legacy LAM configuration file or within the LAMbot:

Netcool Field Name

Moogsoft Onprem Field Name

Data type

Mandatory

ActionType

type

varchar(255)

Yes

Identifier

signature

varchar(255)

Yes

Serial

external_id

incr

Yes

Node

source

varchar(64)

Yes

LocalNodeAlias

source_id

varchar(64)

Yes

Manager

manager

varchar(64)

Yes

Agent

agent

varchar(64)

No

AlertKey

agent_location

varchar(255)

Yes

AlertGroup

agent_location

varchar(255)

No

Severity

severity

integer

Yes

Summary

description

varchar(255)

Yes

FirstOccurrence

first_occurred

integer

Yes

LastOccurrence

agent_time

integer

Yes

Class

class

integer

Yes

OwnerUID

custom_info.ownerUID

integer

Yes

Acknowledged

custom_info.acknowledged

integer

Yes

ExpireTime

custom_info.expireTime

integer

Yes

SuppressEscl

custom_info.suppressEscl

integer

Yes

TaskList

custom_info.taskList

integer

Yes

LocalRootObj

custom_info.localRootObj

varchar(255)

Yes

RemoteNodeAlias

custom_info.remoteNodeAlias

varchar(64)

Yes

RemoteRootObj

custom_info.remoteRootObj

varchar(255)

Yes

ServerName

custom_info.serverName

varchar(64)

Yes

ServerSerial

custom_info.serverSerial

integer

Yes

StateChange

custom_info.stateChange

integer

Yes

InternalLast

custom_info.internalLast

integer

Yes

Tally

custom_info.tally

integer

Yes

Type

custom_info.eventType

integer

No

EventId

custom_info.eventId

varchar(255)

No

NodeAlias

custom_info.NodeAlias

varchar(64)

No

NmosCauseType

custom_info.nmosCauseType

integer

No

NmosSerial

custom_info.nmosSerial

varchar(64)

No

Severity Mapping

By default, severity mapping is identical to the severity values used within Tivoli Netcool/OMNIbus. You can change this if necessary under the severity section of netcool_lam.conf.

Start and Stop the LAM

Restart the Netcool Legacy LAM to activate any changes you make to the configuration file or LAMbot.

The LAM service name is netcoollamd.

See Control Moogsoft Onprem Processes for the commands to start, stop and restart the LAM.

In this section: