Zabbix is enterprise open source monitoring software for networks and applications. It is designed to monitor and track the status of various network services, servers, and other network hardware. Zabbix LAM fetches alarms/events form Zabbix and display it in on Moogsoft AIOps. The alarms and events are called as triggers and problems in Zabbix.
The workflow of gathering alarms/events from Zabbix and publishing it to Moogsoft AIOps is as follows:
- LAM reads the configuration from the zabbix_lam.conf file.
- LAM will connect with Zabbix REST API with given host name.
- We support http and https (SSL) requests with basic user authentication.
- The response is received with event data in json format.
- Zabbix_lam has the option to filter event data based on the filter variable, the final event carries data of events based on the values in the filter fields of the config file.
- The events are parsed and converted into normalized Moogsoft AIOps events.
- The normalized events are then published to MooMS bus.
The Zabbixserver does not need any configuration to connect to the ZabbixLAM, only the following information is required which can be entered in the zabbix_lam.conf file.
- The URL of the Zabbix server along with the port or its host name
- Username and password of the Zabbix console
Zabbix LAM Configuration
The alarms received from Zabbix are processed according to the configurations in the zabbix_lam.conf file. The processed alarms are published to Moogsoft AIOps.
The configuration file contains a JSON object. At the first layer of the object, the LAM has a parameter called config, and the object that follows config has all the necessary information to control the LAM.
The following sections are available for configuration in the Zabbix LAM configuration file.
The Zabbix LAM takes the alarm and event data from the Zabbix Server. The user can configure the parameters here to establish a connection with Zabbix.
name and class: These fields are reserved and should not be changed. The default values are Zabbix Lam Monitor and CZabbixMonitor respectively
host_name: Enter the host name or the IP Address of the Zabbix server
user_name: The username of the user of the Zabbix application
password: The password for the username provided above
encrypted_password: The password encrypted using the moog encryptor
ssl: Enter true here, to enable SSL Communication
ssl_keystore_file_path: Enter the path of the keystore file. This is the path where the generated keystore file is copied in Moogsoft AIOps, e.g. "/usr/local/zabbix_ssl/keystore.jks"
ssl_keystore_password: Enter the password of keystore
ssl_truststore_file_path: Enter the path of the truststore file. This is the path where the generated truststore file is copied in Moogsoft AIOps. e.g. "/usr/local/zabbix_ssl/keystore.jks"
ssl_truststore_password: Enter the password of truststore
The generated keystore.jks file created during the SSL Configuration can be used both as a keystore and a truststore.
event_type: Enter the type of the event that has to be fetched from Zabbix Server. The following event types can be fetched
- Trigger: Enter 0 here to fetch alarms/events raised on triggers
- Discovery_rule: Enter 1 here to get alarms/events from a set discovery rules
- Active_agent: Enter 2 here to get alarms/events from active agents
- Internal_events: Enter 3 here to get internal Zabbix events
filter: Enter true here to enable filters. The following filters are used in combination to filter out the received alarms/events:
host_group_names: Enter the name of host groups from which the alarms/events are to be fetched
host_names: Enter the name of the host present in the above-entered host groups, the alarms/events are then fetched only from the host entered here
application_names: Enter the application of the above host from which the alarms/events are to be fetched. E.g. if CPU is entered, then the alarms/events raised By the defined CPU triggers are only sent to Moogsoft AIOps
trigger_names: Enter the name of the trigger, If in the above-defined application only a specific alarm/event from a trigger is to be fetched
The filter narrows down the alarms/events hierarchically. For example, if the filter is not applied the alarms/events from all the host groups are fetched. If we only apply the host_group_names filter then the alarms/events from the host groups entered here are fetched, if we go down further and enter a hostname, then the alarms/events are fetched from the host/hosts mentioned here. The application names further narrow the alarms/events received from an application e.g. entering CPU only fetches alarms/events related to CPU, if nothing is mentioned in application_names then the alarms/events from all the applications of the defined hosts in host_names are fetched. If we are interested in a specific trigger of the host, then enter the name of the trigger in trigger_names. The alarms/events from the triggers mentioned in trigger_names will be fetched
minimum_trigger_severities: Enter here the minimum level of severity above which alarms/events of all the severities could be fetched. The severities that can be entered here are:
- Not Classified: Enter 0 here to receive all the alarms/events with all the severities including the cleared alarms/events.
- Information: Enter 1 here to receive all the alarms/events with the severity Information and above
- Warning: Enter 2 here to receive all the alarms/events with the severity Warning and above
- Average: Enter 3 here to receive all the alarms/events with the severity Average and above
- High: Enter 4 here to receive all the alarms/events with the severity High and above
- Disaster: Enter 5 here to receive all the alarms/events with the severity Disaster
polling_interval: The polling time interval between the requests after which the event data is fetched from Zabbix. The polling interval is entered in seconds
The default value is set to 10 seconds, if 0 is entered in this field then the time interval is by default set to 10 seconds
max_retries: The maximum number of retry attempts to reconnect with Zabbix in case of a connection failure
The default value is set to 10, if 0 is entered in this field then the LAM by default takes the value 10 and will try at least 10 times to reconnects
If all the number of retries are exhausted, then an alarm is sent to Moogsoft AIOps about the connection failure. For re-establishing the connection the LAM has to be restarted
retry_interval: The time interval between two successive retry attempts
The default value is set to 60 seconds, if 0 is entered in this field then the time interval is by default set to 60 seconds
timeout: If for any reason the response is not received from the Server against a request, then the LAM discards the request after waiting for some time. The time that the LAM waits before discarding is given here in the timeout field. For example, If the timeout field has 120 entered in it, then the LAM will wait for 120 seconds for a response from the server, against a request. If no response is received for 120 seconds, then the LAM discards the request and sends a new request
The entry in the fields port, polling_interval, max_retries, retry_interval and timeout should be an integer, therefore enter the values in these fields without quotation marks
Agent allows the user to define two parameters:
The above example specifies:
- name: This is the agent name, the events sent to MooMs by the Zabbix LAM are identified by the agent name in the log. In this example the agent name is Zabbix
- log: In this instance, the Zabbix LAM will write its ingress contents to zabbix_lam.log located at /var/log/moogsoft/
Refer the document HA Configuration of LAM
Variables section is not required in Zabbix LAM; a user can directly map the alarm/event fields of Zabbix with moogsoft fields. The parameters of the received alarm/event are displayed in Moogsoft AIOps according to the mapping done here.
The above example specifies the mapping of the Zabbix alarm fields with the Moogsoft AIOps fields. The stringToInt is used to convert the time received in the string format into an integer format.
The signature field is used by the LAM to identify correlated alarms
The following images show the Zabbix Alarms, Events and alarms/events displayed on Moogsoft AIOps.
Constants and Conversions
Constants and Conversions allow the user to convert formats of the received data defined users.
The above example specifies:
Severity and sevConverter: The severity field has a conversion defined as sevConverter in the Conversions section, this looks up the value of severity defined in the severity section of constants and returns back the mapped integer corresponding to the severity
- stringToInt: It is used in a conversion, which forces the system to turn a string token into an integer value
- timeConverter: It is used in conversion which forces the system to convert time. If epoc time is to be used, then timeFormat mentioned in timeConverter should be commented. Otherwise, the user should provide the timeFormat
The alarms/events are displayed in the Moogsoft AIOps, the data in the fields of the alarm or event mapped in the mapping section are shown in the respective columns of Moogsoft AIOps columns. The fields of alarms and events which are not mapped in the mapping section are displayed in the Custom Info field of the alarm. An example of Custom Info:
The attribute that is never referenced in a rule is collected and placed as a JSON object in a variable called overflow defined here and passed as part of the event.
The Zabbix field external id is sent to Zabbix LAM. Since it is not mapped to a field in the zabbix_lam.conf file, it is placed in the overflow JSON object. The fields that are placed in the overflow variable can be viewed in the Zabbix LAM log file.
An example of an overflow JSON object created in the Zabbix LAM log file:
In some instances, the attribute strings are quoted. Our JSON parser ignores it, but the standard requires quoting for all strings, so Moogsoft recommends that user quote all strings.
A user can comment out lines by appending them with a hash.
Starting the Zabbix LAM
To start the Zabbix LAM enter the following command:
To stop the Zabbix LAM enter the following command:
To view the status of Zabbix LAM, enter the following command:
Command line attributes
The zabbix_lam is a command line executable that can be run as a service daemon, and takes four attributes, which can be viewed by typing:
Points to a pathname to find the configuration file for the LAM. This is where the entire configuration for the LAM is specified
Displays all the command line options
Displays the component’s version number
Specifies the level of debugging. By default, User gets everything. In common with all executables in MOOG, having it set at that level can result in a lot of output (many messages per event message processed).
In all production implementations, it is recommended that log level is set to WARN, which only informs user of matters of importance
Expected to Work
This LAM was tested on a system with the following configurations:
|Operating System||CentOS Linux release 6.7|
The system must at least have the above mentioned system requirements to run the LAM.