Configure the Fluentd LAM

The Fluentd LAM receives and processes Fluentd alerts forwarded to Moogsoft Enterprise. The LAM parses the data into Moogsoft Enterprise events.

You can install a basic Fluentd integration in the UI. See Fluentd for integration steps.

Configure the Fluentd LAM if you want to configure custom properties, set up high availability or configure advanced options that are not available in the UI integration.

Before You Begin

The Fluentd LAM has been validated with Fluentd v0.12. Before you start to set up the LAM, ensure you have met the following requirements:

You have installed Fluentd.
You have the permissions to edit the Fluentd configuration file.
You have installed Ruby Gems for Fluentd.
Fluentd can make requests to external endpoints over port 443. This is the default.

If you are configuring a distributed deployment refer to High Availability Overview first. You will need the details of the server configuration you are going to use for HA.

Configure the LAM

Edit the configuration file to control the behavior of the Fluentd LAM. You can find the file at $MOOGSOFT_HOME/config/fluentd_lam.conf.

The Fluentd LAM is a REST-based LAM as it provides an HTTP endpoint for data ingestion. Note that only the generic REST LAM properties in fluentd_lam.conf apply to integrating with Fluentd; see the LAM and Integration Reference for a full description of all properties.

Some properties in the file are commented out by default. Uncomment properties to enable them.

Configure the connection properties for the REST connection:
- address: Address on the Moogsoft Enterprise server that listens for REST messages. Defaults to all interfaces.
- port: Port on the Moogsoft Enterprise server that listens for REST messages. Defaults to 48008.
Configure authentication:
- authentication_type: Type of authentication used by the LAM. Defaults to none.
- authentication_cache: Whether to cache the username and password for the current connection when the authentication type is Basic.
Configure the LAM behavior:
- accept_all_json: Allows the LAM to read and process all forms of JSON.
- lists_contain_multiple_events: Whether Moogsoft Enterprise interprets a JSON list as multiple events.
- num_threads: Number of worker threads to use when processing events.
- rest_response_mode: When to send a REST response. See the LAM and Integration Reference for the options.
- rpc_response_timeout: Number of seconds to wait for a REST response.
- event_ack_mode: When Moogfarmd acknowledges events from the Fluentd LAM during the event processing pipeline.
Configure the SSL properties if you want to encrypt communications between the LAM and the REST connection:
- use_ssl: Whether to use SSL certification.
- path_to_ssl_files: Path to the directory that contains the SSL certificates.
- ssl_key_filename: The SSL server key file.
- ssl_cert_filename: The SSL root CA file.
- use_client_certificates: Whether to use SSL client certification.
- client_ca_filename: The SSL client CA file.
- auth_token or encrypted_auth_token: Authentication token in the request body.
- header_auth_token or encrypted_header_auth_token: Authentication token in the request header.
- ssl_protocols: Sets the allowed SSL protocols.
Optionally configure the LAM identification and capture logging details:
- name: Maps to $Laminstancename, so that the agent field indicates events Moogsoft Enterprise ingests from this LAM.
- capture_log: Name and location of the LAM's capture log file, which it writes to for debugging purposes.
Optionally configure severity conversion. See Severity Reference for further information and "Conversion Rules" in Tokenize Source Event Data for details on conversions in general.
Optionally configure the process logging details:
- configuration_file: Name and location of the LAM's process log configuration file. See Configure Logging for more information.Configure Logging

Example

An example Fluentd LAM configuration is as follows.

monitor:
{
    name                          : "Fluentd Lam",
    class                         : "CRestMonitor",
    port                          : 48008,
    address                       : "0.0.0.0",
    use_ssl                       : false,
    #path_to_ssl_files            : "config",
    #ssl_key_filename             : "server.key",
    #ssl_cert_filename            : "server.pem",
    #use_client_certificates      : false,
    #client_ca_filename           : "ca.crt",
    #auth_token                   : "my_secret",
    #encrypted_auth_token         : "dfJtTQMGiFHfiq7sCmxguBt6Jv+eytkoiKCquSB/7iWxpgGsG2aez3z2j7SuBtKj",
    #header_auth_token            : "my_secret",
    #encrypted_header_auth_token  : "dfJtTQMGiFHfiq7sCmxguBt6Jv+eytkoiKCquSB/7iWxpgGsG2aez3z2j7SuBtKj",
    #ssl_protocols                : [ "TLSv1.2" ],
    authentication_type           : "none",
    authentication_cache          : true,
    accept_all_json               : false,
    lists_contain_multiple_events : true,
    num_threads                   : 5,
    rest_response_mode            : "on_receipt",
    rpc_response_timeout          : 20,
    event_ack_mode                : "queued_for_processing"
},
agent:
{
    name                          : "Fluentd",
    capture_log                   : "$MOOGSOFT_HOME/log/data-capture/fluentd_lam.log"
},
log_config:
{
    configuration_file            : "$MOOGSOFT_HOME/config/logging/fluentd_lam_log.json"
{,

Configure for High Availability

Configure the Fluentd LAM for high availability if required. See High Availability Overview for details.

Configure LAMbot Processing

The Fluentd LAMbot processes and filters events before sending them to the Message Bus. You can customize or bypass this processing if required. You can also load JavaScript files into the LAMbot and execute them.

See LAMbot Configuration for more information. An example Fluentd LAM filter configuration is shown below.

filter:
{
    presend: "FluentdLam-SolutionPak.js",
    modules: [ "CommonUtils.js" ]
}

Start and Stop the LAM

Restart the Fluentd LAM to activate any changes you make to the configuration file or LAMbot.

The LAM service name is fluentdlamd.

See Control Moogsoft Enterprise Processes for the commands to start, stop and restart the LAM.

You can use a GET request to check the status of the Fluentd LAM. See "Check the LAM Status" in Configure the REST LAM for further information and examples.

Configure Fluentd

After you have the Fluentd LAM running and listening for incoming requests, you can configure Fluentd. See "Configure Fluentd" in Fluentd.

In this section: