Skip to main content

Configure the Splunk LAM

Splunk is used for application management, security, and compliance, as well as business and web analytics.

It captures, indexes and correlates real-time data in a searchable repository from which it can generate graphs, reports, alerts, dashboards, and visualizations.

See Splunk for UI configuration instructions.

Before You Begin

The Moogsoft Enterprise Splunk integration has been validated with Splunk versions 6.5, 6 6, 7.0, 7.1, 7.2, 7.3, 8.0, 8.1, and 8.2. Before you start to set up your integration, ensure you have met the following requirements:

  • You have an active Splunk account.

  • Splunk can make requests to external endpoints over port 443.

Configure the LAM

Edit the configuration file to control the behavior of the Splunk LAM. You can find the file at $MOOGSOFT_HOME/config/splunk_lam.conf.

The Splunk LAM is a REST-based LAM as it provides an HTTP endpoint for data ingestion. Note that only the generic REST LAM properties in splunk_lam.conf apply to integrating with Splunk; see the LAM and Integration Reference for a full description of all properties.

Some properties in the file are commented out by default. Uncomment properties to enable them.

  1. Configure the connection properties for the REST connection:

    • address: Address on the Moogsoft Enterprise server that listens for REST messages. Defaults to all interfaces.

    • port: Port on the Moogsoft Enterprise server that listens for REST messages. Defaults to 48007.

  2. Configure authentication:

    • authentication_type: Type of authentication used by the LAM. Defaults to none.

    • authentication_cache: Whether to cache the username and password for the current connection when the authentication type is Basic.

  3. Configure the LAM behavior:

    • accept_all_json: Allows the LAM to read and process all forms of JSON.

    • lists_contain_multiple_events: Whether Moogsoft Enterprise interprets a JSON list as multiple events.

    • num_threads:Number of worker threads to use.

    • rest_response_mode: When to send a REST response. See the LAM and Integration Reference for the options.

    • rpc_response_timeout: Number of seconds to wait for a REST response.

    • event_ack_mode: When Moogfarmd acknowledges events from the Splunk Webhook LAM during the event processing pipeline.

  4. Configure the SSL properties if you want to encrypt communications between the LAM and the REST connection:

    • use_ssl: Whether to use SSL certification.

    • path_to_ssl_files: Path to the directory that contains the SSL certificates.

    • ssl_key_filename: The SSL server key file.

    • ssl_cert_filename: The SSL root CA file.

    • use_client_certificates: Whether to use SSL client certification.

    • client_ca_filename: The SSL client CA file.

    • auth_token or encrypted_auth_token: Authentication token in the request body.

    • header_auth_token or encrypted_header_auth_token: Authentication token in the request header.

    • ssl_protocols:Sets the allowed SSL protocols.

  5. Optionally configure the LAM identification and capture logging details:

    • name: Maps to $Laminstancename, so that the agent field indicates events Moogsoft Enterprise ingests from this LAM.

    • capture_log: Name and location of the LAM's capture log file, which it writes to for debugging purposes.

  6. Optionally configure severity conversion. See Severity Reference for further information and "Conversion Rules" in Tokenize Source Event Data for details on conversions in general.

  7. Optionally configure the process logging details:

    • configuration_file: Name and location of the LAM's process log configuration file. See Configure Logging for more information.Configure Logging

Example

An example Splunk LAM configuration is as follows:

monitor:
{
    name                          : "Splunk LAM",
    class                         : "CRestMonitor",
    port                          : 8888,
    address                       : "0.0.0.0",
    use_ssl                       : false,
    #path_to_ssl_files            : "config",
    #ssl_key_filename             : "server.key",
    #ssl_cert_filename            : "server.pem",
    #use_client_certificates      : false,
    #client_ca_filename           : "ca.crt",
    #auth_token                   : "my_secret",
    #encrypted_auth_token         : "dfJtTQMGiFHfiq7sCmxguBt6Jv+eytkoiKCquSB/7iWxpgGsG2aez3z2j7SuBtKj",
    #ssl_protocols                : [ "TLSv1.2" ],
    authentication_type           : "basic_auth_static",
            basic_auth_static:
            {
                username: "user",
                password: "pass"
                #,encrypted_password    : "dfJtTQMGiFHfiq7sCmxguBt6Jv+eytkoiKCquSB/7iWxpgGsG2aez3z2j7SuBtKj"
            },
    authentication_cache          : true,
    accept_all_json               : true,
    lists_contain_multiple_events : true,
    num_threads                   : 5,
    rest_response_mode            : "on_receipt",
    rpc_response_timeout          : 20,
    event_ack_mode                : "queued_for_processing"
},
agent:
{
    name                          : "Splunk",
    capture_log                   : "$MOOGSOFT_HOME/log/data-capture/splunk_lam.log"
},
log_config:
{
     configuration_file           : "$MOOGSOFT_HOME/config/logging/splunk_log.json"
{,

Configure for High Availability

Configure the Splunk LAM for high availability if required. See High Availability Overview for details.

Confgure LAMbot processing

The Splunk LAMbot processes and filters events before sending them to the Message Bus. You can customize or bypass this processing if required. You can also load JavaScript files into the LAMbot and execute them.

See LAMbot Configuration for more information. An example Splunk LAM filter configuration is shown below.

filter:
{
    presend: "Splunk.js",
    modules: [ "CommonUtils.js" ]
}

Start and Stop the LAM

Restart the Splunk LAM to activate any changes you make to the configuration file or LAMbot.

The LAM service name is splunklamd.

See Control Moogsoft Enterprise Processes for the commands to start, stop and restart the LAM.

You can use a GET request to check the status of the Splunk LAM. See "Check the LAM Status" in Configure the REST LAM for further information and examples.

Configure the Splunk Add-On

Log in to Splunk and install the Moogsoft Enterprise Add-On in order to send alerts from Splunk to Moogsoft Enterprise.

The Add-On uses the Splunk search to fetch data from Splunk and send it to Moogsoft Enterprise. If you are installing the Add-On in a distributed deployment, you only need to do so on the search head.

  1. Install the add-on from Apps in the console or from Splunkbase, the Splunk marketplace.

    If using on-premises versions of Splunk and Moogsoft Enterprise, copy the server.pem file to <splunk_home>/etc/apps/TA-Splunk-Moogsoft/bin.

  2. Configure the triggers for Splunk alerts to be forwarded to the integration as follows:

    Field

    Value

    URL

    <url of the integration>

    For example: https://<localhost>/events/splunk_lam_splunk1

    Alert Severity

    Enter a severity. Clear, Indeterminate, Minor, Major, Critical.

    Moogsoft Enterprise Certificate

    Enter your certificate location if using an on-premises version of Moogsoft Enterprise and Splunk. Otherwise leave empty.

  3. Save the changes.

After you complete the configuration, Splunk sends new alerts to Moogsoft Enterprise. See the following topic for more information about sending alerts.

In this section: