Moogsoft Docs

Configure the Datadog Polling LAM

The Datadog Polling LAM allows you to retrieve alerts from Datadog. The Datadog Polling LAM is an HTTPS client that polls your Datadog server at configurable intervals. It parses the JSON responses it receives into Moogsoft AIOps events.

You can install a basic Datadog Polling integration via the UI by supplying connection details and configuring a few parameters. See Datadog Polling for integration steps.

Configure the Datadog Polling LAM if you want to configure custom properties, set up high availability or configure advanced options that are not available in the UI integration.

Before You Begin

The Datadog Polling integration has been validated with Datadog v. 2018. Before you set up your integration, ensure you have met the following requirements:

  • The port for your Datadog server is open and accessible from Moogsoft AIOps.
  • Your Datadog system can accept HTTPS requests.
  • You know your Datadog server URL.
  • You know your Datadog API key.
  • You know your Datadog Application key.

Configure the LAM

Edit the configuration file to control the behavior of the Datadog Polling LAM. You can find the file at $MOOGSOFT_HOME/config/datadog_client_lam.conf

See the Datadog Polling LAM Reference for a full description of all properties. Some properties in the file are commented out by default. Uncomment properties to enable them.

  1. Configure the connection properties for each Datadog target:
    • url : Datadog request URL including host and port.
    • user : Datadog account user.
    • password or encrypted password : Datadog account password or encrypted password.

  2. Determine how to select and process Datadog events for each target:
    • enable_epoch_converter : You can use an epoch timestamp instead of a machine timestamp.
    • params_date_format : Date format to include in Datadog query.
    • request_query_params : SQL query to select Datadog events. See the Datadog Polling LAM Reference for an example.
    • overlap_identity_fields : List of payload tokens the LAM uses to identify duplicate events when Datadog returns all open events and not just updated events.
    • requests_overlap : Period of time to delay processing duplicates.
    • results_path : Location of the JSON results objects in the data structure. Default to results .

  3. Configure the LAM behavior for each target:
    • request_interval : Length of time to wait between requests, in seconds.
    • timeout : Length of time to wait before halting a connection or read attempt, in seconds.
    • num_threads : Number of worker threads to use.

  4. Configure the SSL properties if you want to use SSL:
    • disable_certificate_validation : Whether to disable SSL certificate validation.
    • path_to_ssl_files : Path to the directory that contains the SSL certificates.
    • server_cert_filename : Name of the SSL root CA file.
    • client_key_filename : Name of the SSL client key file.
    • client_cert_filename : Name of the SSL client certificate.

  5. Optionally configure the LAM identification and the log file details:
    • name : Identifies the event sent to the message bus by the Datadog LAM.
    • log : Name and location of the Datadog LAM log file.

  6. Review the severity conversion rules and modify if required. See Severity Reference for details.

Example

You can configure the Datadog LAM to retrieve events from one or more targets. The following example demonstrates a configuration that targets two Datadog sources. For a single source, comment out the target2 section. If you have more than two sources, add a target section for each one and uncomment properties to enable them.

In the following example, the Datadog LAM is configured to poll two different Datadog instances. The LAM uses the tokens NodeID and EventID to identify duplicate events. These configurations specify use variables $from and $to for the query time window; the LAM specifies UNIX epoch values for these fields when it sends a poll request.

monitor:
	{
        name: "Datadog REST Client Monitor",
        class: "CRestClientMonitor",
        request_interval: 60,
        targets:
        {
           target1:
            {
                url: "https://instance1.datadoghq.com/api/v1/events",
                proxy:
                 {
                    host: "localhost",
                    port: 8181,
                    user: "user",
                    password             : "pass"
                    # ,encrypted_password : "ieytOFRUdLpZx53nijEw0rOh07VEr8w9lBxdCc7229o="
                },
                request_interval: 60,
                timeout: 120
                disable_certificate_validation: false,
                path_to_ssl_files: "config",
                server_cert_filename: "server.crt",
                client_key_filename: "client.key",
                client_cert_filename: "client.crt",
                requests_overlap: 10,
                enable_epoch_converter: true,
                results_path: "events",
                overlap_identity_fields: ["NodeID", "EventID"],
                request_query_params     :  
                {  
                    start: "$from",
                    end: "$to",
                    api_key: "1234",
                    application_key: "1234"
                },
                params_date_format: "%s"
            }
           target2:
            {
                url: "https://instance2.datadoghq.com/api/v1/events",
                user: "user2",
                host: "localhost",
                port: 8181,
                request_interval: 60,
                timeout: 120
                disable_certificate_validation: false,
                path_to_ssl_files: "config",
                server_cert_filename: "server.crt",
                client_key_filename: "client.key",
                client_cert_filename: "client.crt",
                path_to_ssl_files: "config",
                requests_overlap: 10,
                enable_epoch_converter: true,
                results_path: "events",
                overlap_identity_fields: ["NodeID", "EventID"],
                request_query_params     :  
                {  
                    start: "$from",
                    end: "$to",
                    api_key: "1234",
                    application_key: "1234"
                },
                params_date_format: "%s",
            }
		}
	}

Configure for High Availability

Configure the Datadog LAM for high availability if required. See Integrations HA Configuration for details.

Configure LAMbot Filtering

The Datadog Polling LAMbot filters and processes Datadog LAM events using the "DatadogClientLam.js" stream, then sends them to the message bus.

The LAMbot moves overflow properties to custom info and performs any filtering you configure in the LAMbot file. See Data Parsing for details.

If you don’t want to map overflow properties, you can comment out the presend property in the filter section to bypass the LAMbot and send events straight to the message bus. This speeds up processing if you have a high volume of incoming Datadog alerts.

Alternatively, you can define a custom stream to receive Datadog LAM events. See Alert Builder Moolet for details.

filter: 
{
	#
	# Specify the default stream this LAM will send events to.
	# If this configuration is not specified then the generic
	# stream "events" is used, i.e. no specific stream.
	#
	# stream: "myStream",
	presend: "DatadogClientLam.js",
	modules: ["CommonUtils.js"]
}

Map LAM Properties

You can configure custom mappings in the Datadog Polling LAMbot. See Advanced Integration information for details.

By default, the following DataDot event properties map to the following Moogsoft AIOps Datadog Polling LAM properties:

Datadog Event Property

Datadog LAM Event Property

$host:$device_name:$id:$source signature
$device_name source_id
$id external_id
Datadog:$source manager
$host source
Datadog Event class
$source agent
$host agent_location
$alert_type severity
$title description
$priority", conversion: "sevConverter" severity
$date_happened agent_time

The overflow properties are mapped to "custom info" and appear under custom_info in Moogsoft AIOps alerts. This mapping requires Event Monitor tag values in the correct format ( {{event.tags.example-tag}} ) as described in the Datadog documentation .

DataDog Event Property

Datadog LAM Overflow Property

url eventDetails.url
is_aggregate eventDetails.is_aggregate
tags eventDetails.tags
resource eventDetails.resource
priority eventDetails.priority
text eventDetails.text
value eventDetails.value
threshold eventDetails.threshold

Start and Stop the LAM

Restart the Datadog LAM to activate any changes you make to the configuration file or LAMbot.

The LAM service name is datadoglamd .

See Control Moogsoft AIOps Processes for the commands to start, stop and restart the LAM.