Defining Custom Integrations

The Data Config > Integrations > Create your own Integration page lists the custom integrations defined in your Moogsoft instance. Each integration specifies the following:

  • A user-defined endpoint for ingesting events or metrics from a third-party service

  • A set of mappings from the third-party metric/event fields to their Moogsoft equivalents.

  • A deduplication key that defines the fields used to identify duplicate events.

Before you begin

Before you create a custom API, do the following:

  1. Verify that you can create a webhook in the third-party service.

  2. Verify that the third-party service can post REST requests with JSON payloads over port 443.

  3. Review the JSON payloads generated by the external service and compare them with the Moogsoft Events API or Metrics API schema.

Workflow description

The following steps describe the basic workflow:

  1. Create the custom endpoint

  2. Send data to the custom endpoint

  3. Set your deduplication key (advanced)

  4. If needed, create an event workflow to further process these events into a format that supports the correlations and incidents you want.Event Workflows

Create the custom endpoint

  1. Choose Data Config > Integrations > Create your own Integration.

  2. In the Create your Integration window, click Add New Integration.

  3. Enter the endpoint, description, and data type.

    Note

    • Do not try to define one custom API for multiple external systems. Each custom API should be based on one and only one service.

    • Each custom API ingests events only or metrics only. To ingest both metrics and events from one service, you need to define two custom APIs for that service.

  4. Click Save.

Send data to the custom endpoint

After you create the custom endpoint, the next step is to send data from your external tool to the endpoint you just created. Set up your external service to send data based on the following requirements.

Port

Send data over TCP port 443.

URL

Go to Data Config > Integrations > Create your own Integration and select the endpoint you just created. You can copy and paste the URL and the API Key directly from the UI.

API Authentication

Moogsoft supports basic and bearer token authentication for programmatic access to our public APIs.

Basic authentication

Embed the API key in the URL using the format api_key@domain_name. For example:

curl -k "https://mynamespace_!bmf0u446-7345-297j-3769-g57rs8qqztm@api.moogsoft.ai/v1/integrations/metrics"\
    -H "Content-Type: application/json"
    -d '{ "metric": "your-metric", "data": 10, "source": "www.your-source.com", "key": "dev", "tags": { "key": "value" }, "utc_offset": "GMT-07:00"}'

Bearer token authentication

Include the API key as a header in the body of the HTTP request. For example:

curl -k "https://api.moogsoft.ai/v1/integrations/metrics"\
    -H "Content-Type: application/json"
    -H 'apiKey: mynamespace_!bmf0u446-7345-297j-3769-g57rs8qqztm'
    -d '{ "metric": "your-metric", "data": 10, "source": "www.your-source.com", "key": "dev", "tags": { "key": "value" }, "utc_offset": "GMT-07:00"}'

Payload

Your payloads should include all relevant data that you want to include in your alerts and incidents. We recommend the following.

  • Review the Events API or Metrics API schema, depending on the type of endpoint you are defining.

  • Identify the required fields in the Moogsoft schema, such as description, severity, and service for events and metric, data, and source for metrics.

  • Identify the fields in your external schema that correspond to the Moogsoft schema.

  • Define the payload to include all the data you want to observe in your Moogsoft alerts and incidents. You will map the external fields to Moogsoft fields in the following step.

  • In some cases you might need to send different payloads to include all the data you want to observe in Moogsoft. This might happen if (for example) a service generates different payloads based on the event or metric type.

  • The payload can be in any valid JSON format.

Map third-party data to the Moogsoft schema

After you set up your external service to send data to the endpoint, the next step is to map your external data fields to their equivalents in the Moogsoft schema.

  1. Go to Data Config > Integrations > Create your own Integration and select the endpoint you created.

  2. Go to 2 Map Your Data.

  3. Select a payload from the table.

  4. Map each source field in the payload to its corresponding Moogsoft target field.

    • You can add or delete mappings as needed.

    • If one payload does not include all the source fields you need to map, select another payload in the list that does. You can select different payloads in sequence and map all your source fields in one operation.

    • If there is no source field that corresponds to a required Moogsoft target field, enter a plain-text placeholder such as "TBD" instead. You can then create an event workflow to populate this field.

    • The Moogsoft events schema requires a severity field. Click Map Values to map the severity values in your source data to their target equivalents.

Mapping multiple source fields to the same target field

You might want to do this when the relevant data for a target field is in multiple source fields. The mapping behavior is different for target fields that are simple strings, such as source, and target fields that are lists, such as service.

String fields

For target fields that are simple strings, such as source and class, the mapper uses an if-then-else mechanism. Suppose your payloads encode the event source as follows:

Given this, the mapper populates the target field source based on the order of mappings in the table:

  • An fqdn field for application events

  • An ip field for network events

  • All events include a labels tag, which includes the hostname and a lot of other information about the event.

In the third case, the source field is still unmapped but the Moogsoft event still has the source name: it is embedded in the tags.labels string.

In this case, you can map the fields as follows.

  1. Event has an fqdn field => map fqdn to source

  2. Event has no fqdn field but it does has an ip field => map ip to source

  3. Event has no fqdn or ipe field => map string "TBD" to source. You can then create an automated workflow that extracts the hostname from the labels tag and maps it to the source target field.

Note that the mapping behavior is determined by the order in which these mappings are defined in the mapping table:

map-mult-source-fields-to-source.png

List

Suppose your payload has the following fields:

  • An app field that defines the user-facing name for an app

  • A microservice field that defines the back-end service that supports the app

In this case, you can map both fields to the service target field. Because the target field is a list, it gets populated with both names: "service" : ["myApp", myMicroService"]

Set your deduplication key (advanced)

To reduce noise, Moogsoft uses a deduplication key to group similar events into alerts. This key is an aggregation of relevant event fields. You can preview the current deduplication and edit the deduplication key as needed. 

Customizing deduplication is an advanced feature. You should use the recommended fields unless the default deduplication does not meet your needs. See Event deduplication: how-to and best practices.