Use batch processing with custom integrations

When creating a custom integration using Create your own integration (CYOI) ►, you can identify the top-level object and map event fields for individual events if your payloads include multiple events sent in a "batch" under a single object.

This example request posts two events inside an object called events and an object called key:

curl --request POST \
     --url CYOI-ENDPOINT-URL \
     --header 'Accept: application/json' \
     --header 'Content-Type: application/json' \
     --header 'apiKey: CYOI-ENDPOINT-API-KEY' \
     --data '
     {
    "events": {
        "key": [{
                "description": "408 http request timeout - login",
                "source": "def.com/login",
                "service": [
                    "retail",
                    "support"
                ],
                "alert_category" : "application",
                "alert_type" : "login response 4xx",
                "severity" : "major",
                "tags": {
                    "origin": "db"
                },
                "utc_offset": "GMT-08:00"
            },
            {
                "description": "500 http internal server error - login",
                "source": "def.com/login",
                "service" : [
                    "retail",
                    "support"
                ],
                "severity" : "major",
                "alert_category" : "application",
                "alert_type" : "login response 5xx",
                "tags": {
                    "origin": "web"
                },
                "utc_offset": "GMT-08:00"
            }
        ]
    }
}'

NOTE: You only need to enable batch processing if your payloads include nested events, as shown in this example.

To map the event fields, do the following:

Batch processing example

  1. Configure your integration as shown in Define a custom integration. Before you complete the steps to map your third-party data to the Moogsoft schema, click Enable Batch Processing.

    Enable_Batch_Processing.png
  2. Click Root Element and select the path to the events in your payloads.

    This option configures the root path for your events. In this example, it is events.key because that is the name of the object path which holds the list of events.

  3. Finish mapping your source fields.

With batch processing enabled and a root element selected, the fields repeated between multiple events are listed once. If there is no defined root element, each field in each event is presented separately for mapping.

With root element selected

Without root element selected

The fields repeated between multiple events are listed once:

Each field in each event is presented separately for mapping:

batch_processing_source_fields.png
batch_processing_no_root_element.png

When you select a root element, other top-level objects are available for mapping at the same level as the root.

In this example, other is at the same level as the root element events.key. The other object could be a field that is included with each event.

batch_processing_top_level_fields.png