Skip to main content

Ingestion methods

Watch a product demo: Ingestion methods in Incident Management ►

APEX AIOps Incident Management supports a number of ingestion methods making it easier to ingest data from a variety of monitoring tools. Depending on the integration you chose, ingested data may take the form of events or metrics. Incident Management centralizes your data and lets you use the features available to add value to the data you're collecting.

The types of ingestion methods are described in the following sections. You can also read Compare Ingestion methods to see tables describing ingestion methods by product name, by Collector and Collector plugin, and by API.

In addition to inbound integrations, Incident Management also includes outbound integrations. Refer to the related topics for more information.

Collector ingestion

The collector ingestion method ingests data through an Incident Management collector agent configured to scrape or receive data from a data source such as New Relic or Pingdom and others. Collectors can be installed on a physical system or a virtual system. The Collector ingestion UI also enables you to set up a local configuration with Advanced Configuration. This allows you to create or use a local JSON-based configuration file for more flexibility in adding data sources.

The following ingestion characteristics also apply to collector plugins.

  • UI-based

  • Supports events and metrics

  • Can be push or pull depending upon the plugin

  • Mapping schema is specified

  • Deduplication key is not required but can be specified

Collector plugin ingestion

Collector plugin-based ingestion extends and simplifies data ingestion for more data sources by offering add-on functionality to collector-based ingestion methods. Collector plugins work in tandem with a collector to make the integration quicker and easier for plugin specific apps. Plugins use product specific agents configured to receive data.

Pre-configured inbound integrations

Incident Management provides a number of pre-configured integrations that help you get data from your application into Incident Management. These integrations provide some UI-based forms and application specific instructions to help you get data from your application into Incident Management. These integrations may be pull or push-based and may also be cloud-based apps. If listed under Integrations > Ingestion Services > Integrations tab, they are supported. A few integrations feature instructions instead of forms but the instructions are provided in the UI. Some example of inbound integrations are:

  • AppDynamics - this pre-configured integration also can be set up using the AppDynamics plugin with a collector.

  • Pingdom - This integration features forms and instructions to help you set-up and ingest data from your Pingdom app.

See the Compare ingestion methods tables for a comparison of these pre-configured ingestion methods.

Create Your Own Integration (CYOI) ingestion

The CYOI ingestion service enables you to ingest data from third-party systems with different schemas and gives you more flexibility over the fields to ingest, as well as the mapping. CYOI Ingestion includes:

  • Pre-configured CYOI - This webhook-based CYOI integration is a user-defined endpoint that ingests JSON objects and converts them to Incident Management events or metrics. CYOI lets you use a webhook configured in an external service to send events or metrics to Incident Management.

  • CYOI API - CYOI API lets you programmatically:

    • Create a custom integration to process multi-event data payloads

    • Use Incident Management documentation to build sample data to test your integration

    • Understand the JSON structure of batch event payloads.

    • Process batch events using the Create Your Own Integration API. You can send data from any tool or service that can POST data in JSON format.

CYOI ingestion has the following characteristics:

  • UI-based

  • Supports events and metrics

  • Mapping schema

  • Deduplication key is not required but can be specified

API-based ingestion

Incident Management includes a set of APIs that enable you to perform data ingestion programmatically using the Incident Management APIs. API-based ingestion has the following characteristics:

  • Include an Events API that supports events integration and a Metrics API that supports metrics integration.

  • Mapping schema

View the API documentation

Supported data ingestion types

Incident Management ingestion methods let you ingest data from a single host system, multiple systems and from a Cloud-based applications into Incident Management.

Cloud-to-Cloud ingestion

The Cloud-to-Cloud method ingests data from a source cloud platform. Incident Management connects directly to another system with provided credentials to ingest events and metrics. This is a pre-configured ingestion method using product specific tools. Cloud-to-Cloud ingestion has the following characteristics:

  • UI-based

  • Supports events and metrics

  • Push-based ingestion

  • Mapping schema is specified

  • Deduplication key is not required but can be customized

Local or host system ingestion

You can install the pre-configured integrations or collectors to to ingest data from a single system, Linux, Mac, and Windows OSs are supported. In most cases, your app should be running on the same host system as the integration or collector. Each individual pre-configured integration or collector contains details about the requirements. See the individual integrations in the Compare ingestion methods tables for a comparison of the ingestion methods and their characteristics.

Pull and push-based ingestion

Pull-based ingestion enables you to configure Incident Management to pull (scrape or poll) data from a third party product (such as Prometheus). Pull-based ingestion has the following characteristics:

  • Only CYOI ingestion is UI-based. Supports events and metrics

  • Mapping schema is specified and deduplication key is not required.

Push-based ingestion enables you to configure a third party product to push to Incident Management . Push integrations are done via webhook. Push-based ingestion has the following characteristics:

  • Only CYOI ingestion is UI-based. Supports events and metrics

  • Mapping schema is specified and deduplication key is not required.

Documented push ingestion

Documented Push ingestions consist of documented instructions for configuring both a third-party monitoring application and Incident Management to transform data from that application and transmit it to Incident Management. Incident Management also offers API-based push integrations. Examples of documented push integrations include Ansible Tower and Azure.

Discover more

In this section: