Skip to main content

Create data catalogs

This topic describes how to create catalogs with enrichment data from your environment.

Enrichment provides added flexibility for correlating your alerts into incidents — you can correlate using enrichment data, not just data in the raw ingested events. Enrichment can also make your alerts more informative and easier to troubleshoot.

What is a catalog?

A catalog is a collection of data from your environment. You can generate a catalog in APEX AIOps Incident Management by uploading a file in comma-separated value (CSV) format, or by using the Catalogs API. The following example illustrates the format to use for uploading a CSV file:

host, app, aws-region, cluster
ip-172-31-37-159.ec2, music-match, us-west-1, cluster-1
ip-172-23-21-112.ec2, music-maker, us-west-1, cluster-7

The first row contains field names, and each following row contains a single catalog entry.

One of the fields, typically specifying the source where events originate, serves as a lookup field. In this example, the host field contains the lookup values, which must be unique. The values in the other fields are available to add to events.

By default, you can create up to 100 data catalogs with a maximum 5GB compressed file limit. If you want to create more than 100 data catalogs or your file is over the 5GB amount, contact Support.

Create a catalog

To create an enrichment data catalog, do the following:

  1. Prepare a CSV file of enrichment data which includes a lookup field, as described above.

    If your organization stores its infrastructure in a CMDB, registry, spreadsheet, or other centralized repository, the simplest process is to publish or export the relevant data to a CSV file. You can also use the Catalogs API to create, retrieve, update, and delete catalogs and individual catalog entries.

    The catalog CSV file name should not include dollar-sign ($) characters.

  2. Upload your CSV file:

    1. Navigate to Correlate & Automate > Workflow Engine and click the Enrichment Data Catalogs tab. Click Add Catalog.

    2. Provide a name and description for your catalog.

    3. Choose the CSV file to import. Click Save.

    4. Your uploaded catalog data will appear. Review the data to make sure it imported successfully.

Enrich events using a catalog

Once you upload your catalog, you can include it in an event workflow. See Use workflows to enrich event data.

Define an event workflow with a Query Catalog action that matches incoming events to catalog rows. In this action, you map an Incident Management field (typically source) to the catalog's lookup field, which you specify, and then map catalog fields you choose to Incident Management event fields or custom tags.

Update a catalog

When you upload a CSV file, it deletes and overwrites any previous entries in the catalog. If you want to add or remove entries from the catalog, add or remove them from the CSV file and then upload. You can also use the Catalogs API to add or delete individual catalog entries.

In this section: