Skip to main content

Create data catalogs

This topic describes how to create catalogs with enrichment data from your environment.

Enrichment provides added flexibility for correlating your alerts into incidents — you can correlate using enrichment data, not just data in the raw ingested events. Enrichment can also make your alerts more informative and easier to troubleshoot.

See also Catalogs API.

What is a catalog?

A catalog is a collection of data from your environment. The simplest way to create a catalog is to generate a CSV from your environment and then upload it to Moogsoft Cloud. The following example illustrates the format.

host, app, aws-region, cluster
ip-172-31-37-159.ec2, music-match, us-west-1, cluster-1
ip-172-23-21-112.ec2, music-maker, us-east-2, cluster-7

Note the following:

  • The first row defines the keys, the following rows define the values to add to individual events

  • After you define a catalog, you must define a Query Catalog action that matches entries in the catalog with new events. In this action, you define the event and catalog fields that the query uses to look up matching documents.

    Lookup fields most commonly specify the source where the event originated. In this example, the host column contains the lookup values. Thus you would specify a key-value pair with the event and catalog field names:

       "Source" : "host"
  • Each lookup must be unique. Thus in this example, a catalog cannot have multiple documents with the same host value .

  • A catalog consists of one or more documents. A document is equivalent to a single row in a CSV.

  • A Query Catalog action can specify multiple lookup fields. In this case, all the lookup values must match for a document to match an event. You might want to do this if (for example) your catalog spans multiple regions and you want to enrich your sources differently depending on the region.

  • When you upload the Catalogs API CSV, it deletes and overwrites any previous enrichment entries in the catalog. If you want to add or remove entries from the catalog, add or remove them from the CSV and then upload. You can also use the to add or delete rows in a catalog.

  • When a lookup results in a match between an alert and a document, Moogsoft Cloud maps the other values in the document with fields in the alert. You specify how this mapping occurs in the Query Catalog action.

  • In this example, three columns map directly to fields in the alert schema:










    The fourth column, cluster, does not have an equivalent in the alert schema, so we will map it to a custom tag: tags:cluster.

  • If your organization stores its infrastructure in a Catalogs API CMDB, registry, spreadsheet, or other centralized repository, the simplest workflow is to publish or export the relevant data to the expected CSV format shown above. You can also use the to push your updates programmatically.

  • The catalog CSV file name should not include dollar-sign ($) characters.

  • Each data catalog you create can store up to 2GB of data. You can create up to ten data catalogs by default. If you want to create more than ten data catalogs, contact Moogsoft support.

Creating a catalog

Do the following:

Define your enrichment data

To add enrichment data, you specify your data in a .csv file and then upload the file to Moogsoft Cloud.

Upload and review your data catalog

After you define your enrichment data, you can upload the CSV and add it to an event workflow. Do the following:

  1. Navigate to Correlate & Automate > Data Catalog and then click Add Catalog.

  2. Setup tab: Select your CSV file and upload.

  3. Catalog tab: Review the catalog data.

Once you upload your catalog, you can include it in an event workflow. See Create event workflows.

Updating a catalog

The Catalog API enables you to create, retrieve, update, and delete catalogs. It also enables you to add and remove individual documents (rows) in a catalog.