Skip to main content

Query Catalog action

Available for:

• Data pipeline workflows

• Standalone workflows

• Event workflows

• Alert workflows

• Incident workflows

This action queries a data catalog and maps the matching data to the specified event, alert, or incident fields.

  • Catalog Name

    The catalog to query.

  • Lookup Mapping

    Look up the matching catalog entry for an object. Note that when multiple catalog lookups are defined, they must ALL successfully match in order for the mapping to be applied. You can define up to eight lookup mappings.

    Click Add Another to add an additional mapping. Click the trash can icon to delete a mapping.

    To define a new mapping, fill out the following fields:

    • Field or Tag - The field or tag from the event, alert, or incident to look up.

    • Match Type - The type of match to perform:

      • String - Compare strings. If the catalog field = the event, alert, or incident field, the entry matches.

      • Regex Pattern - Assumes that the catalog field contains regex patterns and matches the input field or tag against these patterns. Matching is always case-insensitive. If there are multiple matches, the first match always wins. See Regex matching below for more information on how to define regex in your CSV.

      • Range - Assumes that the catalog field contains min/max range values and matches the input field or tag against these ranges. If there are multiple matches, the first match always wins. See Range matching below for more information on how to define proper ranges in your CSV.

    • Catalog Field - The catalog field to match against.

  • Apply Mapping

    If the action finds a matching entry, copy data from the catalog field to the event, alert, or incident field. You can specify a default value if a catalog field is undefined.

    Click Add Another to add an additional mapping. Click the trash can icon to delete a mapping.

Regex matching

The following example illustrates how to define regex in a CSV. For more detailed guidance on defining proper regex, refer to this external cheatsheet:

Example 1. 

Here is a sample of a CSV file that uses regex:

hostname, location , cpu
.*\-ny\.ex, abc , 20
.*\-lin\-.*, NYC, 40
.*\-(?:lin|mac)\-.*, Bangalore, 20

In the above example, where the hostname field is represented as a series of regular expressions, an input hostname value of host111-lin-dc1 would match the second row of data (excluding the CSV header).



Range matching

Ranges defined in the CSV file should follow the following format, where min and max are numbers that define the minimum and maximum limits of the range, respectively:

Format

Definition

[min;max]

This range includes both min and max.

]min;max]

This range excludes min but includes max.

[min;max[

This range includes min but excludes max.

]min;max[

This range excludes both min and max.

You may also omit exactly one of either min or max to indicate the lack of a lower or upper limit.

For example, [30;] would match all values greater than or equal to 30, and [;53] would match all values less than or equal to 53. However, [;] is not a valid range, as neither min nor max are present.

Example 2. 

Here is a sample of a CSV file that uses ranges:

hostname, location , cpu
rtr1-type1-ny.rt, abc, [20;90]
host111-lin-dc1, NYC, ]40;55]
lin-mac, Bangalore, [20;]

In the above example, where the CPU field is represented as range of values, an input cpu value of 30 would match the first and third row of data (excluding the CSV header).



Event example

Your monitoring service sends metrics to APEX AIOps Incident Management. These metrics have all the data required by the Metrics API, but they do not specify the service running on each source. You create a catalog called mySources, which specifies each source and its associated service, region, and instance type:

instance-id,          service,      region,      instance-type
i-0z267fg498xc23750,  custLogin,    us-east-1f,  t2.large
i-0w690ji481vb38749,  dbQuery,      us-west-1b,  t3a.xlarge
...

To map the catalog data to your monitored sources, you add a Query Catalog action into your workflow and configure it as follows:

  • Catalog Name: mySources

  • Lookup Mapping:

    • Field or Tag: instance-id

    • Match Type: String

    • Catalog Field: Source

  • Apply Mapping:

    Create the following mappings.

    • Catalog Field: service

      Field or Tag: service

    • Catalog Field: region

      Field or Tag: location.region

    • Catalog Field: instance-type

      Field or Tag: tags.instance-type

When this action processes an event from one of these sources, it adds the catalog data to the event.

Here is an example event that was processed by this workflow:

Event before

Event after

{   
   "description":"CPU spike to 75%", 
   "severity":4,
   "source":"i-0z267fg498xc23750",
   "time":1592322348182,
   "check":"cpu",
   "service":[""],   
}
{   
   "description":"CPU spike to 75%", 
   "severity":4,
   "source":"i-0z267fg498xc23750",
   "time":1592322348182,
   "check":"cpu",
   "service":["custLogin"],
   "location": { "region":"us-east-1f"},
   "tags": { "instance-type":"t2.large"}    
}

Alert example

Suppose that you want to assign a tags.team value for all new alerts based on their services field value. You already have a catalog called servicesToTeams, which maps each service to a corresponding team assignment:

Service,    team
support,    Team A
retail,     Team B
payment,    Team C
...

You can accomplish this by creating a data pipeline alert workflow with a Query Catalog action.

After setting up the trigger, you can configure the Query Catalog action as follows:

  • Catalog Name: servicesToTeams

  • Lookup Mapping:

    • Field or Tag: service

    • Match Type: String

    • Catalog Field: Service

  • Apply Mapping:

    Create the following mappings.

    • Catalog Field: team

      Field or Tag: tags.team

When this action processes an alert from with a relevant service, it adds the related catalog data to the alert.

Here is an example alert that was processed by this workflow:

Alert fields before

Alert fields after

{
 ...
     "description": "Server Health Alert: Connection Issue Detected from IP 192.168.2.20",
     "service": [
         "support"
     ],
 ...
     "tags": {
        "integration_type": [
           "eventapi"
           ]
      },
 ...
}
{
 ...
     "description": "Server Health Alert: Connection Issue Detected from IP 192.168.2.20",
      "service": [
         "support"
     ],
...
     "tags": {
        "integration_type": [
           "eventapi"
           ],
        "team": "Team A"
      },
 ...
}
In this section: