Skip to main content

Use case walkthrough: Add external data to events ►

This video explains how to Add External Data to Events in Moogsoft Cloud by using a data catalog to facilitate enrichment.

In this quick video, we’ll go over how to configure enrichment in Moogsoft Cloud.

Here’s our scenario. We want to cluster our alerts by the service impacted,

1_Add_External_Data_Take_One.jpg

But the incoming events do not always have the service information.

2_Add_External_Data_Take_One.jpg

We do have a spreadsheet that stores the source to service connections,

3_Add_External_Data_Take_One.jpg

So we want to query by the source in the data catalog, retrieve the service information, and add it to the events.

4_Add_External_Data_Take_One.jpg

It takes 3 steps to make this happen.

First we need to upload the service data to Moogsoft Cloud.

5_Add_External_Data_Take_One.jpg

Next set up a process to look up the catalog for the matching source information, to grab the service.

6_Add_External_Data_Take_One.jpg

Next set up a process to look up the catalog for the matching source information,  and add to events.

7_Add_External_Data_Take_One.jpg

Lastly, cluster alerts by the service field value using the correlation engine.

8_Add_External_Data_Take_One.jpg

Let’s step through the process from beginning to end.

Here’s our data catalog. It’s a csv file that has the Source and Service information.

9_Add_External_Data_Take_One.png

Let’s go to data catalog, and upload this.

Give a name and description other administrators would recognize.

9_Add_External_Data_Take_One.jpg

Here’s the file we just looked at.

10_Add_External_Data_Take_One.jpg

Good.  Looks like our data made it in.

11_Add_External_Data_Take_One.jpg

Now we need to tell Moogsoft Cloud which field to query by.  

For that, we need to setup a workflow.

12_Add_External_Data_Take_One.jpg

This workflow is for events.

We’ll process all incoming events, so we don’t need to set up a trigger. 

13_Add_External_Data_Take_One.jpg

But let’s say you know only the events from a certain data source are missing the service information, then you can set up a trigger so only the applicable events will trigger this workflow. Once an event enters this workflow, we want to query a catalog.

14_Add_External_Data_Take_One.jpg

Pick the catalog you want to reference.. here’s the csv we just uploaded. (select from the catalog name dropdown)Now we are going to map the fields.

15_Add_External_Data_Take_One.jpg

First, we need to tell Moogsoft to query by the source value. All default fields in Moogsoft are available under the base field category, and the source field is one of them. And, source information is also stored in the data catalog, under the source field.

25_Add_External_Data_Take_One.jpg

Next, specify what field value needs to go where. So in our case, we want to retrieve the service information from the data catalog, and feed that value into the Service field which is one of the base fields in Moogsoft Cloud. Also, IF the data catalog has no value for the particular source, we’ll fill in “unknown."

17_Add_External_Data_Take_One.jpg

Once you name the workflow, you can test it. Testing a workflow is easy. Just pull this up and simulate an input here.

18_Add_External_Data_Take_One.jpg

So let’s send in an event with one of the existing sources in the data catalog.

19_Add_External_Data_Take_One.jpg

OK, it didn’t error out.

And now, although the event we sent in only had a source info, now it has the service information.

20_Add_External_Data_Take_One.jpg

Also note that you can configure enrichment programmatically using our APIs.  Consult the Catalog API and Workflow Service API sections in our documentation.

Finally, set up a correlation engine to cluster by service.  We have a separate tutorial that explains how to add a new correlation setting, so consult that for a step by step instruction.  Basically this correlation will process alerts that match the scope filter, and cluster them if they have identical service information into incidents.

21_Add_External_Data_Take_One.jpg
22_Add_External_Data_Take_One.jpg

We’ve done the entire setup, so now the final test.  We are going to send in fifteen events with varying attributes but with the same source information.  If our setup is correct, all events should be enriched with the same service information, and end up being in one incident.  Here we go.

23_Add_External_Data_Take_One.jpg

Here’s an incident.  It has twelve alerts s in it.  And you can verify that each were properly enriched to have the service information. 

24_Add_External_Data_Take_One.jpg

Thanks for watching!