Try It Yourself: Filter Events

This lab is one of a series of Workflow Engine Labs. In this lab, you will:

  • Use the Workflow Engine to keep and process only the events you want to see.

  • Construct a workflow to stop events with a severity of 'Clear' unless they match a pre-existing alert.

Examine the Data

Load sample data into your lab instance and decide how you want to process it.

  1. Prepare your lab instance by closing any open alerts and all but one Situation. If there is no open Situation, generate one manually. (See the Mini Lab for instructions on how to do this.)

  2. Go to the Collaborate tab of your Situation’s Situation Room. Type @moog get_lab_events Clear into the comment box to bring data into your instance.

  3. Scroll up and down the Open Alerts view. Inspect the alert severities and other details. There seem to be a large number of alerts with a severity of 'Clear'. You decide that you don’t want to process the incoming clear events from your hardware monitor. You won’t need to take action on them, and stopping them will reduce noise and improve Moogsoft AIOps performance.

    You want to stop the clear events from processing, but there is one exception. You want to keep any clear events that match previous events—that is, events that will be deduplicated and added to already existing alerts. This would happen if, for example, a hardware problem generated an event with a higher severity, but then resolved itself and generated a follow-up event with a severity of 'Clear'.

Define a Workflow

Use the Workflow Engine to stop the processing of clear events.

  1. Go to Settings>Automation>Workflow Engine. From this page you can define four types of workflows: alert, enrichment, event, and Situation. Go to the Event Workflows tab.

  2. Click on 'Add Workflow' in the upper right.

  3. You will see the workflow definition screen. On the left is a column where you can add multiple workflow actions which will be executed in order. The first action, 'Delay' is already present. It holds incoming events for the given number of seconds before passing them to the next action for processing. Leave it set to 0.

  4. On the upper right is a slider where you can make your workflow active or inactive. Leave it set to 'Active'.

  5. In the right-hand workflow definition pane, enter 'Don’t Process Clear Events' in the workflow name text box.

  6. Describe your workflow so that other users (and you in the future) know what it does. Enter 'Don’t process events with a severity of 'Clear' unless they will be added to an already existing alert' in the description text box.

  7. The third section of the workflow definition pane lets you design an entry filter for your workflow. The workflow will only process events which match the criteria in the entry filter. To improve performance, make the entry filter as specific as possible. Click on 'Edit' and then 'Add Clause'.

  8. Using the drop-down boxes on the right, construct the filter '"severity" = "Clear"'.

  9. Click 'Apply' and then 'Done.'

Add Workflow Actions

Add an action to your workflow to stop processing the clear events.

  1. Click on 'Add Action' and look at the action definition pane on the right. There are four sections: Action Name, Function, Arguments, and Forwarding Behavior. The Arguments section changes depending on the function you select.

  2. Name your action 'Stop unique clear events'.

  3. Before you choose a function, examine the Forwarding Behavior section. It looks as if you want to 'Stop All Workflows' for the clear events. However, read the the text in this section carefully and you will see that 'Stop All Workflows' will not stop further processing in every case.

    At the bottom of the Forwarding Behavior section there is a note that functions that return 'True' continue processing subsequent actions and workflows. This means that to stop further processing you need to choose 'Stop All Workflows' and you also need to choose a function that will yield a value of 'False' for the events you want to stop. Select 'Stop All Workflows'.

  4. Go to the dropdown list in the Function section and locate the function that identifies events which match existing alerts, 'willDeduplicateAlert'. (You can read descriptions of all of the functions in the Moogsoft documentation.) This function returns 'True' if an event will be deduplicated and added to an existing alert. Otherwise it returns 'False.' Thus it will return 'False' for events without matching pre-existing alerts. Choose 'willDeduplicateAlert' from the dropdown list. The combination of the entry filter, this function, and the 'Stop All Workflows' setting will stop Moogsoft AIOps from processing clear events unless they match pre-existing alerts.

  5. Click on 'Save'.

Test Your Workflow

Test your workflow by resending the event data.

  1. Reset your instance by closing the open alerts and all but one Situation. Note that because there are so many alerts, you will need to scroll to the bottom of the alert list before the selection check box becomes active.

  2. From the comment box on the Collaborate tab of your remaining Situation, resend the event data using the @moog get_lab_events Clear ChatOps command. You should see many fewer alerts, and only three clear alerts.