Skip to main content

validateEvent

A Workflow Engine function that validates an event prior to deduplication to ensure that the event can be processed. If it cannot, then an Ingestion Error alert will be raised containing the details of the failed event. This allows an operator to examine the failed event details and take corrective action (ex. adding a workflow to modify the event).

This function is available as a feature of the Add-ons v2.4 download and later.

This function is available for event workflows only.

Back to Workflow Engine Functions Reference.

Validation Process

The following checks are made.

An existence and empty string check are made on the core event fields:

  • source

  • source_id

  • external_id

  • description

  • manager

  • class

  • signature

  • agent

  • agent_location

  • type

  • severity

  • agent_time

Further checks are made on specific field contents:

  • severity - Must be an integer between 0 and 5.

  • agent_time - Must be an integer and if the optional timeDrift parameter is used, the time must be within timeDrift number of seconds forwards or backwards from the evaluation time. If this agent_time is outside the range, the agent_time will be set to "now".

  • signature - Cannot be set to the default value.

  • source - Cannot be set to the default value.

If an event cannot be validated, an Ingestion Error alert will be raised containing details of the failed event in custom_info.

Arguments

Workflow Engine function validateEvent takes the following arguments:

Name

Required

Type

Description

defaultValue

yes

string

A value to use when a core field is invalid.

timeDrift

no

number

The number of seconds forwards or backwards from "now" which the agent_time has to be within.

Example

The following example demonstrates typical use of Workflow Engine function validateEvent.

Given an event with the following details:

{ 	
	    agent 	        : "RESTLAM",
	    signature		: "02d53b8e-c313-4f41-90e4-f298bf6726ea",
	    source_id		: "",
	    external_id		: "",
	    manager		: "RESTLam1",
	    source 		: "10.0.0.3",
	    class 		: "wqtooling",
	    agent_location      : "rest_test.js",
	    type 		: "RestTest",
	    severity 		: 99, 
	    description 	: "",
	    agent_time 		: 20000000000000,
	    custom_info		: {
	    }
    }

Several core fields contain empty strings (source_id, external_id, description), and the severity is invalid.

Using validateEvent would rectify these missing/incorrect fields to:

{
    "agent": "RESTLAM",
    "agent_location": "rest_test.js",
    "agent_time": 20000000000000,
    "class": "wqtooling",
    "custom_info": {},
    "description": "Unknown",
    "external_id": "Unknown",
    "manager": "RESTLam1",
    "overflow": "{\"custom_info\":{},\"LamInstanceName\":\"DATA_SOURCE\"}",
    "severity": 1,
    "signature": "b9c04776-5ff1-4cec-9939-a84b6d9ff6dc",
    "source": "10.0.0.3",
    "source_id": "Unknown",
    "type": "RestTest"
}

If the timeDrift parameter was used, then the agent_time would be corrected to "now":

{
    "agent": "RESTLAM",
    "agent_location": "rest_test.js",
    "agent_time": 1635251365,
    "class": "wqtooling",
    "custom_info": {},
    "description": "Unknown",
    "external_id": "Unknown",
    "manager": "RESTLam1",
    "overflow": "{\"custom_info\":{},\"LamInstanceName\":\"DATA_SOURCE\"}",
    "severity": 1,
    "signature": "b9c04776-5ff1-4cec-9939-a84b6d9ff6dc",
    "source": "10.0.0.3",
    "source_id": "Unknown",
    "type": "RestTest"
}

An ingestion error alert would be raised:

  • If the source event was missing the source or signature fields.

  • If source or signature were empty.

For example:

In this case, the source event contains no source - this will result in an ingestion alert detailing the failed event. Initially the missing source is replaced with the defaultValue, but fails the subsequent validity check.

{ 	
		agent 			: "RESTLAM",
		signature		: "b0f10236-f4b5-45db-b88a-6cf6eb00bc28",
		source_id		: "",
		external_id		: "",
		source 			: "",
		manager			: "RESTLam1",
		class 			: "wqtooling",
		agent_location 	        : "rest_test.js",
		type 			: "RestTest",
		severity 		: 99, 
		description 		: "",
		agent_time 		: 20000000000000,
		custom_info		: {
		}
	}

This results in an ingestion event:

  • custom_info is set to the original event details.

  • severity is set to 1.

  • signature is set to be unique.

  • description is set to indicate to an operator that there has been an ingestion error.

{
    "active_sig_list": [],
    "agent": "RESTLAM",
    "agent_location": "LAM",
    "alert_id": 6386,
    "class": "Moogsoft",
    "count": 1,
    "custom_info": {
        "severity": 1,
        "agent": "RESTLAM",
        "agent_time": 1635252605,
        "manager": "RESTLam1",
        "signature": "7cc5da1e-ac63-4b55-891f-e2e6a643aeaa",
        "description": "Unknown",
        "external_id": "Unknown",
        "source": "Unknown",
        "type": "RestTest",
        "overflow": "{\"custom_info\":{},\"LamInstanceName\":\"DATA_SOURCE\"}",
        "custom_info": {},
        "agent_location": "rest_test.js",
        "source_id": "Unknown",
        "class": "wqtooling"
    },
    "description": "Event Workflows::validateEvent: Ingestion error: , please check custom_info and the farmd log for the original event details (search for 9396ebc6-61f9-4a3d-ba25-489721831e5c in the farmd logfile)",
    "entropy": 1,
    "event_id": 27600,
    "external_id": "9396ebc6-61f9-4a3d-ba25-489721831e5c",
    "first_event_time": 1635252605,
    "int_last_event_time": 1635252605,
    "last_event_time": 1635252605,
    "last_state_change": 1635252605,
    "manager": "Moogsoft",
    "owner": 2,
    "rc_probability": null,
    "severity": 5,
    "sig_list": [],
    "signature": "IngestError::9396ebc6-61f9-4a3d-ba25-489721831e5c",
    "significance": 3,
    "source": "RESTLAM",
    "source_id": "9396ebc6-61f9-4a3d-ba25-489721831e5c",
    "state": 2,
    "type": "IngestError"
}
In this section: