Skip to main content

Types of Data to Ingest

Before you configure data ingestion, explore the format and quality of the incoming data from each source. The following are the recommended practices around data ingestion.

Tip 1: Don't Feed Time Series Data


Moogsoft Enterprise is designed to deal with fault data. That means you should ingest events that may be worthy of operator attention. Do not forward continuous metrics (time series data) should into Moogsoft Enterprise. Instead, set up those data sources to send events when performance metrics reach a specific threshold of interest to your operations team.

Tip 2: Feed Only the Complete Set of Data


Don't send events with critical data missing from the event payload. For example fault description or hostname. Here's an obvious example: if the source event data is missing value for the description field, you won't be able to cluster by the "Events with similar descriptions" cookbook.

Tip 3: Pay Attention to Consistency

Make sure to populate data consistently across fields that will be used in any downstream processes. For example, attributes you plan to use for clustering or for maintenance windows filters clearly need to be consistent. Cookbooks are remarkably flexible clustering tools, but always be mindful to feed consistent data for maximum accuracy.

Also, given that Moogsoft Enterprise provides event deduplication based on the same context between alerts, make sure that any subsequent event updates to the original occurrence have the fields consistently populated. Otherwise, you will need a mechanism to backfill subsequent updates based on the initial event occurrence. Additional data processing introduces overhead and can slow down the event processing rate.

Tip 4: Have a Strategy to Identify Missing Data

Define your strategy for capturing alerts with missing data. For example, highlight any alerts with missing data and cluster them into specific situations. Later, an administrator can review and refine the ingestion configuration as needed.