Configure Search and Indexing

Moogsoft AIOps uses Elasticsearch to provide search and data indexing functions.

You can control the Elasticsearch service using the following service script:

/etc/init.d/elasticsearch [start|restart|stop]

All Elasticsearch logs are stored in following location:

/var/log/elasticsearch/

Index Alerts and Situations

Two tools are used to index alerts and Situations: the Indexer Moolet and the Moog Indexer utility.

Indexer Moolet

The Indexer listens for new alerts and Situations on the Message Bus and indexes them. Moogsoft AIOps indexes alerts and Situations as soon as they are are created or modified so that they are immediately searchable.

You can configure the Indexer in $MOOGSOFT_HOME/config/moolets/indexer.conf using the following parameters:

enable_private_teams

Set to true if you limit team permissions based upon services, Situations, or alerts assigned to the team. The the indexer applies team permissions to the indexes.

If disabled, the Indexer will index all alerts and Situations present in Moogsoft AIOps.

Type: Boolean

Default: False

full_scan_batch_size

The maximum number of alerts or Situations the Indexer scans in each batch. This is useful because it is not possible to load all alerts to the memory at once.

By default the Indexer scans through batches of one thousand alerts or Situations.

Type: Integer

Default: 1000

full_scan_wait

The number of seconds the Indexer waits between batches. This frees up the CPU and memory used to index each batch.

It is set to zero by default so the Indexer will not wait between batches.

Type: Integer

Default: 0

full_scan_at

Determines the exact time when Indexer runs a full scan. This allows you to ensure the accuracy of search data once per day by performing a full reindex. If left empty, the Indexer does not perform a full scan.

Type: Time (HH:mm:ss)

Default: "02:12:35"

full_scan_at_startup

If enabled, the Indexer performs a full scan when it starts. This is useful if you are not using the scheduled scan and only restart Moogfarmd once a week.

Type: Boolean

Default: false

historic_scan_frequency

Determines how frequently the Indexer performs a full scan of both active and historic databases. By default, the Indexer scans both databases every three days.

Type: Integer

Default: 3

By default the Indexer is configured as follows:

# Set to false to disable private teams indexing.
enable_private_teams: false,

# Maximal full scan batch size
full_scan_batch_size: 1000,

# How many seconds to wait between batches (0 not to wait)
full_scan_wait: 0,

# When to run the full scan (HH:mm:ss) leave empty to disable full scan (HH:mm:ss)
full_scan_at: "02:12:35",

# Do we want to run full scan when the moolet starts?
full_scan_at_startup: false

# Scan the historic data once every how many full scans
historic_scan_frequency: 3
Moog Indexer

Before you can run the indexer utility, you must start Moogfarmd with a running Indexer Moolet. The moog_indexer accepts the following options:

Argument

Input

Description

-h,--help  

-

Displays the help text with arguments that can be used with the utility.

-f, --full

-

Scans both the active and historic data. Use this argument if you want data from both databases to be indexed.

-i,--in <arg> 

Integer

Schedule full index to run in a set amount of time (in hours). This can be a decimal. For example, 0.1 = 6 minutes.

-l,--loglevel <arg> 

WARN|INFO|DEBUG|TRACE

Specify the log level to choose the amount of debug output. Defaults to INFO.

-n,--now   

-

Schedules a full index to run immediately.

-r,--report  

-

Request report from on the last performed full scan index. This report will show the status of previous runs within the lifetime of the moogfarmd process and any runs still in progress. If moogfarmd is restarted, the -r argument will not return any data.

Note

If you use Private Teams mode, meaning one or more Roles do NOT have the all_data permission set, then you must run both the initial 'full index' and the 'incremental index crontab' moog_indexer commands with the -p argument. If not, users in one Team will be able to see search results for other Teams.

Tune your MySQL database to ensure indexing runs as quickly as possible. See either the Percona or MySQL websites for information on tuning and optimization.

An output example is shown below:

[root@myhost home]# moog_indexer -r
Got report:
        05/10/17 13:43:06 - Starting full scan
        05/10/17 13:43:06 - Scanning for alerts
        05/10/17 13:43:07 - Scanned: [177] alerts
        05/10/17 13:43:07 - Scanning for situations
        05/10/17 13:43:07 - Scanned: [44] situations
        05/10/17 13:43:07 - Full scan complete
        05/10/17 13:43:22 - Starting full scan
        05/10/17 13:43:22 - Scanning for alerts
        05/10/17 13:43:22 - Scanned: [204] alerts
        05/10/17 13:43:22 - Scanning for situations
        05/10/17 13:43:23 - Scanned: [55] situations
        05/10/17 13:43:23 - Full scan complete

Warning

Before you upgrade to Moogsoft AIOps V6.2.1 or later, remove or disable the crontab jobs for the old indexer utility.

Elasticsearch Details

Elasticsearch runs on port 9200 by default.

To make Elasticsearch available externally and listen on the external host IP address, run the following command:

$MOOGSOFT_HOME/bin/utils/moog_init_search.sh -r

The script updates the Elasticsearch configuration and restarts the service.