Skip to main content

Advanced Configuration for collectors

APEX AIOps Incident Management Advanced Configuration gives you the ability to create and use a JSON-based local configuration file with your APEX AIOps Incident Management Collector. Incident Management takes that configuration file and merges it with the default config behind the scenes and also makes that configuration visible and accessible through the UI.

With Advanced Config, you can:

  • Create your own JSON config file and install it on the system running your Incident Management Collector. It will be added to the Incident Management at startup.

  • Use the Incident Management UI to create your own JSON config. It can be created via the Incident Management UI and it will automatically get applied to the collector system.

Note

You must install an Incident Management Collector before you can use Advanced Configuration.

Why use Advanced Config?

The Incident Management Advanced Config feature provides the following advantages:

  • More flexibility - Customize the Incident Management Collector and use it to add more data sources, sinks or transforms to fit your needs. You can also use your collectors with a pre-defined set of configurations that you have already set up.

  • Expanded tooling - Run collector plugins and separately run your own more customized Advanced Configuration-based collectors.

  • Greater power - Deploy your collectors in different ways. Instead of deploying one collector at a time through the Incident Management UI, you have the option of deploying your collectors in bulk with corresponding advanced config JSON files.

Advanced Configuration characteristics

The Advanced Config file has the following characteristics:

  • You can have only one advanced config file per Incident Management Collector.

  • If the Incident Management Collector is running, no advanced.json file exists, and you enter JSON in the UI, an advanced.json file is created in the collector/config directory that contains the JSON you created in the UI.

  • The JSON is uploaded to the server only when you start or reload the Incident Management Collector using the start.sh or reload.sh scripts located in the collector/config/scripts directory.

  • The advanced config file in the collector/config directory takes precedence over the UI created config only when the Incident Management Collector is started or reloaded. At startup, the contents of the advanced.json file is uploaded and appears in the Editor pane replacing any code that is already there.

  • On collector startup, what appears in the UI is synchronized with what is local to the collector. If there are JSON statements in the UI and no advanced.json file exists in the collector/config directory, the JSON in the UI is deleted.

Create your Advanced Config file in Incident Management

You can create and enable your own custom advanced configuration file in Incident Management . When you create your Advanced Configuration JSON file in the Incident Management UI, it is saved to the same system where you are running your Incident Management Collector.

Note

You must have Incident Management Collector version 0.20.0 or later to use the Advanced Configuration feature.

  1. If you have not done so, you need install and configure your Incident Management Collector to use Advanced Configuration.

  2. Click Integrations > Ingestion Services > Collectors and click your collector name to display the Advanced Configuration tab. You must have a collector installed to see the Advanced Configuration tab.

  3. Click Configuration > Enable Advanced Configuration to display the JSON edit pane and controls. You can use the following controls for your config file:

    1. Copy - Copies the contents of the JSON editor to the clipboard.

    2. Cancel - Click Cancel to cancel any changes made.

    3. Clear - Click Clear to remove the contents of the editor so you can enter new config information.

    4. Save - Click Save to save any JSON in the editor and apply the JSON config to the advanced.json configuration file co-located with your collector. When you enter JSON statements in the editor, the Save button is enabled. If you enter invalid JSON statements, you can click Save but you will get an error message. Once you save the advanced configuration, you can disable it but you cannot delete it.

  4. Click Save. Clicking Save also sets the Enable Advanced Config toggle to on.

  5. After clicking Save, if you wish to disable the advanced configuration for your collector, set the toggle to disabled. By default, the Enable Advanced Config toggle is set to disabled, but is toggled to Enable when you save a configuration.

Validate your UI-based Advanced Configuration

Once you have set up your Advance Configuration, you can validate that you have set up your advanced config as follows:

  • Navigate to Integrations > Ingestion Services and click the Collectors tab. Review the columnar list titled COLLECTOR, OS, CONFIGURED PLUGINS, and ADVANCED CONFIG. The list displays a checkmark for every Advanced Configuration file that you have set up. Validate that the Advanced Config column displays a checkmark for your collector.

  • Select a collector and click Collector name > Collector Logs to track the logging information for the Advanced Config JSON code.

  • At the command line, go to the system where you installed your Incident Management Collector and look for the advanced.json file in the directory collector/config. If you change JSON statements in this file, the changes also show up in the Advanced Config Editor, but this only happens when you run the Advanced Config /collector/scripts/reload.sh script or you stop and restart the collector with the stop.sh and start.sh scripts located in the /opt/collector/scripts directory.

Create your Advanced Config file at the command line

You can also create your Advanced Config file manually using a text editor. It can be created locally on the collector system and is sent up to the server to be merged with the Incident Management configuration.

When you create your Advanced Config file at the command line, it is displayed in the Advanced Config file Editor.

To create your own file:

  1. In your system, go to the collector/config directory. This directory contains the collector.toml file. Create your advanced config file in this directory.

  2. Open a text editor and enter your JSON statements.

  3. You must name the file advanced.json.

  4. Save your file.

If you delete your advanced.json file at the command line, the corresponding JSON statements are also removed from the Advanced Config Editor in the UI and the Advanced Config toggle is disabled.

Validate your command line-based Advanced Configuration

You can validate that you have set up your advanced.json file as follows:

  • Log into your Incident Management account.

  • Click Collectors and check that the Advanced Config column for your collection displays a checkmark.

  • Click your specific Collector and check that the Advanced Config Editor window displays your JSON statements.

  • Click Collector Logs to track the logging information for the advanced.json file that you created outside the UI.

Configure a source for debugging with logs

When you create your custom JSON configuration, you must have well-formed JSON code. To test the advanced configuration setup, you can use the following short sample JSON in the Advanced Configuration UI Editor. You can cut and paste the code into the UI and test it. This sample may be of interest to developers who want to collect large amounts of data in a local log for debugging purposes. The log file is named collector.log and is located the log directory on the system running the Incident Management Collector.

You can also try the longer Advanced Configuration Nginx sample either in the UI or as a manual advanced.json file.

In the following JSON sample, the manual_metrics_console_sink receives console metrics from moogsoft_common_tag_cardinality and sends them to an internal Incident Management endpoint. The moogsoft_common_tag_cardinality transforms the the metrics and provides some limits on the high volume of data. However, please note that it generates data every 15 seconds and could create very large log files.

{
  "sinks": {
    "manual_metrics_console_sink": {
      "inputs": [
        "moogsoft_common_tag_cardinality"
      ],
      "type": "console",
      "encoding" : {
        "codec" : "json"
      }
    }
  }
}

Note

This sample generates a large number of log entries. Make sure and remove or disable this test sample after using it, if you want to avoid generating a significant amount of log data, you may need to disable advanced configuration and delete the collector.log file after testing your advanced configuration set up.

Advanced Config operations

You can only enable or disable the Advance Config in the Incident Management UI by toggling the Enable switch on and off. However, you can reload, start, and stop your Incident Management Collector and the associated advanced config file at the command line.

Reload Advance Config

The reload script allows you to make changes to the advanced.json file and send those changes to the server. They will appear in the UI editor. To run the script:

  1. Go to your CLI and locate the reload script in the collector directory and run: 

    <collector_home>/collector/scripts/reload.sh

  2. Check the Incident Management UI Advanced Config Editor to see if your changes were uploaded to the server.

Start and Stop Advanced Config

To start or stop the Incident Management Collector and by association the Advanced Config file, you can do the following:

  1. To stop the collector, open a CLI and enter: 

    <collector_home>/collector/scripts/stop.sh 

<collector_home>/collector/scripts/start.sh <collector_home>/collector/scripts/reload.sh

  2. To start the collector, open a CLI and enter:

    <collector_home>/collector/scripts/start.sh

  3. If you installed your collector as a root user, you can run systemctl status to make sure your collector is inactive or active. The systemctl status command only works when you install your collector as a root user.

For more information about starting and stopping the collector, see the Collector Operations topic.

In this section: