Advanced Configuration sample: Nginx metrics

APEX AIOps Incident Management Advanced Config gives you the ability to create and use a JSON-based local configuration file with your Incident Management Collector. To help you understand and complete a test run of Advanced Configuration, the Nginx Metrics sample provides the JSON code and a walkthrough of the sample statements for ingesting Nginx metrics. Nginx is an open-source web server that is also used for load balancing, HTTP caching, as a reverse proxy, and as an email proxy.

Incident Management supports the installation of a collector using with a local configuration file and makes that configuration visible and accessible through the Incident Management UI.

Install Nginx and an Incident Management Collector

You need to perform a few actions to start using the sample.

Install Nginx. To get started you need to to have an existing NGINX server installed on a Mac or Linux system (the host) or download and install one. For additional information, see the Nginx website. Depending upon your operating system, you can download Nginx in several ways, such as using:
1. sudo apt install on Ubuntu Linux systems
2. brew install nginx for a Mac
You also need to set up a location block in your Nginx configuration file. Nginx provides a default config file called nginx.conf that you can modify. For details, read about location blocks in the Nginx documentation. Add the following lines, under your SSL configuration, to your Nginx config file:
```
location /nginx_status {
         stub_status on; 
    }
```
Test that your Nginx metrics are enabled, by running the following Linux command:
```
nginx -V 2>&1 | grep -o with-http_stub_status_module
```
This command returns http_stub_status_module if metrics are enabled. If metrics are not enabled, you may need to re-install and re-configure Nginx. Refer to Nginx documentation for instructions.
Install a Collector on your Nginx system. For instructions on how to install an Incident Management Collector, see the Install the Incident Management Collector instructions.
Set up your Advance Configuration. For instructions read the Advanced Configuration topic.

Advanced Configuration JSON

The following sample shows the JSON statements for an advanced configuration file that is designed to gather Nginx metrics and push them to your Incident Management instance. You can create this as a file co-located with the Incident Management Collector or paste it into the Advanced Configuration UI.

Note

In the sample below, the source, transform, and sink names are all preceded with “local”. These names can be anything you want, but since the built-in plugins use the format “moogsoft_something_something" you should avoid using a "moogsoft" prefix to avoid name collisions.

Nginx sample JSON code

{
  "sources": {
    "local_nginx": {
      "type": "nginx_metrics",
      "endpoints": [
        "https://<hostname or IP address>/nginx_status"
      ],
      "scrape_interval_secs": 15,
      "namespace": "nginx"
    }
  },
  "transforms": {
    "local_common_tags": {
      "type": "moogsoft",
      "inputs": [
        "local_nginx"
      ]
    },
    "local_common_tag_cardinality": {
      "inputs": [
        "local_common_tags"
      ],
      "type": "tag_cardinality_limit",
      "mode": "exact"
    }
  },
  "sinks": {
    "local_common_sink": {
      "inputs": [
        "local_common_tag_cardinality"
      ],
      "healthcheck": {
        "enabled": false
      },
      "type": "prometheus_remote_write",
      "endpoint": "https://api.moogsoft.ai/v1/integrations/metrics/prometheus-remote-write?fqm-override=local:integration:collector",
      "request": {
        "concurrency": 1,
        "rate_limit_num": 9223372036854776000
      },
      "auth": {
        "strategy": "basic",
        "user": "anyusername",
        "password": "fake12ac-6231-4354-ap4i-key0387eda8d"
      }
    }
  }
}

Nginx code walkthrough

The following sections provide a walkthrough of the JSON in this sample.

Sources

In the sources section of this JSON sample, local_nginx is a Vector source JSON object that scrapes metrics from the data source https://<hostname or IP address>/nginx_status. This endpoint is scraped every 15 seconds for metrics. The time is specified by the key value pair scrape_interval_secs : 15 sec. The namespace is optional and by default is set to nginx. The value you specify for namespace is prepended to all of the metric names, such as nginx_connections_active.

Transforms

Inside the Transforms object, we have the following name/value pairs:

local_common_tags:

Takes the metric events from "local_nginx" and applies the required Incident Management tags to the metric. In this sample, the key value pair "type": "moogsoft" applies Incident Management tags to the metrics. The resulting object then undergoes another transformation type.

local_common_tag_cardinality:

Takes the metric events from local_common_tags and provides limits on the number of tags. As shown in lines 19 and 23, when creating a production example, you may wish to add the tag_cardinality_limit transform, which protects the Prometheus endpoint from excessive tags. This is recommended for sources where you are unsure of or cannot control the number of tags being generated.

Sinks

The local_common_sink takes the metrics events from the local_common_tag_cardinality transform and sends the data to the Incident Management Prometheus remote write endpoint where it may undergo additional processing or outputting for visualization.

Request

The request key value pair specifies a concurrency value of 1 (effectively disabling it), to ensure that the metrics and events are sent in the order they were received and cannot be sent out of order. The request block also shows a rate_limit_num key value pair that is set to a value of 9223372036854776000. This value is the highest number value you can specify. This specified value effectively removes any limit on the number of requests sent to the endpoint. A low number would limit the number of requests received and a high number allows more throughput.

Healthcheck

When enabled or set to true the healthcheck performs the necessary checks to make sure the downstream service is accessible and ready to accept data. For this small sample, the healthcheck is disabled by setting the healthcheck value to false.

Auth

The Auth section of this JSON example performs a simple authorization check specified by the key value pair strategy: basic. Basic authorization requires a user name and password combination. The user name may be any alphanumeric name, as it is ignored. You can also just use the default. However, the password must be a valid API key. This API key is also used for user name.

Validate your Nginx sample

You can validate that you have set up your Nginx advanced.json file as follows:

Log into your Incident Management account and navigate to Integrations > Integration Services.
Click Collectors and check that the Advanced Config column for your collection displays a checkmark.
In the Collectors tab, click your specific Incident Management Collector. In the Advanced Configuration tab, check that the Advanced Config Editor window displays your Nginx JSON code.
Click Collector Logs to track the logging information for the advanced.json file that you created outside the UI.
Check the Incident Management Metrics page to see if you are getting metrics from Nginx. For example, you might look for nginx_connections_active in the Incident Management metrics dashboard.

In this section: