After you complete this lab, you will know how to:

  • Configure Moogsoft Enterprise to receive data from a REST-compliant web service using the REST LAM.

  • Test to make sure data can be ingested successfully.

  • Upgrade your security configuration.


  • You are familiar with enterprise IT operations and high availability (HA) concepts.

  • You are comfortable using Linux, Javascript, and Javascript Object Notation (JSON).

  • You are familiar with Moogsoft Enterprise installation, operation, and system architecture.

Before You Begin

Request your lab instance here. After filling out the form, you will receive an email from Moogsoft University with information you need to access your virtual machine. Keep in mind that it may take up to two days before you get your lab instance.

While you wait for your instance, review these materials to prepare for the exercise:


You are a member of a team preparing to install and configure Moogsoft Enterprise version 8.0 for Catchlight Medical Records, Inc. (CMR), a cloud-based electronic medical records company. Your role on the Moogsoft Enterprise implementation team is to prepare for data ingestion while other members of your team conduct discovery sessions with the Catchlight operations division group leads and operators.

In this lab you’ll be configuring data ingestion using JSON configuration files and Javascript programs and utilities (ie you will be using the back end and not the user interface). You’ll verify data ingestion and event mapping using command line output, logs, and the user interface.

Lab Instructions

As you proceed, ask yourself:

  • What task do I need to accomplish next?

  • Which file(s) should I edit to accomplish that task?

  • Where are the files located, and what are the file naming conventions?

  • Which sections of each file do I need to edit?

  • What settings should I use?

  • How can I test that I’m getting the results that I want?

With each task, try to answer these questions for yourself. If you don't know how to complete a step on your own, the Example sections provide suggested code or user interface procedures.

Log In and Review Files

Get started by logging in and examining the file structure for your Moogsoft Enterprise installation.

  1. One of your team members has installed Moogsoft Enterprise on an HA installation comprising three Amazon Web Services virtual machines--a primary server, a secondary server, and a redundancy server. Your team will be configuring this installation for Catchlight user acceptance testing (UAT). The system configuration files have been readied for high availability, but automatic failover hasn't been enabled yet. To start with, you will configure data ingestion on the primary server. Your work will be copied to the secondary server later in the implementation process.

    To get set up for data ingestion, go to the URL of your virtual machine (the primary HA server) and log into the Moogsoft Enterprise user interface as an administrator using the credentials 'admin/admin.' Keep the user interface open so you can check the results of your work later.


    You'll need to click through security warnings to get to the login page. If you are using Chrome with the Mac OS, you may not be able to click through the security warnings, but you can click anywhere on the page and type 'thisisunsafe' and hit return to get to the login page.

  2. Using a terminal program, log into the Linux operating system of your virtual machine using ssh and the credentials you were given. Type sudo su to change to root.

  3. Change to the $MOOGSOFT_HOME directory to explore the file structure for the Moogsoft Enterprise installation. At the top level, there are several directories. Review the contents of each.

    • /config contains Moogsoft Enterprise configuration files in JSON format, including logging and moolets subdirectories. Notice that there are a number of *_lam.conf files. These are default Link Access Module (LAM) data adapter configuration files for various data sources. The directory also includes moog_farmd.CMR.conf, a configuration file for the Moogfarmd core processing service already customized for Catchlight.

    • /bin contains executable shell programs, including files to invoke the LAMs.

    • /bots includes two subdirectories, lambots and moobots. Examine the lambots subdirectory. It contains Javascript files that work in tandem with the various LAM executable files to do additional data processing. (Notice the difference in file naming conventions for this directory compared with the config directory.)

    • /contrib contains some helper utilities, including BotUItility.js which provides standard functions to assist in preparing LAMbot Javascript files.

    • /etc contains general reference and configuration files. The cots subdirectory (an acronym for "commercial-off-the-shelf") contains configuration files for MySql, Nginx, RabbitMQ, and Apache Tomcat open source software.

    • /lib contains Moogsoft Enterprise Java program files.

  4. There are two more directories outside the $MOOGSOFT_HOME directory (or /usr/share/moogsoft) which you should review.

    • /var/log/moogsoft contains some of the log files generated by Moogsoft Enterprise.

    • /etc/init.d contains service wrapper files for Moogsoft Enterprise services. You’ll need to use the moogfarmd script to start and stop the Moogsoft Enterprise alert processing engine. restlamd is the service script for the default REST LAM.

  5. Look at the contents of rest_lam.conf. This JSON file contains the default settings for configuring input from REST (Representational State Transfer) web services, the most common type of input for Moogsoft Enterprise.

  6. Review the contents of the RestLam.js LAMbot Javascript file.

  7. Review the contents of the restlamd service shell script and make note of which parameters you can set in the file.

  8. Check your understanding:

    • Where is the REST LAM configuration file located, and what is it called?

    • Where is the REST LAMbot helper file located, and what is it called?

    • Where is the REST LAM service wrapper script located, and what is it called?

Review Services and HA Configuration

Now that you’ve examined the file structure, check which Moogsoft Enterprise components are currently running.

  1. Check the running services. You should see the UI services (Apache Tomcat and Nginx) the core processing service (Moogfarmd), the database service (MySql), and the messaging service (RabbitMQ).

  2. Use the Moogsoft Enterprise HA control utility to verify the HA cluster name. You’ll need this information to identify log files and to configure the REST LAM later.

  3. Check your understanding:

    • What is the name of the HA cluster for your virtual machine?

    • Is the REST LAM service currently running in your installation?

Route the Data

You’ve done your quick tour. As a new member of the team, you know that you have a lot more to learn about the structure and functioning of Moogsoft Enterprise, but you are ready to take a stab at ingesting data using the default REST LAM configuration files.

  1. As a best practice, make a copy of each of the default REST LAM files before you begin customizing them for Catchlight. For example, make a copy of RestLam.js and rename it RestLam.CMR.js, and copy rest_lam.conf and rename it rest_lam.CMR.conf. You’ll also need to make a Catchlight version of the REST LAM service script (where is it located?).

  2. Now that you have created the Catchlight-specific files, you need to specify how they will be used for routing event data. Carry out each of the following steps to customize the default settings. Try to figure out what to edit on your own before looking at the example code.

    1. Change the listening port in the JSON configuration file to 8222.

    2. Uncomment the HA section of the configuration file. The REST LAM will inherit its cluster name from the system configuration files, but as a best practice enter a setting for 'cluster' explicitly in this section using the cluster name given by ha_cntl. Choose an HA group name to identify the REST LAM components. Finally, to avoid queue naming conflicts, give your primary REST LAM an instance name that distinguishes it from the secondary REST LAM that will be added later.

    3. Examine the Mapping section in the configuration file. You can edit these entries to map the fields in incoming data to Moogsoft event fields, though that is outside the scope of this lab. Leave the entries set to their default values for now.

    4. Change the 'presend' in the configuration file Filter section to point to the Catchlight LAMbot. This setting will cause events to be routed to the LAMbot for further processing before they are released to the message bus.

    5. In the service script, update the 'SERVICE_NAME' and 'CONFIG_FILE' settings.

  3. That suffices for setting up the basic routing of arriving event data. Next you need to start the REST LAM service so that Moogsoft Enterprise can start ingesting data.

    1. Add the service script to the list of system startup services using chkchonfig –add.

    2. Start the service.

  4. Check your understanding:

    • Though they are useful for customizing data parsing, customizing data mapping, and performing other event processing chores, including LAMbots in the data workflow is optional. Which of the settings above would you comment out if you did not want to use the REST LAMbot?

Test Data Ingestion

If you’ve done this configuration successfully, the Catchlight REST LAM should be ready to consume data. There are a number of ways you can check your work. Try each of the methods below. You can look at the examples if you are not sure of the syntax for a command, but realize that you need to be comfortable knowing when and how to apply these methods on your own.

  1. Check to make sure that the service is running.

    1. Verify the log file for the REST LAM has been created. The name for the log file is based on the HA naming scheme, and should have the form cluster_name.group_name.instance_name.log. Check its contents using tail or cat. (Where is the log file located?)

    2. Use systemctl to check the service script status. The service should be reported as "Active:active (running)."

    3. Use netstat to verify that the service is listening to the configured port.

    4. Use ha_cntl to check which Moogsoft Enterprise components and instances are running.

    5. Check self monitoring by going to the user interface Settings page.

  2. Verify that the REST LAM is active by sending a curl command to port 8222.

  3. Check that the LAM is functioning by sending some dummy data through the system.

    1. Change the log level in the service script to 'DEBUG' so that you can inspect the output of the curl command that you’ll use to send the dummy data.

    2. Send a dummy event to the port the REST LAM is listening to using a curl command. You should receive a success message saying the REST LAM processed 1 event.


      The default REST LAM expects to receive Moogsoft Enterprise event data fields in JSON format. Of course, it's likely that Catchlight raw data doesn't match the Moogsoft data schema. After you set up and test the REST LAM, you’ll need to map Catchlight data fields to Moogsoft events.

    3. Examine the log output to verify that the event data was received and processed.

    4. Check the user interface to verify that an event was received. Examine the details of the alert that was created by Moogsoft Enterprise from the dummy data.

  4. Check your understanding:

    • Name three ways you can verify that a LAM is running.

    • Find the dummy event value for 'source' in the Open Alerts view of the user interface. What is its column title in the alert list?

Set Up a Data Capture Log

You have verified that the REST LAM you configured is working and you can use it to ingest data into Moogsoft Enterprise. However, there are still a few chores to do before inspecting and mapping Catchlight’s data. For example, you will want a convenient way to inspect incoming raw data that doesn’t match the default format.

  1. Change the log level for the REST LAM service script from DEBUG back to WARN so the log size stays manageable, and then reload service daemons.

  2. Edit the REST LAM configuration file to activate the data capture log, and name the log file appropriately. Note the file path so you can find the log later.

  3. Restart the REST LAM service so the configuration change will take effect.

  4. Resend the dummy data.

  5. Inspect the REST LAM service log and the data capture log to verify that the changes you made took effect and the dummy data was captured.

  6. Check the user interface to verify that the 'count' value for the open alert has increased to 2. Since the two dummy events that you sent were the same, Moogsoft Enterprise deduplicated them into the same alert.

  7. Check your understanding:

    • Why do you need to set up a data capture log, instead of just examining the incoming Catchlight data in the user interface?

Secure Data Transport

Before ingesting Catchlight data you need to enable encryption so data is protected as it transits from Catchlight's web monitors to Moogsoft Enterprise. While you’ve been preparing the basic configuration files, your colleagues have been discussing data security with the Catchlight staff. They have decided to offload security chores to the Nginx front end and set it up as a reverse proxy server. With that architecture, incoming data will "see" only the Nginx service. Nginx will forward the data to the appropriate Moogsoft Enterprise clusters and instances and handle SSL encryption.

  1. The Nginx configuration file is located in $MOOGSOFT_HOME/etc/cots/nginx. Navigate there and edit the Moog proxies configuration file to add forwarding to the Catchlight REST LAM. Set the 'proxy_pass' value to the URL of your virtual machine and the listening port for the REST LAM, and choose an appropriate location name.

  2. Use the nginx -t command to test the configuration file syntax after your changes.

  3. Restart the Nginx service and check to make sure it is running.

  4. Send a dummy event to test the reverse proxy configuration. The format of the receiving URL for the curl command is now https://your-machine-URL/your-proxy-location using the secure 'https' protocol.

  5. Check the capture log and user interface to verify that the dummy event arrived.

  6. Next make sure that you can reach the Nginx reverse proxy from another computer by sending a curl command to the REST LAM URL. Because you are not using security certificates, you should continue to use the --insecure option, even though you are using the secure 'https' protocol. You can also navigate to the REST LAM URL from a browser. Either way, you should see the same success message.

  7. Check your understanding:

    • Is the dummy event data encrypted when it reaches the Nginx server?

    • Is the dummy event data encrypted between the Nginx server and the REST LAM?


Congratulations! You’ve configured secure data ingestion for the Catchlight UAT implementation using the REST LAM. Your team is grateful for your hard work. To obtain a Proof of Completion, take a screenshot of your data capture log showing your URL and email it to Moogsoft University.