Configure the Email LAM

The Email LAM allows you to retrieve email messages from mail servers using JavaMail API and send them to Moogsoft AIOps as events.

You can install a basic Email integration in the UI. See Email for integration steps.

Configure the Email LAM if you want to configure custom properties, set up high availability or configure advanced options that are not available in the UI integration.

Before You Begin

Before you configure the Email LAM, ensure you have met the following requirements:

  • You have command line (SSH) access to the server where the Email LAM is installed.

  • You know the details of each mail source you want to target (host name, port, username and password, name of messages folder).

  • You know the protocol used by each of your mail servers: IMAP, IMAPS, POP3, or POP3S.

  • If your mail servers use SSL (POP3 or POP3S) you know the file names and locations of the SSL keys and certificates.

  • The port for each mail server is open and accessible from Moogsoft AIOps.

  • You know whether the body of the incoming email messages contain JSON.

  • If you are using the Email integration to connect to Gmail, you must configure the Gmail account to allow access for less secure apps. See the Google Help Center for more information.

Note

The Email LAM does not support Outlook 365. Microsoft do not recommend configuring Outlook 365 with IMAP or POP. See Microsoft support information for more details.

If you are configuring a distributed deployment refer to High Availability Overview first. You will need the details of the server configuration you are going to use for HA.

Configure the LAM

Edit the configuration file to control the behavior of the Email LAM.You can find the file at $MOOGSOFT_HOME/config/email_lam.conf.

See the Email LAM Reference and LAM and Integration Reference for a full description of all properties. Some properties in the file are commented out by default. Uncomment properties to enable them.

  1. Configure the connection properties for each target email source:

    • protocol: IMAP, POP3, IMAPS, or POP3S.

    • host: IP address or host name of the mail server.

    • port: Port of the mail server.

    • folder_path: Name of the folder containing the email messages, for example INBOX.

    • username: Username of the account used to connect to your mail server.

    • password or encrypted password: Password or encrypted password of the account used to connect to your mail server.

  2. Determine how to treat messages for each target:

    • retrieve: Whether to receive all email messages or only unread messages.

    • retrieve_filter: One or more filters to limit the email messages to retrieve.

    • mark_as_read: Marks unread emails as read.

    • delete_on_retrieve: Whether to delete email messages on retrieval.

    • remove_html_tags: Whether to remove HTML tags from email messages.

    • treat_body_as_json: Decodes the email body into a JSON object and makes it available for mapping.

  3. Configure the LAM behavior for each target:

    • num_threads: Number of worker threads to use when processing events.

    • event_ack_mode: When Moogfarmd acknowledges events from the Email LAM.

    • request_interval: Length of time to wait between requests, in seconds.

    • max_retries: Number of times the LAM attempts to reconnect after connection failure.

    • retry_interval: Length of time to wait between reconnection attempts, in seconds.

    • recovery_interval: Length of time to wait between requests, in seconds, when the LAM re-establishes a connection after a failure.

    • max_lookback: Period of time for which to recover missed events, in seconds, when the LAM re-establishes a connection after a failure.

    • timeout: Length of time to wait before halting a connection or read attempt, in seconds.

    • javamail_debug: Enables JavaMail debug mode.

  4. Configure the SSL properties for each target using IMAPS or POP3S protocol:

    • disable_certification_validation: Whether to disable SSL certificate validation.

    • path_to_ssl_files: Path to the directory that contains the SSL certificates.

    • server_cert_filename: Name of the SSL root CA file.

    • client_key_filename: Name of the SSL client key file.

    • client_cert_filename: Name of the SSL client certificate.

    • ssl_protocols: Sets the allowed SSL protocols.

  5. If you want to connect to your Email system through a proxy server, configure the host, port, user, and password or encrypted password properties in the proxy section for the target.

  6. Optionally configure the LAM identification and capture logging details:

    • name: Maps to $Laminstancename, so that the agent field indicates events Moogsoft AIOps ingests from this LAM.

    • capture_log: Name and location of the LAM's capture log file, which it writes to for debugging purposes.

  7. Optionally configure severity conversions. See Severity Reference for further information and "Conversion Rules" in Data Parsing for details on conversions in general.

  8. Optionally configure the process logging details:

    • configuration_file: Name and location of the LAM's process log configuration file. See Configure Logging for more information.

Example

You can configure the Email LAM to retrieve messages from one or more sources. If you use more than one mail server or multiple email folders on a single server, configure multiple targets according to the example.

The following example demonstrates a configuration that targets two email sources. For a single source comment out the target2 section. If you have more than two sources, add a target section for each one and uncomment properties to enable them.

monitor:
{
    name                                        : "Email  Monitor",
    class                                       : "CEmailMonitor",
    request_interval                            : 60,
    max_retries                                 : -1,
    retry_interval                              : 60,
    targets:
    {
        target1:
        {
            protocol                            : "IMAPS",
            host                                : "imap.gmx.com",
            port                                : 993,
            folder_path                         : "INBOX",
            username                            : "support@gmx.com",
            password                            : "93pm73xn",
            retrieve                            : "UNREAD",
            retrieve_filter: 
            {
                to              : [ "support@moogsoft.com", "support1@moogsoft.com" ],
                from            : [ "abc@xyz.com", "pqr@xyz.com" ],
                #recipient      : [ ],
                subject         : [ "Alert", "Event" ],
                #body           : ""  
            },
            mark_as_read                       : false,
            delete_on_retrieve                 : false,
            remove_html_tags                   : true,
            treat_body_as_json                 : false;
            disable_certificate_validation     : true,
            #path_to_ssl_files                 : "config",
            #server_cert_filename              : "server.crt",
            #client_key_filename               : "client.key",
            #client_cert_filename              : "client.crt",
            #ssl_protocols                     : [ "TLSv1.2" ],
            num_threads                        : 5
            event_ack_mode                     : "queued_for_processing",
            request_interval                   : 60,
            max_retries                        : -1,
            retry_interval                     : 60,
            timeout                            : 120,
            #javamail_debug                    : true,
            retry_recovery:
            {
                recovery_interval              : 20,
                max_lookback                   : -1
            }
         },
         target2:
         {
             protocol                          : "IMAPS",
             host                              : "imap.mail.yahoo.com",
             port                              : 993,
             folder_path                       : "INBOX",
             username                          : "support@yahoo.com",
             encrypted_password                : "qJAFVXpNDTk6ANq65pEfVGNCu2vFdcoj70AF5BIebEc=",
             retrieve                          : "ALL",
             mark_as_read                      : true,
             delete_on_retrieve                : false,
             remove_html_tags                  : true,
             treat_body_as_json                : false;
             disable_certificate_validation    : false,
             path_to_ssl_files                 : "config",
             server_cert_filename              : "server.crt",
             client_key_filename               : "client.key",
             client_cert_filename              : "client.crt",
             ssl_protocols                     : [ "TLSv1.1, TLSv1.2" ],
             num_threads                       : 5
             event_ack_mode                    : "event_processed",
             request_interval                  : 60,
             max_retries                       : 20,
             retry_interval                    : 120,
             timeout                           : 180,
             #javamail_debug                   : true,
             proxy:
             {
                 host: "localhost",
                 port: 8080
                 user: "John.Doe",
                 password: "Password123"
                 #encrypted_password: "ieytOFRUdLpZx53nijEw0rOh07VEr8w9lBxdCc7229o="
             },
             retry_recovery:
             {
                 recovery_interval             : 20,
                 max_lookback                  : -1
             }
          }
     }
},
agent:
{
    name                                       : "Email",
    capture_log                                : "$MOOGSOFT_HOME/log/data-capture/email_lam.log"
},
log_config:
{
    configuration_file                         : "$MOOGSOFT_HOME/config/logging/email_lam_log.json"
},

Configure for High Availability

Configure the Email LAM for high availability if required. See High Availability Overview for details.

Configure LAMbot Processing

The Email LAMbot processes and filters events before sending them to the Message Bus. You can customize or bypass this processing if required. You can also load JavaScript files into the LAMbot and execute them.

See LAMbot Configuration for more information. An example Email LAM filter configuration is shown below.

filter:
{
    presend: "EmailLam.js"
}

Map LAM Properties

Email header properties are mapped by default to the following Moogsoft AIOps Email LAM properties. The overflow properties are mapped to "custom info" and appear under Overflow in Moogsoft AIOps alerts. You can configure custom mappings in the Email LAMbot.

Email Header Property

Email LAM Event Property

Agent Host

$x_mailer

Agent Time

$sent_date

Description

$message

External ID

$message_id

From

$from

Host

$hostname

Manager

$from

Severity

$severity

Signature

$hostname::$subject

Source ID

$hostname

Type

$subject

Email Header Property

Email LAM Overflow Property

Content-Type

$content_type

Message-ID

$message_id

Received

$received

Return-Path

$return_path

X-Client-IP

$hostname

X-Mailer

$x_mailer

X-Originating-IP

$originating_ip

X-Priority

$priority

X-WM-AuthUser

$AuthUser

Start and Stop the LAM

Restart the Email LAM to activate any changes you make to the configuration file or LAMbot.

The LAM service name is emaillamd.

See Control Moogsoft AIOps Processes for further details.

If the LAM fails to connect to one or more email sources, creates an alert and writes the details to the process log. Refer to the logging details for LAMs and integrations for more information.