Configure the Email LAM

The Email LAM allows you to retrieve email messages from mail servers using JavaMail API and send them to Moogsoft Onprem as events.

You can install a basic Email integration in the UI. See Email for integration steps.

Configure the Email LAM if you want to configure custom properties, set up high availability or configure advanced options that are not available in the UI integration.

Important

For Gmail users

As of September 30th, 2024, Google has removed the ability to authenticate Gmail using your native username and password.

If you are using Gmail with your Email LAM integration, you must do the following to ensure that your integration remains functional:

Open the configuration file for your LAM by following the steps in the Configure the LAM section below.
For the password field, generate and paste a Google app password in this field instead of your Gmail password.
For detailed instructions on generating an app password, see the Google documentation. You must have two-factor authentication enabled for your Google account in order to enable app passwords.
Save the configuration file.
Restart the Email LAM to activate the changes made.

Before You Begin

Before you configure the Email LAM, ensure you have met the following requirements:

You have command line (SSH) access to the server where the Email LAM is installed.
You know the details of each mail source you want to target (host name, port, username and password, name of messages folder).
You know the protocol used by each of your mail servers: IMAP, IMAPS, POP3, or POP3S.
If your mail servers use SSL (POP3 or POP3S) you know the file names and locations of the SSL keys and certificates.
The port for each mail server is open and accessible from Moogsoft Onprem.
You know whether the body of the incoming email messages contain JSON.
If you are using the Email integration to connect to Gmail, you must configure the Gmail account to allow access for less secure apps. See the Google Help Center for more information.

Note

The Email LAM does not support Outlook 365. Microsoft do not recommend configuring Outlook 365 with IMAP or POP. See Microsoft support information for more details.

If you are configuring a distributed deployment refer to High Availability Overview first. You will need the details of the server configuration you are going to use for HA.

Configure the LAM

Edit the configuration file to control the behavior of the Email LAM. You can find the file at $MOOGSOFT_HOME/config/email_lam.conf.

See the Email LAM Reference and LAM and Integration Reference for a full description of all properties. Some properties in the file are commented out by default. Uncomment properties to enable them.

Configure the connection properties for each target email source:
- protocol: IMAP, POP3, IMAPS, or POP3S.
- host: IP address or host name of the mail server.
- port: Port of the mail server.
- folder_path: Name of the folder containing the email messages, for example INBOX.
- username: Username of the account used to connect to your mail server.
- password or encrypted password: Password or encrypted password of the account used to connect to your mail server.
Determine how to treat messages for each target:
- retrieve: Whether to receive all email messages or only unread messages.
- retrieve_filter: One or more filters to limit the email messages to retrieve.
- mark_as_read: Marks unread emails as read.
- delete_on_retrieve: Whether to delete email messages on retrieval.
- remove_html_tags: Whether to remove HTML tags from email messages.
- treat_body_as_json: Decodes the email body into a JSON object and makes it available for mapping.
Configure the LAM behavior for each target:
- num_threads: Number of worker threads to use when processing events.
- event_ack_mode: When Moogfarmd acknowledges events from the Email LAM.
- request_interval: Length of time to wait between requests, in seconds.
- max_retries: Number of times the LAM attempts to reconnect after connection failure.
- retry_interval: Length of time to wait between reconnection attempts, in seconds.
- recovery_interval: Length of time to wait between requests, in seconds, when the LAM re-establishes a connection after a failure.
- max_lookback: Period of time for which to recover missed events, in seconds, when the LAM re-establishes a connection after a failure.
- timeout: Length of time to wait before halting a connection or read attempt, in seconds.
- javamail_debug: Enables JavaMail debug mode.
Configure the SSL properties for each target using IMAPS or POP3S protocol:
- disable_certification_validation: Whether to disable SSL certificate validation.
- path_to_ssl_files: Path to the directory that contains the SSL certificates.
- server_cert_filename: Name of the SSL root CA file.
- client_key_filename: Name of the SSL client key file.
- client_cert_filename: Name of the SSL client certificate.
- ssl_protocols: Sets the allowed SSL protocols.
If you want to connect to your Email system through a proxy server, configure the host, port, user, and password or encrypted password properties in the proxy section for the target.
Optionally configure the LAM identification and capture logging details:
- name: Maps to $Laminstancename, so that the agent field indicates events Moogsoft Onprem ingests from this LAM.
- capture_log: Name and location of the LAM's capture log file, which it writes to for debugging purposes.
Optionally configure severity conversions. See Severity Reference for further information and "Conversion Rules" in Tokenize Source Event Data for details on conversions in general.
Optionally configure the process logging details:
- configuration_file: Name and location of the LAM's process log configuration file. See Configure Logging for more information.Configure Logging

Example

You can configure the Email LAM to retrieve messages from one or more sources. If you use more than one mail server or multiple email folders on a single server, configure multiple targets according to the example.

The following example demonstrates a configuration that targets two email sources. For a single source comment out the target2 section. If you have more than two sources, add a target section for each one and uncomment properties to enable them.

monitor:
{
    name                                        : "Email  Monitor",
    class                                       : "CEmailMonitor",
    request_interval                            : 60,
    max_retries                                 : -1,
    retry_interval                              : 60,
    targets:
    {
        target1:
        {
            protocol                            : "IMAPS",
            host                                : "imap.gmx.com",
            port                                : 993,
            folder_path                         : "INBOX",
            username                            : "support@gmx.com",
            password                            : "93pm73xn",
            retrieve                            : "UNREAD",
            retrieve_filter: 
            {
                to              : [ "support@moogsoft.com", "support1@moogsoft.com" ],
                from            : [ "abc@xyz.com", "pqr@xyz.com" ],
                #recipient      : [ ],
                subject         : [ "Alert", "Event" ],
                #body           : ""  
            },
            mark_as_read                       : false,
            delete_on_retrieve                 : false,
            remove_html_tags                   : true,
            treat_body_as_json                 : false;
            disable_certificate_validation     : true,
            #path_to_ssl_files                 : "config",
            #server_cert_filename              : "server.crt",
            #client_key_filename               : "client.key",
            #client_cert_filename              : "client.crt",
            #ssl_protocols                     : [ "TLSv1.2" ],
            num_threads                        : 5
            event_ack_mode                     : "queued_for_processing",
            request_interval                   : 60,
            max_retries                        : -1,
            retry_interval                     : 60,
            timeout                            : 120,
            #javamail_debug                    : true,
            retry_recovery:
            {
                recovery_interval              : 20,
                max_lookback                   : -1
            }
         },
         target2:
         {
             protocol                          : "IMAPS",
             host                              : "imap.mail.yahoo.com",
             port                              : 993,
             folder_path                       : "INBOX",
             username                          : "support@yahoo.com",
             encrypted_password                : "qJAFVXpNDTk6ANq65pEfVGNCu2vFdcoj70AF5BIebEc=",
             retrieve                          : "ALL",
             mark_as_read                      : true,
             delete_on_retrieve                : false,
             remove_html_tags                  : true,
             treat_body_as_json                : false;
             disable_certificate_validation    : false,
             path_to_ssl_files                 : "config",
             server_cert_filename              : "server.crt",
             client_key_filename               : "client.key",
             client_cert_filename              : "client.crt",
             ssl_protocols                     : [ "TLSv1.1, TLSv1.2" ],
             num_threads                       : 5
             event_ack_mode                    : "event_processed",
             request_interval                  : 60,
             max_retries                       : 20,
             retry_interval                    : 120,
             timeout                           : 180,
             #javamail_debug                   : true,
             proxy:
             {
                 host: "localhost",
                 port: 8080
                 user: "John.Doe",
                 password: "Password123"
                 #encrypted_password: "ieytOFRUdLpZx53nijEw0rOh07VEr8w9lBxdCc7229o="
             },
             retry_recovery:
             {
                 recovery_interval             : 20,
                 max_lookback                  : -1
             }
          }
     }
},
agent:
{
    name                                       : "Email",
    capture_log                                : "$MOOGSOFT_HOME/log/data-capture/email_lam.log"
},
log_config:
{
    configuration_file                         : "$MOOGSOFT_HOME/config/logging/email_lam_log.json"
},

Configure for High Availability

Configure the Email LAM for high availability if required. See High Availability Overview for details.

Configure LAMbot Processing

The Email LAMbot processes and filters events before sending them to the Message Bus. You can customize or bypass this processing if required. You can also load JavaScript files into the LAMbot and execute them.

See LAMbot Configuration for more information. An example Email LAM filter configuration is shown below.

filter:
{
    presend: "EmailLam.js"
}

Map LAM Properties

Email header properties are mapped by default to the following Moogsoft Onprem Email LAM properties. The overflow properties are mapped to "custom info" and appear under Overflow in Moogsoft Onprem alerts. You can configure custom mappings in the Email LAMbot.

Email Header Property	Email LAM Event Property
Agent Host	`$x_mailer`
Agent Time	`$sent_date`
Description	`$message`
External ID	`$message_id`
From	`$from`
Host	`$hostname`
Manager	`$from`
Severity	`$severity`
Signature	`$hostname::$subject`
Source ID	`$hostname`
Type	`$subject`
Email Header Property	Email LAM Overflow Property
Content-Type	`$content_type`
Message-ID	`$message_id`
Received	`$received`
Return-Path	`$return_path`
X-Client-IP	`$hostname`
X-Mailer	`$x_mailer`
X-Originating-IP	`$originating_ip`
X-Priority	`$priority`
X-WM-AuthUser	`$AuthUser`

Start and Stop the LAM

Restart the Email LAM to activate any changes you make to the configuration file or LAMbot.

The LAM service name is emaillamd.

See Control Moogsoft Onprem Processes for further details.

If the LAM fails to connect to one or more email sources, creates an alert and writes the details to the process log. Refer to the logging details for LAMs and integrations for more information.

In this section: