Moogsoft Docs

Email Integration Reference

The JavaMail API provides a platform independent and protocol independent framework to build mails and messaging applications.

Following are some of the protocols supported by JavaMail API:

POP3: POP3 (Post Office Protocol 3) is an application-layer Internet standard protocol used by local email clients to retrieve emails from a remote server over a TCP/IP connection. POP3 supports simple download-and-delete requirements for access to remote mailboxes . A POP3 server listens to port 110. POP3 over SSL (POP3S) is assigned to port number 995.

IMAP: IMAP (Internet Message Access Protocol), is an Application Layer Internet protocol which allows an email client to access emails on a remote mail server. An IMAP server typically listens to port 143. IMAP over SSL (IMAPS) is assigned to port number 993.

Email LAM will allow you to retrieve emails from mail server using JavaMail API and send it to Moogsoft AIOps events. Emails from mail Server can be injected in Moogsoft using Email LAM.



  1. LAM reads the configuration from the email_lam.conf file.
  2. The LAM will connect with the mail servers with the given protocol and host name.
  3. It will send the request to the mail server to fetch all or just the unread emails.
  4. The Email LAM will receive emails in response.
  5. The emails are then parsed and converted into normalized Moogsoft AIOps events.
  6. The normalized events are then published to MooMS bus.

Email LAM Configuration

The emails received from mail servers are processed according to the configurations in the email_lam.conf file. The processed events are then published to Moogsoft AIOps.

The configuration file contains a JSON object. At the first layer of object, LAM has a parameter called config, and the object that follows config has all the necessary information to control the LAM.

The Email LAM configuration file has the following sections:

Monitor

Agent

HA Configuration

Mapping

Constants and Conversions

Monitor

The Email LAM takes the emails from the email server. To establish a connection with the email server, you can configure the parameters here:

General

Field
Type
Description
Example
name and class String

Reserved fields: do not change. Default values are Email Monitor and CEmailMonitor.


protocol String

Specify the Email protocol used for accessing email on a remote web server from a local client. The email protocols can be "IMAP" or "POP3", and for SSL, the protocol can be "IMAPS" or "POP3S".


host Integer

Enter the IP Address or hostname of mail IMAP/IMAPS/POP3/POP3S server .


port

Integer

This is the port of mail IMAP/IMAPS/POP3/POP3S server. The default port values are:

IMAP: 143

POP3: 110

IMAPS: 993

POP3S: 995


folder_path Integer Specify the email folder path here. The default folder path is INBOX.
username and Password String Enter the username and password of IMAP/POP3 server email account.
encrypted_password String

If you are using an encrypted password, enter the encrypted password in this field and comment the password field. Either password or the encrypted_password field is used. If both the fields are specified, then only the encrypted_password value will be used by the Email LAM.


retrieve

String

Specify whether to retrieve all emails or just the unread ones.

The options to retrieve emails are:

  • UNREAD
  • ALL

Default is UNREAD . If no value is specified, then the retrieve value will set to UNREAD .


mark_as_read Boolean

This is applicable to 'UNREAD" retrieve mode only. When set to true, it will mark unread emails as read.
If this configuration is missing, then by default, true value will be used.


delete_on_retrieve Boolean Specify whether you want to delete emails on retrievel or not. If no value is specified, then true will be used by default .
disable_certificate_validation Boolean

Specify whether you want to disable SSL Certificate Validation or not.
This will only be applicable in case of "POP3S" or "IMAPS" protocol.


server_cert_filename String

This is the root ca file. This should be stored in a directory path defined in the "path_to_ssl_files".


use_client_authentication Boolean

To use client authentication, set this field to true. If no value is specified, false value will be used by default. If this is set to true, then value of protocol should be "POP3S" or "IMAPS".


num_threads
Integer

This is the number of worker threads that should be used to handle incoming connections. It is an optional value that defaults to the number of CPUs available (up to a maximum of 8).


event_ack_mode

String

This determines when an Event from this LAM should be acknowledged by the receiving process.
Default: queued_for_processing. If not specified, then default mode will be used.
Possible modes are:
queued_for_processing = Acknowledge Event once it has been added to the Moolet queue.
event_processed = Acknowledge Event once it has been
processed by a Moolet.


polling_interval Integer The polling time interval, in seconds, between the requests after which the emails are fetched from the Email Server.
Default = 60 seconds. If 0 is entered, the time interval will set to 60 seconds.

max_retries
Integer

The maximum number of retry attempts to re-connect with IMAP/POP3 server in case of failure.
Default = -1, i.e. infinite retry attempts. If the specified value is less than 1, then it will switch to default i.e. -1.

If the specified value is greater than 1, then the LAM will try that many times to reconnect.


retry_interval Integer

The time interval between two successive retry attempts.

Default = 60 seconds, if specified value is less than 1, time interval will set to 60 seconds.


timeout Integer

This is the timeout value in seconds which will be used to timeout a connection or read attempt. If no value is specified, then the time interval will set to to 120 seconds.


Secure Sockets Layer

Field
Type
Description
ssl Boolean

Set to true, to enable SSL Communication:

  • path_to_ssl_files: Enter the location of the SSL certificate files. If you are using "POP3S" or "IMAPS" protocol, and the value of disable_certificate_validation is false, then path_to_ssl files must be enabled and it should be non-empty.

  • client_key_filename: Enter the client ssl key filename.

  • client_cert_filename: Enter the client ssl certificate filename.

Filter

Field
Type
Description
retrieve_filter
Object

Parameters to retrieve emails:

to : ["support@ moogsoft.com ", "support1@ moogsoft.com "],

from : ["abc@ xyz.com ", "pqr@ xyz.com "],

recipient : [],

subject : "Alert",

body : ""

This will concatenate all the filters with AND operator. In case of multiple values for one attribute, OR will be used. For example, if the above filters are configured, the LAM will retrieve emails where '(to = "support@ moogsoft.com " OR "support1@ moogsoft.com ") AND (from = "abc@ xyz.com " OR "pqr@ xyz.com ") AND subject contains 'Alert'.

If any of the AND condition is false, you would not be able to retreive any mails.



Example

Config File
config :
    {

        monitor:
        {
            name                    		  : "Email  Monitor",

            class               		      : "CEmailMonitor",

			protocol    					  : "IMAP",

			host        					  : "localhost",
			
			port                              : 143,

			folder_path 					  : "INBOX",
 
			username    					  : "username",

			password    					  : "password",

			#encrypted_password				  : "qJAFVXpNDTk6ANq65pEfVGNCu2vFdcoj70AF5BIebEc=",
 
			retrieve        				  : "UNREAD",

			retrieve_filter 				  : 
			      								{
			        								#to: ["support@moogsoft.com", "support@moogsoft.com"],
			        								#from: ["abc@xyz.com", "pqr@xyz.com"],
			        								#recipient: [],
			        								#subject: "Alert",
			       			 						#body: "",					
			    								},


			mark_as_read    				  : true,

			delete_on_retrieve  			  : false,
 
			disable_certificate_validation	  : false,
   
			path_to_ssl_files       		  : "config",

			server_cert_filename        	  : "server.crt",
 
			use_client_authentication  		  : false,
  
			client_key_filename        		  : "client.key",
            
			client_cert_filename       		  : "client.crt",

			num_threads						  : 5,

			event_ack_mode 					  : "queued_for_processing",

			polling_interval 				  : 60,
  
			max_retries 					  : -1,

			retry_interval 					  : 60,

			timeout							  : 120,
	
	  },

Agent

Agent allows the you to define two parameters:

Field
name This is the agent name, the events sent to MooMS by the Email LAM are identified by the agent name in the log. In this example, the agent name is Email.
log Email LAM will write its ingress contents in the file email_lam.log located at /var/log/moogsoft/.


HA Configuration

Refer to the document Integrations HA Configuration

Mapping

For events received in JSON format, y ou can directly map the event fields of Email with Moogsoft fields. The parameters of the received alarm/event are displayed in Moogsoft AIOps according to the mapping done here:

mapping :
        {
            catchAll: "overflow",
            rules:
            [
                { name: "signature",rule:      "$hostname::$subject" },
                { name: "source_id",rule:      "$hostname" },
                { name: "external_id",rule:    "$message_id" },
                { name: "manager",rule:        "$from" },
                { name: "source",rule:         "$hostname" },
                { name: "class",rule:          "Mail" },
                { name: "agent",rule:          "$LamInstanceName" },
                { name: "agent_location",rule: "$x-mailer" },
                { name: "type",rule:           "Mail" },
                { name: "severity",rule:       "$severity",  conversion: "sevConverter"  },
                { name: "description",rule:    "$message" },
                { name: "agent_time",rule:     "$sent_date" }
            ]
        },
        filter:
        {
           # stream: "myStream",
            presend: "EmailLam.js"
        }


The above example specifies the mapping of the Email fields with the Moogsoft AIOps fields. Data not mapped to Moogsoft AIOps Fields goes into "Custom Info".

Note

The signature field is used by the LAM to identify correlated alarms.

Constants and Conversions

Constants and Conversions allows you to convert format of the received data.

Field
Description
Example
Severity and sevConverter This has a conversion defined as sevConverter in the Conversions section, this looks up the value of severity defined in the severity section of constants and returns back the mapped integer corresponding to the severity.
severity:
{
"CLEAR" : 0,
"INDETERMINATE" : 1,
"WARNING" : 2,
"MINOR" : 3,
"MAJOR" : 4,
"CRITICAL" : 5,
moog_lookup_default : 1
}
 sevConverter:
{
lookup : "severity",
input : "STRING",
output : "INTEGER"
},

stringToInt Used in a conversion which forces the system to turn a string token into an integer value.
stringToInt:
{
    input  : "STRING",
    output : "INTEGER"
},

Example

Example Constants and Conversions
 constants:
        {
            severity:
            {
                "CLEAR"               : 0,
                "INDETERMINATE"       : 1,
                "WARNING"             : 2,
                "MINOR"               : 3,
                "MAJOR"               : 4,
                "CRITICAL"            : 5,
                moog_lookup_default   : 1
            }
        },
        conversions:
        {
            sevConverter:
            {
                lookup			: "severity",
                input			: "STRING",
                output			: "INTEGER"
            },
            stringToInt:
            {
                input			: "STRING",
                output			: "INTEGER"
            }
        },

Severity Reference

Moogsoft Severity Levels
severity:
        {
            "CLEAR" : 0,
            "INDETERMINATE" : 1,
            "WARNING" : 2,
            "MINOR" : 3,
            "MAJOR" : 4,
            "CRITICAL" : 5,
            moog_lookup_default: 1
        }
Level Description
0 Clear
1 Indeterminate
2 Warning
3 Minor
4 Major
5 Critical

Service Operation Reference

Process Name Service Name
email_lam emaillamd

Start the LAM Service:

service emaillamd start

Stop the LAM Service:

service emaillamd stop

Check the LAM Service status:

service emaillamd status

Command Line Reference

To see the available optional attributes of the email_lam, run the following command:

email_lam --help

The email_lam is a command line executable, and has the following optional attributes:

Option Description

--config

Points to a pathname to find the configuration file for the LAM. This is where the entire configuration for the LAM is specified
--help Displays all the command line options
--version

Displays the component’s version number

--log level

Specifies the level of debugging. By default, user gets everything. In common with all executables in MOOG, having it set at that level can result in a lot of output (many messages per event message processed).

In all production implementations, it is recommended that log level is set to WARN. This ensures only warning, error and fatal messages are recorded.

Performance Information

Minimum requirement
Component Value
CPU 2 core
RAM 4 GB
Operating System CentOS Linux release 6.7

Version

LAM Version Tool Version Verified by
1.0 Winmail Mail Server 6.2 Moogsoft