MailIn design specification

MailIn design specification

Introduction

Andreas Solberg once wrote a piece of software called MailIn, written in Perl, to parse incoming email from third party monitoring systems and to translate them into NAV events.

We would to like to include such a system as part of NAV, but since we do have a long term goal of reducing the number of used programming languages, we would like this to be reimplemented as a Python program. This would enable it to make use of the common Python API that NAV already offers.

The University of Tromsø have already started writing a MailIn replacement in Python, and this is up and running at the University, but with only a single plugin.

This blueprint specifies the design required to include such a software component in NAV, and most of is derived from the good documentation of the Perl version.

System overview

The core of MailIn will be a small plugin-invoking engine with a minimal API offered to its plugins.
The MailIn program will be invoked by an MDA (Mail Delivery Agent), and must expect to be fed an email payload on stdin.
The payload will be offered in turn to each registered plugin.
- Once a plugin accepts the payload, MailIn core will not offer it to any other plugin.
- MailIn core verifies that the plugin offers its own authorize function. If it doesn't, MailIn core attempts to authorize the payload according to global rules.
  - When the plugin offers its own authorize function, this is called. The plugin will check its authorization configuration, to see whether the email is allowed to generate an event. Typically, the headers are inspected to find the source of the email.
- If authorized, MailIn core will ask the plugin to process the payload.
  - The plugin will process the payload in whatever manner it sees fit, and using the nav.event API, one or more events should be generated and posted to the event queue.

Plugin API

A MailIn plugin consists of a Python module, which should provide three functions:

function	required	description
`accept(payload)`	yes	Must return a true value if the payload can be processed by this plugin.
`authorize(payload)`	no	Must return a true value if the payload is authorized to generate an event, according to plugin-internal rules/configuration. If this function is not supplied by the plugin, the MailIn core must use a global authorize function to perform authorization according to globally configured rules.
`process(payload)`	yes	Process the payload, generate and post a single or multiple events. Return a true value if successful.

The plugin must expect the payload object to be a message object from Python's standard email library.

Loading plugins

MailIn core will load plugin modules according to its configuration file.
All plugin modules must be listed in a specific configuration option, separated by spaces or colons.
- suffix wildcards should be allowed, such that specifying “nav.mailin.plugins.*” will result in the loading of all modules in the nav.mailin.plugins package.
A plugin module that fails to import into the Python interpreter shall be ignored by the core, and an exception logged (including a full traceback).

Configuration

Example configuration file:

[mailin]
# What plugins to load
plugins = nav.mailin.plugins.*

# Global authorization check (header regexp pattern)
# check to see that uit.no is in the Received headers
authorization = ^Received:.*uit\.no

# Plugin specific options
[nav.mailin.plugins.whatsup]
plugin_specific_value=foo

API usage

The following Python / NAV libraries should be used for various aspects of the program:

api	for what
`email`	Parsing email payloads into Python objects that will be passed around internally
`logging`	Any logging should be performed using named loggers from Python's logging library
`nav.event`	Creating and posting events to NAV's event queue

As of this writing, there is no API to create custom event type and alert type hierarchies in the NAV database. This will be added as a separate blueprint in Launchpad.

Table of Contents