Post Syndicated from Zip Zieper original https://aws.amazon.com/blogs/messaging-and-targeting/email-journaling-with-ses-mail-manager/

Introduction to Journaling

Email journaling is the practice of preserving comprehensive records of all email communications within an organization. This approach stems from the need to maintain rigid, compliance-driven retention policies focused on auditing an entire organization’s email activities. Because journaled email messages are often required to satisfy on-demand audit and investigation requests, they must be readily searchable, making accessibility a key requirement. Reflecting legal and regulatory requirements, email journaling has historically required expensive, dedicated off-site storage and complex retrieval systems.

Amazon WorkMail is a managed business email service with flexible journaling capabilities that are configurable at both the individual mailbox and organization-wide level. With WorkMail, you can use custom rules to selectively preserve or redirect certain messages using granular journaling controls. This flexibility allows administrators to implement both traditional email journaling and configurations that you can customize to meet specific use cases.

Email journaling is used to capture and retain every email sent to and from an organization, primarily for compliance purposes. In contrast, email archiving is typically used to offload and store emails from an organization’s primary email system, often driven by inbox size limits and data backup or eDiscovery needs. While journaling focuses on preserving a consolidated record of communications separate from live mailboxes, archiving is a more selective process. Journaling is usually driven by regulatory, audit, and compliance requirements. As discussed in this blog post, you can use the Mail Manager archiving feature not only for selective email backup and optimization, but also to fulfill your email journaling requirements. You can learn more about email archiving with Mail Manager in this blog post.

Amazon Simple Email Service (SES) Mail Manager provides comprehensive tools that simplify managing large volumes of email communications within an organization. Mail Manager has a built-in archiving function which can be used as an inexpensive journaling solution for email systems like Amazon WorkMail. Mail Manager’s rules engine allows for the creation of rules that readily satisfy a wide range of email journaling requirements. Additionally, Mail Manager’s archiving capability supports multiple, concurrent archiving destinations that can be independently searched and exported on demand.

In this blog post, we discuss how Amazon WorkMail and Amazon Simple Email Service (SES) Mail Manager make email journaling easier to set up and use, more cost-effective and versatile. We’ll walk the reader through setting up email journaling for an Amazon WorkMail organization that uses SES Mail Manager’s routing, processing, and archiving features.

SES Mail Manager as Journaling Destination for WorkMail

For our purposes, we’ll assume you’ve already set up WorkMail as your mailbox provider, but the process described below will work with the journaling features of most 3rd party email solutions. If you want to explore Amazon WorkMail, visit the getting started documentation here.

In the following sections, we’ll describe how to configure WorkMail journaling to send full email journals to SES Mail Manager’s archives. We’ll define different retention periods for each archive to demonstrate how this solution can be used to meet both short and long-term retention requirements. Finally, we’ll use the AWS SES Mail Manager console to search, export, and manage the email journals and archives.

In our examples, we’ll use Amazon Route 53 to create a new domain called ‘journaling.solutions’ which we’ll configure to send all ‘@journaling.solutions’ emails to an SES Mail Manager Ingest endpoint. To begin, open the AWS Console, navigate to your WorkMail Organization’s settings, and click on the Journaling tab:

Click Edit, enable journaling, and provide a journaling email address (we’re using ‘[email protected]’) to receive journaled content. Provide a report email address, such as the admin email list, to receive journaling reports:

Open the AWS SES console in a new browser window, and navigate to Mail Manager’s Rule sets. Create a new rule set called ‘journaling-rule-demo’. Click Edit and create a new rule called “journal-all”, with an Archive action. Click the create an archive button and create an archive called ‘journaling-archive-demo’:

When creating Mail Manager archives, you have options to set the retention period from 3 months to permanent storage. You can also choose to encrypt your archived messages with your own KMS key. The configuration in our example is for permanent storage and shows the optional text field for using your own KMS key:

Traditional journaling calls for recording every email message to the journal, so for our ‘journal-all’ rule, we will not define filtering behaviors in the rule set. This will instruct Mail manager to send all emails for [email protected] to the journaling-archive-demo archive. It is worth noting that Mail Manager’s rule set can be configured to filter and independently process multiple recipient addresses. Consult the documentation to learn about other ways to customize Mail Manager for your use cases.

Next, create a new traffic policy, called journaling-traffic-demo, and configure it to reject any message not explicitly sent to the journaling destination address ([email protected]):

Create an open ingress endpoint called ‘journaling-demo-IG’, and select the ‘journaling-traffic-demo’ traffic policy and ‘journaling-rule-demo’ rule set:

After you press the create Ingest endpoint button, Mail Manager will create an Ingress endpoint and assign it a DNS A Record to be used in your DNS configurations to route email to Mail Manager:

From the General details page of the Ingress endpoint, copy the Ingress endpoint’s DNS A Record to your clipboard. Open a new browser window to your DNS provider’s MX configuration page (in our example below, we’re using AWS Route53). Edit the MX record for ‘journaling.solutions’ by pasting the Ingress endpoint A record. This configuration will route email sent to any address ‘@journaling.solutions’ to the Mail Manager’s Ingress endpoint for processing by the Traffic policy and Rule set:

To test your new journaling configuration, send several emails to several email addresses in your WorkMail organization (or the alternative inbox provider you configured in the first step). WorkMail (or your alternative inbox provider) will send a full record of all emails to the journaling destination address ([email protected]).

Wait a few minutes after sending the emails above, then open the AWS Mail Manager console’s archiving controls and search for messages sent in the last 12 hours:

The example above shows a search for all messages received in the “last 12 hours”, with no other filters specified. The results show every message inserted into the archive in this timeframe. You’ll see one entry where the from address is different (from toby@tegwj@…). This is an example of mail that was sent directly to the journaling destination address ([email protected]). This works because our traffic policy and rule set configurations don’t include any filters.

A cost effective solution at scale

Using Mail Manager as a journaling solution gives you more direct control over your costs than typical journaling services. While most journaling services in the market today charge a fixed rate per journaled mailbox, Mail Manager pricing is comprised of a monthly fixed fee per ingestion endpoint and consumption pricing for basic message handling, and the amount of data archived.

For example, imagine your organization has 250 mailboxes, each handling 50 messages per day. On a monthly basis this amounts to 375,000 messages. If we assume each message is 40 kilobytes in size, your organization is generating roughly 15 gigabytes of email per month. As you can see from the table below, the total cost in month 1 is about $140, or $0.56/mailbox.

|Item |Unit Price |Volume |Subtotal/Mo |
|— |— |— |— |
|Ingress Endpoint |$50/mo |1 |$50 |
|Core message processing |$0.15/1000 msgs |375 |$56.25 |
|Archive insertion/indexing |$2/GB (one-time) |15 |$30 |
|Archive storage |$0.19/GB/mo |15 |$2.85 |
|Subtotal: | | |$139.10 |
| |Monthly price per mailbox |$0.56 |

If the proposed email rate in our assumptions stays constant, the Mail Manager archive will grow by 15 gigabytes each month. After 36 months, the total monthly storage cost increases to $102.60. This results in a total monthly spend in month 36 of $238.85, or $0.96/mailbox/month.

Conclusion

In this blog post, we’ve explored how Amazon WorkMail and Amazon SES Mail Manager can provide a cost-effective and accessible solution for email journaling. By leveraging the flexible journaling capabilities of WorkMail and the archiving features of SES Mail Manager, organizations can easily satisfy rigorous compliance requirements around email retention and accessibility.

The combination of WorkMail’s journaling controls and SES Mail Manager’s rule-based archiving allows you to tailor your journaling solution to your specific needs. Whether you require short-term retention for audits or long-term preservation for legal and regulatory purposes, SES Mail Manager’s flexible archiving options have you covered with predictable and transparent costs that scale with your organization’s email volume.

If you’re looking for a modern, scalable, and cost-effective solution for your email journaling needs, we encourage you to explore the capabilities of Amazon SES Mail Manager. Get started today by visiting the AWS documentation and begin streamlining your email compliance and retention processes.

About the Authors