Tag Archives: email engagement

How to investigate what happened to the email that was sent via SES but was never received in recipient inbox

2023-06-27 singhzwz

Post Syndicated from singhzwz original https://aws.amazon.com/blogs/messaging-and-targeting/how-to-investigate-what-happened-to-the-email-that-was-sent-via-ses-but-was-never-received-in-recipient-inbox/

How to investigate what happened to the email that was sent via SES but was never received in recipient inbox

Amazon Simple Email Service (SES) is an email platform that provides an easy, cost-effective way for you to send and receive email using your own email addresses and domains.

You can use SES for two major use cases. The first use case is marketing emails, which can include things such as special offers, newsletters, and product updates. The second use case is transactional emails, which can include things such as order confirmation, One Time Password (OTP), and registration confirmation.

One common issue we hear from our customers is that emails are not delivered to the recipient’s inbox. There are three areas in the flow where this could happen:

Emails were not sent successfully by your email application
Emails get dropped within SES after being received from your email application
Emails are dropped after leaving SES

In this blog post, we will explore each of these three scenarios.

Prerequisites:

To investigate each scenario, you need the following:

Understand How email sending works in Amazon SES
Enable verbose logging on your email application
Amazon SES – Set up notifications for bounces and complaints

Scenario 1: Emails were not sent successfully by your email application

The first step in the process is for your application to contact SES to pass it the message. This is one place where problems could occur.
Remember that for each message it successfully enqueues, SES returns a message ID that you can log on your side. An example message ID looks like 01000174f87e0fb4-81b97243-56b0-4cd1-90c7-d6987a00fdc1-000000.

Here is an example of Sendmail logs for successfully relaying the email to SES.

Jun 22 15:29:42 ip-172-12-12-12.ec2.internal sendmail[323257]: 35ABCDEF123123: to=<[email protected]>, delay=00:00:01, xdelay=00:00:01, mailer=relay, pri=30123, relay=email-smtp.us-east-1.amazonaws.com. [3.95.123.123], dsn=2.0.0, stat=Sent (Ok 01000174f87e0fb4-81b97243-56b0-4cd1-90c7-d6987a00fdc1-000000)

To verify that your requests are successful, you can look for the message IDs. If SES never accepted your message, you can be sure it won’t be delivered! If you are able to log a message ID, it means that your email safely arrived at SES.

Actions to troubleshoot client side issue:

1. In order to test that the problem lies between your application and SES, you can try to send a test email from the SES console to an email address and domain where you are experiencing the issue. If that test email arrives on time, it’s a pretty good indicator that the error is occurring before your email even reaches SES.
2. Examine the logs at application side to confirm if the application has successfully passed the email to SES. A success response looks like the following example:

250 Ok 0000012345678e09-123a4cdc-b56c-78dd-b90e-d123be456789-000000

Some common errors include the following:

* Email address is not verified. The following identities failed the check in region region: identity1, identity2, identity3
* Account is paused
* Daily message quota exceeded
* Maximum sending rate exceeded
* Maximum SigV2 SMTP sending rate exceeded

Another common type of error is a connection timeout error. Here is an example of a Java exception that shows a connection timeout error to an SES endpoint:

Caused: com.sun.mail.util.MailConnectException: Couldn't connect to host, port: email-smtp.ap-southeast-2.amazonaws.com , 465; timeout 60000; nested exception is: java.net .ConnectException: Connection timed out (Connection timed out) at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:2209 ) at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:740 )

The way that you observe this error message depends on the way that you call SES.

If you call the SES API directly, the action to interact with SES will return an error like MessageRejected or one of the errors specified in the Common Errors topic of the Amazon Simple Email Service API Reference.

If you call SES through its SMTP interface, the way that you experience the error depends on the application. Some applications might display a specific error message, and others might not. For a list of SMTP response codes that SES returns, see SMTP response codes returned by Amazon SES.

Scenario 2: Emails get dropped within SES after being received from your email application

Once SES accepts the message and provides a message ID, the next step is to process it and hand it over to the recipient ISP or the recipient mailbox provider. However, in the following scenarios SES can drop the email after receiving it:

Rendering Failure: This can happen when you use an email template to send email. Upon accepting the email from your email application, SES validates that the template data you send includes the required variables in the template. If the template data contains non-compliant variables or is missing variables, SES is unable to construct the email and drops the email. This is called a Rendering Failure.
Suppression list: If a recipient is on an SES suppression list, SES will drop the email to protect the sender reputation. There are two types of suppression lists: Account-level suppression list and global suppression list. To learn more about the two types of suppression lists, please refer to Managing lists and subscriptions in Amazon Simple Email Service.
Reject Event: SES accepted the email, but determined that it contained a virus and didn’t attempt to deliver it to the recipient’s mail server. SES drops the email and generates a Reject event.

Actions to troubleshoot such issues:

To identify the root cause of the three preceding scenarios, you need to set up Amazon SES event publishing. Please refer to the prerequisite section to set this up.

Scenario 1: Rendering Failure

Here is an example event publishing record for Rendering Failure

{
"eventType":"Rendering Failure",
"mail":{
"timestamp":"2018-01-22T18:43:06.197Z",
"source":"[email protected]",
"sourceArn":"arn:aws:ses:us-east-1:123456789012:identity/[email protected]",
"sendingAccountId":"123456789012",
"messageId":"EXAMPLE7c191be45-e9aedb9a-02f9-4d12-a87d-dd0099a07f8a-000000",
"destination":[
"[email protected]"
],
"headersTruncated":false,
"tags":{
"ses:configuration-set":[
"ConfigSet"
]
}
},
"failure":{
"errorMessage":"Attribute 'attributeName' is not present in the rendering data.",
"templateName":"MyTemplate"
}
}

In this example, the error message tells you that there is a missing variable attributeName in the template data. To learn more about troubleshooting rendering failure, please refer to Why are emails failing to deliver when I send Amazon SES emails using the SendTemplatedEmail operation?

Scenario 2: Suppression list

SES drops the email if the recipient is on an account-level suppression list or a global suppression list. The following is an example error message for recipient being on a global suppression list:

"diagnosticCode": "Amazon SES has suppressed sending to this address because it has a recent history of bouncing as an invalid address. For more information about how to remove an address from the suppression list, see the Amazon SES Developer Guide: http://docs.aws.amazon.com/ses/latest/DeveloperGuide/remove-from-suppressionlist.html"

The example demonstrates that the email get bounced because the recipient is on the global suppression list. We recommend enabling SES account-level suppression list because the account-level suppression list supersedes the global suppression list.

If the recipient is on your account-level suppression list, you can remove individual email addresses from your Amazon SES account-level suppression list. The following is an example error message for recipient being on a account-level suppression list:

"diagnosticCode": "Amazon SES did not send the message to this address because it is on the suppression list for your account. For more information about removing addresses from the suppression list, see the Amazon SES Developer Guide at https://docs.aws.amazon.com/ses/latest/DeveloperGuide/sending-email-suppression-list.html"

Note: The suppression list feature is in place to avoid emails being sent to recipients who are known to produce bounce or complaint for your emails in past. Enabling this feature protects your reputation as a sender. Only remove the recipient when you are assured that recipients in concern will no longer produce bounce or complaint for your emails.

Scenario 3: Reject Event

SES rejects the email when it scans the email and determines that it contains a virus. The following is an example of a Reject event publishing record.

{
"eventType": "Reject",
"mail": {
"timestamp": "2016-10-14T17:38:15.211Z",
"source": "[email protected]",
"sourceArn": "arn:aws:ses:us-east-1:123456789012:identity/[email protected]",
"sendingAccountId": "123456789012",
"messageId": "EXAMPLE7c191be45-e9aedb9a-02f9-4d12-a87d-dd0099a07f8a-000000",
"destination": [
"[email protected]"
],
"headersTruncated": false,
"headers": [{
"name": "From",
"value": "[email protected]"
},
{
"name": "To",
"value": "[email protected]"
},
{
"name": "Subject",
"value": "Message sent from Amazon SES"
},
{
"name": "MIME-Version",
"value": "1.0"
},
{
"name": "Content-Type",
"value": "multipart/mixed; boundary=\"qMm9M+Fa2AknHoGS\""
},
{
"name": "X-SES-MESSAGE-TAGS",
"value": "myCustomTag1=myCustomTagValue1, myCustomTag2=myCustomTagValue2"
}
],
"commonHeaders": {
"from": [
"[email protected]"
],
"to": [
"[email protected]"
],
"messageId": "EXAMPLE7c191be45-e9aedb9a-02f9-4d12-a87d-dd0099a07f8a-000000",
"subject": "Message sent from Amazon SES"
},
"tags": {
" ses:configuration-set": [
"ConfigSet"
],
"ses:source-ip": [
"192.0.2.0"
],
"ses:from-domain": [
"example.com"
],
"ses:caller-identity": [
"ses_user"
],
"myCustomTag1": [
"myCustomTagValue1"
],
"myCustomTag2": [
"myCustomTagValue2"
]
}
},
"reject": {
"reason": "Bad content"
}
}

To fix this issue, please examine the email and make sure there is no virus in the email. You can test your email content by sending a test email to Amazon SES mailbox simulator.

Scenario 3: Emails are dropped after leaving SES
We sometimes encounter this when we successfully deliver the email to a recipient ISP, only to have the recipient ISP hold back the email and not deliver to the recipient’s inbox. This can happen for a variety of reasons, for example the recipient ISP’s anti-spam filters, or because the sender has not yet built sufficient reputation due to it being a new domain or IP and the recipient does not yet trust the sender.

Actions to troubleshoot such issues:

To identify the root cause of the issue, you need to set up Amazon SNS notification. Please refer to the prerequisite section to set this up.

When the recipient ISP accepts the email, it returns a 250 ok status code to SES. SES includes the detailed status code and response message from the recipient ISP in the SNS notification.The following is an example SNS notification for a success delivery event.

{
"notificationType":"Delivery",
"mail":{
"timestamp":"2016-01-27T14:59:38.237Z",
"messageId":"0000014644fe5ef6-9a483358-9170-4cb4-a269-f5dcdf415321-000000",
"source":"[email protected]",
"sourceArn": "arn:aws:ses:us-east-1:888888888888:identity/example.com",
"sourceIp": "127.0.3.0",
"sendingAccountId":"123456789012",
"callerIdentity": "IAM_user_or_role_name",
"destination":[
"[email protected]"
],
"headersTruncated":false,
"headers":[
{
"name":"From",
"value":"\"John Doe\" <[email protected]>"
},
{
"name":"To",
"value":"\"Jane Doe\" <[email protected]>"
},
{
"name":"Message-ID",
"value":"custom-message-ID"
},
{
"name":"Subject",
"value":"Hello"
},
{
"name":"Content-Type",
"value":"text/plain; charset=\"UTF-8\""
},
{
"name":"Content-Transfer-Encoding",
"value":"base64"
},
{
"name":"Date",
"value":"Wed, 27 Jan 2016 14:58:45 +0000"
}
],
"commonHeaders":{
"from":[
"John Doe <[email protected]>"
],
"date":"Wed, 27 Jan 2016 14:58:45 +0000",
"to":[
"Jane Doe <[email protected]>"
],
"messageId":"custom-message-ID",
"subject":"Hello"
}
},
"delivery":{
"timestamp":"2016-01-27T14:59:38.237Z",
"recipients":["[email protected]"],
"processingTimeMillis":546,
"reportingMTA":"a8-70.smtp-out.amazonses.com",
"smtpResponse":"250 ok: Message 64111812 accepted",
"remoteMtaIp":"127.0.2.0"
}
}

In this example, the SMTP response is "250 ok:Message 64111812 accepted", this confirms the recipient ISP accepted the email. At this point, the email is with the recipient ISP and SES no longer has control over the email.

ISPs use different mechanisms and algorithms to filter emails to place them in either the recipient’s inbox folder or spam folder. In such case, you will need to reach out to the recipient ISP to understand why they accepted the email but did not deliver it to the recipient inbox. Since their servers decide the placement of email, they can give a clear explanation of the cause of the issue and steps to remediate the issue.

The recipient ISP may reject the email. When this happens, the recipient ISP returns a 4xx or 5xx error code to SES. This is also called a bounce event.

There are two kinds of bounce events: Soft bounce and hard bounce. Soft bounce refers to a temporary email delivery failure, this kind of error usually can be retried. Hard bounce refers to a permanent delivery failure, and it cannot be retried. The Simple Mail Transfer Protocol (SMTP) Enhanced Status Codes Registry lists the possible error codes and reasons for email delivery failure. Generally speaking, 4xx error codes are soft bounces, 5xx error codes are hard bounces.

When the recipient ISP bounces the email, SES includes the detailed status code and response message from the recipient ISP in the SNS notification.

The following is an example SNS notification for a bounce event.

{
"notificationType":"Bounce",
"bounce":{
"bounceType":"Permanent",
"reportingMTA":"dns; email.example.com",
"bouncedRecipients":[
{
"emailAddress":"[email protected]",
"status":"5.1.1",
"action":"failed",
"diagnosticCode":"smtp; 550 5.1.1 <[email protected]> User not found"
}
],
"bounceSubType":"General",
"timestamp":"2016-01-27T14:59:38.237Z",
"feedbackId":"00000138111222aa-33322211-cccc-cccc-cccc-ddddaaaa068a-000000",
"remoteMtaIp":"127.0.2.0"
},
"mail":{
"timestamp":"2016-01-27T14:59:38.237Z",
"source":"[email protected]",
"sourceArn": "arn:aws:ses:us-east-1:888888888888:identity/example.com",
"sourceIp": "127.0.3.0",
"sendingAccountId":"123456789012",
"callerIdentity": "IAM_user_or_role_name",
"messageId":"00000138111222aa-33322211-cccc-cccc-cccc-ddddaaaa0680-000000",
"destination":[
"[email protected]",
"[email protected]",
"[email protected]"],
"headersTruncated":false,
"headers":[
{
"name":"From",
"value":"\"John Doe\" <[email protected]>"
},
{
"name":"To",
"value":"\"Jane Doe\" <[email protected]>, \"Mary Doe\" <[email protected]>, \"Richard Doe\" <[email protected]>"
},
{
"name":"Message-ID",
"value":"custom-message-ID"
},
{
"name":"Subject",
"value":"Hello"
},
{
"name":"Content-Type",
"value":"text/plain; charset=\"UTF-8\""
},
{
"name":"Content-Transfer-Encoding",
"value":"base64"
},
{
"name":"Date",
"value":"Wed, 27 Jan 2016 14:05:45 +0000"
}
],
"commonHeaders":{
"from":[
"John Doe <[email protected]>"
],
"date":"Wed, 27 Jan 2016 14:05:45 +0000",
"to":[
"Jane Doe <[email protected]>, Mary Doe <[email protected]>, Richard Doe <[email protected]>"
],
"messageId":"custom-message-ID",
"subject":"Hello"
}
}
}

In the preceding example, the recipient ISP permanently bounced the email. This is concluded from the "bounceType":"Permanent" field. Additionally, the recipient ISP provides the reason they bounced the email in the “diagnosticCode” field, and the reason is "smtp; 550 5.1.1 <[email protected]> User not found". In this case, [email protected] isn’t a valid inbox, the error can’t be resolved by retrying the request. The sender should check the spelling of the email address and confirm with the recipient that this address has a valid inbox before retry sending to this address again. You can also contact the recipient ISP to understand the reasons of the error and steps to remediate the issue when the information provided by the "diagnosticCode" isn’t sufficient or clear.

Another event that can happen is “delivery delay”. Such event happen when the email couldn’t be delivered to the recipient’s mail server because a temporary issue occurred. Delivery delays can occur, for example, when the recipient’s inbox is full, or when the receiving email server experiences a transient issue. These are usually soft bounces in which case SES is programmed to retry such deliveries several times over the next 10 hours. If you have enabled SES event publishing for this event you can review the “delivery delay” object section of SES event record to understand the cause of delay and take corrective actions accordingly. As an example, if you reach out to the recipient ISP and the recipient ISP informs you about maintenance on their side that is causing the issue then you can refrain sending to that ISP till the maintenance completes.

Conclusion:
Now that you know the 3 areas in the email sending flow where errors can occur you can begin to troubleshoot whether your problems are at your email application side, within SES, or at the recipient ISP. Make sure that you have proper logging and that you are receiving the events that provide you the detail you need to determine where your errors are occurring. Errors happening prior to the email being sent out from SES are usually solvable, while challenges in deliverability once an email has been sent from SES require more due diligence and experimentation in your content to determine why your emails are being delivered to spam, or not at all.

Feel free to post any comments or questions for us in the AWS Forum.

References:
[1]https://docs.aws.amazon.com/ses/latest/dg/send-email-concepts-process.html
[2]https://docs.aws.amazon.com/ses/latest/dg/monitor-sending-activity.html
[3]https://docs.aws.amazon.com/ses/latest/dg/monitor-using-event-publishing.html#event-publishing-how-works

Increase your email engagement using AMP.

2023-06-05 Pavlos Ioannou Katidis

Post Syndicated from Pavlos Ioannou Katidis original https://aws.amazon.com/blogs/messaging-and-targeting/increase-your-email-engagement-using-amp/

This blog introduces the use of Accelerated Mobile Pages (AMP) with Amazon Simple Email Service (SES). It provides you a guide on how to configure AMP, a ready AMP email template and a working code example that uses Amazon SES API to send the AMP email. AMP emails support components that recipients can interact without leaving their email client, thus improving customer experience.

Despite the increased use and popularity of mobile messengers and chat apps, email is an integral part of daily online life. In 2020, the number of global email users amounted to four billion and is set to grow to 4.6 billion users in 2025 (source Statistica). Furthermore, in 2020, approximately 306 billion emails were sent and received every day worldwide. This figure is projected to increase to over 376 billion daily e-mails in 2025 (source Statistica).

Businesses use e-mail to communicate with their employees and customers. The email content provides information in the form of text and HTML and recipients can use hyperlinks to take actions such as visit the mobile or web application to perform a purchase, fill a form or browse products.

What is an AMP email?

In 2015, Google launched AMP, which is an open-source technology designed to enhance website performance and user experience on every device regardless of the screen size. AMP is a framework of HTML, CSS, and JS with speed and performance optimization. Using AMP for e-mail allows senders to include interactive components giving the ability to recipients to RSVP to an event, browse image carousels and accordions, and take user input on forms and questionnaires from the email.

Use cases:

Travel: A hotel can send an AMP email allowing customers to check live the rooms’ availability, instead of redirecting them their mobile or web application. Furthermore, customers can view available rooms, see prices, and even select their preferred room type, all within the email itself. This can significantly enhance the user experience and increase conversion rates.
Retail: Customers can browse through a carousel of products, select a product of their choice, customize it if needed, and add it to their cart. The entire shopping experience, except for the payment process, can be completed without leaving the email, making it more convenient for the customer and increasing the likelihood of a purchase.
Delivery of goods: AMP emails can also be used to provide live tracking of deliveries. Once a customer places an order, the delivery company can send an AMP email with a live map showing the current location of the delivery vehicle and the estimated time of arrival. This can enhance customer satisfaction by keeping them informed in real-time about their delivery status.
Finance: Financial institutions can use AMP emails to provide interactive tools to their customers. For example, a bank can include an interactive loan calculator in their email. Customers can input their loan amount, interest rate, and loan term, and the calculator will display their estimated monthly payment. This can help customers make informed financial decisions without having to visit the bank’s mobile or web application.

In this blog you will learn how to send your first AMP email using Amazon SES.

Amazon SES AMP email

Considerations

AMP emails is relatively new and before getting started, it’s important to be aware of the following points:

AMP emails are only supported by the following email clients: FairEmail, Gmail, Mail.ru and Yahoo Mail. Get the most up to date list of email clients supporting AMP emails by visiting this link.
Creating an AMP email can be time consuming as it has its own tags, components, properties and ways of structuring an email. On the other hand, there are third party email template tools such as Stripo, which support AMP components using their “what you see is what you get” (WYSIWG) editors.
Tracking performance of AMP emails goes beyond the common email engagement metrics like delivered, open and click. Interactive components like a form submission requires the implementation of additional tracking methods and metrics.
Unlike plain HTML emails, AMP emails require you to first register with Google for each sender email that you use. Find Google’s registration guide here.

Prerequisites

Before you can start sending AMP emails using Amazon SES you need to:

Register with Google for each sender email that you use. Find Google’s registration guide here.
- Note: The registration with Google can take 5 days or more. If you would like to test sending an AMP email to your Gmail client, you can follow the Gmail documentation Test your AMP emails in Gmail. Additionally, Gmail users can use the Gmail AMP for Email Playground to send an email to themselves for testing.
Have a Gmail email address and access to your inbox via Gmail’s web client.
Have an AWS account – How to create an AWS account.
Have a verified domain in Amazon SES – Creating and verifying identities in Amazon SES.
Move your Amazon SES account out of the sandbox – Moving out of the Amazon SES sandbox.
Have Python and PIP installed on your computer.
Install AWS CLI on your computer – Installing or updating the latest version of the AWS CLI.
Install Boto3 on your computer – Install Boto3.

Sending an AMP email with Amazon SES

AMP emails rely on the MIME type of text/x-amp-html under the parent content type of multipart/alternative. In addition, the AMP content is structured similar to HTML, but it also offers a number of unique tags with a different structure. To start, the coded email body must start with <html ⚡4email> or <html amp4email>. You can find a comprehensive list of email AMP components here. For detailed information on the coding requirements, visit Create your first Amp Email.

To send an AMP email with Amazon SES, this blog uses the Amazon SES send_raw_email API operation. This API operation is more flexible than the send_email API operation, allowing you to send a multipart MIME email.

It’s recommended structuring your MIME part with text/plain first, text/x-amp-html second, and text/html last. This is because some email clients only render last MIME part. Furthermore, when a recipient replies or forwards an AMP email the text/x-amp-html MIME part is being removed, thus providing an alternative content in the HTML part is important.

In this example, the AMP email template makes use of the following two components:

amp-carousel: Displays multiple similar pieces of content along a horizontal axis.
amp-bind: Allows elements to mutate in response to user actions or data changes via data binding and simple JS-like expressions.

Important: Before executing the code below, make sure you have updated the sender and recipient email addresses.

import boto3
from botocore.exceptions import ClientError
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from email.mime.application import MIMEApplication
client = boto3.client('ses')

# Specify the sender email address recipient and subject of the email
sender = '<replace with email address from a verified Amazon SES sending identity>'
recipient = '<replace with an email address of an email client that supports AMP>'
subject = 'My first AMP email from Amazon SES'

# Plaintext email body for non-HTML email clients.
BODY_TEXT = "This is a TEXT email body!"

# The HTML email body for non-AMP email clients.
BODY_HTML = """\
<html>
<body>
  <h1>This is a HTML email body!</h1>
</body>
</html>    
"""

# The AMPHTML email body. This will render only for recipients with email clients that support AMP.
BODY_AMPHTML = """\
<!doctype html>
<html amp4email data-css-strict>
<head>
    <meta charset="utf-8">
    <script async src="https://cdn.ampproject.org/v0.js"></script>
    <style amp4email-boilerplate>body{visibility:hidden}</style>
    <style amp-custom>
    h1 { font-family: arial; margin: 10px;}
    body { background-color: #ffffff; font-size: 16px; color: #333333; font-family: Arial }
    .main-body { max-width: 600px; margin: 0 auto; text-align: center }
    .carousel { background: #f1f2f4; padding: 30px 0px }
    .btn { color: #fff; background: #009fc1; padding: 10px 35px; text-decoration: none; font-size: 22px }
    </style>
    <script async custom-element="amp-carousel" src="https://cdn.ampproject.org/v0/amp-carousel-0.1.js"></script>
    <script async custom-element="amp-bind" src="https://cdn.ampproject.org/v0/amp-bind-0.1.js"></script>
</head>
<body>
    <amp-state id="myState">
    <script type="application/json">
    {
        "CommunicationDeveloperServices": [
            {
            "name": "Amazon SES",
            "description": "Amazon Simple Email Service (Amazon SES) lets you reach customers confidently without an on-premises Simple Mail Transfer Protocol (SMTP) system."
        },
        {
            "name": "Amazon Pinpoint",
            "description": "Amazon Pinpoint is a flexible, scalable marketing communications service that connects you with customers over email, SMS, push notifications, or voice."
        },
        {
            "name": "Amazon Chime SDK",
            "description": "The Amazon Chime SDK allows builders to add real-time voice, video, and messaging powered by machine learning into their applications."
        }
        ]
    }
    </script>
    </amp-state>
    <div class="main-body">
    <h1>This is an AMP email body!</h1>
    <h3> <span [text]="myState.CommunicationDeveloperServices[currentService].name">Amazon SES</span> </h3>
    <p> <span [text]="myState.CommunicationDeveloperServices[currentService].description">Amazon Simple Email Service (Amazon SES) lets you reach customers confidently without an on-premises Simple Mail Transfer Protocol (SMTP) system.</span> </p>
    <amp-carousel class="carousel" width="400" height="200" type="slides" on="slideChange:AMP.setState({ currentService: event.index} )">
    <amp-img layout="fill" src="https://d2908q01vomqb2.cloudfront.net/632667547e7cd3e0466547863e1207a8c0c0c549/2023/06/01/Amazon-SES.png"  alt="Amazon SES"></amp-img>
    <amp-img layout="fill" src="https://d2908q01vomqb2.cloudfront.net/632667547e7cd3e0466547863e1207a8c0c0c549/2023/06/01/Amazon-Pinpoint.png"  alt="Amazon Pinpoint"></amp-img>
    <amp-img layout="fill" src="https://d2908q01vomqb2.cloudfront.net/632667547e7cd3e0466547863e1207a8c0c0c549/2023/06/01/Amazon-Chime-SDK.png"  alt="Amazon Chime SDK"></amp-img>
    </amp-carousel>
    <div class="btn"><a href="https://aws.amazon.com/business-applications/cds/" class="btn"> <span [text]="myState.CommunicationDeveloperServices[currentService].name"> Amazon SES </span> </a></div>
    </div>
</body>
</html>
"""

# The character encoding for the email.
CHARSET = "utf-8"

# Create a multipart/mixed parent container.
msg = MIMEMultipart('mixed')

# Adding the Subject, From and To lines.
msg['From'] = sender
msg['To'] = recipient
msg['Subject'] = subject 

# Create a multipart/alternative child container.
msg_body = MIMEMultipart('alternative')

# Encode the text and HTML content and set the character encoding.
text_part = MIMEText(BODY_TEXT.encode(CHARSET), 'plain', CHARSET)
amphtml_part = MIMEText(BODY_AMPHTML.encode(CHARSET), 'x-amp-html', CHARSET)
html_part = MIMEText(BODY_HTML.encode(CHARSET), 'html', CHARSET)

# Attach text first, AMP HTML second and HTML last.
msg_body.attach(text_part)
msg_body.attach(amphtml_part)
msg_body.attach(html_part)
msg.attach(msg_body)

try:
    #Provide the contents of the email.
    response = client.send_raw_email(
        Source=sender,
        Destinations=[
            recipient
        ],
        RawMessage={
            'Data':msg.as_string(),
        }
    )
# Display an error if something goes wrong.    
except ClientError as e:
    print(e.response['Error']['Message'])
else:
    print('Email sent! Message ID: ' + response['MessageId'])
    print(response)

The successful execution of the above code should return a response similar to the one below.

{
   "MessageId":"010001887b628794-5be12c16-b77f-4a72-968a-a65058239742-000000",
   "ResponseMetadata":{
      "RequestId":"a4fd35ef-d3a3-44a1-8c53-1d2f85224fb4",
      "HTTPStatusCode":200,
      "HTTPHeaders":{
         "date":"Fri, 02 Jun 2023 09:13:55 GMT",
         "content-type":"text/xml",
         "content-length":"338",
         "connection":"keep-alive",
         "x-amzn-requestid":"a4fd35ef-d3a3-44a1-8c53-1d2f85224fb4"
      },
      "RetryAttempts":0
   }
}

Conclusion

In this blog you learned what is an AMP email, what you should take into consideration before incorporating it to your email strategy and how to send one using Amazon SES.

Using AMP components in your emails can increase interactivity and consequently improve customer experience. The enhanced functionality offered by AMP comes with additional effort building such emails. That said email template tools have started incorporating AMP into their WYSIWG editors, making it easier for non-technical users to design interactive email templates with AMP components.

Analyze your customer database to see if their email clients support AMP and follow the steps outlined in this blog to start sending interactive AMP emails for a better customer experience.

About the Authors

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis is an Amazon Pinpoint and Amazon Simple Email Service Senior Specialist Solutions Architect at AWS. He enjoys diving deep into customers’ technical issues and help in designing communication solutions. In his spare time, he enjoys playing tennis, watching crime TV series, playing FPS PC games, and coding personal projects.

Improving email deliverability with new virtual deliverability manager features

2023-05-25 Vinay Ujjini

Post Syndicated from Vinay Ujjini original https://aws.amazon.com/blogs/messaging-and-targeting/improving-email-deliverability-with-new-virtual-deliverability-manager-features/

Background:

Email deliverability is a critical aspect of email as it directly impacts the success of campaigns by ensuring that messages reach the intended recipients’ inboxes. It encompasses factors like avoiding spam filters, maximizing open rates, and minimizing bounces. Companies often encounter deliverability problems due to various reasons, such as poor sender reputation, inadequate list hygiene, incorrect authentication, or engaging in spam-like behavior. Managing these problems can be challenging for companies because existing tools and techniques often fall short in addressing the ever-evolving email ecosystem. Factors such as evolving spam filters, complex email client algorithms, and constantly changing best practices necessitate continuous monitoring, optimization, and fine-tuning of email marketing strategies. Companies struggle to keep pace with these dynamic challenges and require advanced tools and expertise to effectively navigate deliverability obstacles.

Virtual deliverability manager (VDM) is an Amazon SES feature that helps you enhance email deliverability, like increasing inbox deliverability and email conversions, by providing insights into your sending and delivery data, and giving advice on how to fix the issues that are negatively affecting your delivery success rate and reputation. Virtual deliverability manager offers three components to help customers increase their email deliverability rates: deliverability insights, optimization recommendations, and detailed dashboards.

Visit this blog for details on the initial launch of virtual deliverability manager. After listening to our customers, we have added new features to virtual deliverability manager that this blog will address.

Graph & time view:

VDM now features a graph and time based view. The improvements in monitoring and diagnosing issues include the ability to view the data in graph form and with a timeline.

Graph view

The new metrics pane displays VDM metrics as time series graphs with Volume progression on the left and Rate progression on the right. Hovering over a date interval in the graphs will show the exact volume count or rate percentage based on a daily aggregation. You can use the ‘select metrics’ dropdown menu to filter and choose the metrics you want to view.

This depicts metrics like Sent, delivered, complaints.

Use case:

A time series graph displaying volume progression and rate progression can provide valuable insights to improve email deliverability for customers. Let’s consider a specific use case to illustrate this:

Use Case: Monitoring Engagement Metrics for Email Deliverability

Scenario: A business wants to enhance their email deliverability and ensure that their messages reach their subscribers’ inboxes. They are experiencing issues with low open rates and high spam complaints. To address these challenges, they decide to use a time series graph with volume progression and rate progression.

Implementation:

Volume Progression: By tracking the volume of emails sent over time, the business can observe any significant changes or spikes. A sudden increase in email volume might indicate a new marketing campaign or a change in email practices. Monitoring volume helps identify potential issues or anomalies that could impact deliverability. It allows businesses to correlate changes in volume with engagement metrics.

Rate Progression: Rate progression refers to tracking various engagement metrics over time, such as open rates, click-through rates, and unsubscribe rates. By analyzing these rates alongside email volume, the business can identify patterns or trends. For instance, a sudden drop in open rates may suggest that emails are landing in spam folders or being ignored by recipients. Conversely, a gradual increase in engagement rates indicates that emails are resonating with subscribers and likely to be delivered to their inboxes.

Benefits:

Early Detection of Issues: With a time series graph, businesses can detect delivery issues promptly. If a sudden drop in open rates or an increase in spam complaints is observed alongside a spike in email volume, it could indicate that the emails are being marked as spam or sent to the promotions tab. Identifying such issues early allows the business to take corrective measures, such as adjusting email content, optimizing subject lines, or reviewing their sender reputation.

Data-Driven Decision Making: By analyzing volume progression and rate progression, businesses gain valuable data to make informed decisions. They can assess the impact of changes in email strategies, content, or sending practices on deliverability. For instance, if a change in email design correlates with a decrease in open rates, the business can revert to a previous design or experiment with variations to optimize engagement.

Proactive Monitoring: By regularly monitoring volume and engagement metrics, businesses can proactively address potential deliverability issues. They can identify trends and take corrective action before they escalate. It enables businesses to maintain a positive sender reputation, improve email engagement, and optimize deliverability over time.

BIMI check in VDM Advisor:

Brand indicators for Message Identification (BIMI) is an email specification that enables email inboxes to display a brand’s logo next to the brand’s authenticated email messages within supported email clients. BIMI is an email specification that’s directly connected to authentication, but it’s not a standalone email authentication protocol as it requires all your email to comply with DMARC authentication.

While BIMI requires DMARC, DMARC requires your domain to have either SPF or DKIM records to align, however, it should be noted that both SPF and DKIM are recommended for best practice. Some email service providers (ESPs) require both when using BIMI. VDM’s Advisor now will let users know if BIMI records are not found or are not configured properly.

Image showing BIMI records missing and for how long

Breakdown by accounts, ISP, sending identity or config set:

Accounts: The VDM Account statistics table gives an overview of your deliverability and reputation metrics showing the total volume, rate percentage, and difference percentage for permanent/transient bounces, complaints, opens, clicks, sends, and delivered email.

Graph with account statistics

ISP: You can now get these metrics by accounts, Internet Service Providers (ISP), sending identities and configuration sets. ISP specific data is important when you have a deliverability issue to a specific ISP or mailbox provider. Instead of trying to adjust your entire account which may otherwise be doing well, you can focus on the problematic endpoint and align with its best practices to improve sender reputation to that ISP and restore good inbox deliverability in order to reach your recipients.

Sending identities: Sending identity table shows the deliverability metrics like send volume, delivered, bounces (transient & permanent), complaints, opens & clicks, broken down by each sending identity.

Image with breakdown by sending identities

When you can click on the sending identity, graphic cards will appear for the deliverability metrics and an ISP table will be displayed listing all the ISPs the sending identity sent mail to with bounce/complaint, open/click, and send/delivery rates given for each ISP within the date range entered.

Deliverability metrics by ISP

Configuration sets: The Configuration sets table displays deliver rates for each configuration set that’s been used to send mail for the date range entered in the account overview panel. To get specific configuration set details, you can click on any configuration set to see its graphics card with deliverability details pertaining to that configuration set. The corresponding ISP table will be displayed listing all the ISPs the configuration set was used to send mail to with bounce/complaint, open/click, and send/delivery rates given for each ISP within the date range entered.

Metrics per configuration set

Conclusion:

By adding VDM to Amazon SES, now customers can gather greater insight into their deliverability through more granular deliverability metric aggregations, automated detection of deliverability configuration problems (with resolution instructions), and optimization of sending through high reputation channels.

About the author:

Vinay Ujjini

Vinay Ujjini is an Amazon Pinpoint and Amazon Simple Email Service Worldwide Principal Specialist Solutions Architect at AWS. He has been solving customer’s omni-channel challenges for over 15 years. He is an avid sports enthusiast and in his spare time, enjoys playing tennis & cricket.

New capabilities make sending emails with dedicated IPs even easier

2023-05-24 gleholm

Post Syndicated from gleholm original https://aws.amazon.com/blogs/messaging-and-targeting/new-capabilities-make-sending-emails-with-dedicated-ips-even-easier/

SES’ New Managed Dedicated IP Capabilities

In the world of email, sending emails over dedicated IPs can provide a number of benefits, including greater control over reputation and the ability to manage email deliverability. However, the heavy lifting involved in setting up and managing a dedicated IP can be intimidating, especially for those who are new to the process.

In this blog post, we’ll explain the benefits of using managed dedicated IPs from Amazon SES and how the new features, whether you’re a seasoned email professional or just getting started, will help take your email sending to the next level. Throughout the post, we’ll explain how managed dedicated IPs removes all of the heavy lifting of setting up, provisioning, monitoring and managing sending over dedicated IPS, how it helps ensure that IPs warm up efficiently, effectively and quickly, how you can get insights into how your IP Pool is performing, and how you can easily convert your current standard dedicated IPs to managed dedicated IPs. By the end of this post, you’ll have a solid understanding of the benefits of using a managed dedicated IPs for your email sending.

With the latest release of managed dedicated IPs, SES customers now have insights into the performance of their dedicated IPs, can easily convert their current dedicated IPs (standard) Pools into managed dedicated IP pools and can now create up to 50 dedicated IP pools.

What are managed dedicated IPs?

With standard IPs, users are provided an aggregate sending capacity, which is a combination of the capacity of all receiving Internet Service Providers (ISPs) such as Gmail, Yahoo, Outlook etc. However, each receiving ISP has a different capacity that depends on the reputation they assign to your dedicated IPs. For example, if you have a recipient list that consists of 800,000 individuals and your IP has a daily capacity of 1M emails. In this case you may assume that you have enough capacity to send all of your daily emails via your dedicated IP. However, the daily IP capacity may be made up of 400,000 of Gmail capacity and 600,000 of Yahoo capacity. If your recipient list consists of 500,000 Gmail addresses and 300,000 Yahoo addresses then your capacity on Yahoo is fine but you are over your capacity on Gmail and you will require another dedicated IP to manage your sending volume to Gmail. This is a simplistic example consisting of two major ISPs. Now consider how many different ISPs they are and how many different recipient email domains that your list consists of and you can easily see how complex this problem becomes. To solve this, many customer over-provision dedicated IPs i.e. they lease extra just in case they need the capacity. However, this approach doesn’t work as to build reputation (and capacity) on an IP you must send regularly and at a sufficient volume. You can’t just lease and hold a dedicated IP just in case you need it sometime.

With dedicated IP’s managed, your sending capacity is calculated at an ISP level, not at an aggregate level. What this means is that if your recipient list contains a large proportion of emails that get sent to a single ISP, then dedicated IPs managed will allocate you the optimal number of IPs to manage your sending capacity for that ISP. Customers don’t have to monitor the volume and manually calculate how many IPs they need to request, managed dedicated IPs does it all for you. When you need more IPs to manage your volume, managed dedicated IPs automatically provisions them, removing all of the heavy lifting and guesswork involved. And to give you insights into this, there is a range of metrics, accessible from within the SES console.

There are now two simple ways to get started with managed dedicated IPs. Users can just log on the AWS Console and load the configuration section of the SES Console. Select “Dedicated IPs”, and click “Enable dedicated IPs”.

How to set up managed dedicated IPs in SES If you have previously leased several standard dedicated IP addresses to use with Amazon SES, you would have created groups of those addresses, called dedicated IP pools. Grouping dedicated IPs together in a pool makes them easier to manage. A common scenario is to create one pool for sending marketing communications, and another for sending transactional emails. Your sender reputation for transactional emails is then isolated from that of your marketing emails. In this scenario, if a marketing campaign generates a large number of complaints, the delivery of your transactional emails is not impacted. If you already have dedicated IPs (standard) pools you can now convert these pools to managed IP pools in one click. Converting from a dedicated IP (standard) pool to a dedicated IP (managed) pool automatically transfers all of your standard IPs over to managed. So, you keep the same set of IPs, but you no longer have to worry about monitoring and managing them. You will no longer be charged per IP that you lease, but rather by the volume of emails that you send. For more information on pricing see the SES pricing page. There is one thing to be aware of before you switch; As managed dedicated IPs is optimized to allocate the number of IPs that you need for your sending volumes, if you have leased more dedicated IPs (standard) than you need then the redundant IPs will be removed. The only use case this may affect is where you have allowlisted specific dedicated IPs in downstream applications. You will need to be careful as managed dedicated IPs may automatically remove these IPs and you will no longer be sending emails from them. In this case, we recommend that you don’t switch to managed dedicated IPs.

standard vs managed dedicated IP pools

How to create managed dedicated IPs

To create your dedicated IP (managed) pool, in Scaling mode select “Managed (auto managed)”, enter a name for your IP pool and associate it with a configuration set. Click ”Create Pool“ and your pool will be created. Previously, you could only create 5 managed dedicated IPs pools but with this release, you can now create 50 dedicated IP Pools (this can be 50 managed dedicated IPs pools or 50 dedicated IPs (standard) pools or a combination of both not exceeding 50).

Once you have created your pool you can access it from the Managed IP pools tab on the Dedicated IPs page. Once you have started to send email via your managed dedicated IPs pool then you can access metrics on the performance of your IPs. when you first start sending via managed dedicated IPs, your email volume will be split between your dedicated IPs and the shared SES IP pool. This is because when IPs are new, they have no reputation with receiving ISPs and therefore, have little capacity. Over time, receiving ISPs, based on variables such as the quality of content, the engagement rates, and your sending patterns, will increase capacity. If you try to send too much volume immediately via your dedicated IPs, receiving ISPs may flag this as a spam attempt and block, or throttle, the IP. managed dedicated IPs ensures that this will never happen by only sending the capacity that each receiving ISPs is willing to accept from the IPs and routes the excess to a shared IP pool to ensure that your email is still sent. This helps to build trust and quickly increase the capacity of the IPs, leading to a better reputation and quicker warm-up times. Over time, you will see a decrease in your email volume being sent via shared until all of it gets routed via dedicated IPs. Once your IPs are fully warmed up, if there is a spike in your email volume that exceeds the receiving ISP capacity, instead of routing the excess volume through shared, managed dedicated IPs will queue the excess volume and retry when there is capacity. This ensures that, when you have established reputation on your dedicated IPs, then all of your sending will be routed through them. You can easily see how much of your traffic is being sent via dedicated IPs versus how much is being routed through shared IPs in the Managed pool volume and Percentage of send volume via dedicated IPs metrics.

dedicated IPs metrics and reporting

One of the key benefits of managed dedicated IPs is the ability to see your IP performance at an ISP level. Senders are able to see how much volume each ISP will accept from their IP pool. This helps to reassure that you are building reputation with the receiving ISPs and also gives you insights into the available contingent capacity should you have a sudden spike in volume. The ISP Capacity metric provides insights to your sent email volume versus the available capacity of the top 10 most popular ISPs.

dedicated IP ISP capacity metric

Dedicated IPs (managed) is in all regions supported by SES. Visit the Amazon SES documentation for more information.

Increase email sending success rates with Amazon Simple Email Services (SES) new Virtual Deliverability Manager

2022-11-03 sakoppes

Post Syndicated from sakoppes original https://aws.amazon.com/blogs/messaging-and-targeting/increase-email-sending-success-rates-with-amazon-simple-email-services-ses-new-virtual-deliverability-manager/

Email is a ubiquitous channel for businesses to communicate with their customers. Businesses rely on email to market products and services, communicate transactional information, and keep customers in-the-know about product changes and events. To realize the full value of this rich and established communication channel, businesses need tools to monitor and optimize their email delivery success rates to make every email count.

Amazon Simple Email Service (Amazon SES) is a cost-effective, flexible, and scalable email service that enables customers to send mail from within any application. SES now offers a new feature, Virtual Deliverability Manager, which helps customers optimize their sending strategies to increase delivery success rates and email campaign return on investment. Virtual Deliverability Manager offers three components to help customers increase their email deliverability rates: deliverability insights, optimization recommendations, and automatic implementation.

Virtual Deliverability Manager’s dashboard gives customers deeper insights into deliverability metrics such as bounce, complaint, open, click-through, and successful delivery rates in their accounts at multiple levels such as by sending email address, by email provider, or by SES configuration set. This makes it easy to quickly check the status and trend of sending health.

Virtual Deliverability Manager analyzes sending configurations and provides automatic recommendations about how to increase sending success. This helps customers make changes such as DKIM configuration (Domain Keys Identified Mail) to increase the likelihood of successful delivery.

Virtual Deliverability Manager can automatically adjust email sending in real time, to reduce the chances that emails are marked as spam. This helps protect against transient changes in email provider spam management, greatly reducing the impact of sudden increases in provider rejection rates.

It’s easy to get started with SES VDM, just log on the AWS Console and load the configuration section of the SES Console. Select “Configuration”, and click “Get started with Virtual deliverability manager”.

You will be prompted to make several configuration choices when enabling Virtual Deliverability Manager. First, you can choose whether you want to be able to see when HTML emails you send through SES are opened, or when customers click on links within those emails. You will be able to see open and click rates in Virtual Deliverability Manager, grouped at the account, mailbox provider (e.g Gmail), sending email address (sender identity), and configuration set level. SES monitors open rates by inserting an image pixel in the HTML message body, which sends a notification to SES when email clients download the image while processing the message. SES monitors clicks on hyperlinks in messages, by wrapping links embedded in HTML messages with a link the SES service. When clicked, SES counts the click, and sends the request to the original destination. To see these metrics, turn on engagement tracking in Step 1.

Next you can choose whether you want Virtual Deliverability Manager to dynamically select the best IP addresses available for sending your emails. If you are using shared IP addresses, this makes sure that your outbound emails are sent over the highest reputation shared IP addresses available in SES. This increases the likelihood that your messages will be delivered to your customers’ inboxes. When using dedicated IP addresses, this ensures that outbound traffic spilling over to shared IP addresses while dedicated IP addresses are warming up use the highest reputation addresses available. This makes it less likely that your messages will be marked as spam while dedicated IP addresses warm up, and it works for both managed and unmanaged dedicated IP addresses.

Follow the prompts on the next screen to enable Virtual Deliverability Manager, and you can change your choices for these two configuration options at any time. Virtual Deliverability Manager is available in all regions supported by SES, visit the Amazon SES documentation for more information.

Tracking email engagement with AWS Analytics services

2022-08-31 Justin Morris

Post Syndicated from Justin Morris original https://aws.amazon.com/blogs/messaging-and-targeting/tracking-email-engagement-with-aws-analytics-services/

Email at scale is one of the most valuable forms of reaching and engaging with customers, but creating the most engaging email content means frequent tracking, monitoring, and analyzing bounces, complaints, opens, and clicks. Managing your reputation as a sender and understanding user engagement have never been more important and play a crucial role in email deliverability and inbox placement.

Amazon Simple Email Service (Amazon SES) is a cost-effective, flexible, and scalable email service that enables customers to send mail from within any application. You can configure Amazon SES quickly to support several email use cases, including transactional, marketing, or mass email communications. An additional benefit of using Amazon SES is event publishing which enables you to track your email sending, feedback, and user engagement at a granular level. AWS also offers analytics tools like Amazon Athena and Amazon QuickSight, which can be connected to SES for even deeper insights capabilities.

In this post, we will walk you through how to create an end-to-end solution for event analysis and how to build custom dashboards to analyze SES utilization, user engagement, and sender reputation.

Prerequisites

The solution presented in this blog requires the following prerequisites to be satisfied before continuing:
An AWS Account that provides access to AWS services.
An AWS Identity and Access Management (IAM) user with the permissions to create an IAM role and policies, and create stacks in AWS CloudFormation.
An Amazon Simple Email Service (Amazon SES) verified identity. You can create and verify a sending identity by following the steps described in the documentation.
Your Amazon SES account is out of the sandbox.
Use a region where AWS Glue DataBrew is available.

Solution Overview

The Figure below describes the architecture diagram for the proposed solution.

Amazon SES publishes email sending events to Amazon Kinesis Data Firehose using a default configuration set.
Amazon Kinesis Data Firehose Delivery Stream stores event data in an Amazon Simple Storage Service (Amazon S3) bucket, known as the Destination bucket.
AWS Glue DataBrew processes and transforms event data in the Destination bucket. It applies the transformations defined in a recipe to the source dataset and stores the output using a different prefix (‘/partitioned’) within the same bucket. Output objects are stored in the Apache Parquet format and partitioned.
An AWS Lambda function copies the resulting output objects to the Aggregation bucket. The Lambda function is invoked asynchronously via Amazon S3 event notifications when objects are created in the Destination bucket.
An AWS Glue crawler runs periodically over the event data stored in the Aggregation bucket to determine its schema and update the table partitions in the AWS Glue Data Catalog.
Amazon Athena queries the event data table registered in the AWS Glue Data Catalog using standard SQL.
Amazon QuickSight dashboards allow visualizing event data in an interactive way via its integration with Amazon Athena data sources.

Figure 1. Serverless architecture for processing Amazon SES Email Sending Events at scale

Solution Deployment

After all the prerequisites listed above are met, the sequence of steps required to deploy the solution is summarized as follows:

Deploy the CloudFormation template in your desired AWS Region.
Configure your Amazon SES verified identity.
Send emails via the Amazon SES API or the Amazon SES Simple Mail Transfer Protocol (SMTP) interface.
Configure AWS Glue DataBrew to transform the source dataset as required.
Create an Amazon QuickSight dashboard to visualize email sending events.

Step 1: Deploy the CloudFormation template in your desired AWS Region

The serverless pipeline described in this post is available as a CloudFormation template at this link.

To deploy the template using the CloudFormation console:

Browse to this URL – make sure to switch to desired AWS Region using the navigation bar.
In the Stack name section, enter a name for the Stack.
Choose Create Stack.

This template will deploy the required resources to implement the serverless pipeline. The deployment takes less than 5 minutes to complete.

Step 2: Configure your Amazon SES verified identity

Amazon SES leverages configuration sets to define groups of rules that you can apply to your verified identities. In this case, the CloudFormation template deployed in the previous step defines a configuration set that uses an event destination based on Amazon Kinesis Data Firehose. In this step you are going to configure your verified identity to use the configuration set “SESConfigurationSet” by default.

Open the Amazon SES console.

In the navigation pane, under Configuration, choose Verified identities.
In the Identity column, select the verified identity that you want to edit.
On the identity’s detail page, select the Configuration set tab and then choose Edit.
In the Default configuration set page, check the Assign a default configuration set box.
For the Default configuration set dropdown, choose SESConfigurationSet.

Step 3: Send emails via the Amazon SES API or the Amazon SES Simple Mail Transfer Protocol (SMTP) interface

To test the pipeline, you can start sending emails using the Amazon SES API or the SMTP interface. During Step 2 you have associated the SESConfigurationSet as the default configuration set for the verified identity. As a result, all the emails you are going to send from the verified identity will generate email sending events that will flow through the Amazon Kinesis Data Firehose Delivery Stream and persisted in the Amazon S3 Destination bucket under the ‘/raw’ prefix.

As an alternative to the Amazon SES API or the SMTP interface, it is possible to quickly send test emails using the Amazon SES mailbox simulator:

Open the Amazon SES console.
In the navigation pane, under Configuration, choose Verified identities.
In the Identity column, select the verified identity that you want to send a test email from.
Choose Send Test Email.
On the Send test email page, for From-address enter the desired value, for example sender.
For Scenario, choose the email sending scenario that you want to simulate, for example Success Delivery.
For Subject, enter the desired email subject.
For Body, type an optional body text.
For Configuration set, you can leave the field empty if you have associated a default configuration set to this identify in Step 2. Choose the SESConfigurationSet otherwise.
Choose Send test email to send the email.

Figure 2. Sending a test email using the Amazon SES mailbox simulator

The Amazon Kinesis Data Firehose Delivery Stream has been configured to buffer data for up to 5 MiB or 60 seconds before delivering it to the Amazon S3 Destination bucket. To verify that the delivery is working as expected, wait for 60 seconds and then:

Open the Amazon S3 console.
In the navigation pane, choose Buckets.
Choose the bucket named <ACCOUNT-ID>-<REGION>-ses-events-destination (replace <ACCOUNT-ID> and <REGION> based on your actual environment).
Navigate to the ‘/raw‘ prefix and verify it contains some compressed objects (the number of objects depends on the volume of email sending events triggered).

As part of this blog post, we are providing you with sample email sending events that will be used to populate the Amazon QuickSight dashboard in Step 5. You can use AWS CloudShell in your own AWS account to run the command below and synchronize the sample events to your Amazon S3 bucket:

aws s3 sync s3://ses-blog-assets/sample-ses-email-events/ s3://<ACCOUNT-ID>-<REGION>-ses-events-destination/

Remember to replace <ACCOUNT-ID> and <REGION> based on your actual environment.

Step 4: Configure AWS Glue DataBrew to transform the source dataset as required

In this step we will go over the configuration of DataBrew, a visual data preparation tool, to clean and prepare the data for analysis in QuickSight. The CloudFormation template you deployed in previous steps already created a DataBrew dataset for you. In this step, you will import a DataBrew recipe that contains the transformations necessary to analyze the SES email sending events, and then create a DataBrew job to process the data.

Upload a DataBrew recipe
To upload a recipe, you should follow the steps below:
1. Download this JSON file that contains the recipe.
2. Navigate to the DataBrew console, on the left panel choose Recipes.
3. In the Recipes console, on the top left choose Upload recipe.
4. For Recipe name provide the following ses-clean-recipe.
5. For Upload recipe choose Upload and select the file you downloaded in Step 1.
6. Choose Create and publish recipe.
Create and Schedule a DataBrew job
Create a job to apply the recipe you created above on the dataset and configure it to output a parquet file. To do that, complete the following steps:
1. In the DataBrew console, at top right of the project editor, choose the Create job button.
2. For Job name, enter ses-event-transformed.
3. In the Job Input section, for Choose dataset choose the dataset named SESDataBrewDataset. Select the recipe imported in the previous step.
4. For output choose Amazon S3.
5. For File type, choose PARQUET and compression as GZIP.
6. For S3 location provide the following S3 Bucket that has been created for you by the CloudFormation template: ‘s3://<ACCOUNT-ID>-<REGION>-ses-events-destination/partitioned/‘ (replace <ACCOUNT-ID> and <REGION> based on your environment).
7. Choose Settings. For File output storage choose Replace output files for each job run. For Custom partition by column values choose Enabled, and on Columns to partition by add the following: year, month, day, hour (in this order). Choose Save.
8. Expand the Associated schedules section, choose Create new schedule and create a schedule that runs every hour and every day. Choose Add.
9. Under Permissions, for Role name, choose an existing role or create a new one.
10. Choose Create and run job.
11. Navigate to the Jobs page and wait for the ses-event-transformed job to complete.
12. Choose the Destination link to navigate to Amazon S3 to access the job output.

You have now created a DataBrew job that would transform your dataset based on the recipe you created above.

The SESDataBrewDataset dataset has been created to fetch only the email sending events generated in the past 1 hour to allow for a more efficient differential processing. At the end of each job execution, a Lambda function copies the latest transformed data to the Aggregation bucket named <ACCOUNT-ID>-<REGION>-ses-events-destination-aggregated. Every hour, an AWS Glue crawler scans the Aggregation bucket to update the Glue Catalog accordingly. Before moving to the next Step, either wait for the next scheduled crawler run or browse to the AWS Glue console and run the crawler called SESEventDataCrawler manually.

Step 5: Create an Amazon QuickSight dashboard to visualize email sending events

Now that your cleaned dataset is ready, you are ready to deploy your dashboard. For this blog post we are providing you with a sample QuickSight dashboard. Follow the steps below to deploy it as is. However, if you would like to have a custom dashboard with your own metrics and KPIs and want to use your BI tool of choice, you can leverage the output of the DataBrew job that is stored in S3 to build your own analysis and deploy it in a dashboard.

To deploy the sample dashboard, you first need to create a QuickSight data source, a QuickSight dataset and then a QuickSight dashboard. These resources are going to be deployed using the AWS CLI. You can use AWS CloudShell in your own AWS account to run the commands below. Remember to replace <ACCOUNT-ID> and <REGION> in the following commands based on your actual environment.

Create a QuickSight data source
1. 1. Download this JSON file which contains the data source definition.
  2. Open the file and change the following:
    1. Principal: the QuickSight users authorized to use the dataset. You can use the procedure described in the documentation to retrieve the QuickSight Principal ARN.
  3. Apply the command below to create the data source.
aws quicksight create-data-source --aws-account-id <ACCOUNT-ID> --cli-input-json file://create-data-source.json --region <REGION>
1. Record the data source ARN.
Create a QuickSight dataset
1. 1. Download this JSON file which contains the dataset definition.
  2. Open the file and change the following:
    1. DataSourceArn: data source ARN from Step a.
    2. ImportMode: if you use QuickSight Enterprise, you can use SPICE. Otherwise, use DIRECT_QUERY.
    3. Principal: the list of QuickSight users ARNs authorized to use the dataset.
  3. Apply the command below to create the data source
aws quicksight create-data-set --aws-account-id <ACCOUNT-ID> --cli-input-json file://create-dataset.json --region <REGION>
1. Record the dataset ARN
Create the Quicksight Dashboard
1. 1. Download this JSON file which contains the dataset definition.
  2. Open the file and change the following:
    1. Principal: the list of QuickSight users ARNs authorized to view the dashboard.
    2. DataSetArn: Dataset ARN from Step b.
  3. Apply the command below to create the data source.
aws quicksight create-dashboard --aws-account-id <ACCOUNT-ID> --cli-input-json file://create-dashboard.json --region <REGION>

After completing the three steps above, you will see a dashboard like the one below populated with your own data.

Figure 3. The sample QuickSight dashboard visualizing SES Email Sending Events

Cleaning up

You should have now successfully produced a first analysis of your Amazon SES email sending events. To avoid incurring any extra charges, remember to delete any resources created manually following the instructions in this blog post and delete the CloudFormation stack that you deployed.

Conclusion

In this blog post we showed you how you can collect your SES email sending events through Firehose, prepare them with DataBrew and visualize them with QuickSight. This solution provides you with a starting point that you can customize at your will to meet your organization requirements. This solution is set to run on a scheduled basis, every hour, making it easier for you to leverage incremental data processing.

You can also augment your event data with your business data by joining them through DataBrew. Finally, you can also leverage Amazon AppFlow (a SaaS integration service) with DataBrew to easily integrate data from third party solutions that you are already using.

About the Authors

Luca Iannario is a Sr. Solutions Architect at AWS within the Public Sector team in EMEA. He works with customers of all sizes across Government, Education, Healthcare and NPO verticals, helps them deploying AWS Services securely at scale and facilitates their cloud adoption journey. His goal is to build better societies by bringing the benefits of the cloud to all citizens. In his spare time, Luca enjoys traveling and watching movies.

Lotfi Mouhib

Lotfi Mouhib is a Senior Solutions Architect working for the Public Sector team with Amazon Web Services. He helps public sector customers across EMEA realize their ideas, build new services, and innovate for citizens. In his spare time, Lotfi enjoys cycling and running

Justin Morris

Justin Morris is a Email Deliverability Manager for the Simple Email Service team. With over 10 years of experience in the IT industry, he has developed a natural talent for diagnosing and resolving customer issues and continuously looks for growth opportunities to learn new technologies and services.

How to investigate what happened to the email that was sent via SES but was never received in recipient inbox

Prerequisites:

Scenario 1: Emails were not sent successfully by your email application

Scenario 2: Emails get dropped within SES after being received from your email application

Actions to troubleshoot such issues:

What is an AMP email?

Considerations

Prerequisites

Sending an AMP email with Amazon SES

Conclusion

Pavlos Ioannou Katidis

Background:

Graph & time view:

Use case:

Use Case: Monitoring Engagement Metrics for Email Deliverability

Implementation:

Benefits:

BIMI check in VDM Advisor:

Breakdown by accounts, ISP, sending identity or config set:

Conclusion:

SES’ New Managed Dedicated IP Capabilities

What are managed dedicated IPs?

How to create managed dedicated IPs

Prerequisites

Solution Overview

Solution Deployment

Step 1: Deploy the CloudFormation template in your desired AWS Region

Step 2: Configure your Amazon SES verified identity

Step 3: Send emails via the Amazon SES API or the Amazon SES Simple Mail Transfer Protocol (SMTP) interface

Step 4: Configure AWS Glue DataBrew to transform the source dataset as required

Step 5: Create an Amazon QuickSight dashboard to visualize email sending events

Cleaning up

Conclusion

About the Authors

The collective thoughts of the interwebz