All posts by Dustin Taylor

An Overview of Bulk Sender Changes at Yahoo/Gmail

Post Syndicated from Dustin Taylor original https://aws.amazon.com/blogs/messaging-and-targeting/an-overview-of-bulk-sender-changes-at-yahoo-gmail/

In a move to safeguard user inboxes, Gmail and Yahoo Mail announced a new set of requirements for senders effective from February 2024. Let’s delve into the specifics and what Amazon Simple Email Service (Amazon SES) customers need to do to comply with these requirements.

What are the new email sender requirements?

The new requirements include long-standing best practices that all email senders should adhere to in order to achieve good deliverability with mailbox providers. What’s new is that Gmail, Yahoo Mail, and other mailbox providers will require alignment with these best practices for those who send bulk messages over 5000 per day or if a significant number of recipients indicate the mail as spam.

The requirements can be distilled into 3 categories: 1) stricter adherence to domain authentication, 2) give recipients an easy way to unsubscribe from bulk mail, and 3) monitoring spam complaint rates and keeping them under a 0.3% threshold.

* This blog was originally published in November 2023, and updated on January 12, 2024 to clarify timelines, and to provide links to additional resources.

1. Domain authentication

Mailbox providers will require domain-aligned authentication with DKIM and SPF, and they will be enforcing DMARC policies for the domain used in the From header of messages. For example, gmail.com will be publishing a quarantine DMARC policy, which means that unauthorized messages claiming to be from Gmail will be sent to Junk folders.

Read Amazon SES: Email Authentication and Getting Value out of Your DMARC Policy to gain a deeper understanding of SPF and DKIM domain-alignment and maximize the value from your domain’s DMARC policy.

The following steps outline how Amazon SES customers can adhere to the domain authentication requirements:

Adopt domain identities: Amazon SES customers who currently rely primarily on email address identities will need to adopt verified domain identities to achieve better deliverability with mailbox providers. By using a verified domain identity with SES, your messages will have a domain-aligned DKIM signature.

Not sure what domain to use? Read Choosing the Right Domain for Optimal Deliverability with Amazon SES for additional best practice guidance regarding sending authenticated email. 

Configure a Custom MAIL FROM domain: To further align with best practices, SES customers should also configure a custom MAIL FROM domain so that SPF is domain-aligned.

The table below illustrates the three scenarios based on the type of identity you use with Amazon SES

Scenarios using example.com in the From header DKIM authenticated identifier SPF authenticated identifier DMARC authentication results
[email protected] as a verified email address identity amazonses.com email.amazonses.com Fail – DMARC analysis fails as the sending domain does not have a DKIM signature or SPF record that matches.
example.com as a verified domain identity example.com email.amazonses.com Success – DKIM signature aligns with sending domain which will cause DMARC checks to pass.
example.com as a verified domain identity, and bounce.example.com as a custom MAIL FROM domain example.com bounce.example.com Success – DKIM and SPF are aligned with sending domain.

Figure 1: Three scenarios based on the type of identity used with Amazon SES. Using a verified domain identity and configuring a custom MAIL FROM domain will result in both DKIM and SPF being aligned to the From header domain’s DMARC policy.

Be strategic with subdomains: Amazon SES customers should consider a strategic approach to the domains and subdomains used in the From header for different email sending use cases. For example, use the marketing.example.com verified domain identity for sending marketing mail, and use the receipts.example.com verified domain identity to send transactional mail.

Why? Marketing messages may have higher spam complaint rates and would need to adhere to the bulk sender requirements, but transactional mail, such as purchase receipts, would not necessarily have spam complaints high enough to be classified as bulk mail.

Publish DMARC policies: Publish a DMARC policy for your domain(s). The domain you use in the From header of messages needs to have a policy by setting the p= tag in the domain’s DMARC policy in DNS. The policy can be set to “p=none” to adhere to the bulk sending requirements and can later be changed to quarantine or reject when you have ensured all email using the domain is authenticated with DKIM or SPF domain-aligned authenticated identifiers.

2. Set up an easy unsubscribe for email recipients

Bulk senders are expected to include a mechanism to unsubscribe by adding an easy to find link within the message. The February 2024 mailbox provider rules will require senders to additionally add one-click unsubscribe headers as defined by RFC 2369 and RFC 8058. These headers make it easier for recipients to unsubscribe, which reduces the rate at which recipients will complain by marking messages as spam.

There are many factors that could result in your messages being classified as bulk by any mailbox provider. Volume over 5000 per day is one factor, but the primary factor that mailbox providers use is in whether the recipient actually wants to receive the mail.

If you aren’t sure if your mail is considered bulk, monitor your spam complaint rates. If the complaint rates are high or growing, it is a sign that you should offer an easy way for recipients to unsubscribe.

How to adhere to the easy unsubscribe requirement

The following steps outline how Amazon SES customers can adhere to the easy unsubscribe requirement:

Add one-click unsubscribe headers to the messages you send: Amazon SES customers sending bulk or potentially unwanted messages will need to implement an easy way for recipients to unsubscribe, which they can do using the SES subscription management feature.

Mailbox providers are requiring that large senders give recipients the ability to unsubscribe from bulk email in one click using the one-click unsubscribe header, however it is acceptable for the unsubscribe link in the message to direct the recipient to a landing page for the recipient to confirm their opt-out preferences.

To set up one-click unsubscribe without using the SES subscription management feature, include both of these headers in outgoing messages:

  • List-Unsubscribe-Post: List-Unsubscribe=One-Click
  • List-Unsubscribe: <https://example.com/unsubscribe/example>

When a recipient unsubscribes using one-click, you receive this POST request:

POST /unsubscribe/example HTTP/1.1
Host: example.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 26
List-Unsubscribe=One-Click

Gmail’s FAQ and Yahoo’s FAQ both clarify that the one-click unsubscribe requirement will not be enforced until June 2024 as long as the bulk sender has a functional unsubscribe link clearly visible in the footer of each message.

Honor unsubscribe requests within 2 days: Verify that your unsubscribe process immediately removes the recipient from receiving similar future messages. Mailbox providers are requiring that bulk senders give recipients the ability to unsubscribe from email in one click, and that the senders process unsubscribe requests within two days.

If you adopt the SES subscription management feature, make sure you integrate the recipient opt-out preferences with the source of your email sending lists. If you implement your own one-click unsubscribe (for example, using Amazon API Gateway and an AWS Lambda function), make sure it designed to suppress sending to email addresses in your source email lists.

Review your email list building practices: Ensure responsible email practices by refraining from purchasing email lists, safeguarding opt-in forms from bot abuse, verifying recipients’ preferences through confirmation messages, and abstaining from automatically enrolling recipients in categories that were not requested.

Having good list opt-in hygiene is the best way to ensure that you don’t have high spam complaint rates before you adhere to the new required best practices. To learn more, read What is a Spam Trap, and Why You Should Care.

3. Monitor spam rates

Mailbox providers will require that all senders keep spam complaint rates below 0.3% to avoid having their email treated as spam by the mailbox provider. The following steps outline how Amazon SES customers can meet the spam complaint rate requirement:

Enroll with Google Postmaster Tools: Amazon SES customers should enroll with Google Postmaster Tools to monitor their spam complaint rates for Gmail recipients.

Gmail recommends spam complaint rates stay below 0.1%. If you send to a mix of Gmail recipients and recipients on other mailbox providers, the spam complaint rates reported by Gmail’s Postmaster Tools are a good indicator of your spam complaint rates at mailbox providers who don’t let you view metrics.

Enable Amazon SES Virtual Deliverability Manager: Enable Virtual Deliverability Manager (VDM) in your Amazon SES account. Customers can use VDM to monitor bounce and complaint rates for many mailbox providers. Amazon SES recommends customers to monitor reputation metrics and stay below a 0.1% complaint rate.

Segregate and secure your sending using configuration sets: In addition to segregating sending use cases by domain, Amazon SES customers should use configuration sets for each sending use case.

Using configuration sets will allow you to monitor your sending activity and implement restrictions with more granularity. You can even pause the sending of a configuration set automatically if spam complaint rates exceed your tolerance threshold.

Conclusion

These changes are planned for February 2024, but be aware that the exact timing and methods used by each mailbox provider may vary. If you experience any deliverability issues with any mailbox provider prior to February, it is in your best interest to adhere to these required best practices as a first step.

We hope that this blog clarifies any areas of confusion on this change and provides you with the information you need to be prepared for February 2024. Happy sending!

Helpful links:

How to send your first email on SES

Post Syndicated from Dustin Taylor original https://aws.amazon.com/blogs/messaging-and-targeting/how-to-send-your-first-email-on-ses/

Introduction

Sending your first email on any service can be complicated. In this blog we will walk you through how to send your first email on Amazon Simple Email Service (SES) through the SES Console and to direct you to examples of how you can send email through the AWS SDK. Our public documentation includes additional information on how you can configure SES. We encourage you to read through these documents to learn about these other mechanisms in the future.

Getting Started

Getting started with sending an email on SES requires three actions which are: 1) verifying a domain or email address 2) requesting production access to SES and 3) sending your first email. Let’s walk through each of these steps and send our first email.

Verifying an Identity

To start, you will configure what email address or domain your customers will receive emails from. As part of this verification, you will need to be able to either receive a confirmation email at the email address you are trying to setup, or to publish CNAME records for your intended domain. Generally, we recommend using a domain for your email sending as this gives you the ability to set up SPF, DKIM, and DMARC alignment which will increase recipient trust in your emails. Email addresses can be used for account-specific email sending where a customer may not own a domain, but this type of use-case is prone to receiving entities having low trust in the sender and a lower probability of inbox placement. For more in-depth instructions please review our public documentation as I will briefly touch on the most important pieces to verifying a domain or email address.

To verify an identity, you can go to the SES Console and click the ‘Verified identities’ link on the left-hand side of the screen. It will then present you with a list of verified domains or email addresses currently in your account if they were previously verified. There is a yellow button that states ‘Create identity’, when you click this you will be presented with a screen to choose whether to verify an email address or domain.

Email Address Verification

To verify an email address, you will be prompted with the following dialog:

The dialog presented to a sender when they choose to verify an email address in the SES console.

To verify an email address to use as your sending identity, you will include the address in the ‘Email address’ field and then click the ‘Create identity’ button. This will trigger an automated email to the address with a verification link that will need to be clicked to verify ownership of the email address. Once verified, you can begin sending emails from your new email address identity.

Domain Verification

To start verifying a domain you will click the ‘Verified identities’ option from the ‘Configuration’ dropdown which can be found on the left side of the screen. When choosing to verify a domain, you will be presented with a series of dialogs which include:

The dialog presented to a sender that prompts a decision to verify a domain or email address.

Here you will need to include the domain you intend to use for email sending. If you are keeping to a basic configuration on SES this will be the only data you need to add to this dialog. However, it is recommended to also use a custom mail-from. A custom mail-from is a way for you to remove the amazonses.com domain from your mail-from header to ensure domain alignment throughout your headers. You can find more information about the custom mail-from addresses in our documentation.

After finishing your changes in the first dialog you will then be presented with a second dialog that looks like the following:

The dialog which allows a sender to verify the domain they intend to use to send email.

To verify the domain, you will need to utilize either the Easy DKIM feature, or to provide a DKIM authentication token if you plan to DKIM sign your own messages. In selecting the ‘Easy DKIM’ option, you will be presented with the option to use either 1024 bit or 2048 bit signing key length. We would recommend utilizing the 2048 bit signing key length for most customers as this is the more secure key.

If you use Amazon Route53 as your DNS provider, SES can automatically publish DNS records for your domain. If not, this step will require you to edit your DNS records to include three CNAME records which are used for the DKIM signature process and as a mechanism to prove domain ownership. An example of the CNAME records is as follows:

An example dialog of the CNAME records that are generated when attempting to verify an identity.

Once you have placed these DNS records SES will periodically attempt to look-up the records to change the status of your domain verification. If SES doesn’t automatically update the status, you are presented with the option to force another check to verify the records are present.

After your domain verification is successful, you are now ready to send emails from any email address for your domain.

Requesting Production Access

Now that you’ve verified an identity, the next step is to be able to send an email to an unverified identity you will need to request production access. If you only want to test to your own domain or email address you can skip this step until you are ready to send to unverified recipients.

Note: This is region-based, a request for production access is limited to the region in which you are requesting. 

To begin this process, you will navigate to the SES Console and the ‘Account dashboard’ section. Once you are on this page you will be presented with the following dialog at the top of your screen.

Clicking the ‘Request production access’ button will then navigate to the ‘Request details’ page which you can reference below.

The dialog from the SES console showing that the SES account is still in the sandbox.
Fill out each section with the details of your mail-type, website URL, use case description, and then acknowledging that you have read and agree to the AWS Service Terms and Acceptable Use Policy (AUP). When filling out the use case description, provide as much detail as you can for your request as our teams will review to determine if we need more information before approving or denying your request. An example of a good use case description would look like the following:

“Example.com is the domain my company intends to use to send our transactional emails. Our recipients are all customers who have either signed up for an account, requested a new password, or have made purchases through our website. We require confirmation of opt-in for all our new accounts and if no confirmation is received, we do not attempt to send an email to that address.”

Note: SES will review your production access request and will provide feedback on your use case and whether it could pose a risk to the sending reputation of SES, our customers, or your own sending domain.

Finally, click the ‘Submit request’ button to submit your request for production access. This will create an AWS Support case and will be reviewed by our team. These requests are reviewed with a 24-hour Service Level Agreement (SLA). While you are waiting for production access you can send test emails to any of the Mailbox Simulator endpoints or to your own verified domain(s) or email address(es).

Sending Your First Email

From the Console

To send your first email from the SES Console you will need to start by clicking the ‘Verified identities’ option from the ‘Configuration’ dropdown which can be found on the left side of the screen. From here you will select the domain and/or email address you want to send your email from and then click the ‘Send test email’ button, which will open the following screen:

The message details dialog where a sender can send an email from the SES console

From here you will fill out the ‘From-address’ box with the local name (anything before the @ sign) that you want to use to send the email. If you want to test SES functionality you can choose any of the dropdown events present, or you can choose the ‘Custom’ option which will allow you to set a ‘Custom recipient’ address of your choosing. Then you will fill out the ‘Subject’ and ‘Body’ fields with the content you will use for this first test email and then click the ‘Send test email’ button.

Congratulations, you’ve sent your first email from the SES Console! Now, utilizing SES to send single emails from the console isn’t the most scalable way to send email. In the next section, I will provide you links to our documentation for the 5 programming languages supported with the AWS SDK so that you can begin building your integration with SES.

From Code

The AWS Documentation includes some code snippets on how to send an email with SES via the AWS SDK. You can find examples of how to send an email from languages such as: .NET, Java, PHP, Ruby, and Python. We highly recommend reviewing our documentation to see these introductory code snippets to get you started.

Conclusion

Hopefully this blog post has aided you in your journey to send your first email through SES. From verifying a domain, requesting production access, and finally sending an email through the console. Take this knowledge and build upon it for future success in sending email through SES. Happy sending!

How to test email sending and monitoring

Post Syndicated from Dustin Taylor original https://aws.amazon.com/blogs/messaging-and-targeting/how-to-test-email-sending/

Introduction

When setting up your email sending infrastructure and connections to APIs it is necessary to ensure proper setup. It is also important to ensure that after making changes to your sending pipeline that you verify that your application is working as expected. Not only is it important to test your sending processes, but it’s also important to test your monitoring to ensure that sending event tracking is working as intended. A common pitfall for email senders is that when they attempt to test their email sending infrastructure or event monitoring they send to invalid addresses and/or test accounts that generate no, or negative, reputation as a result of these sends.

The Amazon Simple Email Service (SES) provides you with an easy-to-use mechanism to accomplish these tests. Amazon SES offers the mailbox simulator feature which enables a sender the ability to test different sending events to ensure your service is working as expected. Using the mailbox simulator you can test: delivery success, bounces, complaints, automated responses (like out of office messages), and when a recipient address is on the suppression list.

In this blog we will outline some information about the mailbox simulator and how to interact with the feature to test your email sending services.

What is the mailbox simulator?

The mailbox simulator is a feature offered to help Amazon SES senders test their sending services to verify normal operation. It provides mechanisms to test their monitoring and event notification services. This feature gives a sender the ability to test their service and email monitoring to verify that it is working as expected without the risk of negatively impacting their sending reputation. The mailbox simulator is an MTA operated by SES that is set to receive mail and to simulate different sending events based on the recipient address used.

Why use the mailbox simulator?

The mailbox simulator provides an easy-to-use mechanism to test your integration with Amazon SES. This gives senders the ability to test their sending environment without triggering actual bounces or complaints, which negatively impact their account sending reputation, as well as not counting against a sender’s email sending quotas.  It is important to test these events to ensure that event monitoring is properly setup and function. A gap in monitoring these events could lead to a decrease in sender reputation from bounces or complaint events going unnoticed. The mailbox simulator gives a sender the ability to programmatically evaluate whether their event monitoring process has been set up properly without the negative impact to their sending reputation that would occur if sending test emails to differing mailbox providers or invalid email addresses.

How do I use the mailbox simulator?

Your first step is setting up a destination for your event notifications. This can be done using Amazon Simple Notification Service (SNS) or by using event publishing depending on your use-case.  Once you have set up an event destination and configured it for your sending identity (either an email address or domain) you are ready to proceed to testing the configuration.

Using the Amazon SES mailbox simulator is simple. In practice, you will be sending an email to an Amazon SES owned mailbox. This mailbox will respond based on the event-type you want to test. Below is a map of the event types and the corresponding email addresses to test the events:

Event Type Email Destination
Delivery Success [email protected]
Bounce [email protected]
Complaint [email protected]
Suppression List [email protected]
Automatic Responses (OOTO) [email protected]

If you are using the Amazon SES console to test these events, SES has already included the addresses to simplify the testing experience and you can find these under the ‘Scenario’ dropdown.

After sending an email to one of the five destinations, you should soon receive a notification, or event, to your publishing destination. This is an example of a success event.

{
    "notificationType": "Delivery",
    "mail": {
        "timestamp": "2023-05-05T21:00:23.244Z",
        "source": "[email protected]",
        "sourceArn": "arn:aws:ses:us-west-2:012345678910:identity/example.com",
        "sourceIp": "192.168.0.1",
        "callerIdentity": "root",
        "sendingAccountId": "012345678910",
        "messageId": "01010187edb7434c-4187f4b8-3e2b-404c-a5f6-72b9b64e5d66-000000",
        "destination": ["[email protected]"]
    },
    "delivery": {
        "timestamp": "2023-05-05T21:00:24.300Z",
        "processingTimeMillis": 1056,
        "recipients": ["[email protected]"],
        "smtpResponse": "250 2.6.0 Message received",
        "remoteMtaIp": "54.165.247.113",
        "reportingMTA": "a62-102.smtp-out.us-west-2.amazonses.com"
    }
}

If you have not received confirmation of the event, it is likely there is a problem with your monitoring configuration. We recommend reviewing the documentation on SNS topic setup and/or event publishing to uncover if an error was made during initial setup.

Note: A sender may have verified an email address and a domain to use for testing. The domain may have the appropriate configuration while the email address does not. When sending an email from SES, SES will use the most specific identity (email address is used before the domain) and will use the configuration associated with that identity. This means that in this instance you can either remove the email address verification for that domain and re-test or set up the same configuration for that email address that is verified.

What next?

Now that your initial setup of event publishing is complete and you have tested your first event through the mailbox simulator, it is time to set up automated testing using the mailbox simulator. Testing email events after a successful update to your application is recommended to confirm that updates have not caused bugs in your event ingestion mechanisms.

Happy sending!