Post Syndicated from Vinay Ujjini original https://aws.amazon.com/blogs/messaging-and-targeting/migrating-to-amazon-ses-a-comprehensive-guide/

Amazon SES: Email remains a powerful tool for businesses, whether for marketing campaigns, transactional notifications, or other communications. Amazon Simple Email Service (Amazon SES) is a cloud email service provider that can integrate into any application for bulk email sending. Amazon SES is an email service that supports a variety of deployments like transactional emails, system alerts, marketing/promotional/bulk emails, streamlined internal communications, and emails triggered by CRM system as a few examples. When you use Amazon SES to send transactional emails, marketing emails, or newsletter emails, you only pay for what you use. Analytics on sender statistics along with managed services like Virtual Deliverability Manager help businesses make every email count with Amazon SES. You can get reliable, scalable email to communicate with customers at the best industry prices. If you are considering Amazon SES for its scalability, cost-effectiveness, and reliability, this guide will walk you through a systematic migration process.

Scenarios to consider:

When considering a migration to Amazon SES, let’s assess the specific scenarios to consider. These scenarios represent different contexts or situations that a business or individual find themselves in, and each scenario has its unique challenges and considerations. By identifying the appropriate scenario for your situation, you can tailor your migration strategy, anticipate potential challenges, and streamline the transition process. Few common scenarios:

• Migrating from on-Prem to SES

  • Advantages:

    • Scalability: SES automatically scales with your needs, thus ensuring you don’t face downtimes or need to regularly upgrade your infrastructure.
    • Maintenance/overhead: Maintaining on-Prem email system can be complex and resource-intensive. Some of the tasks include hardware maintenance and scalability, back up or disaster recovery, security, and compliance (relevant to email storage and transmission).
    • Cost-Effectiveness: You only pay for what you send, eliminating overhead costs associated with maintaining and upgrading on-Prem email infrastructure.
    • Security: SES offers built-in security features like email encryption in transit and at rest, and DKIM authentication with automated key rotation, allowing for sending DMARC compliant email.

  • Considerations:

    • Email Sending Limits: SES has sending limits to protect customers from deliverability events resulting from unexpected sending volumes. Customers monitor when they have reached or are approaching their anticipated sending volumes, and may request the limits to be increased.
    • Migration Time: Depending on the volume and complexity migration has to be planned and executed to minimize downtime, maintain data & sending integrity, and maintain high deliverability. This blog goes in detail on the migration process.
    • Email authentication: Setting up email authentication records such as DKIM, SPF, DMARC and BIMI: Ensure you set up domain authentication to allow mailbox providers to build a trusted model based on the messages from your domain. Sending authenticated mail is the best path to deliverability. Additionally adding trust factors to your messages like BIMI (brand indicators for message identification) will help with brand recognition both by the mailbox provider and the end-recipient (ISPs & mailbox providers use DKIM as the authenticated identifier for the trust models to determine if to show the BIMI logo).

• Migrating from another cloud solution to SES

  • Advantages:

    • Cost Savings: Amazon SES is cost-effective, especially at high volumes.
    • Integration with AWS Services: If you’re using other AWS services, integration is easier with Amazon SES.
    • Expert help: Amazon SES provides email expertise from architectural advise, help with the technical aspects of migrating from one service to another, in addition to email industry experts including deliverability focused specialists.

  • Considerations:

    • Transition Period/migration: Follow the migration path to mitigate transition risks.
    • Update Integrations: Any software or applications integrated with your previous cloud service will need to be reconfigured to work with Amazon SES (ex: SMTP, events, capturing feedback, metrics, etc.).
    • Avoid downtime: You can avoid downtime by ramping up sending gradually by moving each use case into configuration sets and applying warm-up patterns to each campaign as you shift traffic from existing service to Amazon SES.

• Migrating portion of the load and running a hybrid solution

  • Advantages:

    • Flexibility: You can maintain operations on your existing platform while testing and transitioning to SES, ensuring there’s no disruption.
    • Risk Mitigation: You can monitor your migration progress in multiple steps rather than one single step.
    • Phased Implementation: You can migrate in stages, reducing the complexity of the move.

  • Considerations:

    • Complexity: Running two systems simultaneously will introduce operational & management complexities (For example, maintaining customer opt-out preferences and suppressed email addresses need to be synced into the source lists/database).
    • Cost Implications: While you’re transitioning, you will be paying for two services, which has a cost implication.
    • Consistent Branding: Ensure consistent branding and email design across both platforms to provide a uniform experience for recipients and leverage the same domain identities authenticated with DKIM so that their prior sending reputation is carried over.

Steps for migration:

1. Identify use cases: Before the technicalities, understand and breakdown the types of emails you plan on migrating:

1. 1. Marketing Campaign emails (e.g., cross-sell, up-sell, new product released)
   2. Transactional Emails (e.g., order confirmations, password resets)
   3. Regular business communications
   4. Inbox use cases
   5. Others (ex: OTP, acquisition, etc.)

2. Architect the flow by splitting marketing and transactional traffic: Differentiate between marketing and transactional emails, ensuring they are distinctly separated. This helps improve email management, deliverability monitoring, and ensures high-priority transactional emails aren’t delayed by large marketing campaigns. It is highly recommended is to split the transactional and marketing email traffic through separate subdomains. Choose whether to use your primary domain (example.com) or a sub-domain (mail.example.com) for sending emails. Using a sub-domain can help divide email traffic and manage domain reputations separately, like marketing.example.com and transactional.example.com. You can create configuration sets, which are sets of rules that are applied to the emails that you send. For example, you can use configuration sets to specify where notifications are sent when an email is delivered, when a recipient opens a message or clicks a link in it, when an email bounces, and when a recipient marks your email as spam. For more information, see Using configuration sets in Amazon SES.

3. Domain verification: Sending authorization policies act as the gatekeeper for authorizing use of a domain identity. Domain verification is a process for Amazon SES to verify the customer owns the domain and causes messages to be signed with a DKIM signature aligned to the domain in the “From” header address of outbound messages. It is a foundational step towards a secure, reputable, and efficient email-sending program. Here’s why domain verification is essential and how it benefits users:

Why is Domain Verification Needed?

1. Ownership Assurance: Domain verification ensures that the customer is authorized to send emails from the specified domain. By confirming ownership, only customers who have verified a domain identity will have their messages authenticated with a DKIM signature belonging to the domain.
2. Reduce Spam and Phishing: Ensuring that only verified domain owners can send emails contributes to a trustworthy email ecosystem. Using a verified domain identity ensures that the message is signed with a DKIM signature aligned to the domain in the from header, which means that the message will pass DMARC-style policy enforcement (describes how unauthenticated messages claiming to be from the domain).
3. Maintain Domain Reputation: If anyone were able to send emails from any domain, it will damage the domain’s reputation that they are sending from, unless they are the owners of it. By sending from a verified domain, it ensures that your domain’s reputation remains intact and is not misused by others.
4. Compliance with SES Policies: Amazon has set policies to maintain the integrity and reputation of its SES service. Domain verification is in line with these policies, ensuring that all users follow best email practices.

How does domain verification help you?

1. Enhanced Deliverability: Emails from verified domains are more likely to reach the recipient’s inbox rather than being flagged as spam. Internet Service Providers (ISPs), mailbox providers and email clients trust emails that come from verified sources.
2. Builds Trust with Recipients: The ability to verify a domain and send from it by proving domain ownership, where recipients trust the messages are actually coming from who they are purporting to be coming from.
3. Enables Additional Features: In Amazon SES, once your domain is verified, you can also set up domain authentication mechanisms like Sender Policy Framework (SPF), DomainKeys Identified Mail (DKIM), Domain-based Message Authentication Reporting and Conformance (DMARC), and Brand Indicators for Message Identification (BIMI). These further enhance email deliverability and security.
4. Monitoring and Reporting: By verifying your domain, you can access granular metrics specific to your domain in the SES dashboard. You can use VDM and its out of the box dashboards, which includes metrics specific to verified identities. This helps in monitoring and improving your email sending practices.

4. Testing in sandbox: Amazon SES starts users in a sandbox environment. Here, you can test sending to only verified email without affecting your production environment or domain reputation. Sandbox has a limit of number of emails you can send per day.

5. Request production access: Once ready, request access to production box by following the steps outlined here: https://docs.aws.amazon.com/ses/latest/dg/request-production-access.html

6. Configure domain authentication:  You can configure your domain to use authentication systems such as DKIM and SPF. This step is technically optional, but highly recommended. By setting up either DKIM or SPF (or both) for your domain, you can improve the deliverability of your emails, and increase the amount of trust that your customers have in you. Here are key resources:

• • Authenticating email with SPF in Amazon SES
  • Authenticating email with DKIM in Amazon SES
  • Blog to set up easyDKIM
  • How to use BIMI
  • Complying with DMARC

7. IP management: When you create a new Amazon SES account, by default your emails are sent from IP addresses that are shared with other SES users. You can use dedicated IP addresses that are reserved for your exclusive use by leasing them for an additional cost. This gives you complete control over your sender reputation and enables you to isolate your reputation for different segments within email programs. Amazon SES 4 ways of IP Management outlined below:

1. Shared: Emails are sent through shared IPs.
2. Dedicated: Emails are sent through dedicated IPs.
3. Managed dedicated: Emails are sent through dedicated IPs and Amazon SES will determine how many dedicated IPs you require based on your sending patterns. Amazon SES will create them for you, and then manage how they scale based on your sending requirements.
4. BYOIP: Amazon SES includes a feature called Bring Your Own IP (BYOIP), which makes it possible to use your own IP addresses to send email through Amazon SES. If you already use a range of IP addresses to send email, you can request that we make your IP range (minimum range allowed is /24) available for sending email through Amazon SES.

Based on your use case and need, you can make a decision on how to proceed on IPs after reviewing the comparison matrix.

8. IP Warm up: IP warm-up is a crucial process when introducing a new IP address for sending emails. The goal is to progressively increase email volume sent through the new IP address, allowing mailbox providers to gradually recognize and trust this IP as a legitimate email sender. Sending reputation is built with a combination of sending domain and the IP addressed through which they are delivered.

• Why is IP warm-up necessary? When an (or a set of) IP address is new (or has been dormant for a while), it lacks a reputation with mailbox providers. If you suddenly start sending large volumes of emails from this new IP, mailbox providers perceive this behavior as suspicious, potentially categorizing these emails as spam or even blocking them. Warming up the IP helps establish a positive sending reputation over time so that mailbox providers can build a positive profile for your sending which includes IP reputation.
• IP warm-up process:
  • Start Small: Begin by sending a low volume of emails on the first day.
  • Gradually Increase Volume: Each subsequent day, increase the volume. A common strategy is to double the volume every other day, but this depends on your ultimate email volume needs.
  • Target Engaged Users First: In the initial stages, send emails to your top engaged users—those who are more likely to open, click, and not mark your emails as spam. Their positive engagement will bolster the IP’s reputation.
  • Monitor Deliverability Metrics: Keep a close eye on key metrics like delivery rates, open rates, bounce rates, and complaint rates. If you notice issues, you need to slow down the warm-up process.
  • Respond to Feedback: Some mailbox providers offer feedback loops where you can see if recipients marked your emails as spam. This feedback is invaluable during the warm-up phase to adjust your email practices.
  • Spread Sends Throughout the Day: Instead of sending all your emails at once, distribute them throughout the day. This creates a more consistent sending pattern that mailbox providers favor.
  • Continue Best Email Practices: While warming up your IP, it’s crucial to maintain best practices like segmenting your list, regularly cleaning your email list, and sending relevant content.
  • Understand your Mailbox Provider and domain distribution breakdown. For example if you send to 65% gmail.com users, you will want to focus heavily on the Gmail postmaster page and also setup tooling available for that specific Mailbox Provider. In the case of Gmail, it would be Google Postmaster Tools.
  • Identify and track any available reputation tooling for Mailbox Providers you send to. Example: Google Postmaster Tools, Hotmail SNDS, Yahoo Performance Feeds.
  • During warm-up, monitor these daily to track reputation progress.

9. Additional considerations:

• If you are planning on using a dedicated IP, warming up is crucial. For dedicated or managed dedicated IPs, you need to either manually warm them up or you can leverage Amazon SES’s auto warm-up feature. Shared IP pools (used by ESPs for smaller senders) don’t require individual warm-ups since they have an established reputation.
• The warm-up duration varies. For some, it might be a 3-4 weeks, while for others, it could stretch to a couple of months, depending on the final email volume you intend to reach.
• Let’s use an example scenario:
  • Number of emails to be migrated – 10M emails/day.
  • Peak volume throughput – 2M/hour.
  • The below table shows a sample warm-up schedule.

10. Generate SMTP credentials: If you plan to send email using an application that uses SMTP, you have to generate SMTP credentials. Your SMTP credentials are different from your regular AWS credentials. These credentials are also unique in each AWS Region. For more information on generating your SMTP credentials, see Obtaining Amazon SES SMTP credentials.

11. Connect to SMTP endpoint: If you use a message transfer agent such as postfix or sendmail, you have to update the configuration for that application to refer to an Amazon SES SMTP endpoint. For a complete list of SMTP endpoints, see Connecting to an Amazon SES SMTP endpoint. Note that the SMTP credentials that you created in the previous step are associated with a specific AWS Region. You have to connect to the SMTP endpoint in the region that you created the SMTP credentials in.

12. Monitor email send: When you send email through Amazon SES, it’s important to monitor the bounces and complaints for your account. You can do one or more of the below for monitoring your email send:

1. Reputation metrics: Amazon SES includes a reputation metrics console page that you can use to keep track of the bounces and complaints for your account. For more information, see Using reputation metrics to track bounce and complaint rates.
2. CloudWatch alarms: You can also create CloudWatch alarms that alert you when these rates get too high. For more information about creating CloudWatch alarms, see Creating reputation monitoring alarms using CloudWatch.
3. Virtual Deliverability Manager (VDM): Deliverability, or ensuring your emails reach recipient inboxes instead of spam or junk folders, is a core element of a successful email strategy. Virtual Deliverability Manager is an out of the box Amazon SES feature that helps you enhance email deliverability. It can help in increasing inbox deliverability and email conversions, by providing insights into your sending and delivery data, and giving advice on how to fix the issues that are negatively affecting your delivery success rate and reputation. VDM has dashboards and advisor features that are built-in, Visit this VDM blog to see how you can improve your email deliverability using VDM.

13. Ramp-up ramp-down strategy: Sending email communication along with maintaining the domain and send reputation is key to any business. The ramp-up ramp-down strategy in the context of email migration, especially to a new email sending platform or a new IP address, is a best practice to ensure that your emails maintain a high deliverability rate and don’t end up being flagged as spam. Let’s delve deeper into what this strategy entails and why it’s crucial:

1. Gradual volume increase: Start by sending a small number of emails (refer to table below in #12 – IP warm up) and then gradually increase this number over days or weeks. This slow increase allows mailbox providers to recognize and trust your new sending source. Ramp up gradually by moving each use case and applying warm-up pattern to each campaign as you shift traffic. Closely monitor deliverability metrics as you ramp-up. If the metrics show any signs of issue, freeze the warm-up to assess the root cause. Sending stable, predictable patters are the key, avoiding unexpected spikes.
2. Prioritize engaged recipients: Begin your email sends by targeting recipients who are most likely to open and engage with your emails, like your top active subscribers or customers. Positive interactions, like email opens or link clicks, can boost your new IP’s reputation.
3. Monitor Feedback loops: Utilize feedback loops offered by mailbox providers to understand if recipients are marking your emails as spam. This immediate feedback can help you tweak your sending practices.
4. Maintain consistency: While you’re ramping up, maintain consistency in your sending patterns. Avoid erratic sending volumes, which can be red flags for mailbox providers.
5. Maintain Domain/IP Reputation: Even if you’re sending fewer emails, ensure those emails still adhere to best practices to maintain your domain or IP reputation.

14. Final cut over: After rigorous testing, ramping up, and ensuring your emails are being delivered reliably, you can fully transition to Amazon SES. Monitor continuously, especially during the initial days, to catch and address any potential issues promptly.

Deliverability resources:

• • SES
    • https://docs.aws.amazon.com/ses/latest/dg/send-email-concepts-deliverability.html
  • Best Common Practices for email senders:
    • https://www.m3aawg.org/sites/default/files/document/M3AAWG_Senders_BCP_Ver3-2015-02.pdf
  • BCP for email vetting:
    • https://www.m3aawg.org/documents/en/maawg-vetting-best-common-practices-bcp
  • BCP for complaint handling:
    • https://www.m3aawg.org/documents/en/m3aawg-recommendations-for-senders-handling-of-complaints
  • BCP for Sending Domain reputation:
    • https://www.m3aawg.org/SendingDomsBCP
  • Spamhaus best practices
    • https://www.spamhaus.com/campaign/email-deliverability/
  • Gmail Postmaster Best Practices:
    • https://support.google.com/mail/answer/81126?hl=en
  • Yahoo: https://senders.yahooinc.com/best-practices/?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAAJCI2ATkg8WfVA-TS7rsGYfXYrUZwSFhMV0eXxfH3faQMQuK5Jt9hAs8EklQkNCEgTEtY-V_LRGv3A5YEKN7NSyVtYEgew8gRCcSWXe7ee6ECi-gYyd_9URT5RSaAhiw56-tXEB8rV2TABnRW5q9fHJoBYoR7mXRNoyPjc8sF-3I
  • MSFT (this is geared more toward office365 but holds true as an overall best practice guide ): https://learn.microsoft.com/en-us/dynamics365/marketing/get-ready-email-marketing

Conclusion:

Migrating to Amazon SES offers a host of benefits, but like all IT endeavors, it requires careful thought and execution. By following this comprehensive guide, you can pave a path for a smooth transition, allowing your business to leverage the power of Amazon SES effectively.

About the author:

Vinay Ujjini is an Amazon Pinpoint and Amazon Simple Email Service Worldwide Principal Specialist Solutions Architect at AWS. He has been solving customer’s omni-channel challenges for over 15 years. He is an avid sports enthusiast and in his spare time, enjoys playing tennis & cricket.