Tag Archives: messaging

Automate the Creation & Rotation of Amazon Simple Email Service SMTP Credentials

Post Syndicated from Zip Zieper original https://aws.amazon.com/blogs/messaging-and-targeting/automate-the-creation-rotation-of-amazon-simple-email-service-smtp-credentials/

[Amazon Simple Email Service] provides a secure email solution that scales with your business needs. Unfortunately, all email systems, including Amazon SES, remain the primary target for spammers and bad actors due to email’s widespread use and accessibility.

While SES offers powerful features for application-based email sending, its SMTP credentials require careful management to prevent unauthorized access. Compromised credentials enable bad actors to send malicious emails through legitimate domains, which can bypass security filters and damage sender reputation.

To protect your SES implementation, you must encrypt SMTP credentials during storage and transmission. Additionally, implementing role-based access controls helps restrict credential access to authorized personnel only. Regular credential rotation at fixed intervals, typically every 90 days, minimizes potential security breaches. Automating this rotation process eliminates human error and ensures consistent security practices across your organization.

Problem Statement

Imagine you are the administrator for a large financial institution. You recently began using Amazon SES to send email from two dozen on-premises servers. Your email servers authenticate with SES using SMTP credentials to access the SES SMTP interface. Your organization’s security policies mandate regular credential rotation, including the ability to rotate them on-demand. How can you automate SMTP credential rotation such that you can meet your organization’s security policies?

This blog post will present two solutions that automate the secure management and automatic rotation of SMTP credentials for Amazon SES. Each will help enhance email security, comply with regulations, and minimize operational overhead.

Both solutions provide SES customers who use SMTP with additional tools to improve email security, ensure compliance, and reduce operational overhead. You can deploy the option that best suites your needs by following the guidance in this blog post.

If your environment supports automated rotation, AWS Systems Manager Documents (SSM Documents) can help by providing pre-defined or custom automation workflows for securely managing secrets rotation, deploy Option 1.

If your environment does not support automated rotation, you can still implement an auditable, managed rotation solution by storing your secrets in AWS Systems Manager Parameter Store by deploying Option 2.

As a pay-per-use platform, the underlying AWS services used in either deployment option will only charge you for the resources that you actually consume. You can leverage the AWS Pricing Calculator to estimate the run-time costs for your specific workload. Alternatively, you can work directly with your AWS account team to understand the pricing for these solutions.

Getting SES SMTP Credentials

To send emails through the Amazon SES SMTP interface, email servers must first authenticate with SES using dedicated SES SMTP credentials. Typically, a systems administrator logs into the AWS SES console, clicks the Create SMTP Credentials button, and navigates to the AWS Identity and Access Management] (IAM) console. There, the administrator creates an IAM user with permissions for SES. The administrator then uses the IAM user’s secret access key to generate the SES SMTP password, which they use to configure their email servers or SMTP-enabled applications for use with SES.

Multiple SMTP Credentials

The SES SMTP interface authenticates requests using an SMTP credential derived from an IAM user’s access key ID and secret access key. Since temporary access keys cannot be used to derive SES SMTP credentials, you must deploy and regularly rotate a long-lived key.

While the manual process of creating SES SMTP credentials works for a small number of credentials, it becomes cumbersome for customers with numerous email servers or strict password rotation policies. These customers may find the automated credential rotation mechanisms described in the following solutions better suited to their production needs.

Option 1 – Fully Automated Credential Rotation:

The fully automated version of this solution uses a custom Lambda function to create an SMTP password, which is stored in AWS Secrets Manager. AWS Secrets Manager’s built-in rotation feature then triggers the rotation of SES SMTP credentials. AWS Systems Manager Documents utilize AWS Systems Manager Agents to automatically make the changes to the authentication configuration on email servers.

The key advantages of using AWS Systems Manager to make the email server configuration changes include:

  • Ability to deploy changes to on-premises and Amazon EC2 hosts, allowing rotation of secrets across a hybrid estate.
    Customization of the document to specific email software configurations.
    Targeting the secret (SMTP credential) rotation document on all email servers based on tags.

Let’s dive deep into Option 1 – Fully Automated Credential Rotation.

Option 1 - Fully Automated Credential Rotation

How Option 1 works:

Refer to the image above for the workflow:

  1. AWS Secrets Manager initiates a rotation request, either on a schedule or via an authorized user’s request, triggering the “rotation Lambda” to rotate the SES SMTP credentials.
  2. The SES Secret Rotation Function Lambda (see figure x above):
    • a. Creates a new IAM secret access key for the designated SES IAM user, derives the new SES SMTP password, and stores it in AWS Secrets Manager.
    • b. Connects to SES to verify the new SMTP password can authenticate.
    • c. Initiates an AWS Systems Manager Run Command to update the new SMTP password on target email servers using a pre-configured Systems Manager Document.
    • d. (and e.) Monitors the status of the Systems Manager Document execution until all updates complete successfully
    • f. Deletes the old IAM access and secret access keys.

With this fully automated solution, SES SMTP credentials can be rotated on a schedule or triggered manually, with no impact to email service uptime.

Deploying the Fully Automated Solution in Your AWS Account (Option 1)

Prerequisites for the Fully Automated Solution

  1. AWS Account Access, typically with admin-level permission to allow for the deployment.
  2. Your preferred IDE with AWS CLI version 2 and named profile setup.
    • Alternatively, you can use the AWS CLI from the AWS CloudShell in your browser.
  3. Clone the Github repository (for this solution, you only need the README.md and sesautomaticrotation.yaml files found in /ses-credential-rotation/automatic-rotation).
    • git clone -b ses-credential-rotation https://github.com/aws-samples/serverless-mail.git
    • Note – We follow the principles of least privilege in this solution. The CloudFormation templates we’ve supplied require you to specify an identity, or configuration-set resource to use in the SES sending operation. You can find guidance on defining these values at Actions, resources, and condition keys for Amazon SES. Additionally, we’ve limited the IAM User to the ses:SendRawEmail action, which you can adjust as appropriate).
  4. Console access to your AWS SES account that is properly configured to send emails via at least one verified identity.
  5. Target email server(s) properly configured to send email via SES using SES SMTP authentication.
    • The AWS Systems Manager agent(s) must be correctly installed and configured on your target email server(s) as detailed in Setting up AWS Systems Manager.
    • The target email servers must be decorated with the tag (“SSMServerTag“) and value (“SSMServerTagValue“). These values allow the Systems Manager Document to identify them.
      • We use the tag “EmailServer” and the value “True” in our example, but you can use any tag and value that you wish).
  6. An email address (or list) to receive SMTP rotation notifications.
  7. Console access to your AWS Secrets Manager.
  8. Console access to your AWS Systems Manager.

Deployment Steps

  1. Clone the GitHub repository to your IDE
    • If using AWS CloudShell, ensure you are in the same region as your AWS Systems and Secrets Manager
    • run: git clone https://github.com/aws-samples/serverless-mail.git
    • Navigate to the directory ses-credential-rotation/automatic-rotation
  2. Follow the steps in README.md to
    • Create a S3 bucket to deploy the CloudFormation Template.
    • Package the Lambda functions and upload them to Amazon S3.
    • Deploy the Cloud Formation Template.
    • Update the appropriate AWS Systems Manager sample document created by the CloudFormation Template to reflect your email server environments. These can be found in the AWS Systems Manager console under Documents > Owned by me
      • The ExampleWindowsIISSMTPSESpasswordrotator sample provides an example for Microsoft Windows hosts using the runPowerShellScript action to update the server’s SMTP credentials.
      • The ExamplePostfixSESpasswordrotator sample provides an example for Linux hosts using the runShellScript action to update the server’s SMTP credentials.

Testing Option 1 – Fully Automated Credential Rotation

To test the Fully Automated Credential Rotation solution, have Secrets Manager perform an immediate rotation by following these steps:

  1. Open AWS Secrets Manager console
  2. Locate the secret SESSendSecret
  3. Select the Rotation tab
  4. Click the “Rotate Secret immediately” button.

You can track the progress of the rotation by locating the logs of the Lambda that is deployed to manage the rotation.

  1. In the AWS console, go to CloudFormationStack’s Resources tab
  2. Find the LogicalID = SESSecretRotationFunction
  3. Click the PhysicalID link to open the Lambda
  4. Under the Monitor Tab, select the “View CloudWatch logs” button in the top right
  5. The logs should show the rotation flow through 4 stages below (more details of each stage are available here):
    1. create_secret
    2. set_secret
    3. test_secret
    4. finish_secret

Option 2 – Partially Automated Credential Rotation:

The partially automated version uses a custom AWS Lambda function to create an SMTP password, which is stored in AWS Systems Manager Parameter Store. This solution simplifies credential rotation, where manual changes must be conducted by support staff. By wrapping the manual change process with AWS Step Functions, you can ensure a robust and auditable process to regularly rotate the SES SMTP credential.

How Option 2 works:

  1. The credential rotation AWS Step Function creates a new SES SMTP credential and updates it in AWS Systems Manager Parameter Store.
  2. It retrieves a list of servers from an Amazon DynamoDB table and launches a manual confirmation AWS Step Function execution for each server to initiate and track the manual step.
  3. The manual confirmation AWS Step Function emails the designated address, requesting support staff to arrange the rotation. The email includes a link specific to that server.
  4. The person completing the manual change confirms back to the AWS Step Function via the link that the rotation is complete.
  5. Once the rotation on a server is confirmed, the manual confirmation AWS Step Function for that server is marked as complete.
  6. After all server rotations are complete, the credential rotation AWS Step Function continues, disabling the old SES SMTP credential and deleting it after a few days.

AWS Step Function executions can last up to 365 days, providing sufficient time for the manual rotation and confirmation.

The screenshot below shows a graphical representation of the credential rotation AWS Step Function execution status, providing a real-time view of the rotation progress.

SMTP credential rotation AWS Step Function

You can also track the status of individual servers via the manual rotation step function execution list.

SMTP manual rotation step function execution list

The partially automated solution for rotating Amazon SES SMTP credentials is illustrated and detailed below:

Option 2 - partially automated solution

Refer to the image above for the option 2 workflow:

  1. EventBridge Scheduler Trigger: An EventBridge scheduler rule triggers a custom Starter Function Lambda (SF Lambda) on the last day of every 3rd month (this can be adjusted to suit your needs in the CloudFormation template).
  2. Credential Rotation Step Function: The Starter Function Lambda triggers the Credential Rotation AWS Step Function, providing a clearly defined name to facilitate auditing (“password-rotation-dd-mm-yy“).
  3. Credential Rotation Step Function Actions:
    1. Creates a new IAM (Identity and Access Management) secret access key for the SES IAM user.
    2. Triggers the SMTP Credential Generator Lambda to derive the SES SMTP password from the newly created IAM secret access key (using the algorithm provided in the SES documentation.
    3. Stores the new SES SMTP credential in AWS Systems Manager Parameter Store.
    4. Reads a list of servers that are utilizing this credential from a DynamoDB table.
  4. Manual Confirmation Step Function:
    1. For each server, a manual confirmation AWS Step Function is triggered, sending a message on the Amazon Simple Notification Service (SNS) topic.
    2. The SNS notification prompts the server administrator via email to manually rotate the SMTP credentials on the on-premises email server.
    3. The server administrator uses a link in the email to confirm the credential has been rotated and tested on the server.
    4. The link triggers the Confirmation Lambda exposed via API Gateway, which marks the ManualConfirmation step function as complete.
  5. Credential Rotation Completion: The CredentialRotation step function waits until all manual confirmation step functions have completed before proceeding.
  6. Old IAM Access Key Deletion: Once confirmation has been received for all servers, the step function deletes the old IAM access key.

Deployment

To deploy the partially automated solution in your AWS account, you will need the following prerequisites:

Prerequisites for the Partially Automated Solution

  1. AWS Account Access, typically with admin-level permission to allow for the deployment.
  2. Your preferred IDE with AWS CLI version 2 and named profile setup. Alternatively, you can use the AWS CLI from the AWS CloudShell in your browser.
  3. SES enabled, configured, and properly sending emails.
  4. External email server(s) currently configured to use SES with SMTP.
  5. Administrator email address to receive notifications.
  6. AWS Secrets Manager and AWS Systems Manager set up.
  7. AWS Systems Manager agent(s) correctly installed and configured on your target email servers as detailed in Setting up AWS Systems Manager.
  8. Amazon EC2 instance with Postfix configured to send emails through SES
  9. Target email servers must be decorated with a tag (“SSMServerTag“) and value (“SSMServerTagValue“) that will be used to identify them by the Systems Manager Document (we used “server” and “email”)
  10. AWS Parameter Store and AWS Step Functions.

Once you have the prerequisites in place, follow the instructions in the GitHub project.

Conclusion

Implementing an automated credential rotation process for Amazon SES SMTP enhances security and compliance, streamlines operations, and reduces the risk of downtime and human error. By leveraging AWS Secrets Manager and AWS Systems Manager (option 1) or AWS Systems Manager Parameter Store and Step Functions (option 2), organizations can centralize SES SMTP credential management, maintain an audit trail, and quickly update email application servers with new SMTP credentials.

Need additional guidance?

A Guide to SMS Short Codes with AWS End User Messaging

Post Syndicated from Tyler Holmes original https://aws.amazon.com/blogs/messaging-and-targeting/guide-to-sms-short-codes-with-aws-end-user-messaging/

In today’s digital age, SMS messaging remains a crucial communication channel for businesses. One of the most effective ways to leverage SMS is through the use of short codes – those brief, memorable numbers that make it easy for customers to interact with your brand. This comprehensive guide will walk you through everything you need to know about obtaining and using SMS short codes.

What Are Short Codes and Why Use Them?

While short codes aren’t required for sending SMS in most countries(not all) they are synonymous with high throughput and reliability.

They offer several advantages:

  • High throughput – They start out at 100 messages per second
  • Ability to scale to thousands of messages per second – If you need the kind of scale to send out mass alerts in a short amount of time, the best option is likely a short code
  • Higher deliverability rates – Because of the increased upfront scrutiny in the registration process they have less issues with filtering and blocking. Critical use cases such as One-Time Password(OTP)/Two-Factor Authentication(2FA) are use cases that, even if you don’t need the high throughput, you may want to consider using a short code to deliver them
    • Short codes investigate issues before blocking through proactive audits, while other registered originator types such as toll-free or 10DLC block/filter first and investigate later when issues arise
  • Easy for customers to remember and identify – Short codes are 5-6 digit numbers, which is easier for recipients to remember than 10+ and lends credibility and trust to your message

Short Codes Shortcomings

While short codes offer many advantages for SMS messaging, they also come with some things to be aware of

  • Short codes are limited to SMS/MMS functionality only, unlike other number types such as long codes, 10DLC, or toll-free numbers that can also support voice messages. This limitation may restrict your communication options with customers.
  • Carriers consider short codes a premium product, which can lead to higher costs for businesses.
  • The process of obtaining a short code can be more time-consuming and complex than other originators, requiring strict adherence to carrier requirements and a thorough application process that can take several weeks to months to complete. This is not specific to AWS, these processes are controlled by the carriers and countries in which you are applying. For example, some countries require specific supporting documents such as a Letter of Authorization (LOA) or even a copy of a passport of an executive at the company. Be ready to provide these as there are usually no exceptions to these requirements.
  • While it’s an edge case, there are some prepaid mobile plans, like those offered by T-Mobile in the US, that don’t allow their users to receive messages from short codes, potentially limiting your reach.

Do You Need a Short Code?

If you answer yes to any of the below, you might want to consider a short code

  • You need over 20 MPS
  • You need daily volumes over 400K
  • You need higher success rates
  • You want an easily recognizable number for your brand
  • The country you need to send to ONLY allows short codes
  • Ensuring maximum reliability is critical
  • You operate in a highly regulated industry (like finance)

However, keep in mind that short codes:

  • Do not support voice like long codes can
  • Require a more rigorous application process and take longer to obtain
  • Have an increased cost compared to other originator types (Long codes, Toll-Free, 10DLC, Sender ID, etc.)
  • While a small edge case, they may not be supported by some prepaid mobile plans

If you’ve decided that your use case requires a short code, you have to do some planning before you request one. The next few sections guide you through some of the requirements that must be in place in order to obtain and use a short code.

Planning Your Short Code Application

Before applying for a short code, you need to have several elements in place:

1. Understand Consent Requirements

Carriers impose their own, more strict, requirements beyond the minimum requirements of law in many countries. A non-exhaustive list of consent requirements include:

  • Explicit consent is required before sending any messages
    • It does not matter if you are messaging to your employees, sending to 6 people on an IT team, or sending millions of messages promoting your brand, there are no exceptions to this rule
  • Consent must be specific to each message type (no blanket consent)
    • You must obtain explicit consent for each distinct use case such as OTP/2FA, Marketing, and the various flavors of transactional messaging. You can find examples of the distinct use cases that US 10DLC numbers adhere to here which is a good place to start when trying to classify your use cases
  • Consent can’t be transferred between companies
    • The concept of consent lies with who owns the relationship to the recipient. Just because you have received consent for a brand within your portfolio does not mean that you can message someone for a different brand or use case.
    • If you are a SaaS or multi-tenant type customer where you are offering SMS capabilities to your customers, then proof of consent must be supplied by your customer and their information submitted for review.
    • Never sell/rent your list of opted-in customers, and never use purchased or rented lists.
  • Receiving SMS can’t be a requirement for using your service or opting out
    • If your use case requires that you verify your customer’s phone number, provide them an alternative to receiving text messages. For example, provide the option to receive a voice call or an email and opt-out via different channels as well

2. Design Compliant Opt-in Workflows

An in-depth explanation of how to design a compliant opt-in process can be found here. Your opt-in process does not have to be online. You can use verbal scripts, paper forms, or online signups but the location at which someone is opting in must include:

  • A description of message types you’ll send
  • The phrase “Message and data rates may apply”
  • Message frequency information
    • This is a statement such as:
      • “You will receive (1-10) messages per (day/week/month/other)” or “message frequency varies”
  • Links to Terms & Conditions and Privacy Policy or a printed copy if not publicly accessible
  • Opt-out information (example: “Text STOP to opt-out of future messages.”)
  • Communication options
    • If your use case is 2FA/OTP you must supply a secondary channel for receiving the codes, such as email. SMS must be optional and another option must be provided. This is mandated by Verizon in the US.
    • In all other use cases SMS must be optional. It cannot be a requirement for a subscriber to opt in to SMS in order to sign up for a service. As an example: you sign up for a rewards program at a coffee shop and it is mandatory to sign up for SMS in order to join the program. This is prohibited
  • An explicit opt-in
    • Checking a box, signing your name and date, capturing it verbally and logging it. These are all valid forms of explicit opt-in. If you pre-check boxes on your forms that is NOT considered an explicit opt-in and your registration will be denied, increasing the time it will take to get approved.

The image below is an example of an online form. This is a more complex flow but you could also have a single screen flow, as long as it contains all of the above required components. Keep in mind that you can also use verbal scripts and paper forms that include all of the required components above.

Image 1 – Complex

3. Create/Update SMS-specific Privacy Policy and Terms & Conditions

Data privacy is an important component of any SMS program and when applying for a short code the US mobile carriers set the most stringent requirements regarding specific language in your Privacy Policy and Terms & Conditions. If you comply with the below for your Privacy Policy and Terms & Conditions this should put you in compliance for other countries that you may want to apply for.

For a more in depth discussion of the topic of opt-in and compliance please visit

NOTE: You are allowed to create an SMS only section within these two documents to call out different data sharing or other terms that apply to SMS but not to other parts of your business, but these items are non-negotiable and must be present or your registration will be denied by the carriers in the US

Terms & Conditions

Below is the boilerplate language that covers minimum requirements from the US carriers who are the most stringent:

  1. {Program name}
  2. {Insert program description here; this is simply a brief description of the kinds of messages users can expect to receive when they opt-in.}
  3. You can cancel the SMS service at any time. Just text “STOP” to the short code. After you send the SMS message “STOP” to us, we will send you an SMS message to confirm that you have been unsubscribed. After this, you will no longer receive SMS messages from us. If you want to join again, just sign up as you did the first time and we will start sending SMS messages to you again.
  4. If you are experiencing issues with the messaging program you can reply with the keyword HELP for more assistance, or you can get help directly at {support email address or toll-free number}.
  5. Carriers are not liable for delayed or undelivered messages
  6. As always, message and data rates may apply for any messages sent to you from us and to us from you. You will receive {message frequency}. If you have any questions about your text plan or data plan, it is best to contact your wireless provider.
  7. If you have any questions regarding privacy, please read our privacy policy: {link to privacy policy}

Privacy Policy

One of the key items carriers look for in a Privacy Policy is the sharing of end-user information with third-parties. If your privacy policy mentions data sharing or selling to non-affiliated third parties, there is a concern that customer data will be shared with third parties for marketing purposes.

Express consent is required for SMS; therefore, sharing data is prohibited. Privacy policies must specify that this data sharing excludes SMS opt-in data and consent. Privacy policies can be updated (or draft versions provided) where the practice of sharing personal data to third parties is expressly omitted from the number registration.

Example: “The above excludes text messaging originator opt-in data and consent; this information will not be shared with any third parties.”

It can also be an option to put the privacy statement in the Call-to-action mockup if you do not want to have to put it in your privacy policy page.

4. Prepare Message Templates

When submitting your registration, you must provide examples of the messages you will be sending. This includes automated responses triggered by specific keywords from end-users such as “Help“ as well as the outbound messages you will be sending to your recipients based on your use case. While we’ll provide English examples below, note that keywords and responses should be localized based on your recipient countries. You may (and should) configure multiple keywords depending on your audience demographics. All messages must be under 160 characters and meet the specific requirements detailed below:

  • Standard Outbound Messages
    • These will be specific to your program and what your use case is. You do not need to provide all of your messages that you plan on sending but you should provide an example of each type of message that you plan on sending. If you are sending in multiple languages you should provide a version for each language.
  • Opt-in Confirmation Message
    • This is the message that must be sent whenever you opt someone into your program
  • Help
    • This message must be sent when any of the required keywords for help in the country you are registering for are sent back to you. In the US this is “help” and in Mexico this is “ayuda” for example.
  • Stop
    • The “Stop message” is the response that is required to be sent to end-users when they text the keyword “STOP” (or similar keywords). End-users are required to be opted out of further messages when they text the STOP (or equivalent) keyword to your number and you are required to confirm with them that they will no longer receive messages for the program.

Let’s review the specific requirements for each in detail

Standard Outbound Message Types:

Your short code application must include examples of all of the message TYPES that you plan to use but it does not need to include ALL of the possible examples. This is especially true with promotional messages that can have a lot more variability. You need to give the reviewers enough examples that you are illustrating how you will be using the short code.

The two main types of messages are Promotional and Transactional, let’s break each down.

Promotional: This includes any type of messaging that has content related to sales or other offers. This does not only have to be related to content that requires a purchase. This could also be marketing new features, announcing the launch of a new program, or other messaging that could be construed as marketing. Remember, there are humans reviewing these registrations so make sure that any messages that you are using will not be misconstrued as a type of message that you are not registering for.

Transactional: There are a lot of different types of messages that can be considered transactional. The simple way of thinking about it is that anything that is NOT promotional, is transactional. Some examples of transactional messages include:

  • Account Notifications
  • Status notifications about an account that the recipient is a part of or owns
  • Customer Care
  • Communications related to support or account management
  • Delivery Notifications
  • Notifications about the status of a delivery of a product or service
  • Fraud Alert Messaging
  • Notifications related to potential fraudulent activity
  • Security Alerts
  • Notifications related to a compromised software or hardware system that requires recipients to take an action
  • Two Factor Authentication(2FA) or One-Time Password(OTP)
  • Authentication, account verifications, or one-time passcode

While these two types can be mixed on a single short code we strongly recommend against this:

  • Mixing message types on a single code is not as resilient should something happen to your code or if your code is ever audited.
  • It could prevent your recipients from accessing their account if they opt out of a code delivering OTP and marketing messages and you are not self managing your opt out process.
  • It can also impacts the throughput — if, during a marketing campaign using the same short code, that campaign is exhausting the throughput of the short code, the transactional use case will be impacted by that capacity limitation.

Best practices:

  • Include your Program (brand) name in each message
  • If you have multiple templates, include an example of all of the types you are registering for.
  • Read them all and make sure that they are clearly a part of the message type/use case that you are registering for
  • If your messages will include variables, it’s fine to use either placeholder values or variables.
    • For example, both of the following are acceptable: “Hello John. Your one-time password is 654321” and “Hello [first_name]. Your one-time password is [otp code] .”
  • Make sure they are all less than 160 characters
  • Do not use unbranded link shorteners
    • If you have to use a shortener make sure that it is branded and that there are not multiple redirects. Also make sure to provide examples of the branded shortened domain so that it is on file with the carriers

Confirmation:

Provide the exact message that will be sent back to your end-users letting them know that they have successfully registered.

Example:
“Welcome to AnyCo! Reply “YES” to confirm your subscription and get special offers once a month. Msg & data rates may apply. Text ‘STOP’ to opt out.”

You must include:

  • The message has to be a minimum of 20 characters and a maximum of 160 characters
  • Brand Name
  • Opt-in confirmation messages are required for all use cases, even one-time passwords. However, for OTP-type use cases, the act of end users requesting an OTP is essentially an opt-in and associated confirmation message which means you don’t need another unique confirmation message on top of the OTP as long as the OTP message is compliant.
    • A double opt-in confirmation message flow, where the recipient will text back “YES” to confirm that they did want to register, is only required for shopping cart reminder use cases. However we do recommend it as a best practice for all use cases since it ensures you are maintaining a clean database of numbers.
  • Include “Msg & data rates may apply” as seen in the example
  • Include opt-out language as seen in the example

Help:

Provide the exact message that will be sent back to your end-users when they request “help” via keyword response.

Example:
“ExampleCorp Account Alerts: For help call 1-888-555-0142 or go to example.com. Msg&data rates may apply. Text STOP to cancel.”

You must include:

  • The message has to be a minimum of 20 characters and a maximum of 160 characters
  • The message must include:
    • Program (brand) name OR product description.
    • Additional customer care contact information.
      • It is mandatory to include a phone number and/or email for end-user support
  • Include “Msg & data rates may apply” as seen in the example

Stop:

Provide the exact message that will be sent back to your end-users when they request to be opted out of your program. You are allowed to include instructions on how to resubscribe but be sure to keep the total message length under 160 characters while still including all of the required components.

Example:
“You are unsubscribed from ExampleCorp Account Alerts. Text JOIN to resubscribe. No more messages will be sent. Reply HELP for help or call 1-888-555-0142.“

You must include:

  • The message has to be a minimum of 20 characters and a maximum of 160 characters
  • The message must include:
    • Program (brand) name OR product description
    • Confirmation that no further messages will be delivered

How to Apply for a Short Code

Step 1: Create a Case in AWS Support Center

Currently short codes are only available by opening a case. The first step to register a short code is to open a short code request case in the AWS Support Center, providing detailed information about your use case and opt-in policies. A request must be opened for each country in which you want a short code. Each country has their own registration process and requirements so having a case for each allows for much easier tracking and updating as there are communications going back and forth between you, the customer, support, and any downstream clarifications that may come back.

Step 2: Complete the Short Code Application

Each country may have different requirements for the information you need to fill out as well as supporting documents you may need to provide. Make sure that everything you submit is 100% correct to your knowledge, as any missing information or information that needs to be corrected extends the time it takes to receive your short code. The minimum information you will need to provide is:

  • Company information such as legal name and tax ID
  • Use case details
  • Opt-in workflow documentation
  • Privacy Policy and Terms of Service
  • Message templates (including CONFIRMATION, HELP, and STOP responses)

The short code application form contains all of the information that we send to the carriers to let them know about your use case. For this reason, the form must be filled in completely, and the responses must all be compliant with the requirements of the carriers. The high-level process looks like the below:

  1. Submit your request for a short code through the AWS case console
  2. AWS will respond back with the appropriate form and required supporting documentation for the country you are applying for.
  3. Fill out the document with all of the information and include any supporting documents required. Don’t leave anything blank, there are no exceptions to these forms.
  4. AWS will validate that the form is complete and supporting documentation is included. We do not review your documents, so make sure that to your knowledge they are correct.
  5. AWS will submit your form to be carefully reviewed.
  6. The registrar reviews documentation to ensure that it is compliant with all carrier and country requirements.
    1. This step may have several rounds of revisions. If you requested through an AWS support case, make sure you are CC’d to updates so you can quickly address any feedback from external reviewers as delays in responses increase the timeline.
  7. Once your documentation is considered compliant your company will be submitted for vetting. This is similar to a credit check process where your data will be validated and you will need to prove you are a member of your company through a domain validation email.
  8. Once the vetting process is completed your short code moves to “carrier review” state — this is when the timer starts. Continue to monitor your case for any needed updates.
  9. The short code numbers are confirmed and the number is provisioned to your account and billing starts. This does not mean that the number can be used however as there are configurations with carriers that need to be completed.
  10. Carriers approve and provision the SC to their network to make it active
    1. This is the final external provider phase. This process requires each individual carrier to review your registration, approve it, and then to provision the code to their networks so that it is active when handed over to you. In rare cases, an individual carrier may reject your registration and request changes or additional information before they approve.
  11. Our SC partner informs us the SC is now approved and active on carrier networks
  12. AWS updates your case informing you that the SC is ready to use

Conclusion

Obtaining and using SMS short codes requires careful planning and adherence to the strict requirements set forth by the carriers. By following the guidance provided in this comprehensive guide, businesses can navigate the application process successfully and leverage the unique advantages of short codes for their SMS messaging needs.

Key to the successful procurement and usage of a short code is the establishment of compliant opt-in workflows, SMS-specific privacy policies and terms of service, as well as the preparation of required message templates. By meticulously addressing these prerequisites, businesses can streamline the application process and avoid costly delays or rejections.

The high bar for obtaining a short code exists to protect consumers from spam and abuse, ensuring that only approved and legitimate use cases are granted access to this powerful communication channel. While the application process may be time-consuming and complex, the benefits of using a short code – including high throughput, reliability, and an easily recognizable brand identifier – make it a valuable investment for businesses that need to communicate with their customers via SMS at scale.

Ultimately, the diligence required to obtain and properly utilize a short code pays dividends in the form of a highly effective and trustworthy SMS messaging channel. Following the guidance outlined in this document will position businesses for success in leveraging the power of short codes to connect with their customers in today’s digital landscape.

Update on Support for Amazon Chime

Post Syndicated from Beth Styles original https://aws.amazon.com/blogs/messaging-and-targeting/update-on-support-for-amazon-chime/

Amazon Chime is a communications service that lets you meet, chat, and place business calls inside and outside your organization, all using a single application.

After careful consideration, we have decided to end support for the Amazon Chime service, including Business Calling features, effective February 20, 2026. Amazon Chime will no longer accept new customers beginning February 19, 2025. Existing customers can continue to use Amazon Chime features, including Business Calling, scheduling and hosting meetings, adding and managing users, and other capabilities supported through the Amazon Chime administration console. To qualify as an existing customer, you must have created an Amazon Chime Team or Enterprise account prior to February 19, 2025. After February 20, 2026, you will no longer be able to host Amazon Chime meetings, manage users, or use any of the Business Calling features. If you cannot delete your data prior to February 20, 2026, please reach out to us via Amazon Support for help. Note: This does not impact the availability of the Amazon Chime SDK service.

We know that you may need support to transition to other collaboration solutions. Please review our documentation which provides instructions and best practices for Amazon Chime IT administrators and users to move to alternate collaboration solutions. This may include solutions provided by AWS, such as AWS Wickr, or from AWS partners, such as Zoom from Zoom Video Communications Inc., Webex from Cisco Systems, Inc., and Slack from Salesforce, Inc. Visit the AWS Marketplace for more information about our AWS partners’ solutions. If you need additional assistance transitioning, please contact AWS Support.

Organizations with at least one Amazon Chime Team or Enterprise account can continue to use Amazon Chime and Business Calling features until February 20, 2026, when support for the service will end. Features that will no longer be supported include scheduling and hosting meetings, adding and managing users, and other capabilities available using the Amazon Chime console.

Additional Resources

Improving Security in Amazon WorkMail with MFA

Post Syndicated from Zip Zieper original https://aws.amazon.com/blogs/messaging-and-targeting/improving-security-in-amazon-workmail-with-mfa/

Securing your business email is more critical than ever in today’s digital workplace. To help you protect your users and data in Amazon WorkMail, we have introduced enhanced security features that give organizations more control and protection for their communication platforms. With the integration of AWS Identity and Access Management (IAM) Identity Center, WorkMail now offers robust multi-factor authentication and personal access token capabilities that can help prevent unauthorized access to user accounts and protect sensitive business communications. In this post, we’ll explore how these new security features can strengthen your organization’s email security strategy

Introduction

Email remains a critical business communication channel, yet it’s also one of the most targeted by cybercriminals. When you’re managing an organization’s communications, a single compromised account can lead to significant financial losses, damage your reputation, and serve as a gateway for additional cyber attacks. Traditional username and password protection is no longer adequate against growing cyber threats.

With Amazon WorkMail, you now have powerful tools to enhance your email security. Our support for Multi-Factor Authentication (MFA) and Personal Access Token (PAT) capabilities provides administrators with essential additional security layers to prevent unauthorized account access.

This blog demonstrates WorkMail’s integration with the IAM Identity Center’s default identity store to enable these advanced security features. If you’re using third-party identity providers like Microsoft Entra ID or Okta Universal Directory, you’ll find dedicated integration guides in our documentation.

High-level Overview

Amazon WorkMail’s default authentication is established via a unique username & password:

  1. Users of the WorkMail web-app sign in using their username and password.
  2. Users who access WorkMail from a desktop &/or mobile email application sign in using their username and password.WorkMail standard login

After you integrate WorkMail with IAM Identity Center, Amazon WorkMail can be configured with enhanced authentication that requires:

  1. Users of the WorkMail web-app will log-in via their unique AWS Access Portal using username, password and a MFA token. Upon successful log-in, they select and are redirected into the WorkMail web-app.
  2. Users who access WorkMail from a desktop &/or mobile email app continue to sign in to WorkMail using their username, however they must use a personal access token (PAT) instead of their WorkMail password.

WorkMail via MFA or PAT

More details about WorkMail authentication can be found in our documentation.

Prerequisites

  1. Administrator access to an AWS account
    1. You can evaluate the integration in this post for a limited period of time using an AWS Free Tier Account (link = https://aws.amazon.com/free )
  2. Administrator access to an Amazon WorkMail Organization
    1. Your WorkMail organization should have at least 3 or more users for testing
  3. Administrator access to Amazon IAM Identity Center
  4. Your WorkMail and IAM Identity Center must be in the same AWS region

High-level Configuration Steps

  1. Configure Identity Center (see our documentation for detailed information).
  2. Configure WorkMail to use Identity Center (see our documentation for detailed information).
  3. Assign IAM Identity Center users/groups to WorkMail organization
    1. Associate Amazon WorkMail users with IAM Identity Center users (this step is not necessary if your IdC and WorkMail user names are exactly the same, see our documentation for details)
  4. Check Authentication mode (see documentation) & Personal access token configuration (see documentation).
    1. Allow both WorkMail Directory (no MFA/PAT) and Identity Center (requires MFA & PAT) modes for testing.
  5. Test your users’ access to WorkMail with MFA & PAT
  6. Notify your WorkMail users of upcoming changes to login procedures.
  7. Switch WorkMail Authentication Mode to Identity Center only.
    1. When your users are ready for MFA and PAT, switch authentication mode to require MFA and desktop and mobile email clients to use PAT.
  8. Review additional WorkMail security guidance in AWS blogs and documentation to ensure you are up to date with the latest security guidance.

Detailed Configuration Guidance

Configure AWS IAM Identity Center

    1. Open the IdC console in the same AWS region as your WorkMail organization.
      1. If this is your first time accessing IAM Identity Center, you’ll be greeted with the IdC console home page and “Enable IAM Identity Center”. Click the **`Enable`** button.
      2. Enable IAM Identity Center
      3. Unless you have a reason to use an account instance of IdC, choose Enable.
      4. Org instance of IdC
      5. In a new browser window, open the WorkMail console in the same AWS region as the IdC you created above.
      6. Arrange the IdC console browser next to the WorkMail console browser window so you can easily copy/paste between the two services.
      7. Sync IdC and WorkMail users
      8. In the IdC console’s left navigation rail, choose users and click add user.
        1. Create several IdC user accounts with the same usernames and email addresses as your WorkMail users.
          1. Using identical usernames in Amazon WorkMail and IAM Identity Center simplifies user synchronization and reduces authentication errors during integration. This alignment streamlines troubleshooting and user lifecycle management while ensuring consistent access control across both services.)
        2. Make sure the “Send an email to this user with password setup instructions.” is selected.
          1. The user will receive an email with a link to set up a password and instructions to connect to the AWS access portal. The link will be valid for up to 7 days. You can grant this user permissions to accounts or applications (such as WorkMail) when they sign in to the AWS access portal.
          2. join-idc-email
      9. In IdC’s left navigation rail, choose groups and create a new group called “workmail_users”.
        1. IdC-workmail-users-group
        2. Add the IdC users created above to the IdC workmail_users group.
  1. Configure WorkMail to use Identity Center
    1. In the WorkMail console’s left navigation rail, click the link for Identity Center.
    2. Click the down arrow for Multi-factor authentication setup guide
    3. Click Step 1 – Enable identity center and click Enable.
    4. enable-MFA-workmail.
  2. Assign IAM Identity Center users/groups to WorkMail organization
    1. Click the down arrow for Multi-factor authentication setup guide
    2. Click Step 2 – Add and Assign users and click Next
    3. add-assign-users
    4. Assigning users and groups – Users and groups synced to your Identity Center directory are available to assign to your application. Learn more
      1. Click Get Started 
      2. Type workmail_users and select it from the drop-down list.
      3. assign-group
      4. Click Assign
        1. You will get a message “Successfully assigned group workmail_users. Please continue with step 3 by associating users within this group with WorkMail users.”
  3. Authentication mode & Personal access token configuration
    1. The default Authentication mode is set to WorkMail directory and Identity center. Don’t change this yet.
      1. This will allow WorkMail web-app users to continue to login to the WorkMail client directly, without MFA.
    2. The Personal access token configuration default is set to active, and token lifespan set to 365 days. PATs are used by desktop and mobile email clients to login to WorkMail.
      1. This will allow desktop and mobile email clients to continue to login to the WorkMail with their username and password, without a PAT.
  4. Test WorkMail logins to verify a few users can properly access their WorkMail accounts via both the WorkMail web-app and your organization’s unique AWS access portal URL.
    1. Open the Amazon WorkMail web application and login as one of your test users.
      1. You should have an email invitation to join AWS IAM Identity Center.
      2. Accept the invitation.
      3. Create a IdC password.
      4. Use your username and the new password to login to IdC.
      5. Register an MFA device
      6. enable-mfa
      7. Click Next
      8. You’ll be redirected to the AWS access portal.
        1. Enter your user name and password
        2. Provide your MFA token
      9. Click the tile for Amazon WorkMail to login to the WorkMail web-app
    2. Desktop or mobile email software users need to create PATs to access WorkMail (once the WorkMail administrator disables the WorkMail directory Authentication mode and logins are via the Identity center AWS access portal URL only). Note – PATs are retrieved by individual users from within the WorkMail web-client after logging in via the AWS access portal URL (with MFA).
      1. Open the AWS access portal URL and login
        1. You can find the URL from the Identity Center console > Settings > AWS access portal URL
      2. Login via your username password
      3. Register an MFA device
      4. You’ll be redirected to the AWS access portal.
        1. Enter your user name and password
        2. Provide your MFA token
      5. Click the tile for Amazon WorkMail to login to the WorkMail web-app
      6. In the web-app, click the settings (gear in top right) icon
      7. get-PAT
        1. In settings, click Personal access token and Create token
        2. Enter a token name (typically the device on which you’ll use this PAT) and select create token.
        3. Copy the token value (this is the only time you can retrieve this token value).
        4. Open your desktop or mobile email software, enter your username and your PAT (the PAT replaces your existing user password).
        5. update-email-app-with-pat
  5. Notify your WorkMail users of upcoming changes to login procedures
    1. Once you have tested the integration between Amazon WorkMail and IAM Identity Center with a few test users, you should prepare your WorkMail users for the increased login security. For example, you could send them an email that explains:
      1. The organization is adding an additional login security step to help protect their inboxes.
      2. Inform them that they should anticipate an email from the AWS IAM Identity Center with info about the upcoming implementation of MFA for web-app users and PATs for desktop and mobile client users.
      3. Users should accept the invitation and create a new password for the AWS IAM Identity Center.
      4. Inform them that once WorkMail MFA is enabled, all WorkMail web-app users will be required to use their username, password and MFA.
      5. Inform them that once WorkMail PATs are enabled, all WorkMail desktop and mobile email client users will need to login to the WorkMail web client (with MFA) via the AWS access portal URL, create one PAT per software client (the same PAT can not be used on desktop and mobile). They then must update their desktop or mobile email software to use their username and PAT, instead of their current password. Explain that the PAT is now their email client application password and their personal desktop or mobile email software passwords will no longer work.
      6. Provide users with a way to request support.
  6. Switch WorkMail Authentication Mode to Identity Center only
    1. Once you are satisfied that your WorkMail users have incorporated MFA and/or PATs into their WorkMail login routines, the WorkMail administrator should disable the WorkMail directory Authentication mode found in the WorkMail console under Organization > Identity Center.
  7. Review additional guidance to improve WorkMail security via AWS blogs and documentation.
    1. WorkMail Audit Logging Overview: https://aws.amazon.com/blogs/messaging-and-targeting/an-introduction-to-amazon-workmail-audit-logging/
    2. Custom Security Alarm Setup: https://aws.amazon.com/blogs/messaging-and-targeting/how-to-create-a-big-yellow-taxi-to-help-protect-amazon-workmail/
    3. For comprehensive security guidelines, refer to the Amazon WorkMail Security Documentation: https://docs.aws.amazon.com/workmail/latest/adminguide/security.html

Conclusion – Strengthen your Amazon WorkMail security with IAM Identity Center

By integrating Amazon WorkMail with IAM Identity Center you can more fully protect your organization’s email communications. This integration also centralizes user access management, allowing you to:

  • Manage WorkMail access alongside other AWS applications
  • Reduce security risks in a landscape of constant cyber threats
  • Simplify administrative tasks through a single dashboard

To keep your email environment secure, we recommend you:

Take control of your email communications today with Amazon WorkMail

  • Enable IAM Identity Center Integration
  • Connect your WorkMail organization to centralized access management
  • Configure WorkMail to require:
    • Multi-Factor Authentication (MFA) – Adds an extra layer of security for web-app users
    • Personal Access Tokens (PAT) – Add an extra layer of security for desktop and mobile client access
  • Visit the WorkMail Console (https://aws.amazon.com/workmail/) to begin configuration

Need guidance? Contact your AWS account team or check out our technical documentation.

Join the conversation and connect with other administrators and security professionals on the AWS re:Post community to share insights and learn best practices.

Automatización de flujos de trabajo con WhatsApp usando AWS End User Messaging Social

Post Syndicated from Bruno Giorgini original https://aws.amazon.com/blogs/messaging-and-targeting/automatizacion-de-flujos-de-trabajo-con-whatsapp-usando-aws-end-user-messaging-social/

En el mundo actual, WhatsApp se ha convertido en una aplicación de comunicación ampliamente utilizada, con más de 2.700 millones de usuarios globalmente. Como propietario de un negocio, probablemente tenga clientes que usan WhatsApp regularmente. Esto presenta una oportunidad para usar WhatsApp como un canal adicional para aumentar su alcance e interactuar con sus clientes de manera más efectiva. La mensajería de WhatsApp no necesita limitarse a interacciones uno a uno con sus clientes. También puede usarla para automatizar algunos de sus flujos de trabajo empresariales. Estos flujos de trabajo pueden ser parte de sus procesos corriendo en AWS, o puede construir nuevas automatizaciones usando servicios de AWS.

AWS End User Messaging Social vincula de forma nativa su WhatsApp Business Account (WABA) y su cuenta de AWS. Esto ayuda a simplificar la construcción de integraciones entre la aplicación de mensajería WhatsApp de su cliente y sus cargas de trabajo ejecutándose en AWS. Además, también proporciona una experiencia de facturación unificada dentro de AWS.

Esta publicación explica una solución para establecer esta integración y automatizar sus flujos de trabajo empresariales usando WhatsApp y AWS End User Messaging Social.

Visión general de la solución

La solución utiliza AWS End User Messaging Social para integrar su carga de trabajo implementada en AWS con su portafolio de negocios de Meta. Esto le permite recibir mensajes de WhatsApp que sus clientes finales envían, directamente en su entorno AWS. Un mensaje entrante de WhatsApp que AWS End User Messaging Social recibe activa una notificación de Amazon Simple Notification Service (SNS), que luego activa una función AWS Lambda. Esta función Lambda procesa la solicitud y luego envía un mensaje de WhatsApp como respuesta al usuario. Podrá construir flujos de trabajo empresariales complejos y marketing automatizado de WhatsApp expandiendo esta solución.

La arquitectura de la solución de la publicación muestra cómo el cliente envía un mensaje de WhatsApp a una carga de trabajo alojada en AWS. AWS End User Messaging Social recibe el mensaje de WhatsApp y, a continuación, lo reenvía a un tema de redes sociales. Esto activa una función Lambda. A continuación, la función Lambda envía una respuesta de WhatsApp al teléfono móvil del cliente a través de AWS End User Messaging Social.

Figura 1: Arquitectura de la Solución

 

IMPORTANTE: Tenga en cuenta que cualquier usuario de WhatsApp puede enviar un mensaje a su número WABA y activar el flujo de trabajo. Para evitar incurrir en cargos continuos, complete los pasos en la sección “Limpieza“.

Requisitos previos

Antes de comenzar esta guía, asegúrese de tener:

  1. Una cuenta de AWS
  2. Un portafolio de negocios de Meta. Cree uno siguiendo estas instrucciones
  3. Un número de teléfono para crear una WABA
  4. Un teléfono móvil con la aplicación WhatsApp Messenger instalada para probar la solución. Tenga en cuenta que la aplicación móvil usa un número de teléfono diferente al que está asociado con su WABA.
  5. AWS CLI instalado y configurado para acceder a su cuenta AWS
  6. Serverless Application Model CLI para implementar la aplicación de ejemplo
  7. node.js >= 22.x

Implementación de la solución

Paso 1: Crear un tema SNS

  1. Navegue a la página de SNS.
  2. Use la región de AWS donde planea usar AWS End User Messaging Social.
  3. Seleccione “Standard” como “Type” del tema
  4. Ingrese “WhatsAppIncomingMessages” como “Name” del tema
  5. Haga clic en el botón “Create topic“.
  6. En los detalles del tema que aparecen, tome nota del “ARN” del tema

Paso 2: Agregar WABA a AWS End User Messaging Social

  1. Navegue a la página WABA.
  2. Haga clic en el botón “Add WhatsApp phone number
  3. Haga clic en “Launch Facebook portal” para conectar su cuenta AWS y su portafolio de negocios de Meta.
  4. Siga las instrucciones en la ventana emergente de Facebook para completar el registro. A continuación se muestran capturas de pantalla de los pasos clave en el proceso:
Captura de pantalla de los pasos que debes seguir para conectar tu cuenta de AWS y tu cartera de negocios Meta.

Figura 2: Conectar AWS y el Portafolio de Negocios de Meta

  1. Después de que se cierre la ventana emergente de registro de Facebook, verá este mensaje “Link established” en su consola AWS. Ingrese el ARN del tema SNS creado en el Paso 1 en la sección “Message and event destination”.
Captura de pantalla de la consola de AWS que muestra la conexión establecida entre AWS y la cuenta WABA y el tema de SNS

Figura 3: Conexión exitosa entre la cuenta WABA y AWS

  1. Haga clic en “Add phone number” para completar la vinculación de su cuenta comercial de WhatsApp con su cuenta AWS.
  2. Navegue a la página WABA y verifique que la cuenta agregada esté en estado “Active”.
  3. Haga clic en el ID de cuenta comercial que agregó para navegar a la página de detalles.
  4. En la sección “Phone Numbers“, tome nota del “Phone number ID” del número que agregó.

Paso 3: Implementar la función Lambda para manejar los mensajes de WhatsApp de los clientes

  1. Clone el repositorio de ejemplo e implemente la función Lambda siguiendo los pasos a continuación:
$ git clone https://github.com/aws-samples/aws-end-user-messaging-social-automation.git

$ cd aws-end-user-messaging-social-automation/
  1. Construya el proyecto ejecutando este comando:
$ sam build
  1. Implemente la aplicación ejecutando el comando a continuación y siga las indicaciones:
$ sam deploy --guided
...
# Enter the details for questions asked
Stack Name [sam-app]: EndUserMessagingWhatsApp
AWS Region [us-east-1]: <AWS region where you have created the WABA account>
Parameter SNSTopicArn []: <ARN of the SNS topic created in Step 1>
Parameter PhoneNumberID []: <Phone number ID from Step 2>

Acepte los valores predeterminados del resto de las preguntas para completar la implementación. Esto creará una función Lambda que se activa cuando se envía un mensaje de WhatsApp a su número WABA.

Paso 4: Prueba de la solución

  1. Desde un teléfono que tenga WhatsApp instalado, envíe este mensaje a su número WABA:
Hello
  1. Valide que ve lo siguiente en respuesta al mensaje anterior:
    • Marca azul, indicando que el mensaje ha sido recibido y leído
    • Una reacción 👋 a su mensaje
    • Un mensaje de respuesta diciendo “Hello {nombre del perfil}, how can we help you?”
    • Una lista de opciones para elegir
  2. Seleccione una de las opciones
  3. Recibirá un mensaje estático que lo dirigirá a encontrar más información sobre AWS End User Messaging Social en la página de documentación. La interacción se verá como en la imagen siguiente:
Captura de pantalla de la interacción del usuario final en WhatsApp. El usuario envía un mensaje de inicio "Hello!", al que la empresa responde con 2 opciones para elegir. El cliente selecciona la opción de consultar el estado del pedido, a lo que la empresa responde con un enlace a la documentación sobre el servicio de mensajería para el usuario final.

Figura 4: Interacción del usuario final con WhatsApp con el negocio

Solución de problemas

Acceda a los registros de su función lambda navegando a la página de registros de CloudWatch y buscando “WhatsAppMessageHandler” en el cuadro de búsqueda. Seleccione el grupo de registros de su función Lambda y haga clic en el flujo de registro correspondiente al momento en que ejecutó la prueba. La función Lambda registra información cuando se recibe y procesa un mensaje. Sin embargo, los eventos SNS recibidos por Lambda y los mensajes de usuario extraídos se excluyen deliberadamente de los registros de CloudWatch. El evento contiene información como el número de teléfono del remitente y el nombre del perfil de WhatsApp que su empresa puede clasificar como sensible. Verifique las políticas de manejo de datos de su organización antes de registrar esta información. Encuentre más información sobre la estructura de mensajes de los diferentes tipos de mensajes en la documentación para desarrolladores de Meta.

También puede usar AWS CLI para enviar mensajes a los clientes. Puede enviar mensajes de tipo “text” si su cliente ya ha iniciado una conversación con usted. Ejecute el siguiente comando para enviar una respuesta a un mensaje de WhatsApp:

aws socialmessaging send-whatsapp-message \
--cli-binary-format raw-in-base64-out \
--message '{"messaging_product": "whatsapp", "to": "+15554567890", "text": { "preview_url": false, "body": "Hello! How can we help you?"}}' \
--origination-phone-number-id phone-number-id-beac123456789abcdefgh \
--meta-api-version v20.0

Tenga en cuenta que solo puede enviar mensajes de tipo “text” al cliente si han iniciado una conversación con usted y han pasado menos de 24 horas desde la última vez que le enviaron un mensaje. Use mensajes con plantillas para iniciar conversaciones con sus clientes, por ejemplo, para enviar una OTP (contraseña de un solo uso) o mensajes de marketing. Meta necesita aprobar las plantillas de mensajes antes de que pueda enviarlas a los clientes.

Monitoreo

AWS End User Messaging Social puede publicar datos relacionados con el costo de usar el servicio en las métricas de CloudWatch. Para habilitarlo, ingrese el comando a continuación para crear un rol vinculado al servicio IAM llamado “AWSServiceRoleForSocialMessaging“:

aws iam create-service-linked-role --aws-service-name social-messaging.amazonaws.com

Puede monitorear AWS End User Messaging Social usando CloudWatch para ver métricas:

  • WhatsAppMessageFeeCount
  • WhatsAppConversationFeeCount

También puede recibir notificaciones cuando se alcance un umbral específico configurando una alarma de CloudWatch en estas métricas. Esto puede ayudarlo a realizar un seguimiento de los costos asociados con AWS End User Messaging Social.

Limpieza

Tenga en cuenta que cualquier usuario de WhatsApp puede enviar un mensaje a su número WABA y activar el flujo de trabajo. Para evitar incurrir en cargos continuos, complete los siguientes pasos de limpieza:

  1. Para eliminar la función Lambda y los recursos asociados, ejecute el siguiente comando:
sam delete
  1. Para eliminar los recursos de AWS End User Messaging Social, navegue a la página WABA. Seleccione el ID de cuenta comercial de su WABA y haga clic en el botón “Unlink”.

Conclusión

Esta publicación muestra cómo puede simplificar la integración entre WhatsApp y sus cargas de trabajo ejecutándose en AWS, permitiéndole automatizar las interacciones con sus clientes finales. Muestra cómo puede activar una función Lambda desde un mensaje de WhatsApp iniciado por su cliente final. Puede extender esta solución para construir flujos de trabajo más complejos, por ejemplo, activar AWS Step Functions, invocar sus cargas de trabajo basadas en contenedores ejecutándose en AWS, entre otros.

Para más información, consulte los siguientes recursos:

Automate workflows with WhatsApp using AWS End User Messaging Social

Post Syndicated from Bruno Giorgini original https://aws.amazon.com/blogs/messaging-and-targeting/whatsapp-aws-end-user-messaging-social/

In today’s world, WhatsApp has become a widely used communication app, with over 2.7 billion users globally. As a business owner, you likely have customers who use WhatsApp regularly. This presents an opportunity to use WhatsApp as an additional channel to increase your reach and engage with your customers more effectively. WhatsApp messaging need not be limited to one-on-one interactions with your customers. You can also use it to automate some of your business workflows. These workflows can be part of your existing workloads running on AWS, or you can build new automations using AWS services.

AWS End User Messaging Social natively links your WhatsApp Business Account (WABA) and your AWS account. This helps simplify building integrations between your customer’s WhatsApp messenger app and your workloads running on AWS. Additionally, it also provides unified billing experience within AWS.

This post walks through a solution to establish this integration and automate your business workflows using WhatsApp and End User Messaging Social.

Overview of solution

The solution uses End User Messaging Social to integrate your workload deployed on AWS with your Meta business portfolio. This allows you to receive WhatsApp messages that your end customers send, directly in your AWS environment. An incoming WhatsApp message that End User Messaging Social receives triggers an Amazon Simple Notification Service (SNS) notification, which then activates an AWS Lambda function. This Lambda function processes the request and then sends a WhatsApp message as a response to the user. You can build complex business workflows and automated WhatsApp marketing expanding this solution.

Solution architecture of the post shows customer sending a WhatsApp message to a workload that is hosted on AWS. AWS End User Messaging Social receives the WhatsApp message. It then forwards the message to an SNS topic. This triggers a Lambda function. The Lambda function then sends a WhatsApp reply back to the customer’s mobile phone through End User Messaging Social

Figure 1: Solution Architecture

 

IMPORTANT: Note that any WhatsApp user can send a message to your WABA number and trigger the workflow. To avoid incurring ongoing charges, complete the steps in the “Cleaning up” section.

Prerequisites

Before starting this walkthrough, ensure you have:

  1. An AWS account
  2. A Meta business portfolio. Create one by following these instructions
  3. A phone number to create a WABA
  4. A phone with WhatsApp Messenger app installed to test the solution. Note that the mobile app uses a different phone number than the one that is associated with your WABA.
  5. AWS CLI installed and configured to access to your AWS account
  6. Serverless Application Model CLI to deploy the sample application
  7. node.js >= 22.x

Implementing the solution

Step 1: Create an SNS topic

  1. Navigate to the SNS page.
  2. Use the AWS region where you plan to use End User Messaging Social.
  3. Select “Standard” as the topic “Type”
  4. Enter “WhatsAppIncomingMessages” as the “Name” of the topic
  5. Click on the “Create topic” button.
  6. In the topics details that appear, note the “ARN” of the topic

Step 2: Add WABA to AWS End User Messaging

  1. Navigate to the WABA page.
  2. Click on the “Add WhatsApp phone number” button
  3. Click on “Launch Facebook portal” to connect your AWS account and your Meta business portfolio.
  4. Follow the instructions in the Facebook pop-up to complete the registration. Below are screenshots of the key steps in the process:
Screenshot of the steps you need to follow to connect your AWS account and your Meta business portfolio.

Figure 2: Connect AWS and Meta Business Portfolio

  1. After the Facebook signup popup closes, you will see this “Link established” message on your AWS console. Enter the ARN of the SNS topic created in Step 1 in the “Message and event destination” section.
Screenshot of the AWS console that shows established connection between AWS and WABA account and the SNS topic

Figure 3: Successfully linked WABA and AWS account

[Alt text: Screenshot of the AWS console that shows an established connection between AWS and WABA account and the SNS topic]

  1. Click “Add phone number” to complete linking your WhatsApp business account with your AWS account.
  1. Navigate to the WABA page and verify that the added account is in “Active” status.
  2. Click on the Business account ID you added to navigate to the details page.
  3. From the “Phone Numbers” section, note the “Phone number ID” of the number that you added.

Step 3: Deploy the Lambda function to handle customer WhatsApp messages

  1. Clone the example repository and deploy the Lambda function by following the steps below:
$ git clone https://github.com/aws-samples/aws-end-user-messaging-social-automation.git

$ cd aws-end-user-messaging-social-automation/
  1. Build the project by running this command:
$ sam build
  1. Deploy the application by running the command below and follow the prompts:
$ sam deploy --guided
...
# Enter the details for questions asked
Stack Name [sam-app]: EndUserMessagingWhatsApp
AWS Region [us-east-1]: <AWS region where you have created the WABA account>
Parameter SNSTopicArn []: <ARN of the SNS topic created in Step 1>
Parameter PhoneNumberID []: <Phone number ID from Step 2>

Accept the defaults of the rest of the questions to complete the deployment. This will create a Lambda function that is triggered when a WhatsApp message is sent to your WABA number

Step 4: Testing the solution

  1. From a phone that has WhatsApp installed, send this message to your WABA phone number
Hello
  1. Validate that you see the following in response to the previous message:
    1. Blue check mark, indicating the message has been received and read
    2. A 👋 reaction to your message
    3. A reply message saying “Hello {profile name}, how can we help you?”
    4. A list of options to choose from
  2. Select one of the options
  3. You will receive a static message pointing you to find more information about the End User Social in the documentation page. The interaction will look like the picture below:
Screenshot of the end user’s WhatsApp interaction. User sends a Hello! Message, to which the business responds with 2 options to choose from. Customer selects check order status option, to which the business responds with a link to documentation about End user messaging service.

Figure 4: End user’s WhatsApp interaction with the business

Troubleshooting

Access the logs of your lambda function by navigating to the CloudWatch logs page and searching for “WhatsAppMessageHandler” in the search box. Select the log group of your Lambda function and click on the log stream corresponding to the time you ran the test. The Lambda function logs information when a message is received and processed. However, Lambda’s received SNS events and extracted user messages are deliberately excluded from CloudWatch logs. The event contains information such as sender’s phone number and WhatsApp profile name that can be classified as sensitive by your business. Check your organization’s data handling policies before logging this information. Find more information on the message structure of the different message types in Meta developer documentation.

You can also use AWS CLI to send messages to customers. You can send messages of “text” type if your customer has already initiated a conversation with you. Run the command below to send a reply to a WhatsApp message:

aws socialmessaging send-whatsapp-message \
--cli-binary-format raw-in-base64-out \
--message '{"messaging_product": "whatsapp", "to": "+15554567890", "text": { "preview_url": false, "body": "Hello! How can we help you?"}}' \
--origination-phone-number-id phone-number-id-beac123456789abcdefgh \
--meta-api-version v20.0

Note that you can only send messages of type “text” to customer if they have started a conversation with you and it has been less than 24 hours since the last time they messaged you. Use templated messages to start conversations with your customers, for example, to send an OTP (one-time password) or marketing messages. Meta needs to approve message templates before you can send them to customers.

Monitoring

End User Messaging Social can publish data related to cost of using the service to CloudWatch metrics. To enable this, enter the command below to create a IAM service linked role called “AWSServiceRoleForSocialMessaging”:

aws iam create-service-linked-role --aws-service-name social-messaging.amazonaws.com

You can monitor AWS End User Messaging Social using CloudWatch to view metrics:

  • WhatsAppMessageFeeCount
  • WhatsAppConversationFeeCount

You can get also get notified when a specific threshold is reached by setting up a CloudWatch alarm on these metrics. This can help you keep track of costs associated with End User Messaging Social.

Cleaning up

Note that any WhatsApp user can send a message to your WABA number and trigger the workflow. To avoid incurring ongoing charges, complete the following cleanup steps:

  1. To delete the Lambda function and the associated resources, run the following command:
sam delete
  1. To delete the End User Messaging Social resources, navigate to the WABA page. Select the Business account ID of your WABA and click on the “Unlink” button.

Conclusion

This post shows how you can simplify the integration between WhatsApp and your workloads running on AWS, allowing you to automate interactions with your end customers. It shows how you can trigger a Lambda function from a WhatsApp message initiated by your end customer. You can extend this solution to build more complex workflows, for example triggering an AWS Step Functions , invoking your container based workloads running on AWS among others.

For more information, see the following resources:

Enhance Your Email Campaigns using Amazon SES SendBulkEmail APIs Inline Templates

Post Syndicated from Satya S Tripathy original https://aws.amazon.com/blogs/messaging-and-targeting/enhance-your-email-campaigns-using-amazon-ses-sendbulkemail-apis-inline-templates/

Amazon Simple Email Service (SES) is a cloud-based email sending service provided by Amazon Web Services (AWS), handling both inbound and outbound email traffic for your applications. It allows users to send and receive email using SES’s reliable and cost-effective infrastructure without having to provision email servers yourself. Customers use Amazon SES to send emails like one time passwords (OTPs), transactional emails such as order confirmation, and promotional/marketing emails.

For this post, you should be familiar with the following:

Amazon SES continues to evolve, offering new features that help you simplify and optimize your email campaigns. We’re excited to announce the addition of inline template support for both the SendEmail and SendBulkEmail APIs. This new capability allows you to include template content directly in your API requests, reducing complexity and eliminating the need to manage separate template resources in your SES account.

What are inline templates?

Inline templates allow you to provide the subject, HTML body, and text body of an email directly in the API request, along with dynamic placeholders for personalized content. Instead of creating and storing a separate email template in SES, you can define the template as part of your API call. This feature is especially useful for organizations that need flexibility in managing numerous templates or want to make quick adjustments to email content.

How inline templates simplify your workflow

Previously, Amazon SES required you to create and store email templates in your SES account, which you would then reference by name or Amazon Resource Name (ARN) when sending an email. This process adds some management overhead, particularly for organizations that frequently create new templates or exceed the limit of templates per AWS Region. With inline templates, you can reduce complexity by defining your email content directly in the API payload, avoiding the need to create and manage stored templates. This approach can improve flexibility, allowing you to quickly make changes to your email content without updating stored templates. Additionally, it can simplify your integration by providing template content directly within your application logic, making the process more seamless and efficient. When using the and SendBulkEmail API, you can include personalized content for up to 50 destinations in a single call, making large-scale communication more efficient.

How to use inline templates

To use inline templates, you simply provide the email content (subject, text, HTML) and the replacement data directly in the SendBulkEmail API request payload within TemplateContent parameter.

Inline-template API

Here’s an example for using the SendBulkEmail API with inline templates:

file://mybulkemail-inline-template-conten.json:

{
    "FromEmailAddress": "Mary Major <[email protected]>",
    "DefaultContent": {
        "Template": {
            "TemplateContent": {
                "Subject": "Greetings, {{name}}!",
                "Text": "Dear {{name}},\r\nYour favorite animal is {{favoriteanimal}}.",
                "Html": "<h1>Hello {{name}},</h1><p>Your favorite animal is {{favoriteanimal}}.</p>"
            },
            "TemplateData": "{ \"name\":\"friend\", \"favoriteanimal\":\"unknown\" }"
        }
    },
    "BulkEmailEntries": [
        {
            "Destination": {
                "ToAddresses": [
                    "[email protected]"
                ]
            },
            "ReplacementEmailContent": {
                "ReplacementTemplate": {
                    "ReplacementTemplateData": "{ \"name\":\"Anaya\", \"favoriteanimal\":\"angelfish\" }"
                }
            }
        },
        {
            "Destination": {
                "ToAddresses": [
                    "[email protected]"
                ]
            },
            "ReplacementEmailContent": {
                "ReplacementTemplate": {
                    "ReplacementTemplateData": "{ \"name\":\"Liu\", \"favoriteanimal\":\"lion\" }"
                }
            }
        }
    ],
    "ConfigurationSetName": "ConfigSet"
}

SES SendBulkEmail API call:

aws sesv2 send-bulk-email -cli-input-json file://mybulkemail-inline-template-conten.json

Output:

{
    "BulkEmailEntryResults": [
        {
            "Status": "SUCCESS",
            "MessageId": "010001xxxxxx-xxxxxxxx-xxxx-xxxx-000000"
        },
        {
            "Status": "SUCCESS",
            "MessageId": "020002xxxxxx-xxxxxxxx-xxxx-xxxx-000000"
        }
    ]
}

Backward compatibility

If you’re currently using stored templates, don’t worry – Amazon SES still supports the use of stored templates, and you can continue to use them without any changes. Inline templates are simply an additional option for customers who need more flexibility or wish to avoid managing stored templates altogether. Since inline templates only support the use of simple substitution, stored templates remain the solution for advanced personalization options such as conditional logic or complex formatting. More details in our doc link: How to use Bulk email template.

Get started today

The inline template feature is now available in all AWS Regions where Amazon SES is offered. To start using inline templates, refer to the Amazon SES Developer Guide and what’s new announcement. There are no additional charges applicable for using inline template feature.

Conclusion

The inline templating feature in SendBulkEmail allows you to avoid worrying about template management by updating / creating new email templates whenever a minor modification or alteration is required in the existing templates, as well as cleaning up unused templates on a regular basis. Therefore, if your business has a high number of email template requirements, there are no predefined rules or patterns for creating email templates, and you need to generate many templates simultaneously within Amazon SES, you must use inline templating feature of SendBulkEmail API . If you do not want to use the Inline templating capability, you can continue to use the templated SendBulkEmail API from Amazon SES.

Modernize email sending with Amazon Simple Email Service and Proofpoint SER

Post Syndicated from Zip Zieper original https://aws.amazon.com/blogs/messaging-and-targeting/modernize-email-sending-with-amazon-simple-email-service-and-proofpoint-ser/

As organizations migrate to the cloud, managing email security and delivery becomes increasingly complex. This post explores how Amazon Simple Email Service (SES) and Proofpoint Secure Email Relay (SER) work together to provide a robust solution for modern email sending.

Shifting email workflows to the cloud

For many organizations, Simple Mail Transfer Protocol (SMTP) relay is a critical email delivery mechanism that facilitates the transmission of email messages between different domains and servers. When an email is sent to a recipient outside the sender’s domain, SMTP relay(s) ensures the message is routed correctly and delivered to the intended destination.

Traditionally, on-premises SMTP relay servers were used by messaging and security teams to manage email sending from on-premises applications on behalf of their organizations’ domains. This approach helps to:

  • Guard against brand damage and loss of sensitive data
  • Protect recipients from fraud
  • Provide straightforward control over email sending

However, as applications are modernized and migrated to the cloud, email sending has changed. Many organizations now outsource bulk email sending to cloud service providers such as Amazon SES. This shift has made it challenging for internal teams to regulate their organization’s email effectively.

Addressing modernization challenges

Amazon Web Services (AWS) is a popular choice for developing and modernizing applications, with Amazon SES offering a secure, scalable and cost effective service for applications to send email. To address the need for additional security and control over email, Proofpoint offers their popular Secure Email Relay (SER) on the AWS Marketplace.

Using Amazon SES with Proofpoint SER combines the convenience and features of Amazon SES with Proofpoint SER’s ability to:

  • Regulate and govern outbound application email
  • Support migration from on-premises relay to the cloud
  • Apply security controls to application emails

Email flow and security

SES Mail Manager allows emails to be conditionally routed from Amazon SES to Proofpoint SER. The process includes:

  1. Scanning emails with Proofpoint threat detection technologies
  2. Applying centrally managed Proofpoint SER policies
  3. Performing DKIM-signing and distributing DMARC compliant emails
  4. Sending mail to recipients

Two configuration options are available:

  1. Proofpoint SER handles distribution to final recipients (Figure 1)
  2. Emails are routed back to Amazon SES for distribution after Proofpoint SER processing (Figure 2)

In the first configuration, email is sent to SES and routed through a Mail Manager SMTP relay to Proofpoint SER where it is processed for threat detection, malware, spam, sensitive data, and more. In this configuration, Proofpoint SER distributes the email to recipients through a STMP relay or another email sending service beyond Proofpoint. Deliverability reporting, archiving, and other capabilities are left to Proofpoint or downstream providers.

SES to SER

Figure 1: Proofpoint SER applying security controls to application emails before sending.

In the second configuration, email is sent to SES which routes through a Mail Manager SMTP relay to Proofpoint SER to be processed for threat detection, malware, spam, sensitive data, and more. Proofpoint SER then sends the processed emails back to SES via Mail Manager STMP relay. Deliverability reporting, archiving and other capabilities such as Amazon Q Business can be provided by SES or other AWS services.

SES-SER-SESFigure 2. Security controls applied by Proofpoint SER before routing emails back to Amazon SES for distribution.

Conclusion

The integration of Amazon SES and Proofpoint SER offers a powerful solution for organizations looking to modernize their email sending processes while maintaining robust security. This collaboration allows businesses to leverage the scalability and convenience of cloud-based email services without compromising on control and protection.

Call to Action

Take the next step in modernizing your email infrastructure while enhancing security and control. To learn more about how Amazon SES and Proofpoint SER can benefit your organization:

How to Make Simple Email Service Resilient Across Two AWS Regions with Global Endpoints

Post Syndicated from Zip Zieper original https://aws.amazon.com/blogs/messaging-and-targeting/how-to-make-simple-email-service-resilient-across-two-aws-regions-with-global-endpoints/

Introduction

Amazon Simple Email Service (SES) recently announced Global Endpoints, a major enhancement to its email sending features. This new capability improves the availability and reliability of SES API v2 email sending workloads by automatically distributing messages, in an active/active configuration across the Primary and Secondary AWS regions. When Global Endpoints detects degradation in either the Primary or Secondary SES region, the feature automatically shifts all traffic to the healthy region, no customer intervention is needed.

Multi-Region SES Configuration Challenges

Customers face significant difficulties in correctly implementing a multi-region setup or disaster recovery setup. The process requires careful curation of systems along the failover path and ensuring high availability of these systems. Ironically, the system designed to trigger failover can itself fail when needed most. For many SES customers, the effort required to design, build test, monitor and maintain a two-region SES system outweighs the benefits.

SES Global Endpoints eliminates the need for these complex, custom workarounds. The feature provides a straightforward solution for maintaining email sending during unexpected regional disruptions. Global Endpoints’ built-in safeguards ensure email infrastructure remains resilient when it matters most. Please note that at launch, Global Endpoints does not support SMTP or VPC endpoint access.

SES Global Endpoints: Key Technological Components

Global Endpoints utilizes four new SES components that work together to provide a seamless and reliable multi-region email sending experience:

  1. Multi-Region Endpoint (MREP) is a new type of SES endpoint that automatically distributes email traffic across two AWS regions.
  2. Deterministic Easy DKIM Identities (DEED) makes it easy to setup multi-region identities without having to make any DNS changes.
    1. Read the blog introducing SES DEED for more information.
  3. Updated AWS SES Console Tool walks you through the process and simplifies the duplication of Domain Identities, Configuration sets, and Sending limits across regions.
  4. Readiness Checks in the SES console verify uniformity between configurations of key resources in both the Parent and Secondary SES regions.

How SES Global Endpoints work

GE-MREP-healthy

Figure 1 – SES Global Endpoints with two healthy regions.

Global Endpoints are resources that distribute your SES outbound workloads across two AWS Regions. When you set up Global Endpoints, you select a Primary Region (where the actual Endpoint is created) and a Secondary Region. When configured, a new Global Endpoints resource, called “multi-region endpoint” (MREP) is created and managed. Developers will need to update their SES v2 API enabled applications and services to use their unique MREP as the entry point to SES for their email sending requests.

The Global Endpoints configuration requires that your sending domain identity(s) is verified in both the Primary and Secondary regions. SES Global Endpoints uses DEED to simplify this process. DEED is a new feature that generates consistent DKIM tokens across all AWS Regions based on a Parent Domain Identity that is configured with Easy DKIM. This consistency allows SES to automatically verify a domain identity in the Secondary region once it’s verified in the Primary region, without requiring additional DNS record updates. Customers do not need to make any additional changes other than activating the DEED identity type. When customers expand their sending workload’s geographic footprint, or reconfigure their Global Endpoints settings in the future, their DEED identities will continue to be available and managed automatically. You can learn more about DEED from this post.

Global Endpoints works alongside other SES services, such as Virtual Deliverability Manager (VDM). Once Global Endpoints are enabled, you’ll continue to see per-region data on email sending performance in VDM. If you’ve configured event destinations like CloudWatch, SNS, or Firehose, you can make use of those same monitoring and alerting tools in your second region as soon as you’re ready. As noted below, although Configuration Sets are automatically duplicated in the Secondary region, you must manually duplicate your SES event destinations in those Configuration Sets.

It is important to understand that Global Endpoints is not a failover solution for SES, it’s an active-active implementation; when no regional impairment exists, SES Global Endpoints distributes sending traffic across two AWS regions. Customers who use SES’ shared IP sending pool do not need to make any changes, Global Endpoints will utilize the SES shared IP pool in the Secondary region. Customers who use standard, dedicated IPs must manually set up equivalent number of dedicated IPs in the Secondary region. Once properly configured, Global Endpoints will keep the dedicated IPs warm in both regions as long as you use the MREP and maintain a steady sending volume.

For example, when SES’s regional health monitoring detects degradation in the Primary region (as shown in the diagram below), The MREP automatically shifts all traffic to the healthy region. This illustrates the need for matching configurations in both regions, since all traffic will be sent through a single region, in this example the Secondary region, as long as the impairment exists. When SES’s regional health monitoring detects the impaired region is back to normal, traffic is once again redistributed across both regions. Importantly, no customer intervention is needed; SES Global Endpoints automatically and dynamically monitors and manages the traffic distribution via the MREP.

GE-MREP-impaired

Figure 2 – SES Global Endpoints with one impaired region.

The key benefits of using Global Endpoints include:

  • Simplified multi-region configuration
  • Automatic routing between two regions with no delay
  • More resilient email sending

Global Endpoints: Setup and Use

Using the SES Console, the Global Endpoint setup process assists in duplicating the key artifacts and sending limits from your Primary Region to your Secondary Region. This process ensures that both regions have equivalent verified identities, configuration sets, and approved sending limits sufficient for all of the expected volume. Customers can manually duplicate these key artifacts using the SES v2API or CloudFormation, but we recommend using the SES console because these steps are simplified.

Once the Global Endpoint is ready, key resources duplicated and the application or service has been updated to use the new MREP, SES Global Endpoints automatically routes your outbound traffic evenly between your primary and secondary regions using the multi-region endpoint (MREP). If the MREP detects degraded performance in the Primary or Secondary region, it will automatically route all SES traffic to the healthy region until the impairment is resolved.

Preparing the Parent Region

The high-level steps to setup Global Endpoints using the SES Console are below.  Note – you must already have a primary SES region fully operational with at least 1 fully verified sending identity with production access before setting up Global Endpoints.

Create Global Endpoint

  1. Open the SES console in the Primary AWS region.
  2. In the navigation pane, choose Global Endpoints.
  3. Choose Create Global Endpoint.
  4. Select a Secondary Region from the dropdown menu. (Your Primary Region defaults to the region to which you signed into the console)
  5. Review the configuration and choose Create.

The creation process may take a few seconds. Once completed, the status of your Global Endpoint will change to “Ready.”

Global Endpoints "ready" status

Preparing the secondary region

Once your Global Endpoint is ready, you must now ensure that the your email sending configuration, including all its components (Domain Identities, Configuration sets, email templates, and sending limits), is consistent across the primary and secondary regions before utilizing the Global endpoint for sending emails. This alignment is crucial to avoid potential issues and ensure proper email delivery and tracking.

The new Region duplication feature in the SES console assists you by automatically duplicating resources and duplicating account-level settings from the primary to the secondary region, ensuring that both regions have equivalent configurations.

The high-level steps to setup the secondary region for use with Global Endpoints using the SES Console are below. If you’d like to use the AWS CLI to manually duplicate these resources, consult with the SES v2 API documentation.

Duplicating verified domain identities

Next you’ll use the Duplicate verified domain identities feature in the SES console to create one or more domain identities in the Secondary Region. SES will then use DEED to verify the domain identities in the Secondary Region. Note that DEED can only be used if the Primary Domain Identity is already configured with Easy DKIM. Domain identities that are verified with BYODKIM will need to be created manually in the Secondary Region, as DEED is not applicable in this case.

Important – It’s crucial that both Primary and Secondary Regions have the equivalent verified identities and configuration sets that you intend to send email with, along with matching sending limits to ensure proper functionality of the Global endpoint. Any discrepancies could cause delivery failures, diminished failover reliability, and missing metrics.

To duplicate identities from the SES console:

  1. On the Global endpoints page, choose the Global endpoint you want to duplicate by selecting it from the Name column.
  2. Under the Region duplication tab, choose Duplicate identities.
  3. Select the identities you want to duplicate followed by Confirm.

To duplicate configuration sets from the SES console:

  1. On the Global endpoints page, choose the Global endpoint you want to duplicate by selecting it from the Name column.
  2. From the Region duplication tab, choose Duplicate configuration sets.
  3. Select the configuration sets you want to duplicate followed by Confirm.

Important Notes:

  • When duplicating configuration sets across regions, Event destinations and Reputation settings require manual reconfiguration in the Secondary Region to match the Primary Region’s setup. Since SES event destinations are region-specific, you’ll need to manually recreate these configurations in each region. For cross-regional monitoring and event routing, you can refer to additional AWS documentation for services like CloudWatch (cross-region dashboards), SNS (cross-region message delivery), and EventBridge (cross-region event routing) to develop a comprehensive multi-region event strategy.
  • If you are using SES email template resources, those templates need to be manually duplicated into the Secondary Region (the console is currently unable to perform this action).
  • You must manually synchronize any changes made to the Parent Region’s configuration sets to the Secondary Region to maintain sending integrity. This includes adding or removing standard dedicated IPs to both regions to ensure either region has the required IPs to manage the expected throughput in the case of a regional event or impairment.

The Duplicate production limits feature allows you to:

  • Check if production limits are aligned between primary and secondary regions
  • Request a limit increase in the secondary region if needed

To duplicate production limits from the SES console:

  1. On the Global endpoints page, choose the Global endpoint you want to duplicate by selecting it from the Name column.
  2. From the Region duplication tab, verify the status in the Duplicate production limits card. If the status displays Sending limits not aligned, choose Duplicate production limits.
  3. The Service Quotas page opens in the secondary region where you can request increases to “Sending quota” and “Sending rate” to match the values from the primary region.

Note – SES checks if sending limits are aligned between regions and allows you to request limit increases in the secondary region if needed. We recommend that you request the maximum quota you’re eligible for in both regions. While email traffic is distributed amongst both regions under normal operating conditions, during a failover event, the full volume of email traffic will be sent to one region and its limits should be enough to handle the full volume load.

If any manual steps are required to complete the Global Endpoint creation, they will be shown in the SES Console:

manual steps warning

When the Global Endpoint is fully configured, a MREP will be created with an Endpoint ID (see below). You will use this new endpoint ID when re-configuring your SendEmail and SendBulkEmail API calls. (note – Global Endpoints MREP are only supported by SES APIv2. The feature is not available using SMTP or VPC endpoints).

Now you’re ready to send your first email through SES Global Endpoints and your MREP!

Once you’ve obtained the Endpoint ID of your Global endpoint (this is the MREP), you should update your applications’ SendEmail or SendBulkEmail API calls to include the Endpoint ID value for the -endpoint-id parameter.

Here’s an example of how to specify the Endpoint ID in a SendEmail API call using the AWS CLI (modify the from & to email addresses and the --endpoint-id accordingly):

aws sesv2 send-email \
--from-email-address "[[email protected]](mailto:[email protected])" \
--destination "ToAddresses=[[email protected]](mailto:[email protected])" \
--content "Subject={Data=Test
email,Charset=UTF-8},Body={Text={Data=This is a test email sent using Amazon SES
Global endpoints.,Charset=UTF-8}}" \
--endpoint-id "abcdef12.g3h"

The Global Endpoints console page provides summary observability metrics on the combined workload and a unified view of your email sending volume across both the primary and secondary regions. You can access these metrics through the Cross-region metrics tab on the Global endpoint details page in the SES console..

Conclusion

By using a SES Global Endpoint in their SES API v2 applications and services, SES customers benefit from uninterrupted email delivery during regional service issues. SES Global Endpoints automatically distributes sending workloads across two AWS Regions, significantly enhancing resilience against regional outages. The Global Endpoints feature maintains warmed-up dedicated IP addresses in both regions, when used, and automatically shifts traffic to the healthy region when the other is impaired, without requiring customer intervention. SES Global Endpoints eliminates the pain points typically associated with manually-built, multi-region SES sending systems.

Global Endpoint’s console tools provide quick and easy setup and includes readiness checks to identify and mitigate potential misconfigurations. These enhancements simplify the configuration and management process, making it easier for customers to maintain a robust email sending infrastructure.

Overall, SES Global Endpoints addresses customer needs for a more reliable and easily managed email sending system, automating critical processes and providing robust tools for setup and maintenance. This significant improvement to the email sending experience is expected to benefit AWS customers across various industries and use cases.

Call to Action

Get started with SES Global Endpoints today to enhance your email sending resilience!

  • Visit the AWS Console to enable this feature for your account
  • Review the comprehensive documentation for step-by-step guidance.
  • For personalized assistance, don’t hesitate to contact AWS Support or your AWS account team.

Elevate your email infrastructure now to ensure uninterrupted communication with your customers, even in the face of regional disruptions.

Simplify Multi-Region Email Sending with Simple Email Service’s Deterministic Easy DKIM

Post Syndicated from Zip Zieper original https://aws.amazon.com/blogs/messaging-and-targeting/simplify-multi-region-email-sending-with-simple-email-services-deterministic-easy-dkim/

Introduction

Amazon Simple Email Service (SES) provides customers with a robust, scalable email solution to send large-scale, global email communications. The service offers customers many benefits, including scalability, high deliverability rates, cost effective pay-as-you go pricing, availability in over a dozen AWS regions, and tight integration with other AWS services.

We’ve heard from customers who want to synchronise domain identities across multiple AWS regions that it can be difficult to configure and maintain. These customers have shared that establishing and maintaining AWS region specific domain verifications can be confusing, time-consuming, and that it’s difficult to coordinate the many details across their users, customers and service providers.

In this post, we’ll explore the newly introduced SES feature called Deterministic Easy DKIM (aka “DEED”), which solves critical challenges in multi-region email identity management for SES customers. DEED generates consistent DKIM tokens across multiple AWS Regions based on a Parent (domain) Identity that is configured with SES Easy DKIM. DEED uses the Parent Identity’s Easy DKIM configuration to automatically provide the same DKIM signing configuration for a Replica (domain) Identity in a AWS Replica Region. With DEED, you only need to publish DNS records once for the Parent Identity. The Replica Identity will automatically use these same DNS records to verify domain ownership and manage DKIM signing. DEED streamlines multi-region email operations by simplifying DNS management and ensuring consistent DKIM signing across AWS regions, maintaining best-practice email authentication while reducing operational complexity.

Background and key challenges

SES introduced Easy DKIM over a decade ago as an innovative, streamlined solution to help customers create, verify, and manage domain identities with automated DKIM signing. Easy DKIM is simple to set-up, and works by generating pre-determined tokens that customers add to their DNS configurations. Once Easy DKIM is enabled, SES generates a public/private signing key for each domain identity and updates the verified identity’s CNAME public key. SES’ Easy DKIM simplifies email authentication by managing and rotating DKIM keys on behalf of customers.

Before DEED, customers who wanted to expand their SES email infrastructure across multiple regions faced complexity, operational challenges and a substantial administrative burden configuring and maintaining DKIM across AWS regions. This made it very difficult for customers to scale their email infrastructure efficiently, and often dissuaded customers from fully leveraging the potential of a multi-region email sending strategy. These challenges are more pronounced for Independent Software Vendors (ISVs) and email service providers using SES. Frequently these organizations don’t own or control the domains used by their end-customers, who must manually update their DNS entries each time the ISV expanded or shifted their SES sending infrastructure between AWS regions.

Solution Overview

Deterministic Easy DKIM (DEED) allows SES customers to set up email identities across multiple regions, leveraging an existing domain identity configuration in a single region without the need to make companion DNS changes in all other regions. This innovative approach eliminates the manual overhead of creating region-specific DNS entries, and provides a streamlined solution to configure and maintain global email infrastructure for organizations and ISVs alike.

The key benefits of using DEED include:

  • Simplified DNS Management – Publish DNS records once for the Parent Identity and the replica identity is automatically synced.
  • Easier Multi-Region Operations – Simplifies the process of expanding email sending operations to new AWS regions.
    Reduced Administrative Overhead – Manage DKIM configurations centrally from the Parent Identity.

Terminology:

To understand DEED, let’s explore the key terminology that underpins this innovative approach:

  • Deterministic – a process or system where the same input will always produce the same output, with no randomness or unpredictability involved. In other words, if all the starting conditions are known, the outcome can be precisely predicted or determined.
  • Parent Region – The original AWS Region where the primary email identity is initially established.
  • Parent Identity – A verified email identity configured with Easy DKIM that serves as the authoritative source for DKIM configuration across regions.
  • Replica Region – An AWS Region where an identical email identity is replicated without additional configuration.
  • Replica Identity – An identity that shares identical DNS configuration and DKIM signing configuration of a parent identity.
  • DEED Identity – Any identity that is used as either a parent identity or a replica identity. (When a new identity is created, it is initially treated as a regular (non-DEED) identity. However, once a replica is created, the identity is then considered a DEED identity.)

How DEED works

DEED flow

DEED is built on the existing Easy DKIM framework:

  1. Using Easy DKIM, SES generates a public-private key pair and automatically adds a DKIM signature to every message sent from a SES verified identity in the Parent Region.
  2. To accommodate authenticated SES sending from multiple AWS regions, DEED automatically synchronizes the signing keys from the Parent Region and Parent Identity to the Replica Region and Replica Identity.
  3. This automated process ensures that both the Parent and Replica Identities receive and use identical keys for DKIM signing, maintaining consistent authentication across different AWS regions.
  4. SES manages the complex process of key rotation across Parent Identity and Replica Identities, further simplifying email infrastructure management across multiple AWS regions.
  5. Replica Identities inherit the DKIM signing configuration of the parent identity. Because of this dependency, you cannot delete a Parent Identity until all Replica Identities are deleted.
  6. The receiving email server/service validates DKIM from DNS.

We recommend customers take advantage of DEED for single-Region sending, as this new capability is included, at no additional cost, in the base SES pricing.

Steps to setup SES DEED Replica in a second AWS region

These steps assume that you are already using SES in the Parent Region, and have a fully verified domain identity that is configured to use Easy DKIM. You can also use the AWS CLI.

Step 1 – Update the Parent Identity

  1. Login to the AWS SES Console in the Parent Region
  2. Click on the Identities link in the SES navigation panel (far left) and click on the verified identity you want to use in other AWS regions.
    1. The Parent Identity must have Easy DKIM enabled.
    2. You cannot create Replicas of Identities that use BYODKIM or self-signed identities.
  3. Click on the Authorization tab
  4. Click Create Policy and select Create custom policy from the drop-down menu.
  5. Name the policy (for example, DEED-example_com)
  6. Modify the IAM policy (below) with your AWS account ID and AWS region, and paste it into the IAM Policy document to grant permission on the Parent Identity to allow the desired Replica Region to replicate the Parent Identity’s DKIM Signing Attributes:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowDKIMReplication",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::YOUR_ACCOUNT_ID:root"
},
"Action": "ses:ReplicateEmailIdentityDKIMSigningKey",
"Resource": "arn:aws:ses:us-east-1:123456789124:identity/example.com",
"Condition": {
"ForAllValues:StringEquals": {
"ses:ReplicaRegion": ["us-west-2", "eu-west-1"]
}
}
}
]
}

Note – Use consistent IAM policies that all allow for DKIM replication across intended Replica Regions.

Step -2 – Setup DEED Replica Identity

  1. Login to the AWS SES Console in the Replica Region
  2. Click on the Identities link in the SES navigation panel (far left) and click Create identity
  3. Under Identity details, Identity type, click Domain
  4. Type the domain name exactly as it appears in the Parent Identity
  5. Under Verifying your domain, click Deterministic Easy DKIM
  6. Select the Parent Region in the Parent region drop-down
  7. Ensure DKIM Signature is Enabled
  8. Click Create identity

create DEED ID

  1. The Replica Identity in the Replica Region will now automatically synchronize DKIM with the Parent Identity in the Parent Region and your apps/services can be configured to use either Parent or Replica Region & Identity to send DKIM authenticated email.
  2. Repeat the process if you want to create additional Replica Regions.

You can verify that the replica identity was configured correctly with the parent identity’s DKIM signing configuration by using the get-email-identity command and specifying the Replica’s domain name and region:

`aws sesv2 get-email-identity --email-identity [example.com](http://example.com/) —region us-west-2`

The response will include the value of the Parent Region in the DomainSigningAttributesOrigin parameter, signifying that the Replica Identity has been successfully configured with the Parent Identity’s DKIM signing configuration:

{
  "DkimAttributes": {
    "SigningAttributesOrigin": "AWS_SES_US_EAST_1"
  }
}

Conclusion

Deterministic Easy DKIM (DEED) represents a significant leap forward in multi-region email management for Amazon SES users. By eliminating the need for manually configured region-specific DNS configurations, DEED streamlines the process of expanding email operations across multiple AWS regions. This innovation not only reduces administrative overhead but also opens up new possibilities for organizations and ISVs to implement robust, globally distributed email infrastructures. With DEED, businesses can now leverage the full potential of Amazon SES across regions, ensuring consistent authentication, improved disaster recovery, and optimal performance without the previous complexities of multi-region setup.

Call to Action

Are you ready to simplify your multi-region email strategy with Amazon SES and DEED? Take the next step in optimizing your email infrastructure:

  1. Explore the DEED feature in your Amazon SES console today.
  2. Set up a test environment to experience the seamless multi-region configuration firsthand.
  3. For more detailed information, check out our comprehensive documentation on implementing DEED.
  4. Have questions or need assistance? Reach out to our AWS support team or join the AWS community forums to connect with other users.

Don’t let regional boundaries limit your email capabilities. Embrace the power of Deterministic Easy DKIM and transform your global email strategy with Amazon SES. Start your DEED journey today and unlock new levels of efficiency and scalability in your email operations.

Enhancing Message Reach: An Omnichannel Approach Using WhatsApp, SMS, and Email with AWS

Post Syndicated from Pavlos Ioannou Katidis original https://aws.amazon.com/blogs/messaging-and-targeting/enhancing-message-reach-an-omnichannel-approach-using-whatsapp-sms-and-email-with-aws/

SMS, WhatsApp, and email are essential channels for communication, each offering distinct advantages. SMS is known for its high deliverability and broad accessibility, making it a reliable choice for reaching users even without internet access. However, SMS can be costly, and delivery failures may lead to opportunity costs, such as potential lost sales if users cannot access their account or complete payment verification. WhatsApp provides a richer user experience at a lower cost but depends on internet connectivity and users having signed up for an account, which limits its reach. Email, while cost-effective and suitable for delivering detailed content, often faces issues including spam filtering and lower open rates.

By combining intelligent fallback mechanisms with broadcast capabilities, this versatile solution addresses a wide spectrum of communication needs, from time-sensitive transactional messages to expansive marketing campaigns. It sets the stage for diverse and impactful applications across various industries. The fallback mechanism dynamically switches to a backup channel if the primary one fails, ensuring that messages reach recipients. Meanwhile, the broadcast feature allows the simultaneous delivery of messages across multiple channels, maximizing reach and engagement. Together, these approaches extend message reach, improve user experience, and ensure reliable communication across platforms.

Use Cases

This multi-channel messaging solution is highly versatile, making it suitable for various use cases:

  • OTP/Transactional Messaging: Timely and reliable delivery is critical for one-time passwords (OTP) and transactional notifications. The solution ensures these important messages are promptly delivered, utilizing fallback mechanisms across SMS, WhatsApp, and email in case of channel limitations or failures.
  • Marketing Messages: This solution optimizes message delivery by selecting the most cost-effective and reliable channel while using the broadcast feature to deliver marketing messages across multiple channels, maximizing reach and engagement.
  • Broadcast Messages: The broadcast option can be especially useful in sending high-priority or time-sensitive messages to a wide audience. Messages can be sent across multiple channels at once, increasing the likelihood that recipients will see the message in their preferred communication method.

Technical Solution

The architecture employs several AWS services to ensure reliable, cost-efficient message delivery:

  • Amazon API Gateway: Accepts incoming API requests, including message details like content, recipient, fallback channels, and whether the message is broadcast or single-channel.
  • Amazon Simple Queue Service (SQS): Manages the queuing of messages for both primary and fallback channels, as well as for broadcast messages, ensuring smooth processing and delivery.
  • AWS Lambda: The Primary Message Handler Lambda function sends messages via the selected channel (SMS, WhatsApp, or email). If the primary delivery fails, the fallback process is handled by a secondary Lambda function. For broadcast messages, the message is sent to all specified channels simultaneously.
  • Amazon DynamoDB: Stores message data and delivery statuses, enabling tracking of messages through all channels, including broadcast messages.
  • Amazon Simple Notification Service (SNS): Publishes delivery status updates and triggers the next steps based on whether the message was delivered or failed.
  • Amazon SES & End User Messaging Service: Email messages are sent via Amazon SES, while AWS End User Messaging handles SMS and WhatsApp delivery.

For a deeper technical breakdown of each component and the deployment steps, refer to the Github repository.

Architecture Diagram

omnichannel-fallback-solution

Sending an SMS, WhatsApp, or Email

When sending a message, the system first attempts delivery via the primary channel specified in the API request, whether that’s SMS, WhatsApp, or email. The message details are processed by the API Gateway, which triggers the Primary Message Handler Lambda function:

  • SMS: The message is sent via the End User Messaging Service. A delivery report is awaited to confirm whether the message was successfully delivered.
  • WhatsApp: Similarly, the message is delivered via End User Messaging. Real-time read receipts and delivery statuses are tracked and updated in DynamoDB.
  • Email: Email messages are sent through Amazon Simple Email Service (SES). The system tracks delivery confirmation events (like success or bounce) and updates the status in DynamoDB.
    In all cases, if delivery fails or confirmation isn’t received within a specified period, the system triggers the fallback mechanism specified in the API request. This mechanism attempts to deliver the message through the secondary channel, ensuring message delivery via an alternate method.

For broadcast messages, the system simultaneously delivers the message across all specified channels, ensuring the widest possible reach. For example, an emergency notification might be broadcast via SMS, WhatsApp, and email, allowing recipients to receive the message through whichever channel they prefer.

Monitoring Delivery Status

The system tracks delivery statuses for all three channels:

  • SMS: Delivery confirmations are provided by the SMS provider and published to an SNS topic. This triggers a Lambda function, which updates the message status in DynamoDB.
  • WhatsApp: Real-time delivery and read receipts from WhatsApp are received and processed similarly to SMS. SNS triggers a Lambda function to update DynamoDB.
  • Email: Delivery status for emails is tracked via events from Amazon SES (such as success, bounce, or rejection). These events are published to SNS, which triggers an update of the delivery status in DynamoDB.

The monitoring configured by this solution is solely for the fallback mechanism. It’s recommended to configure separately monitoring that you might require for storing and analyzing message engagement events.

Fallback to Secondary Channel

If the initial message delivery fails, the fallback mechanism is initiated, based on the fallback channel specified in the API request. The Primary Message Handler Lambda function places a request in the Secondary Channel Fallback Queue via SQS. The Secondary Message Handler Lambda function retrieves this request and sends the message through the secondary channel (for example, sending the message via WhatsApp if SMS delivery fails).

Deployment

Visit this GitHub repository for a detailed ReadMe and deployment guide of this solution.

Conclusion

This multi-channel messaging solution ensures reliable and efficient communication across SMS, WhatsApp, and email. By leveraging AWS services, the system guarantees message delivery through fallback mechanisms and a broadcast option. Whether sending one-time passwords, marketing communications, or emergency broadcasts, this solution adapts to different needs while optimizing costs and improving reach across multiple channels.

Build a Two-Way Email-to-SMS Service with Amazon SES and Amazon End User Messaging

Post Syndicated from Cheng Wang original https://aws.amazon.com/blogs/messaging-and-targeting/build-a-two-way-email-to-sms-service-with-amazon-ses-and-amazon-end-user-messaging/

Introduction

Businesses and organizations today struggle to effectively communicate with their customers, employees, or other stakeholders across the diverse range of digital channels they now use. One common problem arises when the requirement to exchange information quickly and reliably extends beyond traditional email. This issue challenges organizations where recipients lack immediate access to email. This applies to field workers, remote teams, or customers who prefer to communicate via text messages. It is vital to bridge this gap between email and SMS communication for timely updates, urgent notifications, and seamless collaboration. However, separate management of these disparate channels independently proves cumbersome and leads to inefficiencies.

To address this challenge, one approach is to leverage Amazon Simple Email Service (SES) and Amazon End User Messaging services to create a robust, scalable, and cost-effective messaging system. This system seamlessly bridges the gap between email and SMS, enhances the reach and delivery of your messages and streamlines your communication workflows. Ultimately, this approach delivers a superior experience for your audience, ensuring that critical information reaches recipients through their preferred channels in a timely and efficient manner.

This blog post will delve into the step-by-step process of building a solution that enables both Email-to-SMS and SMS-to-Email communication. This solution allows you to send SMS messages using email and receive any SMS replies on the same email address you used to send the original message. Furthermore, you can continue the conversation by replying to the email you receive in response. By the end, you’ll have the knowledge and tools necessary to revolutionize your communication strategy and deliver a superior experience to your audience.

Here are some of the use cases for this solution:

  • Real estate agents can use this solution to send listing updates to clients via SMS, and then receive client inquiries and responses as emails.
  • Customer service team can leverage the Email to SMS functionality to proactively reach out to customers with important notifications. Customers are able to respond directly via SMS.
  • Retailers can use this solution to send order confirmations, shipping updates, and promotional offers to customers via SMS. Customers are able to respond with inquiries or feedback that are then received as emails.
  • Medical practices and hospitals can leverage the Email to SMS functionality to quickly notify patients of appointment reminders, prescription refills, or other time-sensitive information. Patients can then respond via SMS, which gets converted to an email that the healthcare staff can access.

Solution Overview

The following figure shows the high level architecture for this solution.

Figure 1: Two-Way Email-To-SMS architecture

  1. Email Users send an email to the email address formatted as “mobile-number@verified-domain”. Amazon SES email receiving receives the email and triggers a receipt rule.
  2. The email is published to Amazon Simple Notification Service (SNS) topic (EmailToSMS) based on the receipt rule action, which triggers an AWS Lambda function (ConvertEmailToSMS). The ConvertEmailToSMS Lambda function performs the following actions:
    1. Receives the event from SNS and constructs a text message using the email body content.
    2. The constructed message is then sent to the “mobile-number” in the destination email address using the “SendTextMessage” API from AWS End User Messaging SMS. This is achieved by using a phone number in AWS End User Messaging SMS as the origination identity.
    3. The SMS message ID and source email address are stored as items in the Amazon DynamoDB table (MessageTrackingTable). This tracks email addresses for replies from SMS users.
  3. Mobile Users receive the SMS, and they have the option to reply to the phone number with two-way SMS messaging
  4. AWS End User Messaging receives the incoming SMS message from the Mobile Users. It then publishes this message to a SNS topic (SMSToEmail) for two-way SMS integration, which triggers a Lambda function (ConvertSMSToEmail). The ConvertSMSToEmail Lambda function performs the following actions:
    1. Retrieves the item from “MessageTrackingTable” using “previousPublishedMessageId” (SMS message ID) from the SNS event, and locate the corresponding email address.
    2. Sends the SMS message body to the Email Users using SES. This step uses “mobile-number@verified-domain” as the source email address, and the email address retrieved from the previous step as the destination.
  5. Email Users receive the email, and they have the option to reply to the email to continue the conversation. Mobile Users will receive the latest reply from Email Users.

Walkthrough

This section will dive into the step-by step process for the deployment. There are 4 steps to deploy this solution:

  1. Configure SES verified identity for email receiving and sending.
  2. Deploy the CloudFormation stack for the Email to SMS functionality.
  3. Deploy the CloudFormation stack for the SMS to Email functionality.
  4. Set up two-way SMS messaging in AWS End User Messaging SMS.

Note: the Lambda code for this solution is developed based on phone numbers and long code as the supported origination identity in Australia. You need to adjust the Lambda code (“format_phone_number” function) accordingly for this to work in your country.

Refer to this GitHub repository for the solution source code.

Prerequisites

Prerequisites for this walkthrough:

  1. Administrator-level access to an AWS account
  2. A domain or subdomain that you own to create SES verified identity
  3. An origination identity that supports two-way messaging, following Choosing an origination identity for two-way messaging use cases. Simulator phone numbers are available if you are in the US
  4. A mobile phone to send and receive SMS
  5. An email address to send and receive emails

Step 1: Configure SES Verified Identity

Follow the steps outlined in Creating a domain identity to create a verified identity for your domain in your AWS account. Confirm your domain identity is in the “Verified” status before proceeding to the next step:

Figure 2: Verified Identity

Step 2: Deploy Email to SMS functionality

The following steps create a CloudFormation stack to deploy the required components for Email to SMS functionality:

  1. Sign in to your AWS account.
  2. Download the email-to-sms.yaml for creating the stack.
  3. Navigate to the AWS CloudFormation page.
  4. Choose Create stack, and then choose With new resources (standard).
  5. Choose Upload a template file and upload the CloudFormation template that you downloaded earlier: email-to-sms.yaml. Then choose Next.
  6. Enter the stack name Email-To-SMS.
  7. Enter the following values for the parameters:
    • RuleName: The name of your SES Rule Set and receipt rule.
    • Recipient1: Domain name used for recipient condition in the SES Rule Set.
    • Recipient2: Domain name used for recipient condition in the SES Rule Set if you need additional recipients.
    • PhoneNumberId: Phone number ID of the phone number to send SMS messages.
  8. Choose Next, and then optionally enter tags and choose Submit. Wait for the stack creation to finish.

Now you have the required components to convert email to text, and sending it as SMS to a phone number using AWS End User Messaging SMS.

Note: if required, modify the following code in email-to-sms.yaml to format your phone numbers accordingly:

def format_phone_number(email_address):

    # Extract the local part of the email address (before @)

    local_part = email_address.split('@')[0]   

    # Remove the leading '0' and add '+61' for phone number (Australia)

    if local_part.startswith('0'):

        formatted_number = '+61' + local_part[1:]

    return formatted_number

Step 3: Deploy SMS to Email functionality

The following steps create a CloudFormation stack to deploy the required components for SMS to Email functionality:

  1. Sign in to your AWS account.
  2. Download the sms-to-email.yaml for creating the stack.
  3. Navigate to the AWS CloudFormation page.
  4. Choose Create stack, and then choose With new resources (standard).
  5. Choose Upload a template file and upload the CloudFormation template that you downloaded earlier: sms-to-email.yaml. Then choose Next.
  6. Enter the stack name SMS-To-Email.
  7. Enter the following values for the parameters:
    • EmailDomain: The email domain to use for the SMS-to-Email function
  8. Choose Next, and then optionally enter tags and choose Submit. Wait for the stack creation to finish.

Note: if required, modify the following code in sms-to-email.yaml to format your phone numbers accordingly:

def format_phone_number(phone_number):

    # Replace the '+61' with '0' from the phone number (Australia)

    formatted_number = f"0{mobile_number[3:]}"

    return formatted_number

Step 4: Set up Two-Way Messaging in AWS End User Messaging SMS

Follow the steps 1 – 5 outlined in Set up two-way SMS messaging for a phone number in AWS End User Messaging SMS.

For step 6:

  • For Destination type, choose Amazon SNS.
  • Choose Existing SNS standard topic.
  • For Incoming messages destination, choose the SNS topic created from Step 3 (default topic name is SMSToEmailTopic).
  • For Two-way channel role, choose Use SNS topic policies.
  • Choose Save changes.

This allows your origination identity (phone number) to receive incoming messages, which is then published to an SNS topic and converted into emails by Lambda.

Testing

To test the solution, send an email with the destination address of “mobile-number@verified-domain”. You should receive a SMS on your mobile with the following information:

Note: AWS End User Messaging SMS has character limit for SMS messages depending on the type of characters the message contains. This solution takes the first 160 characters of the email body by default, you can adjust the EmailToSMS Lambda function as required.

Reply directly to the SMS, you should receive an email at the same email address that sent the original email, with the following information:

  • Subject: Re:mobile-number
  • Body: SMS message content
  • Source email address: mobile-number@verified-domain

If you are not receiving the email or SMS, check the Lambda CloudWatch logs for troubleshooting.

Cleaning up

To remove unneeded resources after testing the solution, follow these steps:

  1. In the CloudFormation console, delete the Email-To-SMS stack
  2. In the CloudFormation console, delete the SMS-To-Email stack
  3. If applicable, in Amazon SES, delete the verified identities
  4. If applicable, in AWS End User Messaging, release the unused phone numbers

Additional Consideration

Conclusion

This blog has explored how organizations can leverage AWS services to build a flexible, two-way communication solution bridging the gap between email and SMS channels. By integrating Amazon SES and Amazon End User Messaging, businesses can reach their audience through multiple channels, ensuring timely and effective delivery of critical messages.

The detailed guide provided the knowledge to create a scalable, cost-effective system tailored to evolving communication needs – whether facilitating email-to-SMS or SMS-to-email exchanges. This unified approach to email and SMS capabilities empowers companies to address the common challenge of managing disparate communication platforms, streamlining workflows and enhancing responsiveness.

If you run into issues or want to submit a feature request, use the New issue button under the issues tab in the GitHub repository.

How Amazon SES Mail Manager customers can prevent EchoSpoofing

Post Syndicated from Zip Zieper original https://aws.amazon.com/blogs/messaging-and-targeting/how-ses-mail-manager-customers-can-prevent-echospoofing/

Customers not using Amazon SES Mail Manager, or those leveraging the authenticated SMTP functionality, are not at risk of EchoSpoofing. In such cases, no further action is required.

However, customers currently using or evaluating the unauthenticated SMTP relay feature of Amazon SES Mail Manager are strongly advised to review and implement the guidance provided in this blog post.

A new type of email spoofing attack

In July 2024, the researchers at Guardio Labs disclosed a new type of email spoofing (authentication bypass) attack they called “EchoSpoofing”. The attackers successfully sent spoofed emails by redirecting them through a virtual SMTP server, Office365 Exchange Online server, and a trusted third-party SMTP relay service. This path provided the fraudulent messages a means to pass standard authentication checks. Fortunately, the Guardio Labs researchers responsibly disclosed the issue to the targeted email security provider, leading to a speedy and effective remedy.

Unfortunately, before addressing the vulnerability completely, cybercriminals executed a series of sophisticated phishing campaigns. These campaigns involved sending millions of fraudulent emails that had valid Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM) from well-known consumer brands.

The EchoSpoofing incident reminds both providers and customers to adopt a “trust-but-verify” approach to email security. This is especially true when mail routing functions have been, or are, in the process of being decoupled from single-tenant, on-premise (or cloud) email infrastructures.

As a leading provider of global managed email infrastructure, the Amazon SES service team went to work immediately after the Guardio Labs announcement. The Amazon SES team scrutinized the announcement and remediations undertaken by both Microsoft and the third-party SMTP relay service to fully understand the EchoSpoofing exploits and devised methods to swiftly safeguard Amazon SES customers.

Although we won’t delve into Guardio Labs’ in-depth analysis, it is crucial to grasp the attack’s main elements and examine how malicious individuals could exploit supposedly “secure” email relay systems. Amazon SES has taken steps to safeguard against EchoSpoofing and similar attacks, urging its customers to do the same.

Understanding the EchoSpoofing attack

The bad actors who implemented EchoSpoofing were able to send millions of well camouflaged malicious messages through the trusted delivery path of targeted organizations by preserving the SPF and DKIM attributes of the targeted sender’s domain. This greatly increased the likelihood of recipients trusting and acting upon the fake messages.

The attacker first set up a tenant in Microsoft Office365 and then delivered a spoofed email to that tenant, falsifying the from email headers. The attacker-controlled tenant was configured to relay the email to a security relay point linked to the forged sending identity. As the forged email came from trusted IP addresses belonging to Microsoft, the security relay point signed the message and relayed to the final recipients.

The attackers had amassed a large inventory of high-profile domains, and spread the EchoSpoof campaigns out across them to smooth the traffic and avoid sending spikes from any single domain. They carried out this attack for several months undetected, sending as many as 14 million messages per day, targeting the users of the compromised domains’ email services. This made the attack easy to automate at scale, difficult to detect via automated means, and highly successful in delivering malicious emails to unsuspecting recipients.

Guardio Labs’ discovery highlights the risks associated with an insecure SMTP relay model in a trusted domain configuration. This is of particular concern when permissive security policies allow fraudulent emails to be injected into the email flow without raising alarms.

The AWS shared security model for SES

As an email sender, Amazon SES is one of the largest on the internet, operating a worldwide fleet of trusted mail relays. Amazon SES maintains high IP reputations for this large fleet by maintaining a tight focus on robust, evolving security practices.

At AWS we operate under the shared security model. For Amazon SES, this means AWS takes responsibility for securing the underlying email delivery infrastructure, including the email servers, networks, and physical data centers. Customers take responsibility for securing their configurations, email content, sender authentication, and email lists they use with Amazon SES.

To ensure we meet our obligations in the shared security model, Amazon SES has recently added new features to Mail Manager SMTP relays that provide an increased level of protection to help guard against exploits like EchoSpoofing. Theses features are live in every AWS Region where Mail Manager is accessible.

We have outlined our recommendations and updated Amazon SES Mail Manager configurations in this blog to help customers meet their obligations and strengthen their Amazon SES email infrastructure against EchoSpoofing. As noted above, authenticated SMTP relays are not subject to this exploit.

Prevent EchoSpoofing when relaying email out of MailManager

If you need to relay email to a third party system that cannot enforce SMTP authentication, our recommendation is to limit access to the IP addresses used by Mail Manager in your region.

As of this writing, Mail Manager is generally available, and has its own IP range, in six commercial regions (below). As Amazon expands SES Mail Manager availability into more regions the IP ranges will be updated in the Amazon SES documentation .

SES Mail Manager regions

SES Mail Manager IP ranges as of 10/23/2024

When unable to enforce SMTP authentication, we recommend configuring your SMTP servers, or third party software for the new MIME header"X-MAIL-MANAGER-ORIGINATOR-ORG". This new Mail Manager header is now automatically inserted into messages relayed by Mail Manager. The X-MAIL-MANAGER-ORIGINATOR-ORG will be set to the customer’s unique SMTP relay ID, which can be found via the Mail Manager console or the ListSmtpRelays API.

In addition to added security, the MIME header feature can also be used in message search and filtering behaviors for a wide range of MIME header name:value pairs.

If the original email already contains an X-MAIL-MANAGER-ORIGINATOR-ORG header, it will be replaced with the last MailManager SMTP relay ID to relay the email. Here is an example of an email relayed by MailManager with the header:

MIME-Version: 1.0
From: [email protected]
To: [email protected]
Subject: Test
X-SES-REDIRECT-MESSAGE-ID: <[email protected]>
X-MAIL-MANAGER-ORIGINATOR-ORG: rl-usmoots8mgmfgfaeijckxhqx
X-SES-Outgoing: 2024.08.26-76.223.191.14

--===============1760803815732220490==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit

This is a sample message. Have a nice day.
--===============1760803815732220490==--

This approach elevates your security posture because the IP access control lists on your third party system ensures only mail from Amazon SES is accepted, and the MIME header check can be trusted and checked.

Prevent EchoSpoofing when relaying into MailManager

When relaying email from a third party into Amazon SES Mail Manager, you will similarly need to configure an IP allowlisting, and if the email comes from a shared or cloud environment, you will need an additional header check to disambiguate among the multiple tenants it hosts. Those IPs and headers are provider specific, for example, emails coming from Office365 will have a header called X-OriginatorOrg.

You can use the rule editor screen in MailManager to configure the check in Mail Manager for the IPs and 3rd party headers before executing any action.

Mail Manager Rules Detail Page

The verification of a MIME header is not necessary if the third party relaying into MailManager uses an IP dedicated for your tenant. In that case, there is no possibility that an attacker tenant injects email into MailManager using your IP.

Conclusion

While the conditions that made the EchoSpoofing exploit were highly specific, they reminded us all of the importance of taking a proactive approach to email security.

The chances of your Amazon SES Mail Manager unauthenticated SMTP relay being compromised are low, but we highly advise you follow the recommendations in this blog post promptly. You’ll find more information in the Amazon SES documentation ( here ).

If you need help with securing your Amazon SES Mail Manager SMTP relay actions against EchoSpoofing, contact AWS support, or leave us a comment in community section of the blog post.

Call to Action:

If you are using SES Mail Manager’s unauthenticated SMTP relay today, follow the guidance in this blog to secure your email infrastructure today by configuring the recommended ACLs and MIME header verification in AWS SES Mail Manager.“

Stay ahead of emerging threats by subscribing to this AWS blog where we post the latest security updates as well as new features and interesting use cases for SES.

Join the conversation and share your best practices for email security with the AWS community.

Explore the new MIME header evaluation feature in AWS SES Mail Manager and share your creative use cases with us and the SES community via the community comments section of the blog.

About the Authors

Toby Weir-Jones

Toby Weir-Jones

Toby is a Principal Product Manager for Amazon SES and WorkMail. He joined AWS in January 2021 and has significant experience in both business and consumer information security products and services. His focus on email solutions at SES is all about tackling a product that everyone uses and finding ways to bring innovation and improved performance to one of the most ubiquitous IT tools.

Zip

Zip

Zip is a Sr. Specialist Solutions Architect at AWS, working with Amazon Pinpoint and Simple Email Service and WorkMail. Outside of work he enjoys time with his family, cooking, mountain biking, boating, learning and beach plogging.

Leandro Batista Lameiro

Leandro Batista Lameiro

Leandro is a Sr. Software Dev Engineer at AWS.

Linzhou Zhong

Linzhou Zhong

Linzhou is a software engineer at AWS.

Automate AWS End User Messaging US toll-free Number Registrations

Post Syndicated from Bruno Giorgini original https://aws.amazon.com/blogs/messaging-and-targeting/automate-us-tfn-registrations/

Introduction:

AWS End User Messaging enables customers to send SMS messages to recipients located across the globe.
If you are planning on rolling out SMS across multiple countries, we recommend reviewing this blog. When you send SMS or MMS messages using AWS End User Messaging SMS, you must use a specific origination identity that supports the sending of SMS. In the US the originators that are supported are Toll-Free(TFN), 10DLC, and short codes and each of these has its own registration process.

This blog discusses the process of programmatically registering Toll-Free numbers (TFNs), which can be used exclusively within the United States. TFNs have a max throughput of 3 Messages Per Second(MPS) and are typically used for low-volume/throughput use cases. You can learn more about TFNs here.

TFNs have a relatively simple registration process, however, for Independent Software Vendors (ISVs), SaaS providers, or large organizations with many teams or use cases, completing dozens, or hundreds of TFN registrations manually can be time-consuming, tedious, and error-prone. AWS End User Messaging APIs can be used to streamline the TFN registration process by enabling AWS customers to programmatically register TFNs in the USA. You can learn more about the TFN registration requirements and process here.

In this post, we will discuss the AWS End User Messaging APIs required for programmatically registering TFNs. We’ll explore a simple Bash script you can use to automate the registration of a single TFN, as well as a Python script that you can use to bulk register multiple TFNs. Both of these scripts help simplify the TFN registration process and can save both time and effort for businesses looking to leverage AWS End User Messaging for their communication needs.

Outline of the APIs used in the scripts:

There are seven actions that need to be taken in order for a TFN registration to be created and submitted. The two scripts in this post automate these actions:

  1. create-registration
    • Creates a new registration
    • The “RegistrationType” field controls whether this is a registration for a Toll-Free, or 10DLC or a SenderID.
  2. describe-registration-field-definitions
    • Retrieves the specified RegistrationType field definitions.
    • You can use DescribeRegistrationFieldDefinitions to view the requirements for creating, filling out, and submitting each registration type. In this post we have chosen Toll-Free and provided the values.
  3. create-registration-attachment
    • The carriers require you to provide a mockup that closely resemble the opt-in experience that your customers will complete to ensure end-users have consented to receiving SMS messages from this toll-free number. This should be a screenshot of your website or mobile application’s opt-in workflow.
    • The maximum file size is 500kb, and valid file extensions are PDF, JPEG, or PNG.
    • You need to wait until the attachment is uploaded before moving to step 4.
  4. put-registration-field-value
    • This action needs to be repeated for all required fields (retrieved in step 2)
  5. request-phone-number
    • Request an origination phone number for use in your account (needed for step 6)
  6. create-registration-association
    • This associates the registration with the origination identity (phone number) you requested in step 5.
    • You will need the PhoneNumberId for the origination identity.
  7. submit-registration-version
    1. This will submit the specified registration for review and approval
    2. IMPORTANT:
      • Make sure that all of your data is correct, especially before attempting to submit multiple registrations using the second script option below.
      • Once your script has submitted the registration, it’s initial status will be “CREATED” and should change to “REVIEWING” within 24 hours.
      • Once submitted, you will be unable to edit or delete this registration until it is approved or rejected by the third-party registrar that controls the registration process for the type of registration you are submitting (in this blog we’re registering TFNs).
      • You begin incurring costs for the TFNs as soon as you successfully submit the registration.

No matter which script you decide to use, it’s important to keep a these things in mind:

  1. The registration information you provide will be reviewed by a third-party company. This is standard, no matter which provider you use for SMS
  2. While AWS submits the registration on your behalf, we do not participate in the actual review process, nor can we influence the third-party.
  3. The review process for TFNs can take up to 15 business days and if the information provided in the registration process is incomplete, inaccurate, or the use case falls into a forbidden category (Guidelines for using toll-free numbers) , the application can be rejected.
  4. If your toll-free number registration is rejected, the status of your registration will change to “Requires Updates“. We recommend monitoring your registration status frequently to ensure that you do not miss an update and extend the time it takes to complete.

Prerequisites for either approach:

Before running either script, you’ll need:

  1. An AWS account with permission to use/provision the AWS End User Messaging service (link) in the target region.

The following information for each TFN you wish to register/request (you can retrieve this list with describe-registration-field-definitions API):

Company Info
  • Company Name
  • Website
  • Address 1
  • Address 2 (optional)
  • City
  • State / Province
  • Zip Code / Postal Code
  • Country Code
Contact Info
  • First Name
  • Last Name
  • Support Email
  • Support Phone Number
Messaging Use Case
  • Monthly Message Volume (you must use one of the options below)
    • “10”
    • “100”
    • “1,000”
    • “10,000”
    • “100,000”
    • “250,000”
    • “500,000”
    • “750,000”
    • “1,000,000”
    • “5,000,000”
    • “10,000,000+”
  • Use Case Category (you must use one of the options below)
    • “Two-factor authentication”
    • “One-time passcodes”
    • “Notifications”
    • “Polling and surveys”
    • “Info on demand”
    • “Promotions & marketing”
    • “Other”
  • Use Case Details
  • Opt-In Description – The primary purpose of the Opt-in Description is to demonstrate that the end user explicitly consents to receive text messages and understands the nature of the program. Your application is being reviewed by a 3rd party reviewer, so make sure to provide clear and thorough information about how your end-users opt-in to your SMS service and any associated fees or charges. If the reviewer cannot determine how your opt-in process works then your application will be denied and returned. If your Opt-in process requires a log-in, is not yet published publicly, is a verbal opt-in, or if it occurs on printed sources such as fliers and paper forms then make sure to thoroughly document how this process is completed by the end-user receiving messages. Provide a screenshot and host the screenshot on a publicly accessible website (like OneDrive or Google Drive) and provide the URL in the text as well as in the Opt-In Image below.
  • Opt-In Image – This is optional, but highly recommended, even if you provide a description above:
    • If your experience is not publicly available, the carriers will require you to provide one to ensure end-users have consented to receiving SMS messages from this toll-free number.
    • You are encouraged to provide a screenshot (PDF, JPEG, or PNG) of your website or mobile application’s opt-in workflow. This could also be a screenshot of a paper form, a verbal opt-in script, or an as yet published website experience in development.
    • The maximum file size is 500Kb.
Message Samples
  • Message Sample 1
  • Message Sample 2 (optional)
  • Message Sample 3 (optional)

Option 1 – Execute a script to register a single TFN

The script requires the additional prerequisites (if you use AWS CloudShell to run the script, the AWS CLI and jq are already installed).

  • AWS CLI installed and configured with appropriate AWS account credentials and region (link)
  • jq (a lightweight and flexible command-line JSON processor) installed (link)
  1. Set environment variables for each of the required fields.

Before running the registration script, you must first set the necessary variables, replacing the placeholder values with your company actual data (these values are listed as a prerequisite above). Ensure that the opt-in screenshot image file optInImage.png is in the same directory as the script, or provide the correct path to the image file in the attachment_body variable. Note – the allowed file types for the opt-in screenshot are PNG, JPG, and PDF with a max file size of 500 kb.

Run the environment variable exports:

export AWS_REGION="us-west-2"
export COMPANY_NAME="Your Company Name"
export COMPANY_WEBSITE="www.yourcompany.com"
export COMPANY_ADDRESS1="123 Main Street"
export COMPANY_ADDRESS2="Suite 200"
export COMPANY_CITY="Your City"
export COMPANY_STATE="Your State"
export COMPANY_ZIPCODE="12345"
export COMPANY_COUNTRY_CODE="US"
export CONTACT_FIRSTNAME="John"
export CONTACT_LASTNAME="Doe"
export CONTACT_EMAIL="[email protected]"
export CONTACT_PHONE="+1234567890"
export USECASE_DETAILS="Details about your use case"
export OPTIN_DESCRIPTION="Description of opt-in process"
export MONTHLY_MESSAGE_VOLUME="10"
export USECASE_CATEGORY="Other"
export MESSAGE_SAMPLE1="Sample message"
export ATTACHMENT_BODY="fileb://optInImage.png"
  1. Copy and save the below script as register_us_toll_free_number.sh into your current directory. Adjust the region variable in the script to the one you’re using to register the US TFN.
#!/bin/bash
set -euo pipefail

# Check dependencies
command -v aws >/dev/null 2>&1 || { echo "AWS CLI is required but it's not installed. Aborting."; exit 1; }
command -v jq >/dev/null 2>&1 || { echo "jq is required but it's not installed. Aborting."; exit 1; }

# Initialize variables from environment or use default values
region="${AWS_REGION:-'us-west-2'}"
companyInfo_companyName="${COMPANY_NAME:-"Default Company Name"}"
companyInfo_website="${COMPANY_WEBSITE:-"https://defaultcompany.com"}"
companyInfo_address1="${COMPANY_ADDRESS1:-"123 Default St."}"
companyInfo_address2="${COMPANY_ADDRESS2:-""}"  # Optional
companyInfo_city="${COMPANY_CITY:-"Default City"}"
companyInfo_state="${COMPANY_STATE:-"Default State"}"
companyInfo_zipCode="${COMPANY_ZIPCODE:-"00000"}"
companyInfo_isoCountryCode="${COMPANY_COUNTRY_CODE:-"US"}"
contactInfo_firstName="${CONTACT_FIRSTNAME:-"John"}"
contactInfo_lastName="${CONTACT_LASTNAME:-"Doe"}"
contactInfo_supportEmail="${CONTACT_EMAIL:-"[email protected]"}"
contactInfo_supportPhoneNumber="${CONTACT_PHONE:-"+10000000000"}"
messagingUseCase_useCaseDetails="${USECASE_DETAILS:-"Default use case details"}"
messagingUseCase_optInDescription="${OPTIN_DESCRIPTION:-"Default opt-in process description"}"
messagingUseCase_monthlyMessageVolume="${MONTHLY_MESSAGE_VOLUME:-"10"}"
messagingUseCase_useCaseCategory="${USECASE_CATEGORY:-"Other"}"
messageSamples_messageSample1="${MESSAGE_SAMPLE1:-"This is a sample message."}"
attachment_body="${ATTACHMENT_BODY:-"fileb://optInImage.png"}"  # Path to the image, can be overridden

# Log to file
log_file="registration.log"
echo "Logging to $log_file"
exec > >(tee -a "$log_file") 2>&1

# Retry configuration
max_retries=10
retry_interval=2
retry_count=0
upload_complete=false

# Helper function to put registration field values
put_registration_field_value() {
    local field_path="$1"
    local value="$2"
    aws pinpoint-sms-voice-v2 --region "$region" put-registration-field-value \
         --registration-id "$registration_id" \
         --field-path "$field_path" \
         --text-value "$value" || { echo "Failed to update $field_path"; exit 1; }
}

# Helper function to validate registration field values
validate_input() {
    local required_vars=(
        "AWS_REGION"
        "COMPANY_NAME"
        "COMPANY_WEBSITE"
        "COMPANY_ADDRESS1"
        "COMPANY_CITY"
        "COMPANY_STATE"
        "COMPANY_ZIPCODE"
        "COMPANY_COUNTRY_CODE"
        "CONTACT_FIRSTNAME"
        "CONTACT_LASTNAME"
        "CONTACT_EMAIL"
        "CONTACT_PHONE"
        "USECASE_DETAILS"
        "OPTIN_DESCRIPTION"
        "MONTHLY_MESSAGE_VOLUME"
        "USECASE_CATEGORY"
        "MESSAGE_SAMPLE1"
        "ATTACHMENT_BODY"
    )

    for var in "${required_vars[@]}"; do
        if [ -z "${!var}" ]; then
            echo "Error: $var is missing or empty" >&2
            return 1
        fi
    done

    # Validate zip code
    if [[ ! "$COMPANY_ZIPCODE" =~ ^[0-9]+$ ]]; then
        echo "Error: Invalid or missing zip code" >&2
        return 1
    fi

    # Validate monthly message volume
    local valid_volumes=("10" "100" "1,000" "10,000" "100,000" "250,000" "500,000" "750,000" "1,000,000" "5,000,000" "10,000,000+")
    local volume_valid=false
    for volume in "${valid_volumes[@]}"; do
        if [ "$MONTHLY_MESSAGE_VOLUME" = "$volume" ]; then
            volume_valid=true
            break
        fi
    done
    if [ "$volume_valid" = false ]; then
        echo "Error: Invalid or missing monthly_message_volume" >&2
        return 1
    fi

    # Validate use case category
    local valid_categories=("Two-factor authentication" "One-time passcodes" "Notifications" "Polling and surveys" "Info on demand" "Promotions & marketing" "Other")
    local category_valid=false
    for category in "${valid_categories[@]}"; do
        if [ "$USECASE_CATEGORY" = "$category" ]; then
            category_valid=true
            break
        fi
    done
    if [ "$category_valid" = false ]; then
        echo "Error: Invalid or missing use_case_category" >&2
        return 1
    fi

    # Additional validations (you can add more as needed)
    if [[ ! "$COMPANY_WEBSITE" =~ ^https?:// ]]; then
        echo "Error: COMPANY_WEBSITE must start with http:// or https://" >&2
        return 1
    fi
    
    if [[ ! "$CONTACT_EMAIL" =~ ^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$ ]]; then
        echo "Error: CONTACT_EMAIL is not a valid email address" >&2
        return 1
    fi
    
    if [[ ! "$CONTACT_PHONE" =~ ^\+[1-9][0-9]{10,14}$ ]]; then
        echo "Error: CONTACT_PHONE must be in E.164 format (e.g., +10000000000)" >&2
        return 1
    fi
    
    echo "All required variables are set and valid"
    return 0
}


# Step 1: Create registration
echo "Creating registration..."
create_registration_output=$(aws pinpoint-sms-voice-v2 --region "$region" create-registration \
      --registration-type 'US_TOLL_FREE_REGISTRATION' \
      --tags Key=Name,Value='Registration friendly name') || { echo "Failed to create registration"; exit 1; }

# Extract RegistrationId
registration_id=$(echo "$create_registration_output" | jq -r '.RegistrationId')
if [ -z "$registration_id" ]; then
    echo "Error: Registration ID not found!"
    exit 1
fi
echo "Registration ID: $registration_id"

# Step 2: Create registration attachment
echo "Creating registration attachment..."
create_attachment_output=$(aws pinpoint-sms-voice-v2 --region "$region" create-registration-attachment \
      --attachment-body "$attachment_body") || { echo "Failed to create registration attachment"; exit 1; }

# Extract RegistrationAttachmentId
attachment_id=$(echo "$create_attachment_output" | jq -r '.RegistrationAttachmentId')
if [ -z "$attachment_id" ]; then
    echo "Error: Attachment ID not found!"
    exit 1
fi
echo "Attachment ID: $attachment_id"

# Step 3: Wait for the attachment to be fully uploaded
echo "Waiting for attachment upload to complete..."
while [ "$upload_complete" = false ] && [ $retry_count -lt $max_retries ]; do
    # Describe the registration attachment
    describe_attachment_output=$(aws pinpoint-sms-voice-v2 --region "$region" describe-registration-attachments \
        --registration-attachment-ids "$attachment_id")
    
    # Extract the attachment status
    attachment_status=$(echo "$describe_attachment_output" | jq -r '.RegistrationAttachments[0].AttachmentStatus')

    # Check if the status is 'UPLOAD_COMPLETE'
    if [ "$attachment_status" == "UPLOAD_COMPLETE" ]; then
        echo "Attachment upload complete."
        upload_complete=true
    else
        echo "Attachment status: $attachment_status. Retrying in $retry_interval seconds... (attempt $((retry_count + 1))/$max_retries)"
        sleep "$retry_interval"
        retry_count=$((retry_count + 1))
    fi
done
if [ "$upload_complete" = false ]; then
    echo "Attachment upload did not complete within the retry limit."
    exit 1
fi
echo "Attachment upload complete."

# Step 4a: Put registration field values from environment variables
text_field_paths=(
    "companyInfo_companyName"
    "companyInfo_website"
    "companyInfo_address1"
    "companyInfo_address2"
    "companyInfo_city"
    "companyInfo_state"
    "companyInfo_zipCode"
    "companyInfo_isoCountryCode"
    "contactInfo_firstName"
    "contactInfo_lastName"
    "contactInfo_supportEmail"
    "contactInfo_supportPhoneNumber"
    "messagingUseCase_useCaseDetails"
    "messagingUseCase_optInDescription"
    "messageSamples_messageSample1"
)

for text_field_path in "${text_field_paths[@]}"; do
    echo "Putting registration field value for: $text_field_path"
    value=$(eval "echo \$$text_field_path")
    if [ -z "$value" ]; then
        echo "Error: $text_field_path is empty or not set. Skipping."
        continue
    fi
    echo "Value for $text_field_path is: '$value'"
    put_registration_field_value "${text_field_path//_/.}" "$value"
done

# Step 4b: Put other registration field values (Choice fields)
choice_field_paths=(
    "messagingUseCase_monthlyMessageVolume"
    "messagingUseCase_useCaseCategory"
)

for choice_field_path in "${choice_field_paths[@]}"; do
    echo "Putting registration field value for: $choice_field_path"
    value=$(eval "echo \$$choice_field_path")
    if [ -z "$value" ]; then
        echo "Error: $choice_field_path is empty or not set. Skipping."
        continue
    fi
    echo "Value for $choice_field_path is: '$value'"
    aws pinpoint-sms-voice-v2 --region "$region" put-registration-field-value \
        --registration-id "$registration_id" \
        --field-path "${choice_field_path//_/.}" \
        --select-choices "$value" || { echo "Failed to update $choice_field_path"; exit 1; }
done

# Step 5: Associate registration attachment with registration
echo "Associating registration attachment with registration..."
aws pinpoint-sms-voice-v2 --region "$region" put-registration-field-value \
      --registration-id "$registration_id" \
      --field-path 'messagingUseCase.optInImage' \
      --registration-attachment-id "$attachment_id" || { echo "Failed to associate registration attachment"; exit 1; }

# Step 6: Request phone number
echo "Requesting phone number..."
request_phone_number_output=$(aws pinpoint-sms-voice-v2 --region "$region" request-phone-number \
      --iso-country-code 'US' \
      --number-type 'TOLL_FREE' \
      --number-capabilities 'SMS' 'MMS' 'VOICE' \
      --message-type 'TRANSACTIONAL') || { echo "Failed to request phone number"; exit 1; }

# Extract PhoneNumberId
phone_number_id=$(echo "$request_phone_number_output" | jq -r '.PhoneNumberId')
if [ -z "$phone_number_id" ]; then
    echo "Error: Phone number ID not found!"
    exit 1
fi
echo "Phone Number ID: $phone_number_id"

# Step 7: Associate phone number with registration
echo "Associating phone number with registration..."
aws pinpoint-sms-voice-v2 --region "$region" create-registration-association \
      --registration-id "$registration_id" \
      --resource-id "$phone_number_id" || { echo "Failed to associate phone number with registration"; exit 1; }

# Step 8: Submit registration
echo "Submitting registration..."
aws pinpoint-sms-voice-v2 --region "$region" submit-registration-version \
     --registration-id "$registration_id" || { echo "Failed to submit registration"; exit 1; }

echo "Registration submitted successfully!"
  1. Run chmod command to make it executable. Run the script with the environment variable values you provided in Step 1 above.

chmod u+x register_us_toll_free_number.sh
./register_us_toll_free_number.sh
  1. The script completes with the following lines:

Submitting registration...
{
    "RegistrationArn": "arn:aws:sms-voice:us-west-2:637400000000:registration/registration-244aa3f91fcc49b598bdf1234569",
    "RegistrationId": "registration-244aa3f91fcc49b598bdf1234569",
    "VersionNumber": 1,
    "RegistrationVersionStatus": "SUBMITTED",
    "RegistrationVersionStatusHistory": {
        "DraftTimestamp": "2024-09-11T19:44:41+00:00",
        "SubmittedTimestamp": "2024-09-11T19:45:16+00:00"
    }
}
Registration submitted successfully!

Option 2 – Execute a Python script to register more than one Toll-Free number

For ISVs, SaaS providers, or large organizations registering multiple phone numbers on behalf of customers or business units, automating the process can save time and reduce errors. We’ve chosen to use Python because it’s CSV library handles file parsing complexities, and its boto3 library enables seamless interaction with AWS services, making it well-suited for programmatically managing these registrations.

  1. Below is a sample CSV file with the variables for 5 registrations, copy it and replace the sample data with your actual data (these data were listed in the prerequisites above). Save your file as data.csv:
companyInfo_companyName,companyInfo_website,companyInfo_address1,companyInfo_address2,companyInfo_city,companyInfo_state,companyInfo_zipCode,companyInfo_isoCountryCode,contactInfo_firstName,contactInfo_lastName,contactInfo_supportEmail,contactInfo_supportPhoneNumber,messagingUseCase_monthlyMessageVolume,messagingUseCase_useCaseCategory,messagingUseCase_useCaseDetails,messagingUseCase_optInDescription,messageSamples_messageSample1,attachmentFilePath
AnyCompany,https://example.com/example1,123 Any Street,Suite 200,Anytown,WA,98109,US,FirstName,LastName,[email protected],15553331234,10,One-time passcodes,Dev/Demo - Internal testing only,Internal testing only,Your AWS End User Messaging internal testing one time passcode is 123456,./companyA.png
Example Corp,https://example.net/example2,123 Main Street,,Anywhere,WA,98765,US,FirstName2,LastName2,[email protected],15553332222,10,Notifications,Dev/Demo - Internal testing only,Internal testing only,Your AWS End User Messaging internal testing one time passcode is 654321,./companyB.png
AnyDepartment,https://example.org/example3,123 Oak Avenue,Apt B,Nowhere,WA,54321,US,FirstName3,LastName3,[email protected],15553333333,10,Two-factor authentication,Dev/Demo - Internal testing only,Internal testing only,Your AWS End User Messaging internal testing one time passcode is 987654,./companyC.png
AnyOrganization,https://example.com/example4,123 Pine Blvd,,Anywhere Else,WA,12345,US,FirstName4,LastName4,[email protected],15553334444,100,One-time passcodes,Dev/Demo - Internal testing only,Internal testing only,Your AWS End User Messaging internal testing one time passcode is 654789,./companyD.png
AnyGovernment,https://example.org/example5,123 Oak St,,Nowhere Town,WA,67890,US,FirstName5,LastName5,[email protected],15553335555,"100",One-time passcodes,Dev/Demo - Internal testing only,Internal testing only,Your AWS End User Messaging internal testing one time passcode is 987654,./companyE.png
  1. Prepare opt-in screenshot image files for each registration, and upload them in the same directory as you will use for the data.csv and script files. Note: the allowed file types for the opt-in screenshot are PNG, JPG, and PDF with a max file size of 500 kb.
  2. Below is a Python script to create registrations using the data.csv file (above). Save this file as register_phone_numbers_bulk.py into your current directory.
import csv
import boto3
import sys
import argparse
import time

# Parse command line arguments
parser = argparse.ArgumentParser(description="AWS End User Messaging SMS Registration Script")
parser.add_argument("-r", "--region", default="us-west-2", help="AWS region")
parser.add_argument("-c", "--csv_file", default="data.csv", help="Path to the Comma-Separated Values file")
args = parser.parse_args()

# Initialize AWS End User Messaging SMS client
client = boto3.client('pinpoint-sms-voice-v2', region_name=args.region)

def process_csv(csv_file):
    with open(csv_file, newline='', encoding='utf-8') as f:
        reader = csv.DictReader(f)
        
        # Strip BOM from the first key (header) if it exists
        if '\ufeff' in reader.fieldnames[0]:
            reader.fieldnames[0] = reader.fieldnames[0].replace('\ufeff', '')
        
        for row in reader:
            process_row(row)

def process_row(row):
    print(f"================================================================================================")
    
    if 'companyInfo_companyName' not in row:
        print(f"Error: 'companyInfo_companyName' field is missing in the CSV file.")
        return
    
    print(f"Processing entry for: {row['companyInfo_companyName']}")
    print(f"================================================================================================")

    # Validate required fields
    required_fields = [
        'companyInfo_companyName', 'companyInfo_website', 'companyInfo_isoCountryCode', 
        'attachmentFilePath'
    ]
    for field in required_fields:
        if field not in row or not row[field]:
            print(f"Error: Required field '{field}' is missing or empty for entry: {row.get('companyInfo_companyName', 'Unknown Company')}")
            return

    # Validate field values
    if 'companyInfo_zipCode' not in row or not row['companyInfo_zipCode'].isdigit():
        print(f"Error: Invalid or missing zip code for entry: {row['companyInfo_companyName']}")
        return

    valid_volumes = ["10", "100", "1,000", "10,000", "100,000", "250,000", "500,000", "750,000", "1,000,000", "5,000,000", "10,000,000+"]
    if 'messagingUseCase_monthlyMessageVolume' not in row or row['messagingUseCase_monthlyMessageVolume'] not in valid_volumes:
        print(f"Error: Invalid or missing monthly_message_volume for entry: {row['companyInfo_companyName']}")
        return

    valid_categories = ["Two-factor authentication", "One-time passcodes", "Notifications", "Polling and surveys", "Info on demand", "Promotions & marketing", "Other"]
    if 'messagingUseCase_useCaseCategory' not in row or row['messagingUseCase_useCaseCategory'] not in valid_categories:
        print(f"Error: Invalid or missing use_case_category for entry: {row['companyInfo_companyName']}")
        return

    try:
        # Step 1: Create registration
        print("Creating registration...")
        create_registration_output = client.create_registration(
            RegistrationType='US_TOLL_FREE_REGISTRATION'
        )
        registration_id = create_registration_output['RegistrationId']

        # Step 2: Put registration field values from CSV
        print("Putting registration field values...")
        field_mappings = {
            "companyInfo.companyName": {"value": row['companyInfo_companyName'], "type": "text"},
            "companyInfo.website": {"value": row['companyInfo_website'], "type": "text"},
            "companyInfo.address1": {"value": row['companyInfo_address1'], "type": "text"},
            "companyInfo.address2": {"value": row['companyInfo_address2'], "type": "text"},            
            "companyInfo.city": {"value": row['companyInfo_city'], "type": "text"},
            "companyInfo.state": {"value": row['companyInfo_state'], "type": "text"},
            "companyInfo.zipCode": {"value": row['companyInfo_zipCode'], "type": "text"},
            "companyInfo.isoCountryCode": {"value": row['companyInfo_isoCountryCode'], "type": "text"},
            "contactInfo.firstName": {"value": row['contactInfo_firstName'], "type": "text"},
            "contactInfo.lastName": {"value": row['contactInfo_lastName'], "type": "text"},
            "contactInfo.supportEmail": {"value": row['contactInfo_supportEmail'], "type": "text"},
            "contactInfo.supportPhoneNumber": {"value": row['contactInfo_supportPhoneNumber'], "type": "text"},
            "messagingUseCase.useCaseDetails": {"value": row['messagingUseCase_useCaseDetails'], "type": "text"},
            "messagingUseCase.optInDescription": {"value": row['messagingUseCase_optInDescription'], "type": "text"},
            "messageSamples.messageSample1": {"value": row['messageSamples_messageSample1'], "type": "text"},
            "messagingUseCase.monthlyMessageVolume": {"value": row['messagingUseCase_monthlyMessageVolume'], "type": "select"},
            "messagingUseCase.useCaseCategory": {"value": row['messagingUseCase_useCaseCategory'], "type": "select"},
        }

        for field, field_data in field_mappings.items():
            if field_data['type'] == 'text' and field_data['value']:
                client.put_registration_field_value(
                    RegistrationId=registration_id,
                    FieldPath=field,
                    TextValue=field_data['value']
                )
            elif field_data['type'] == 'select' and field_data['value']:
                client.put_registration_field_value(
                    RegistrationId=registration_id,
                    FieldPath=field,
                    SelectChoices=[field_data['value']]
                )

        # Step 3: Create registration attachment
        print("Creating registration attachment...")
        with open(row['attachmentFilePath'], 'rb') as attachment_file:
            create_attachment_output = client.create_registration_attachment(
                AttachmentBody=attachment_file.read()
            )
            attachment_id = create_attachment_output['RegistrationAttachmentId']

        # Step 4: Wait for the attachment upload to finish
        print("Verifying attachment upload complete...")
        attachment_id = create_attachment_output['RegistrationAttachmentId']
        upload_complete = False
        max_retries = 10
        retry_count = 0

        while not upload_complete and retry_count < max_retries:
            try:
                describe_attachment_output = client.describe_registration_attachments(
                    RegistrationAttachmentIds=[attachment_id]
                )
                attachment_status = describe_attachment_output['RegistrationAttachments'][0]['AttachmentStatus']
                if attachment_status == 'UPLOAD_COMPLETE':
                    upload_complete = True
                else:
                    print(f"Attachment status: {attachment_status}. Waiting 2 seconds before retrying... (max {max_retries} retries)")
                    time.sleep(2)
                    retry_count += 1
            except Exception as e:
                print(f"Error occurred while checking attachment status: {str(e)}")
                time.sleep(10)
                retry_count += 1

        if not upload_complete:
            print(f"Error: Attachment upload did not complete for {row['companyInfo_companyName']}")
            return

        # Step 5: Associate registration attachment with registration
        print("Associating registration attachment with registration...")
        client.put_registration_field_value(
            RegistrationId=registration_id,
            FieldPath='messagingUseCase.optInImage',
            RegistrationAttachmentId=attachment_id
        )
        
        # Step 6: Request phone number
        print("Requesting phone number...")
        request_phone_number_output = client.request_phone_number(
            IsoCountryCode='US',
            NumberType='TOLL_FREE',
            NumberCapabilities=['SMS', 'MMS', 'VOICE'],
            MessageType='TRANSACTIONAL'
        )
        phone_number_id = request_phone_number_output['PhoneNumberId']

        # Step 7: Associate phone number with registration
        print("Associating phone number with registration...")
        client.create_registration_association(
            RegistrationId=registration_id,
            ResourceId=phone_number_id
        )
                
        # Step 8: Submit registration
        print("Submitting registration...")
        client.submit_registration_version(
            RegistrationId=registration_id
        )
        print(f"Processing completed for: {row['companyInfo_companyName']}\n")

    except Exception as e:
        print(f"Error occurred while processing {row['companyInfo_companyName']}: {str(e)}")

if __name__ == "__main__":
    process_csv(args.csv_file)
  1. Run the script: python register_phone_numbers_bulk.py -c data.csv -r us-west-2
    The 2 parameters are the name of the CSV file containing your data (shown here as data.csv) and specify the target AWS region (shown here as us-west-2) as parameters. Adjust these parameters as needed to match your environment and data location.

Check status:

Once you have successfully ran the script, you can check the status of your registration requests via AWS End User Messaging Console or with the API using the DescribeRegistrations API. If you want to use the console log in to the AWS End User Messaging Console in the target region and choose Configurations > Registrations from the left navigation. The registration status for each request will initially display “CREATED” and it will change to “REVIEWING” within 24 hours. As noted in the introduction, the actual registration review process is conducted by a third-party company and it can take up to 15 days to receive an acceptance (“COMPLETE”) or request for additional information or corrections (“Requires Update“) for each request.

Registration Status:

Registrations

Phone Number Status:

Phone Number status

Conclusion:

In this post you have learned how to automate the registration process for Toll-Free numbers with a Bash script or Python script and the AWS End User Messaging APIs. Using the API can significantly improve efficiency and reduce manual errors, but keep in mind that you will still need to wait up to 15 business days for the registrations to be approved (or not), and for the numbers to be set to “ACTIVE” status before you can begin sending messages with your TFN(s).

The AWS End User Messaging V2 API for SMS and Voice has other useful actions, and we encourage you to explore how it can further help you simplify and automate your applications.

Resources to help you plan for your SMS program:

Use this spreadsheet to plan for the countries you need to send SMS/MMS

To restrict your sending to only certain countries read this blog

Confirm the origination IDs you will need here

Check out the support tiers comparison

Leverage IAM Roles for email sending via SES from EC2 and eliminate a common credential risk

Post Syndicated from Zip Zieper original https://aws.amazon.com/blogs/messaging-and-targeting/leverage-iam-roles-for-email-sending-via-ses-from-ec2-and-eliminate-a-common-credential-risk/

Sending automated transactional emails, such as account verifications and password resets, is a common requirement for web applications hosted on Amazon EC2 instances. Amazon SES provides multiple interfaces for sending emails, including SMTP, API, and the SES console itself. The type of SES credential you use with Amazon SES depends on the method through which you are sending the emails.

In this blog post, we describe how to leverage IAM roles for EC2 instances to securely send emails via the Amazon SES API, without the need to embed IAM credentials directly in the application code, link to a shared credentials file, or manage IAM credentials within the EC2 instance. By adopting the approach outlined in this blog, you can enhance security by eliminating the risk of credential exposure and simplify credential management for your web applications.

Solution Overview

Below we provide step-by-step instructions to configure an IAM role with SES permissions to use on your EC2 instance. This allows the EC2 hosted web application to securely send emails via Amazon SES without storing or managing IAM credentials within the EC2 instance. We present an option for running EC2 and SES in the same AWS account, as well as an option to accommodate running EC2 and SES in different AWS accounts. Both options offer a way to enhance security and simplify credential management.

Either option begins with creating an IAM role with SES permissions. Next, the IAM role is attached to your EC2 instance, providing it with the necessary permissions for SES without needing to embed IAM credentials in your application code or on a file in the EC2 instance. In option 2, we’ll add cross-account permissions that allow the code on the EC2 instance in account “A” to send email via the SES API in account “B”. We also provide a sample Python script that demonstrates how to send an email from your EC2 instance using the attached IAM role.

Option 1 – SES and EC2 are in a single AWS account

In a typical scenario where an EC2 instance is operating in the same AWS account as SES, the process of using an IAM role to send emails via SES is straightforward. In the steps below, you’ll configure and attach an IAM role to the EC2 instance. You’ll then update a sample Python script to use the permissions provided by the attached IAM role to send emails via SES. This direct access simplifies the SES sending process, as no explicit credential management is required in the code, nor do you need to include a shared credentials file on the EC2 instance.

Option_1-Single_AWS_Account

EC2 & SES in the same AWS Account

Prerequisites – single AWS account for EC2 and SES

  • A single AWS account in a region that supports SES
  • Verified domain or email identity in Amazon SES.
    • Make note of a verified sending email address here: ___________
  • EC2 instance (Linux) in running state
    • If you don’t have a EC2 instance create one (Linux)
  • Administrative Access to Amazon SES, IAM and EC2 consoles.
  • Access to a recipient email address to receive test emails from the python script.
    • Make note of a SES verified recipient email address to send test emails here: ___________

Step 1 – Create IAM Role for EC2 instance with SES Permissions

To start, create an IAM role that grants the necessary permissions to send emails using Amazon SES by following these steps:

  • Sign in to the AWS Management Console and open the IAM console.
  • In the navigation pane, choose “Roles,” and then choose “Create role.”
  • Choose the trusted entity type as “AWS service” and select “EC2” as the service that will use this role, then click ‘Next
  • Search for and select the “AmazonSESFullAccess” policy from the list (or create a custom policy with the necessary SES permissions), then click ‘Next’.
  • Provide a name for your role (e.g., EC2_SES_SendEmail_Role).
  • Click “Create role“.

Step 2 – Attach the IAM Role to EC2 instance.

Next, attach the IAM role to your EC2 instance:

  • Open the EC2 Management Console.
  • In the navigation pane, choose “Instances,” and select the running EC2 instance to which you want to attach the IAM role.
  • With the instance selected, choose “Actions,” then “Security,” and “Modify IAM role.
  • Choose the IAM role you created (EC2_SES_SendEmail_Role) from the drop-down menu and click “Update IAM role.”

Step 3 – Create a sample python script that sends emails from the EC2 instance with the attached role.

  • Now that your EC2 instance is configured with the necessary permissions, you can set up an example Python script to send emails via Amazon SES using the IAM Role. Here, we’re using the AWS SDK for Python (Boto3), a powerful and versatile library to interact with the SES API endpoint. Before running the example script, ensure that Python, pip (the package installer for Python), and the Boto3 library are installed on your EC2 instance:
    • Run the ‘python3 –version‘ command to check if Python is installed on your EC2 instance. If Python is installed, the version will be displayed, otherwise you’ll receive a ‘command not found’ or similar error message.
      • If python is not installed, run the command ‘sudo yum install python3 -y
    • Run the ‘pip3 --version‘ command to check if pip is installed on your EC2 instance. If pip3 is installed, is installed, the version will be displayed, otherwise you’ll receive a ‘command not found’ or similar error message.
      • If pip3 is not installed, run the command ‘sudo yum install python3-pip
    • Install the Boto3 Library which allows Python scripts to interact with AWS services including SES. Run the command ‘pip3 install boto3‘ to install (or update) Boto3 using pip.
  • Save the code below as a Python file named ‘sesemail.py‘ on your EC2 instance.
  • Edit 'sesemail.py‘ and replace the placeholder values of SENDER, RECIPIENT, and AWS_REGION with your values (see prerequisites). Do not modify any “” marks.

[copy]

import boto3
from botocore.exceptions import ClientError

SENDER = "[email protected]"
RECIPIENT = "[email protected]"
#CONFIGURATION_SET = "ConfigSet"
AWS_REGION = "us-west-2"
SUBJECT = "Amazon SES Test Email (SDK for Python) using IAM Role"
BODY_TEXT = ("Amazon SES Test (Python)\r\n"
             "This email was sent with Amazon SES using the "
             "AWS SDK for Python (Boto)."
            )
            
BODY_HTML = """<html>
<head></head>
<body>
  <h1>Amazon SES Test (SDK for Python) using IAM Role</h1>
  <p>This email was sent with
    <a href='https://aws.amazon.com/ses/'>Amazon SES</a> using the
    <a href='https://aws.amazon.com/sdk-for-python/'>
      AWS SDK for Python (Boto)</a>.</p>
</body>
</html>
            """            

CHARSET = "UTF-8"

client = boto3.client('ses',region_name=AWS_REGION)

try:
    response = client.send_email(
        Destination={
            'ToAddresses': [
                RECIPIENT,
            ],
        },
        Message={
            'Body': {
                'Html': {
                    'Charset': CHARSET,
                    'Data': BODY_HTML,
                },
                'Text': {
                    'Charset': CHARSET,
                    'Data': BODY_TEXT,
                },
            },
            'Subject': {
                'Charset': CHARSET,
                'Data': SUBJECT,
            },
        },
        Source=SENDER,
    )   
except ClientError as e:
    print(e.response['Error']['Message'])
else:
    print("Email sent! Message ID:"),
    print(response['MessageId'])
  • Run ‘python3 sesmail.py‘ to execute the Python script.
  • When ‘python3 sesmail.py‘ runs successfully, an email is sent to the RECIPIENT(check the inbox), and the command line will display the sent Message ID.


Option 2 – SES and EC2 are in different AWS accounts

In some scenarios, your EC2 instance might operate in a different AWS account than SES. Let’s call the EC2 AWS account “A” and SES AWS account “B”. Because the AWS resources in account A don’t automatically have permission to access AWS resources account B, we need some way to allow the code on EC2 to assume a role in the SES Account using the AWS Security Token Service (STS). This involves a method that generates temporary credentials that include an access key, secret access key, and session token, which are only valid for a limited time.

option-2

EC2 & SES in different AWS Accounts

In the steps below, you’ll configure and attach an IAM role to the EC2 instance in account “A” such that it can run an example Python script. This Python script can use the permissions provided by the attached IAM role to send emails via SES in account “B”. This approach leverages cross-account access and simplifies sending email from the EC2 in account A via SES in account B. As with Option 1, no explicit credential management is required in the code running on EC2, nor do you need to include a shared credentials file on the Ec2 instance.

Prerequisites – different AWS accounts for EC2 and SES (use cross-account access)

  • An AWS account “A” with:
    • EC2 instance (Linux) in running state. (If you don’t have a EC2 instance, create one using Amazon Linux)
    • Administrative Access to Amazon IAM and EC2 consoles.
    • Make note of your “A” AWS account ID here: ________________
  • An AWS account “B” with:
    • Verified domain (or email identity for testing only) in Amazon SES
      • Make note of a verified sending email address here: ___________
    • Administrative Access to Amazon SES and IAM consoles.
      • Make note of your “B” AWS account ID here: ________________
    • In the steps below, you will create a “SES_Role_for_account_A” role.
      • Make note of the ARN of the “SES_Role_for_account_A” role here: ___________
    • Access to a recipient email address to receive test emails from the python script.
      • Make note of a SES verified recipient email address to send test emails here: ___________

Step 1 – Create IAM Role in the SES “B” account

  • Sign in to the SES “B” account via the AWS Management Console and open the IAM console.
  • In the navigation pane, choose “Roles,” and then choose “Create role“.
  • Choose the trusted entity type as ‘AWS account’ and select ‘Another AWS account’.
  • Add the AWS account ID where your EC2 instance resides (AWS account “A” in the prerequisites) and click ‘Next’.
  • Search for and select the “AmazonSESFullAccess” policy or create a custom policy with the necessary SES permissions, then click ‘Next’.
  • Provide a name for your role (e.g., ‘SES_Role_for_account_A').
  • Click “Create role“.
  • Copy the arn for the new SES_Role_for_account_A (you’ll need the arn in the next step).

Step 2 – Create a IAM policy in the EC2 “A” account that allows this role to assume the SES_Role_for_account_A role you just created in the SES “B” Account.

  • Sign in to the EC2 “A” account via the AWS Management Console and open the IAM console.
  • In the navigation pane, choose “Policies,” and then choose “Create Policy”.
  • Choose the service as ‘EC2’ and select policy editor as JSON.
  • Copy the policy below, and in the policy editor, replace the Resource with the arn of theSES_Role_for_account_A in the SES account “B” (you created this in step 1).

[copy, paste into policy editor & replace the arn with SES_Role_for_account_A]

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::<SES_Account_ID>:role/<Role_Name>"
}
]
}

  • Click ‘Next’ and provide a name for your role (e.g., EC2_Policy_for_account_B).
  • Click ‘Create the Policy

Step 3 – Create an IAM role in the EC2 “A” account, and attach the previously created IAM policy (EC2_Policy_for_account_B) to it.

  • In the EC2 “A” account IAM console navigation pane, choose “Roles,” and then choose “Create role.”
  • Choose the trusted entity type as “AWS service” and select “EC2” as the service, then click ‘Next’.

  • Filter by type “customer managed”, search for (EC2_Policy_for_account_B) and select that policy and ‘Next’ (note – if you are using AWS Session Manger to remotely connect to your EC2 instance, you may need to add the “AmazonSSMManagedInstanceCore” policy to the role).

  • Provide a name for your role (e.g., EC2_SES_in_account_B_role).
  • Click “Create role“.

Step 4 – Attach the IAM Role (EC2_SES_in_account_B_role) to the EC2 instance in AWS account “A”.

  • Open the EC2 Management Console in AWS account “A”
  •  In the navigation pane, choose “Instances,” and select the instance to which you want to attach the EC2_SES_in_account_B_role IAM role.
  • With the instance selected, choose “Actions,” then “Security,” and “Modify IAM role.”

  • Choose the IAM role you created (EC2_SES_in_account_B_role) from the drop-down menu.
  • Click “Update IAM role.”

Step 5 – Create a sample python script that sends emails via SES in AWS account “B” from the EC2 instance in AWS account “A” using the EC2 attached role.

  1. Now that your EC2 instance is configured with the necessary permissions, you can set up an example Python script to send emails via Amazon SES in AWS Account “B” using the IAM Role on EC2 in AWS Account “A”. We’ll use the AWS SDK for Python (Boto3), a powerful and versatile library to interact with the SES API endpoint. Before running the example script, ensure that Python, pip (the package installer for Python), and the Boto3 library are installed on your EC2 instance:
    • Run the ‘python3 –version‘ command to check if Python is installed on your EC2 instance. If Python is installed, the version will be displayed, otherwise you’ll receive a ‘command not found’ or similar error message.
      • If python is not installed, run the command ‘sudo yum install python3 -y
    • Run the ‘pip3 --version‘ command to check if pip is installed on your EC2 instance. If pip3 is installed, is installed, the version will be displayed, otherwise you’ll receive a ‘command not found’ or similar error message.
      • If pip3 is not installed, run the command ‘sudo yum install python3-pip
    • Install the Boto3 Library which allows Python scripts to interact with AWS services including SES. Run the command ‘pip3 install boto3‘ to install (or update) Boto3 using pip.
  1. Save the code below as a Python file named cross_sesemail.py on your EC2 instance.
    4b. Edit cross_sesemail.py and replace the placeholder values of the ROLE_ARN with ARN of the SES_Role_for_account_A you created in SES Account “B” (see prerequisites), SENDER, RECIPIENT, and AWS_REGION with your values (see prerequisites). Do not modify any “” marks.

[copy, edit & replace the ROLE_ARN]

import boto3
from botocore.exceptions import ClientError

# Replace with your role ARN in SES Account
ROLE_ARN = "arn:aws:iam::<Account_ID>:role/<Role_Name>"

# Create an STS client
sts_client = boto3.client('sts')

# Assume the role
assumed_role = sts_client.assume_role(
    RoleArn=ROLE_ARN,
    RoleSessionName="SESSession"
)

# Extract the temporary credentials
credentials = assumed_role['Credentials']

# Create an SES client using the assumed role credentials
ses_client = boto3.client(
    'ses',
    region_name='us-west-2',
    aws_access_key_id=credentials['AccessKeyId'],
    aws_secret_access_key=credentials['SecretAccessKey'],
    aws_session_token=credentials['SessionToken']
)

# Email parameters
SENDER = "[email protected]"
RECIPIENT = "[email protected]"
SUBJECT = "Amazon SES Test (SDK for Python) using cross-account IAM Role"
BODY_TEXT = ("Amazon SES Test (Python)\r\n"
             "This email was sent with Amazon SES using the "
             "AWS SDK for Python (Boto) using IAM Role."
            )
BODY_HTML = """<html>
<head></head>
<body>
  <h1>Amazon SES Test (SDK for Python) using IAM Role</h1>
  <p>This email was sent with
    <a href='https://aws.amazon.com/ses/'>Amazon SES</a> using the
    <a href='https://aws.amazon.com/sdk-for-python/'>
      AWS SDK for Python (Boto)</a> using IAM Role.</p>
</body>
</html>
            """
CHARSET = "UTF-8"

# Send the email
try:
    response = ses_client.send_email(
        Destination={
            'ToAddresses': [RECIPIENT],
        },
        Message={
            'Body': {
                'Html': {
                    'Charset': CHARSET,
                    'Data': BODY_HTML,
                },
                'Text': {
                    'Charset': CHARSET,
                    'Data': BODY_TEXT,
                },
            },
            'Subject': {
                'Charset': CHARSET,
                'Data': SUBJECT,
            },
        },
        Source=SENDER,
    )
except ClientError as e:
    print(e.response['Error']['Message'])
else:
    print("Email sent! Message ID:"),
    print(response['MessageId'])
  • Run the python script python3 cross_sesemail.py. When the email is sent successfully, the command line output will display the message ID of the sent email, and the recipient will receive an email.


Conclusion:

By implementing IAM roles for EC2 instances with SES permissions, you can securely send emails via the SES APIs from your web applications without the need to store or manage IAM credentials within the EC2 instance or application code. This approach not only enhances security by eliminating the risk of credential exposure, but also simplifies the management of credentials. With the step-by-step guide provided in this blog post, you can easily configure IAM roles for your EC2 instances and start sending emails via the Amazon SES API in a secure and efficient manner, regardless of whether your EC2 and SES resources reside in the same or different AWS accounts.

Next Steps:

  1. Sign up for the AWS Free Tier and try out Amazon SES with IAM roles for EC2 instances as demonstrated in this blog post.
  2. Consult the AWS documentation on IAM Roles for Amazon EC2 and Amazon SES for more detailed instructions and best practices.
  3. Join the AWS Community Forums to ask questions, share experiences, and learn from other AWS users who have implemented similar solutions for secure email sending from their web applications.

About the Authors

Manas Murali M

Manas Murali M

Manas Murali M is a Cloud Support Engineer II at AWS and subject matter expert in Amazon Simple Email Service (SES) and Amazon CloudFront. With over 5 years of experience in the IT industry, he is passionate about resolving technical issues for customers. In his free time, he enjoys spending time with friends, traveling, and exploring emerging technologies.

zip

Zip

Zip is an Amazon Pinpoint and Amazon Simple Email Service Sr. Specialist Solutions Architect at AWS. Outside of work he enjoys time with his family, cooking, mountain biking and plogging.

Streamline SMS and Emailing Marketing Compliance with Amazon Comprehend

Post Syndicated from Koushik Mani original https://aws.amazon.com/blogs/messaging-and-targeting/streamline-sms-and-emailing-marketing-compliance-with-amazon-comprehend/

In today’s digital landscape, businesses heavily rely on SMS and email campaigns to engage with customers and deliver timely, relevant messages. The shift towards digital marketing has increased customer engagement, accelerated delivery, and expanded personalization options. Email and SMS marketing is essential to digital strategies according to 44% of Chief Marketing Officers and they allocate approximately 8% of their marketing towards this. Industries face stringent restrictions on the content they can send due to legal regulations and carrier filtering policies.

Messages related to the subjects listed below are considered restricted and are subject to heavy filtering or even being blocked outright. Failing to comply with these restrictions can result in severe consequences, including legal action, fines, and irreparable damage to a brand’s reputation. Marketers need a solution that will proactively scan their content used in campaigns and flag restricted content before sending it out to their customers without facing penalties and losing trust.:

  • Gambling
  • High-risk financial services
  • Debt forgiveness
  • S.H.A.F.T (Sex, Hate, Alcohol, Firearms, and Tobacco)
  • Illegal substances

In this blog, we will explore how to leverage Amazon Comprehend, Amazon S3, and AWS Lambda to proactively scan text-based marketing campaigns before publishing content . This solution enables businesses to enhance their marketing efforts while maintaining compliance with industry regulations, avoiding costly fines, and preserving their hard-earned reputation, conforming to best practices.

Solution Overview

AWS provides a robust suite of services to meet the infrastructure needs of the booming digital marketing industry, including messaging capabilities through email, SMS, push, and other channels through Amazon Simple Email Service, Amazon Simple Notification Service, or Amazon Pinpoint.

The main goal for this approach is to flag any message that contains restricted content mentioned above before distribution.

Figure 1: Architecture for proactive scanning of marketing content

Figure 1: Architecture for proactive scanning of marketing content

Following are the high-level steps:

  1. Upload documents to be scanned to the S3 bucket.
  2. Utilize Amazon Comprehend custom classification for categorizing the documents uploaded.
  3. Create an Amazon Comprehend endpoint to perform analysis.
  4. Inference output is published to the destination S3 bucket.
  5. Utilize AWS Lambda function to consume the output from the destination S3 bucket.
  6. Send the compliant messages through various messaging channels.

Solution Walkthrough

Step 1: Upload Documents to Be Scanned to S3

  1. Sign in to the AWS Management Console and open the Amazon S3 console
  2. In the navigation bar on the top of the page, choose the name of the currently displayed AWS Region. Next, choose the Region in which you want to create a bucket.
  3. In the left navigation pane, choose Buckets.
  4. Choose Create bucket.
    • The Create bucket page opens.
  5. Under General configuration, view the AWS Region where your bucket will be created.
  6. Under Bucket type, choose General purpose.
  7. For Bucket name, enter a name for your bucket.
    • The bucket name must:
      • Be unique within a partition. A partition is a grouping of Regions. AWS currently has three partitions: aws (Standard Regions), aws-cn (China Regions), and aws-us-gov (AWS GovCloud (US) Regions).
      • Be between 3 and 63 characters long.
      • Consist only of lowercase letters, numbers, dots (.), and hyphens (-). For best compatibility, we recommend that you avoid using dots (.) in bucket names, except for buckets that are used only for static website hosting.
      • Begin and end with a letter or number.
  8. In the Buckets list, choose the name of the bucket that you want to upload your folders or files to.
  9. Choose Upload.
  10. In the Upload window, do one of the following:
    • Drag and drop files and folders to the Upload window.
    • Choose Add file or Add folder, choose the files or folders to upload, and choose Open.
    • To enable versioning, under Destination, choose Enable Bucket Versioning.
    • To upload the listed files and folders without configuring additional upload options, at the bottom of the page, choose Upload.
  11. Amazon S3 uploads your objects and folders. When the upload is finished, you see a success message on the Upload: status page.

Step 2: Creating a Custom Classifiction Model

Custom Classification Model

Out-of-the-box models may not capture nuances and terminology specific to an organization’s industry or use case. Therefore, we train a custom model to identify compliant messages.

A custom classification model is a feature that allows you to train a machine learning model to classify text data based on categories that are specific to your use case or industry. It trains the model to recognize and sort different types of content which is used to power the endpoint. A custom classification model is designed to save costs and promote compliant messages and further prevent marketing companies from potential fines.

Requirements for custom classification:

  • Dataset creation
    • A CSV dataset with 1000 examples of marketing messages, each labeled as compliant (1) or non-compliant (0).
    • Designed to train a model for accurate predictions on marketing message compliance.
Figure 2: Screenshot of dataset – 20 entries of censored marketing messages

Figure 2: Screenshot of dataset – 20 entries of censored marketing messages

  • Creating a Test Data Set
    In addition to providing a dataset to power your customer classification model, a test dataset is also required to test the data that the model will be running on. Without a test dataset, Amazon Comprehend trains the model with 90 percent of the training data. It reserves 10 percent of the training data to use for testing. When using a test dataset, the test data must include at least one example for each unique label (0 or 1) in the training dataset.
  1. Upload the data set and test data set to an S3 Bucket, by following the steps in this user guide.
  2. In the AWS Console, search for Amazon Comprehend.
  3. Once selected, select custom classification on the left panel.
  4. Once there, select Create new model.
  5. Next specify model settings:
    • Model name
    • Specify the version (optional)
    • Language: EnglishModel Setting
  6. Specify the data specifications:
    • Training model type: Plain Text Documents
    • Data format: CSV File
    • Classifier Mode: Using Single-Label Mode
    • Training Dataset: Give the name of the bucket you created in step 1
    • Test Data set: Autosplit, i.e. how much of your data will be used for training and testing.
    • Data Specifications
  1. Specify the location of the model output in S3
    • Output data
  1. Create an IAM Role
    • Permissions to access: Train, Test and output data (if specified in your S3 Buckets)
    • IAM Role
  2. Once all parameters have been identified, select Create.
    • Preferences
  3. Once the model has been created, you can view it under Custom Classification. To check and verify the accuracy and F1 score, select the version number of the model. Here, you can view the model details under the Performance tab.

Step 3: Creating an Endpoint in Amazon Comprehend

Next, an endpoint needs to be created to analyze documents. To create an endpoint:

  • Select endpoint on the left panel in Amazon Comprehend.
  • Select Create endpoint in the left panel.
  • Specify Endpoints Settings :
    • Provide a name
    • Custom model type: Custom Classification
    • Choose a custom model that you want to attach to the new endpoint. From the dropdown, you can search by model name.
    • create_endpointFigure 8: Amazon Comprehend – Endpoint settings
  • Provide the number of inference units (IUs): 1
  • Once all the parameters have been provided, ensure that the Acknowledge checkbox has been selected.
  • Finally, select Create endpoint.
  • inference_units

Step 4: Scanning Text with the Custom Classification Endpoint

Once the endpoint has been successfully created, it can be used for real-time analysis or batch-processing jobs. Below is a walkthrough of how both options can be achieved.

Real-time analysis:

  • On the left panel, select Realtime Analysis.
  • Pick Analysis type: custom, to view real-time insights based on the custom models from an endpoint you’ve created
  • Select custom model type
  • Select your Endpoint
  • Insert your input text.
    • For this example, we have used a non-compliant message: Huge sale at Pine County Shooting Range 25% off for 6mm and 9mm bullets! Lazer add-ons on clearance too
  • Once inserted, click Analyze.input_data
  • Once analyzed, you will see a confidence score under Classes. Because the dataset is labeled as 0 for non-compliant and 1 for compliant. The message that was inserted was non-compliant, the result of the real-time analysis is a high confidence score for non-compliant.insights

Real-time analysis in Amazon Comprehend:

  • On the left panel in Amazon Comprend, select Analysis Jobs.
  • Select the Create Job button.
  • Configure Job settings:
    • Enter the Name
    • Analysis Type: Custom Classification
    • Classifications Model: The model you have created for your Classifier, as well as the version number of that model you would like to use for this job.
    • create_analysis
  • Enter the location of the Input Data and Output Data in the form of an S3 bucket URL.
  • input_data
  • Before creating a job the last thing, we want to do is provide the right access permission, by creating an IAM role that give access permissions to the S3 input and output locations.
  • access_premissions
  • Once the batch processing job shows a status of completed, you can view the results in the output S3 bucket which was identified earlier. The results will be in a json file where each line represents the confidence score for each marketing message.
  • json

Step 5 (optional): Publish message to communication service

The result from the batch processing is automatically uploaded to the output S3 bucket. For each json file uploaded, S3 will initiate an S3 Event Notification which will inform a Lambda function that a new S3 object has been created.

The Lambda function will evaluate the results and automatically identify the messages labeled as compliant (label 0). These compliant messages will then be published to communication services using one of the following three APIs, depending on the desired service:

To automatically trigger the AWS Lambda function, which will read the files uploaded into the S3 bucket and display the data using the Python Pandas library, we will use the boto3 API to read the files from the S3 bucket.

  1. Create an IAM Role in AWS.
  2. Create an AWS S3 bucket.
  3. Create the AWS Lambda function with S3 triggers enabled.
  4. Update the Lambda code with a Python script to read the data and send the communication to customer.

Conclusion

Proactively scanning and classifying marketing content for compliance is a critical aspect of ensuring successful digital marketing campaigns while adhering to industry regulations. Leveraging the powerful combination of Amazon Comprehend, Amazon S3, and AWS Lambda enables the automatic analysis of text-based marketing messages and flagging of any non-compliant content before sending them to your customer. Following these steps provides you with the tools and knowledge to implement proactive scanning for your marketing content. This solution will help mitigate the risks of non-compliance, avoiding costly fines and reputational damage, while freeing up time for your content creation teams to focus on ideation and crafting compelling marketing messages. Regular monitoring and fine-tuning of the custom classification model should be conducted to ensure accurate identification of non-compliant language.

To get started with proactively scanning and classifying marketing content for compliance, see Amazon Comprehend Custom Classification.

About the Authors

Caroline Des Rochers

Caroline Des Rochers

Caroline is a Solutions Architect at Amazon Web Services, based in Montreal, Canada. She works closely with customers, in both English and French, to accelerate innovation and advise them through technical challenges. In her spare time, she is a passionate skier and cyclist, always ready for new outdoor adventures.

Erika_Houde_Pearce

Erika Houde Pearce

Erika Houde-Pearce is a bilingual Solutions Architect in Montreal, guiding small and medium businesses in eastern Canada through their cloud transformations. Her expertise empowers organizations to unlock the full potential of cloud technology, accelerating their growth and agility. Away from work, she spends her spare time with her golden retriever, Scotia.

Koushik_Mani

Koushik Mani

Koushik Mani is a Solutions Architect at AWS. He had worked as a Software Engineer for two years focusing on machine learning and cloud computing use cases at Telstra. He completed his masters in computer science from University of Southern California. He is passionate about machine learning and generative AI use cases and building solutions.

Email Journaling with SES Mail Manager

Post Syndicated from Zip Zieper original https://aws.amazon.com/blogs/messaging-and-targeting/email-journaling-with-ses-mail-manager/

Introduction to Journaling

Email journaling is the practice of preserving comprehensive records of all email communications within an organization. This approach stems from the need to maintain rigid, compliance-driven retention policies focused on auditing an entire organization’s email activities. Because journaled email messages are often required to satisfy on-demand audit and investigation requests, they must be readily searchable, making accessibility a key requirement. Reflecting legal and regulatory requirements, email journaling has historically required expensive, dedicated off-site storage and complex retrieval systems.

Amazon WorkMail is a managed business email service with flexible journaling capabilities that are configurable at both the individual mailbox and organization-wide level. With WorkMail, you can use custom rules to selectively preserve or redirect certain messages using granular journaling controls. This flexibility allows administrators to implement both traditional email journaling and configurations that you can customize to meet specific use cases.

Email journaling is used to capture and retain every email sent to and from an organization, primarily for compliance purposes. In contrast, email archiving is typically used to offload and store emails from an organization’s primary email system, often driven by inbox size limits and data backup or eDiscovery needs. While journaling focuses on preserving a consolidated record of communications separate from live mailboxes, archiving is a more selective process. Journaling is usually driven by regulatory, audit, and compliance requirements. As discussed in this blog post, you can use the Mail Manager archiving feature not only for selective email backup and optimization, but also to fulfill your email journaling requirements. You can learn more about email archiving with Mail Manager in this blog post.

Amazon Simple Email Service (SES) Mail Manager provides comprehensive tools that simplify managing large volumes of email communications within an organization. Mail Manager has a built-in archiving function which can be used as an inexpensive journaling solution for email systems like Amazon WorkMail. Mail Manager’s rules engine allows for the creation of rules that readily satisfy a wide range of email journaling requirements. Additionally, Mail Manager’s archiving capability supports multiple, concurrent archiving destinations that can be independently searched and exported on demand.

In this blog post, we discuss how Amazon WorkMail and Amazon Simple Email Service (SES) Mail Manager make email journaling easier to set up and use, more cost-effective and versatile. We’ll walk the reader through setting up email journaling for an Amazon WorkMail organization that uses SES Mail Manager’s routing, processing, and archiving features.

SES Mail Manager as Journaling Destination for WorkMail

For our purposes, we’ll assume you’ve already set up WorkMail as your mailbox provider, but the process described below will work with the journaling features of most 3rd party email solutions. If you want to explore Amazon WorkMail, visit the getting started documentation here.

In the following sections, we’ll describe how to configure WorkMail journaling to send full email journals to SES Mail Manager’s archives. We’ll define different retention periods for each archive to demonstrate how this solution can be used to meet both short and long-term retention requirements. Finally, we’ll use the AWS SES Mail Manager console to search, export, and manage the email journals and archives.

In our examples, we’ll use Amazon Route 53 to create a new domain called ‘journaling.solutions’ which we’ll configure to send all ‘@journaling.solutions’ emails to an SES Mail Manager Ingest endpoint. To begin, open the AWS Console, navigate to your WorkMail Organization’s settings, and click on the Journaling tab:

Organization settings Journaling tab

Organization settings Journaling tab

Click Edit, enable journaling, and provide a journaling email address (we’re using ‘[email protected]’) to receive journaled content. Provide a report email address, such as the admin email list, to receive journaling reports:

Provide a Journaling email address

Provide a Journaling email address

Open the AWS SES console in a new browser window, and navigate to Mail Manager’s Rule sets. Create a new rule set called ‘journaling-rule-demo’. Click Edit and create a new rule called “journal-all”, with an Archive action. Click the create an archive button and create an archive called ‘journaling-archive-demo’:

Create a new Rule Set called ‘Journaling-rule-demo’

Create a new Rule Set called ‘Journaling-rule-demo’

When creating Mail Manager archives, you have options to set the retention period from 3 months to permanent storage. You can also choose to encrypt your archived messages with your own KMS key. The configuration in our example is for permanent storage and shows the optional text field for using your own KMS key:

you can encrypt the archived messages with your own KMS key

you can encrypt the archived messages with your own KMS key

Traditional journaling calls for recording every email message to the journal, so for our ‘journal-all’ rule, we will not define filtering behaviors in the rule set. This will instruct Mail manager to send all emails for [email protected] to the journaling-archive-demo archive. It is worth noting that Mail Manager’s rule set can be configured to filter and independently process multiple recipient addresses. Consult the documentation to learn about other ways to customize Mail Manager for your use cases.

Next, create a new traffic policy, called journaling-traffic-demo, and configure it to reject any message not explicitly sent to the journaling destination address ([email protected]):

Create a new Traffic policy, called ‘Journaling-traffic-demo’

Create a new Traffic policy, called ‘Journaling-traffic-demo’

Create an open ingress endpoint called ‘journaling-demo-IG’, and select the ‘journaling-traffic-demo’ traffic policy and ‘journaling-rule-demo’ rule set:

Create an Open Ingress endpoint called ‘Journaling-demo-IG’,

Create an Open Ingress endpoint called ‘Journaling-demo-IG’,

After you press the create Ingest endpoint button, Mail Manager will create an Ingress endpoint and assign it a DNS A Record to be used in your DNS configurations to route email to Mail Manager:

Mail Manager Ingress endpoint DNS A Record to be used in your DNS configurations

Mail Manager Ingress endpoint DNS A Record to be used in your DNS configurations

From the General details page of the Ingress endpoint, copy the Ingress endpoint’s DNS A Record to your clipboard. Open a new browser window to your DNS provider’s MX configuration page (in our example below, we’re using AWS Route53). Edit the MX record for ‘journaling.solutions’ by pasting the Ingress endpoint A record. This configuration will route email sent to any address ‘@journaling.solutions’ to the Mail Manager’s Ingress endpoint for processing by the Traffic policy and Rule set:

Using AWS Route53 to edit MX record for ‘journaling.solutions’ to Ingress endpoint A record

Using AWS Route53 to edit MX record for ‘journaling.solutions’ to Ingress endpoint A record

To test your new journaling configuration, send several emails to several email addresses in your WorkMail organization (or the alternative inbox provider you configured in the first step). WorkMail (or your alternative inbox provider) will send a full record of all emails to the journaling destination address ([email protected]).

Wait a few minutes after sending the emails above, then open the AWS Mail Manager console’s archiving controls and search for messages sent in the last 12 hours:

AWS Mail Manager console’s archiving controls

AWS Mail Manager console’s archiving controls

The example above shows a search for all messages received in the “last 12 hours”, with no other filters specified. The results show every message inserted into the archive in this timeframe. You’ll see one entry where the from address is different (from toby@tegwj@…). This is an example of mail that was sent directly to the journaling destination address ([email protected]). This works because our traffic policy and rule set configurations don’t include any filters.

A cost effective solution at scale

Using Mail Manager as a journaling solution gives you more direct control over your costs than typical journaling services. While most journaling services in the market today charge a fixed rate per journaled mailbox, Mail Manager pricing is comprised of a monthly fixed fee per ingestion endpoint and consumption pricing for basic message handling, and the amount of data archived.

For example, imagine your organization has 250 mailboxes, each handling 50 messages per day. On a monthly basis this amounts to 375,000 messages. If we assume each message is 40 kilobytes in size, your organization is generating roughly 15 gigabytes of email per month. As you can see from the table below, the total cost in month 1 is about $140, or $0.56/mailbox.

|Item |Unit Price |Volume |Subtotal/Mo |
|— |— |— |— |
|Ingress Endpoint |$50/mo |1 |$50 |
|Core message processing |$0.15/1000 msgs |375 |$56.25 |
|Archive insertion/indexing |$2/GB (one-time) |15 |$30 |
|Archive storage |$0.19/GB/mo |15 |$2.85 |
|Subtotal: | | |$139.10 |
| |Monthly price per mailbox |$0.56 |

If the proposed email rate in our assumptions stays constant, the Mail Manager archive will grow by 15 gigabytes each month. After 36 months, the total monthly storage cost increases to $102.60. This results in a total monthly spend in month 36 of $238.85, or $0.96/mailbox/month.

Conclusion

In this blog post, we’ve explored how Amazon WorkMail and Amazon SES Mail Manager can provide a cost-effective and accessible solution for email journaling. By leveraging the flexible journaling capabilities of WorkMail and the archiving features of SES Mail Manager, organizations can easily satisfy rigorous compliance requirements around email retention and accessibility.

The combination of WorkMail’s journaling controls and SES Mail Manager’s rule-based archiving allows you to tailor your journaling solution to your specific needs. Whether you require short-term retention for audits or long-term preservation for legal and regulatory purposes, SES Mail Manager’s flexible archiving options have you covered with predictable and transparent costs that scale with your organization’s email volume.

If you’re looking for a modern, scalable, and cost-effective solution for your email journaling needs, we encourage you to explore the capabilities of Amazon SES Mail Manager. Get started today by visiting the AWS documentation and begin streamlining your email compliance and retention processes.

About the Authors

Toby Weir-Jones

Toby Weir-Jones

Toby is a Principal Product Manager for Amazon SES and WorkMail. He joined AWS in January 2021 and has significant experience in both business and consumer information security products and services. His focus on email solutions at SES is all about tackling a product that everyone uses and finding ways to bring innovation and improved performance to one of the most ubiquitous IT tools.

Zip

Zip

Zip is a Sr. Specialist Solutions Architect at AWS, working with Amazon Pinpoint and Simple Email Service and WorkMail. Outside of work he enjoys time with his family, cooking, mountain biking, boating, learning and beach plogging.

Andy Wong

Andy Wong

Andy Wong is a Sr. Product Manager with the Amazon WorkMail team. He has 10 years of diverse experience in supporting enterprise customers and scaling start-up companies across different industries. Andy’s favorite activities outside of technology are soccer, tennis and free-diving.

Bruno Giorgini

Bruno Giorgini

Bruno Giorgini is a Senior Solutions Architect specializing in Pinpoint and SES. With over two decades of experience in the IT industry, Bruno has been dedicated to assisting customers of all sizes in achieving their objectives. When he is not crafting innovative solutions for clients, Bruno enjoys spending quality time with his wife and son, exploring the scenic hiking trails around the SF Bay Area.

How we improved translation experience with cost efficiency

Post Syndicated from Grab Tech original https://engineering.grab.com/improved-translation-experience-with-cost-efficiency

Introduction

As COVID restrictions were fully lifted in 2023, the number of tourists grew dramatically. People began to explore the world again, frequently using the Grab app to make bookings outside of their home country. However, we noticed that communication posed a challenge for some users. Despite our efforts to integrate an auto-translation feature in the booking chat, we received feedback about occasional missed or inaccurate translations. You can refer to this blog for a better understanding of Grab’s chat system.

An example of a bad translation. The correct translation is: ‘ok sir’.

In an effort to enhance the user experience for travellers using the Grab app, we formed an engineering squad to tackle this problem. The objectives are as follows:

  • Ensure translation is provided when it’s needed.
  • Improve the quality of translation.
  • Maintain the cost of this service within a reasonable range.

Ensure translation is provided when it’s needed

Originally, we relied on users’ device language settings to determine if translation is needed. For example, if both the passenger and driver’s language setting is set to English, translation is not needed. Interestingly, it turned out that the device language setting did not reliably indicate the language in which a user would send their messages. There were numerous cases where despite having their device language set to English, drivers sent messages in another language.

Therefore, we needed to detect the language of user messages on the fly to make sure we trigger translation when it’s needed.

Language detection

Simple as it may seem, language detection is not that straightforward a task. We were unable to find an open-source language detector library that covered all Southeast Asian languages. We looked for Golang libraries as our service was written in Golang. The closest we could find were the following:

  • Whatlang: unable to detect Malay
  • Lingua: unable to detect Burmese and Khmer

We decided to choose Lingua over Whatlang as the base detector due to the following factors:

  • Overall higher accuracy.
  • Capability to provide detection confidence level.
  • We have more users using Malay than those using Burmese or Khmer.

When a translation request comes in, our first step is to use Lingua for language detection. If the detection confidence level falls below a predefined threshold, we fall back to call the third-party translation service as it can detect all Southeast Asian languages.

You may ask, why don’t we simply use the third-party service in the first place. It’s because:

  • The third-party service only has a translate API that also does language detection, but it does not provide a standalone language detection API.
  • Using the translate API is costly, so we need to avoid calling it when it’s unnecessary. We will cover more on this in a later section.

Another challenge we’ve encountered is the difficulty of distinguishing between Malay and Indonesian languages due to their strong similarities and shared vocabulary. The identical text might convey different meanings in these two languages, which the third-party translation service struggles to accurately detect and translate.

Differentiating Malay and Indonesian is a tough problem in general. However, in our case, the detection has a very specific context, and we can make use of the context to enhance our detection accuracy.

Making use of translation context

All our translations are for the messages sent in the context of a booking or order, predominantly between passenger and driver. There are two simple facts that can aid in our language detection:

  • Booking/order happens in one single country.
  • Drivers are almost always local to that country.

So, for a booking that happens in an Indonesian city, if the driver’s message is detected as Malay, it’s highly likely that the message is actually in Bahasa Indonesia.

Improve quality of translation

Initially, we were entirely dependent on a third-party service for translating our chat messages. While overall powerful, the third-party service is not perfect, and it does generate weird translations from time to time.

An example of a weird translation from a third-party service recorded on 19 Dec 2023.

Then, it came to us that we might be able to build an in-house translation model that could translate chat messages better than the third-party service. The reasons being:

  • The scope of our chat content is highly specific. All the chats are related to bookings or orders. There would not be conversations about life or work in the chat. Maybe a small Machine Learning (ML) model would suffice to do the job.
  • The third-party service is a general translation service. It doesn’t know the context of our messages. We, however, know the whole context. Having the right context gives us a great edge on generating the right translation.

Training steps

To create our own translation model, we took the following steps:

  • Perform topic modelling on Grab chat conversations.
  • Worked with the localisation team to create a benchmark set of translations.
  • Measured existing translation solutions against benchmarks.
  • Used an open source Large Language Model (LLM) to produce synthetic training data.
  • Used synthetic data to train our lightweight translation model.

Topic modelling

In this step, our aim was to generate a dataset which is both representative of the chat messages sent by our users and diverse enough to capture all of the nuances of the conversations. To achieve this, we took a stratified sampling approach. This involved a random sample of past chat conversation messages stratified by various topics to ensure a comprehensive and balanced representation.

Developing a benchmark

For this step we engaged Grab’s localisation team to create a benchmark for translations. The intention behind this step wasn’t to create enough translation examples to fully train or even finetune a model, but rather, it was to act as a benchmark for translation quality, and also as a set of few-shot learning examples for when we generate our synthetic data.

This second point was critical! Although LLMs can generate good quality translations, LLMs are highly susceptible to their training examples. Thus, by using a set of handcrafted translation examples, we hoped to produce a set of examples that would teach the model the exact style, level of formality, and correct tone for the context in which we plan to deploy the final model.

Benchmarking

From a theoretical perspective there are two ways that one can measure the performance of a machine translation system. The first is through the computation of some sort of translation quality score such as a BLEU or CHRF++ score. The second method is via subjective evaluation. For example, you could give each translation a score from 1 to 5 or pit two translations against each other and ask someone to assess which they prefer.

Both methods have their relative strengths and weaknesses. The advantage of a subjective method is that it corresponds better with what we want, a high quality translation experience for our users. The disadvantage of this method is that it is quite laborious. The opposite is true for the computed translation quality scores, that is to say that they correspond less well to a human’s subjective experience of our translation quality, but that they are easier and faster to compute.

To overcome the inherent limitations of each method, we decided to do the following:

  1. Set a benchmark score for the translation quality of various translation services using a CHRF++ score.
  2. Train our model until its CHRF++ score is significantly better than the benchmark score.
  3. Perform a manual A/B test between the newly trained model and the existing translation service.

Synthetic data generation

To generate the training data needed to create our model, we had to rely on an open source LLM to generate the synthetic translation data. For this task, we spent considerable effort looking for a model which had both a large enough parameter count to ensure high quality outputs, but also a model which had the correct tokenizer to handle the diverse sets of languages which Grab’s customers speak. This is particularly important for languages which use non-standard character sets such as Vietnamese and Thai. We settled on using a public model from Hugging Face for this task.

We then used a subset of the previously mentioned benchmark translations to input as few-shot learning examples to our prompt. After many rounds of iteration, we were able to generate translations which were superior to the benchmark CHRF++ scores which we had attained in the previous section.

Model fine tuning

We now had one last step before we had something that was production ready! Although we had successfully engineered a prompt capable of generating high quality translations from the public Hugging Face model, there was no way we’d be able to deploy such a model. The model was far too big for us to deploy it in a cost efficient manner and within an acceptable latency. Our solution to this was to fine-tune a smaller bespoke model using the synthetic training data which was derived from the larger model.

These models were language specific (e.g. English to Indonesian) and built solely for the purpose of language translation. They are 99% smaller than the public model. With approximately 10 Million synthetic training examples, we were able to achieve performance which was 98% as effective as our larger model.

We deployed our model and ran several A/B tests with it. Our model performed pretty well overall, but we noticed a critical problem: sometimes, numbers got mutated in the translation. These numbers can be part of an address, phone number, price etc. Showing the wrong number in a translation can cause great confusion to the users. Unfortunately, an ML model’s output can never be fully controlled; therefore, we added an additional layer of programmatic check to mitigate this issue.

Post-translation quality check

Our goal is to ensure non-translatable content such as numbers, special symbols, and emojis in the original message doesn’t get mutated in the translation produced by our in-house model. We extract all the non-translatable content from the original message, count the occurrences of each, and then try to match the same in the translation. If it fails to match, we discard the in-house translation and fall back to using the third-party translation service.

Keep cost low

At Grab, we try to be as cost efficient as possible in all aspects. In the case of translation, we tried to minimise cost by avoiding unnecessary on-the-fly translations.

As you would have guessed, the first thing we did was to implement caching. A cache layer is placed before both the in-house translation model and the third-party translation. We try to serve translation from the cache first before hitting the underlying translation service. However, given that translation requests are in free text and can be quite dynamic, the impact of caching is limited. There’s more we need to do.

For context, in a booking chat, other than the users, Grab’s internal services can also send messages to the chat room. These messages are called system messages. For example,our food service always sends a message with information on the food order when an order is confirmed.

System messages are all fairly static in nature, however, we saw a very high amount of translation cost attributed to system messages. Taking a deeper look, we noticed the following:

  • Many system messages were not sent in the recipient’s language, thus requiring on-the-fly translation.
  • Many system messages, though having the same static structure, contain quite a few variants such as passenger’s name and food order item name. This makes it challenging to utilise our translation cache effectively as each message is different.

Since all system messages are manually prepared, we should be able to get them all manually translated into all the required languages, and avoid on-the-fly translations altogether.

Therefore, we launched an internal campaign, mandating all internal services that send system messages to chat rooms to get manual translations prepared, and pass in the translated contents. This alone helped us save roughly US$255K a year!

Next steps

At Grab, we firmly believe that our proprietary in-house translation models are not only more cost-effective but cater more accurately to our unique use cases compared to third-party services. We will focus on expanding these models to more languages and countries across our operating regions.

Additionally, we are exploring opportunities to apply learnings of our chat translations to other Grab content. This strategy aims to guarantee a seamless language experience for our rapidly expanding user base, especially travellers. We are enthusiastically looking forward to the opportunities this journey brings!

Join us

Grab is the leading superapp platform in Southeast Asia, providing everyday services that matter to consumers. More than just a ride-hailing and food delivery app, Grab offers a wide range of on-demand services in the region, including mobility, food, package and grocery delivery services, mobile payments, and financial services across 428 cities in eight countries.

Powered by technology and driven by heart, our mission is to drive Southeast Asia forward by creating economic empowerment for everyone. If this mission speaks to you, join our team today!

Introducing quorum queues on Amazon MQ for RabbitMQ

Post Syndicated from Chris Munns original https://aws.amazon.com/blogs/compute/introducing-quorum-queues-on-amazon-mq-for-rabbitmq/

This post is written by Vignesh Selvam (Senior Product Manager – Amazon MQ), Simon Unge (Senior software development engineer – Amazon MQ).

Amazon MQ for RabbitMQ announced support for quorum queues, a type of replicated queue designed for higher availability and data safety. This post presents an overview of this queue type, describes when you should use it, and best practices you can follow. The post also describes how Amazon MQ has also improved quorum queues in the open-source RabbitMQ community.

Overview of quorum queues

A quorum queue is a replicated first in, first out queue type offered by open-source RabbitMQ that uses the Raft consensus algorithm to maintain data consistency. Each quorum queue has a leader and multiple followers (replicas), which ensure that messages are replicated and persisted across a majority of nodes, thus providing resilience against node failures. Quorum queues only need a majority of member nodes (a quorum) to make decisions about data. If a RabbitMQ node hosting a leader becomes unavailable, another node hosting one of the followers is automatically elected as the leader. Once the node becomes available again, the node will become a follower for the quorum queue and catch up or synchronize with the new leader. Quorum queues can detect network failures faster and recover quicker than classic mirrored queues, thus improving the resiliency of the message broker as a whole.

Quorum queues share most of the fundamental features that are key to RabbitMQ replicated queue types such as consumption, consumer acknowledgements, cancelling consumers, purging and deletion. Poison message handling is a unique feature of quorum queues which help developers manage unprocessed messages more efficiently. A poison message is a message that cannot be processed and ends up being repeatedly requeued. Quorum queues keep track of the number of unsuccessful delivery attempts and expose it in the ‘x-delivery-count’ header that is included with any redelivered message. A delivery limit can be set using a policy argument for ’delivery-limit’. If the limit is reached, the message can be dropped or put in a dead-letter queue. This feature further improves the data reliability of a quorum queue.

You can get started with quorum queues by explicitly specifying the ‘x-queue-type’ parameter as ’quorum’ on a RabbitMQ broker running version 3.13 and above. We recommend that you change the default vhost queue type to ’quorum’ to ensure that all queues are created as quorum queues by default inside a vhost.

RabbitMQ queues console

RabbitMQ queues console

When should you use quorum queues?

You should use quorum queues when you need higher availability and consistency for their messaging infrastructure. Quorum queues are ideal for scenarios where data durability and fault tolerance are critical, such as financial transaction systems, e-commerce data processing systems, or any application requiring high reliability. They are particularly beneficial in environments where node failures are more likely or where maintaining data consistency across distributed systems is essential.

When should you NOT use quorum queues?

Quorum queues are not meant to be temporary. They do not support transient or exclusive queues and are not meant to be used in scenarios with high queue churn (declaration and deletion rates). They are also not recommended for unreplicated queues.

Best practices for quorum queues

Quorum queues perform better when the queues are short. You can set the maximum queue length using a policy or queue arguments to limit the total memory usage by queues (max-length, max-length-bytes).

Add a new queue dialog

Add a new queue dialog

Amazon MQ recommends publishers to use publisher confirms and consumers to use manual acknowledgements on quorum queues. Publisher confirms will only be issued once a published message has been successfully replicated to a quorum of nodes and is considered safe within the context of the system. Publisher confirms can also serve as a form of back pressure and protect the availability of the broker during periods of high workload. Manual acknowledgements are used to ensure messages that are not processed can be returned to the queue for reprocessing.

Open-source improvements by Amazon MQ

Amazon MQ contributed multiple improvements to the open-source RabbitMQ community to improve quorum queues for operators and users.

Automatic membership reconciliation
Quorum queues depend on a majority of replicas being available for the Raft consensus algorithm. Amazon MQ identified that many users and operators would prefer to maintain a certain minimal number of replicas (generally 3 or 5) at all times to ensure a majority always exists. The quorum queue replica management was also initially available only via CLI tools. Amazon MQ engineers introduced automatic membership reconciliation to improve this experience. Now, RabbitMQ can be configured to identify any queues that are below a target group member size, and automatically grow or add a node to the queue members. Thus ensuring a certain minimum number of replicas always exist.

Voter status
RabbitMQ considers a quorum queue member node to be a full member even if the member has not caught up or fully synced to the quorum. The CLI command rabbitmq-queues check_if_node_is_ quorum_critical can provide a false positive, and indicate a node is safe to remove, even though another node has queue members that are still synchronizing to the quorum. Amazon MQ introduced a new ‘non-voter’ state for a queue member node to indicate a member that is still catching up or synchronizing to the quorum. If a queue has a member in this state, it is not considered a full member. Once the member is fully synchronized, it is automatically promoted to the voter status, and is considered a full member. The command rabbitmq-queues check_if_node_is_quorum_critical now takes this into account and correctly reports if a node can be safely terminated without any queues becoming unavailable due to a loss of majority.

Inconsistent state management
When a broker is overloaded, a quorum queue can end up in an inconsistent state, where the quorum queue membership state stored in the Raft state machine differs from the RabbitMQ internal state for the queue. Amazon MQ introduced a periodic check per quorum queue that identifies if a queue has an inconsistent state and takes action to fix it.

Default queue type
The default queue type for a RabbitMQ broker vhost was classic queues. You could declare a different queue type by explicitly stating the ’x-queue-type’ as a queue creation argument. Amazon MQ introduced a global default queue type in the configuration file (rabbit.conf) that provides the ability to define a default queue type at the broker level. Now, an operator can change the default queue type to quorum queues if not specified during creation.

Membership management permissions
RabbitMQ users are able to configure the quorum queue membership using the management API. This can interfere with automatic membership reconciliation. Amazon MQ introduced the ability for an operator to turn off the membership management permissions available through the management API. Thus, preventing customers from accidentally affecting their broker.

Conclusion

Quorum queues on RabbitMQ provide a robust solution for scenarios requiring high availability and resilience. By leveraging the Raft consensus protocol, quorum queues ensure that messages are safely stored and replicated across a quorum of nodes, making them an excellent choice for modern, distributed message queuing systems.

Amazon MQ recommends that you adopt quorum queues as the preferred replicated queue type on RabbitMQ 3.13 brokers. For more details, see Amazon MQ documentation. To know more about the open-source feature, see quorum queues.

Get started with quorum queues on Amazon MQ for RabbitMQ 3.13 with a few clicks.

Reaching your global players – message on a single, scalable communication hub

Post Syndicated from Richard Perez original https://aws.amazon.com/blogs/messaging-and-targeting/reaching-your-global-players-message-on-a-single-scalable-communication-hub/

Introduction

AWS provides solutions to address industry challenges, enabling customers to address the risk of potentially significant financial and reputational damage. This blog discusses how customers that have adopted Amazon Pinpoint have been able to deliver time-sensitive messages reliably and securely. Specifically, it can mitigate the risk associated with reaching players who have voluntarily self-excluded from marketing communications.

Regulated workloads in the betting and gaming industry refer to specific activities, systems, and processes that are subject to government regulations, oversight, and compliance requirements. The role of the gambling regulator is to ensure adherence to these requirements and maintain fairness, security, and responsible behavior within the industry. Regulators also exist to protect players, subject to their jurisdiction, by regulating the businesses that provide gambling services, imposing controls for operators, the Internet Software Vendor’s (ISVs) and players themselves. By way of example, the UK Gambling Commission (GC) is one such organization with the responsibility of regulating the individuals and businesses that provide gambling services in Great Britain. The UK GC and their counterparts across the globe regulate and license betting and gaming businesses to ensure they adhere to standards, and investigate and prosecute those that breach regulations and their conditions of licensure.

Player engagement

Player engagement in the Betting and Gaming industry refers to the level of involvement, enjoyment, and interaction that individuals experience when participating in various gambling activities. It encompasses aspects like personalized gaming experiences, social interactions, rewards, and challenges. Maintaining a strong relationship with players is vital for operators as it fosters loyalty, increases player retention, and drives engagement. Engaged players are more likely to return, deposit, and recommend the brand to others. Additionally, a positive operator-regulator relationship that promotes responsible gambling and regulatory compliance creates a trustworthy reputation, a key component for an operator’s long-term success in the industry. Player expectations and betting options are constantly changing and require operators to develop more engaging and sophisticated games to maintain participation and loyalty. At their core, they typically include:

Responsible Gaming

Proactively preventing messages being delivered to players that have self-excluded, mitigating the risk of incurring fines, and sending an SMS message for the purpose of interrupting gambling behavior or sending an email with guidance or support.

Transactional history

Sending statements of fact about the transactions that have completed and providing information to the player about the bets placed, and suspicious account activity. Send Real-time notifications that can include game results or betting odd changes. These can be delivered in multi-channel formats, subject to the player preferences.

Player Engagement

Deliver personalized notifications that are tailored to the gaming experience of the player. This can include games suggestions based on player’s preference, offering customized bonuses, or providing recommendations for in-game actions or new games. After confirming that a player is not on an exclusion list and opted to receive messaging with you can send notifications with bonuses and promotions offering attractive incentives to maintain player engagement. These can include welcome bonuses, free game access, and loyalty programs.

Account settings & Security messaging

This can be in the form of SMS to provide seamless account creation, onboarding verification and deliver one-time-passwords to accelerate player identification. Access codes can be delivered where a unique code is delivered to the player in a time sensitive window to provide access to a particular game.

Responsible Gaming

Operators face several challenges when it comes to maintaining player engagement while adhering to regulatory mandated responsible gambling standards, requiring they balance engagement and retention strategies with player well-being and safe-play requirements. A common challenge faced by operators is ingesting multiple self-exclusion lists to ensure self-excluded users do not receive engagement and retention offers. Operators do employ various activities, such as educational initiatives, such as providing players with information about responsible gambling, offering responsive customer support to address player issues and queries promptly, and use machine learning to identify problematic gambling and take proactive actions. It is also worth noting that operators collaborate with problem gambling support organizations to provide resources and assistance to affected individuals.

Players can also take decisive action themselves and are free to opt-out of all gambling notifications for a set length of time, usually between 6 months and five years. The gambling operators in that jurisdiction are required to observe this request by law or face million-dollar fines and reputational damage. With increased player participation, ensuring you have the right tools and services in place in critical, as highlighted by an AWS customer.

Betting and Gaming has become one of the fastest growing sectors in the last 20 years. Everyday millions of people use these platforms for their betting and entertainment needs. tombola recognize the importance of having a reliable and scalable outbound communication hub for engaging with and protecting our players. Reliable communication is essential for a safe, engaging and sustainable customer experience.

James Conway, Technology Director, tombola

How to build Player Engagement solutions in the Betting and Gaming industry with AWS

AWS offers robust solutions that can benefit the Betting and Gaming industry. With the challenges discussed in this blog, AWS provides the infrastructure and solutions necessary to address these challenges effectively.

In a series of blogs, we will be sharing how AWS and Amazon Pinpoint can support industry focused use cases. Other solutions in this series are;

  • How you can use AIML and GenAI to create player recommendations to improve player engagement
  • How you use Amazon Pinpoint for improved onboarding and user authentication
  • Identify player trends and next best actions using Amazon Pinpoint and AIML

The first in this four-part series is how we are supporting customers with responsible gaming, and how they can mitigate the risk of contacting players that have chosen not to receive any more communication.

Amazon Pinpoint is a key service in this solution. Pinpoint is a purpose-built, customer engagement service with multi-channel capabilities, message delivery, management, and optimization. This allows AWS customers to send personalized messages, promotions, and notifications to users via various channels, through a single console. Customers view Amazon Pinpoint as their communication hub because they can consolidate the services and tools used for their outbound channels to just one. It becomes the central service for managing and facilitating communications between different stakeholders. It streamlines and creates efficiency for front and back-office teams as they are able to concentrate on the player’s experience. Customers are looking for an easy, optimized, and consolidated solution to manage their digital communications. Customers often have different services or tools for each communication channel, causing operational complexity that can lead to system issues, inefficiencies and breaches in regulatory obligations. Operators face issues when different teams and stakeholders have to sync between their customer data repository and these services and tools. Maintaining the integrity of these systems can be cumbersome and increase the risk in the event of a sync or systems error. By maintaining a single service for outbound communications and simplifying the customer data, you are able to innovate and deliver player centric solutions. Below we have provided a selection of use cases focusing on player engagement.

Responsible Gaming – Mitigate the Risk of Targeting Self-Excluded Players on AWS

Under the UK gaming regulations, operators are required to observe a self-exclusion list, managed by GAMSTOP. Amazon Pinpoint queries GAMSTOP’s API in real time before sending the message to a customer. The above is possible with the use of Amazon Pinpoint’s Campaign Hooks and Journey Hooks. Campaign and Journey hooks can be used for other use cases that require reaching out to a 3rd party service in real time, such as players’ balance check before sending a message. Gamstop has been used in this example but can apply to any self-exclusion site or an internal exclusion list.

Tombola, who operate the largest online bingo website in Europe, successfully adopted Pinpoint across all their territories said,

‘At Tombola, we needed a communication tool that helped us deliver on one of our core values: responsible gambling. Players can opt out of marketing communications from online gambling operators by registering themselves on an independent self-exclusion scheme. We needed a system that helped deliver consistency with evolving changing requirements and ensure, in real-time, we have the controls in place not to contact the wrong people. Amazon Pinpoint allowed us to consolidate the various communication systems we had deployed across territories and allowed us to continually evaluate the content and the recipients of our communication. Only Pinpoint’s system could keep up with our shifting requirements, so we can easily connect with the players who want to hear from us, but never contact those who had self-excluded.’

About the Authors

Richard Perez

Richard Perez

Richard Perez is a Principle Go-to-Market Specialist for Communication Developer Services at AWS. He enjoys diving deep into customers’ issues and helping to deliver communication solutions. In his spare time, he enjoys watching and participating in sports, and reading historical novels.