Tag Archives: AWS

How Amazon SES Mail Manager Elevates Email Security and Efficiency

Post Syndicated from Pavlos Ioannou Katidis original https://aws.amazon.com/blogs/messaging-and-targeting/how-amazon-ses-mail-manager-elevates-email-security-and-efficiency/

In today’s digital landscape, efficient and secure email management is essential for businesses facing the complexities of cyber threats and regulatory compliance. Companies are seeking ways to safeguard against unauthorized access and apply audit rules, while maintaining operational efficiency. Amazon SES Mail Manager is designed to meet these challenges, offering a suite of features that enhance both inbound and outbound email flows.

Mail Manager provides key components such as traffic policies for detailed email filtering, authenticated ingress endpoints that ensure emails are received only from verified senders, and customizable rule sets that enable administrators to precisely manage email traffic. These tools aim to bolster security and streamline the email management process.

The blog explores Mail Manager’s capabilities by demonstrating how each component works and can be utilized in practical business scenarios. Some common use cases include security, where Mail Manager blocks harmful emails based on IP ranges, TLS versions, and authentication checks while leveraging third-party security add-ons. Another use case is email archiving, where you can use Mail Manager to set up multiple archives with customizable retention periods and encryption, ensuring compliance and easy searchability.

Familiarize with some of mail manager’s key components below before proceeding with the customer use cases.

Mail manager components definition:

  • Ingress endpoints:
    • Open ingress endpoint: a SMTP endpoint responsible for accepting connections, and process SMTP conversation key infrastructure. It’s a key component that utilizes traffic polices and rules that you can configure to determine which emails should be allowed into your organization and which ones should be rejected.
    • Authenticated ingress endpoint: Mail sent to your domain has to come from authorized senders whom you’ve shared your SMTP credentials with, such as your on-premise email servers.
  • Traffic policies: let you determine the email you want to allow or block from your ingress endpoint. A traffic policy consists of one or more policy statements where you allow or deny traffic based on a variety of protocols including recipient address, sender IP address and TLS protocol version.
  • Rules sets: A Rule set is a container for an ordered set of rules you create to perform actions on your email. Each rule consists of conditions and rules.
  • Email add-ons: A suite of 3rd party applications that are seamlessly integrated with Amazon SES mail manager. Some of them are Trend Micro Virus Scanning, Abusix Mail intelligence and Spamhaus Domain Block List.

For a deep dive into Mail Manager’s capabilities, ready this blog.

Customer background and use case

Nutrition.co is an online retail business with multiple departments, including marketing, tech, and sales, that send and receive emails. Nutrition.co is looking for a solution to monitor both outbound and inbound emails and apply various controls such as filtering, message processing, and archiving. Nutrition.co uses Outlook as an enterprise mailbox environment for its employees.

Use case 1: Nutrition.co to the world

This use case focuses on the outbound email flow, where Nutrition.co employees are sending emails outside of Nutrition.co. Some of the requirements include the archival of all outbound emails originated by the marketing department, blocking any tech emails exceeding 1mb and scanning the email content of emails originated by sales. These controls should be centrally managed and provide flexibility to edit/create/delete new ones.

Solution: Each department will direct its outbound emails to an authenticated ingress endpoint by configuring an Exchange transport rule. These endpoints ensure that only authorized senders with SMTP credentials can send emails. Each ingress endpoint generates an A record, which is added as an MX record to the DNS provider for each department’s subdomain. Additionally, each ingress endpoint is associated with a specific traffic policy and rule set. According to Nutrition.co’s requirements, all connections between the departments and the ingress endpoints must use TLS 1.3 or higher. Emails that comply with the traffic policies are processed through distinct rule sets. Emails from marketing that comply with DKIM and SPF are first archived and then sent to the recipient via the Send to Internet action. Tech emails have their recipient’s address rewritten to a test email address, while emails from the sales department undergo content scanning before being sent to the final recipients via the Send to Internet action.

SES-Mail-Manager-Outbound

Use case 2: World to Nutrition.co

This use case focuses on the inbound email flow, where third parties send emails to Nutrition.co. Nutrition.co requires inbound emails to align with SPF and DKIM and have TLS 1.3 or higher to be archived. Emails originating from warehouse.com, Nutrition.co’s fulfilment partner, are containing customer order updates. These emails should be processed by Nutrition.co and accordingly update the customers’ order status database. Furthermore, warehouse.com emails should originate from a certain IP range, have TLS 1.3 or higher and align with SPF and DKIM.

Solution: Nutrition.co will use an open ingress endpoint without authentication for all inbound external emails. This is achieved by adding an MX record generated by Mail Manager upon the creation of the ingress endpoint. This ingress endpoint will be associated with a traffic policy that evaluates TLS. If the inbound email conforms to the traffic policy, it will proceed through the rule set condition and actions. The rule set condition is to align with SPF and DKIM and the actions are to be archived and then sent to the final recipient (Nutrition.co employee) via SMTP Relay. Emails containing parcel delivery updates from warehouse.com will be directed to a separate Nutrition.co subdomain, which routes all inbound emails to an authenticated ingress endpoint. Emails from warehouse.com with TLS 1.3 or higher will meet the traffic policy requirements. If they are SPF and DKIM aligned, they will be stored in a Nutrition.co Amazon S3 bucket as part of the rule set. Using Amazon S3 notifications, an AWS Lambda function is invoked upon receiving an email. This function processes the email payload, and performs an API call to update the Nutrition.co customers’ order status database.

SES-Mail-Manager-Inbound

Archiving inbound emails

In the following section, you will use AWS CloudShell and AWS CLI commands to create a traffic policy that rejects emails with TLS versions lower than 1.3, includes an open ingress endpoint, and establishes a ruleset that archives emails that are DKIM aligned.

Prerequisites: Own a domain and have access to its DNS provider, in order to add the MX record.

Navigate to the AWS Management Console and open CloudShell, find CloudShell availability here. Follow the steps below by copying and pasting the AWS CLI commands to the CloudShell terminal. Note that creating and configuring these resources, can also be done from the AWS Console.

# 1. Creating archive

ARCHIVE=$(aws mailmanager create-archive \
  --archive-name NutritionCo \
  --retention RetentionPeriod=THREE_MONTHS \
  --region ${AWS_REGION} \
  --tags Key=Company,Value=NutritionCo | jq -r '.ArchiveId') && echo $ARCHIVE

# 2. Creating traffic policy

TRAFFIC_POLICY=$(aws mailmanager --region ${AWS_REGION} create-traffic-policy \
  --traffic-policy-name ArchiveTrafficPolicy \
  --default-action DENY \
  --policy-statements '[
    {
      "Action": "ALLOW",
      "Conditions": [
        {
          "TlsExpression": {
            "Evaluate": {
              "Attribute": "TLS_PROTOCOL"
            },
            "Operator": "MINIMUM_TLS_VERSION",
            "Value": "TLS1_3"
          }
        }
      ]
    }
  ]'| jq -r '.TrafficPolicyId') && echo $TRAFFIC_POLICY

# 3. Creating Mailmanager RuleSet for archiving

RULE_SET=$(aws mailmanager --region ${AWS_REGION} create-rule-set \
  --rule-set-name ArchiveRuleSet \
  --rules '[
    {
      "Name": "Archive",
      "Actions": [
        {
          "Archive": {
            "TargetArchive": "'"${ARCHIVE}"'"
          }
        }
      ],
      "Conditions": [
        {
          "VerdictExpression": {
            "Evaluate": {
              "Attribute": "DKIM"
            },
            "Operator": "EQUALS",
            "Values": ["PASS"]
          }
        }
      ]
    }
  ]'| jq -r '.RuleSetId') && echo $RULE_SET

# 4. Create ingress endpoint

aws mailmanager --region ${AWS_REGION} create-ingress-point \
--ingress-point-name Archiving \
--type OPEN \
--traffic-policy-id ${TRAFFIC_POLICY} \
--rule-set-id ${RULE_SET}

To view the resources created above, navigate to the Amazon SES console > Mail Manager and view Traffic policies and Rule sets. Below, you can see the rule in edit mode.

Mail-Manager-RulesetNavigate to Amazon SES > Mail Manager > Ingress endpoint, select the ingress endpoint named Archiving and copy the ARecord, which looks like this <unique-id>.fips.wmjb.mail-manager-smtp.amazonaws.com – see screenshot below. Add this value to your MX record.

Mail-Manager-IngressEndpoint

To test if the MX record has been added successfully, open your local terminal and execute the command below:
nslookup -type=MX <your-domain.com>
The response should return the MX preference and mail exchanger containing the A record value.

Testing

To test if the inbound emails are archived successfully, send an email to an address within the domain for which you have added the MX record. Wait for 3-5 minutes to allow for email processing. Then, navigate to the AWS Management Console, go to Amazon SES, and select Mail Manager. Under Email Archiving, select NutritionCo under Archive and click on Search. This should return all the emails you have sent.

MailManager-Archive

Conclusion & Next steps

In this blog, we delved into the essential features of Amazon SES Mail Manager and its application in managing both inbound and outbound email flows. We explored key components such as traffic policies, authenticated ingress endpoints, and customizable rule sets that enhance security and operational efficiency. Through practical use cases, this blog demonstrates how these features can be implemented to meet the specific needs of a business like Nutrition.co. By leveraging Amazon SES Mail Manager, businesses can significantly enhance their email security and management processes, safeguarding against cyber threats while ensuring compliance and efficiency.

Continue exploring Mail Manager’s features such as SMTP relays and Email add-ons.

About the Authors

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis is an Amazon Pinpoint and Amazon Simple Email Service Senior Specialist Solutions Architect at AWS. He enjoys diving deep into customers’ technical issues and help in designing communication solutions. In his spare time, he enjoys playing tennis, watching crime TV series, playing FPS PC games, and coding personal projects.

Alexey Kiselev

Alexey Kiselev

Alexey Kiselev is a Senior SDE working on Amazon Email. Alexey has played a pivotal role in shaping the design, infrastructure, and delivery of MailManager. With years of experience, deep understanding of the industry and a passion for innovation he is enthusiast and a builder with a core area of interest on scalable and cost-effective email management and email security solutions.

Lower Your Risk of SMS Fraud with Country Level Blocking and Amazon Pinpoint

Post Syndicated from Brett Ezell original https://aws.amazon.com/blogs/messaging-and-targeting/lower-your-risk-of-sms-fraud-with-country-level-blocking-and-amazon-pinpoint/

As a developer, marketing professional, or someone in the communications space, you’re likely familiar with the importance of SMS messaging in engaging customers and driving valuable interactions. However, you may have also encountered the growing challenge of artificially inflated traffic (AIT), also known as SMS pumping. A new report co-authored by
Enea revealed that AIT is so widespread within the SMS ecosystem that 19.8B-35.7B fraudulent messages were sent by bad actors in 2023, incurring substantial costs of over $1 billion. In this blog post, we’ll explore how you can use
Protect configurations, a powerful set of capabilities within
Amazon Pinpoint SMS, that provides granular control over which destination countries your SMS, MMS, and voice messages can be sent to.
<img decoding=” width=”1252″ height=”889″>

What is SMS Pumping, aka Artificially Inflated Traffic (AIT)?

AIT, or SMS pumping, is a type of fraud where bad actors use bots to generate large volumes of fake SMS traffic. These bots target businesses’ whose websites, apps, and other assets have forms or other mechanisms that trigger SMS being sent out. Common use cases for SMS such as one-time passwords (OTPs), app download links, promotion signups, etc. are all targets for these bad actors to “pump” SMS and send out fraudulent messages. The goal is to artificially inflate the number of SMS messages a business sends, resulting in increased costs and a negative impact on the sender’s reputation. In the realm of SMS-based artificially inflated traffic (AIT), the prevalent method for bad actors to profit involves colluding with parties in the SMS delivery chain to receive a portion of the revenue generated from each fraudulent message sent.

<img decoding=” width=”1280″ height=”720″>

AIT poses several challenges for businesses:

  1. Overspending: The fake SMS traffic generated by AIT bots results in businesses paying for messages that yield no actual results.

  2. Interrupted service: Large volumes of AIT can force businesses to temporarily halt SMS services, disrupting legitimate customer communications.

  3. Diverted focus: Dealing with AIT can shift businesses’ attention away from core operations and priorities.

  4. Reduced deliverability rates due to the messages never being interacted with and/or large volumes of SMS being sent quickly.

How does Protect mitigate AIT?

Amazon Pinpoint’s Protect feature allows you to control which countries you can send messages to. This is beneficial if your customers are located in specific countries.

With Protect, you can create a list of country rules that either allow or block messages to each destination country. These country rules can be applied to SMS, MMS, and voice messages sent from your AWS account. The Protect configurations you create enable precise control over which destination countries your messages can be sent to. This helps mitigate the impact of AIT by allowing you to tailor where you do or do not send.

Protect offers flexibility in how the country rules are applied. You can apply them at the account level, the configuration set level, or the individual message level. This enables you to customize your AIT mitigation strategy to best fit your business needs and messaging patterns.

By leveraging Protect within Amazon Pinpoint, you can help ensure the integrity and cost-effectiveness of your SMS, MMS, and voice communications.

Account-level Protect Configuration

The simplest approach is to create a Protect configuration and associate it as the account default. This means the allow/block rules defined in that configuration will be applied across all messages sent from your account, unless overridden. This is an effective option if you only need one set of country rules applied universally.

Configuration set-specific Protect configuration

You can associate a Protect configuration with one or more of your Pinpoint SMS configuration sets. This allows you to apply different country rules to distinct messaging flows or use cases within your application without changing your existing code if you are already using Config Sets. It also enables more detailed logging and monitoring of the Protect configuration’s impact, such as:

  1. Error Logs: Logging of any errors or issues encountered when messages are sent, providing insights into how the Protect configuration is affecting message delivery.
  2. Audit Logs: Records of all configuration changes, access attempts, and other relevant activities related to the Protect configuration, allowing for comprehensive auditing and monitoring.
  3. Usage Metrics: Tracking of usage statistics, such as the number of messages sent to different countries, the impact of the Protect configuration on message volumes, and other usage-related data.
  4. Compliance and Policy Enforcement Logs: Documentation of how the Protect configuration is enforcing compliance with messaging policies and regulations, including any instances where messages are blocked or allowed based on the configuration rules.

Dynamic Protect configuration per message

If your needs are even more specific, you can create a Protect configuration without any association, and then specify its ID when sending messages via the Pinpoint APIs (e.g. SendMediaMessage, SendTextMessage, SendVoiceMessage). This gives you the ability to dynamically choose the Protect configuration to apply for each individual message, providing the ultimate flexibility.

Regardless of the approach, the core benefit of Protect configurations is the ability to precisely control which destination countries your messages may be sent to. Blocking countries where you don’t have a presence or where SMS pricing is high eliminates your exposure to fraudulent AIT traffic originating from those regions. This helps protect your messaging budget, maintain service continuity, and focus your efforts on legitimate customer interactions.

Who should use Protect configurations?

Protect configurations are designed to benefit a wide range of AWS customers, particularly those who:

  1. Send SMS messages to a limited number of countries: If your business primarily operates in a few specific countries, Protect configurations can help you easily block SMS messages to countries where you don’t have a presence, reducing the risk of AIT.
  2. Have experienced AIT issues in the past: If your business has been a target of SMS pumping, Protect configurations can help you regain control over your SMS communications and prevent future AIT attacks.
  3. Want to proactively protect their SMS messaging: Even if you haven’t encountered AIT issues yet, Protect configurations can help you stay ahead of the curve and maintain the integrity of your SMS communications.

How to create a country rules list with Protect configuration

To begin building a list of country rules that allow or block messages to specific destination countries, you start by creating a new Protect configuration. There are two ways to accomplish this, either by using the console, or the API.

Option 1 – Using the AWS Console

Console Scenario: My account is out of the sandbox and I only want to send to 1 country – United Kingdom (iso:GB) using the SenderID “DEMOTP”.

At a high level, we will follow the three steps outlined below for each method. In our examples, we used a SenderID as our Originator. However, it should be noted that the same process can be achieved using any originator you’d like. i.e. SenderID, Phone pool, Phone number, 10DLC, short code, etc.

  1. Create SenderID (Optional if you already have one)
  2. Create Protect Configuration
  3. Send Test Message via console

Using the AWS Console

1) Create SenderID for United Kingdom (GB)

  • Pinpoint SMS Console – Request Originator
    • Select United Kingdom (GB) and follow the prompts for a SenderID. DO NOT select Two-way SMS Messaging
    • Enter Sender ID – Example: DEMOTP
    • Confirm and Request

2) Create default Protect Configuration

<img decoding=” width=”863″ height=”521″>

    • Search for Country=United Kingdom then deselect United Kingdom

<img decoding=” width=”865″ height=”582″>

    • Set as Account Default and select Create protect configuration

<img decoding=” width=”1497″ height=”1173″>

3) Send a test message with SMS simulator

Note: The Pinpoint SMS Simulator provides special phone numbers you can use to send test text messages and receive realistic event records, all within the confines of the Amazon Pinpoint service. These simulator phone numbers are designed to stay entirely within the Pinpoint SMS ecosystem, ensuring your test messages don’t get sent over the carrier network.

You can use these simulator phone numbers to send both SMS and MMS messages, allowing you to thoroughly validate your message content, workflow, and event handling. The responses you receive back will mimic either success or fail depending on which destination simulator number you send to.

  • From the Pinpoint SMS Console SMS Simulator page,
    • For Originator, Choose Sender ID, and select your Sender ID created from earlier.
    • Under Destination number, select Simulator numbers and choose United Kingdom (GB). Enter a test message in the Message body.
    • Finally, choose send test message. This should prompt a green “Success” banner at the top of your page.

<img decoding=” width=”1336″ height=”1313″>

    • Conversely, follow the previous test message steps, and instead attempt to send to anywhere other than the United Kingdom (GB). In this example, Australia (AU) 
    • As shown below in the screenshot this one is blocked since you have configured to only send to GB.

<img decoding=” width=”1333″ height=”1364″>

Option 2 – Using the V2 API and CLI

V2 API Scenario: 
My account is out of the sandbox and I want to BLOCK only 1 country – Australia (AU) while using the SenderID “DEMOTP”.

1) Create SenderID for GB

Note: before using the CLI remember to configure your access and secret key using

aws configure

Windows users should use PowerShell over cmd to test

  • Use RequestSenderId to create the same Sender Id as previously made via the console.
aws pinpoint-sms-voice-v2 request-sender-id --iso-country-code "GB" --sender-id "DEMOTP"

Response:

{
   "DeletionProtectionEnabled": False,
   "IsoCountryCode": "GB",
   "MessageTypes": [ "TRANSACTIONAL" ],
   "MonthlyLeasingPrice": "0.00",
   "Registered": False,
   "SenderId": "DEMOTP",
   "SenderIdArn": "string"
}

2) Create default Protect Configuration

aws pinpoint-sms-voice-v2 create-protect-configuration --tags Key=Name,Value=CLITESTING

Response:

{
   "AccountDefault": False,
   "CreatedTimestamp": number,
   "DeletionProtectionEnabled": False,
   "ProtectConfigurationArn": "string",
   "ProtectConfigurationId":  "string",
   "Tags": [ 
      { 
         "Key": "Name",
         "Value": "CLITESTING"
      }
   ]
}

  • Add AU as BLOCKED on protect configuration.
aws pinpoint-sms-voice-v2 update-protect-configuration-country-rule-set --protect-configuration-id protect-string --number-capability 'SMS' --country-rule-set-updates '{\"AU\":{\"ProtectStatus\":\"BLOCK\"}}'

Response:

{
   "CountryRuleSet": { 
      "string" : { 
         "ProtectStatus": "ALLOW | BLOCK"
      }
   },
   "NumberCapability": "SMS",
   "ProtectConfigurationArn": "string",
   "ProtectConfigurationId": "string"
}

  • Set as account default configuration.
aws pinpoint-sms-voice-v2 set-account-default-protect-configuration --protect-configuration-id protect-string

Response:

{
   "DefaultProtectConfigurationArn": "string",
   "DefaultProtectConfigurationId": "string"
}

3) Send test message

  • Use SendTextMessage to test your Protect Configuration via the V2 API.
  • Test sending to GB Simulator Number should be successful.
aws pinpoint-sms-voice-v2 send-text-message --destination-phone-number "string" --message-body "string"

Response:

{
   "MessageId": "string"
}

  • Test sending to AU Simulator Number should be blocked.
aws pinpoint-sms-voice-v2 send-text-message --destination-phone-number "string" --message-body "string"

Response – (ConflictException):

{
An error occurred (ConflictException) when calling the 
SendTextMessage operation: Conflict Occurred - 
Reason="DESTINATION_COUNTRY_BLOCKED_BY_PROTECT_CONFIGURATION" ResourceType="protect-configuration" ResourceId="string"
}

Conclusion

As SMS messaging continues to play a crucial role in customer engagement and authentication, protecting your communications from AIT is more important than ever. Amazon Pinpoint Protect provides a powerful and user-friendly solution to help you mitigate the impact of SMS pumping, ensuring the integrity of your SMS channels and preserving your business’ reputation and resources. Whether you’re a small business or a large enterprise, Pinpoint Protect is a valuable tool to have in your arsenal as you navigate the evolving landscape of SMS messaging.

To get started with Pinpoint SMS Protect, visit the Amazon Pinpoint SMS documentation or reach out to your AWS account team. And don’t forget to let us know in the comments how Protect configurations has helped you combat AIT and strengthen your SMS communications.

A few resources to help you plan for your SMS program:

About the Author

Brett Ezell

Brett Ezell is your friendly neighborhood Solutions Architect at AWS, where he specializes in helping customers optimize their SMS and email campaigns using Amazon Pinpoint and Amazon Simple Email Service. As a former US Navy veteran, Brett brings a unique perspective to his work, ensuring customers receive tailored solutions to meet their needs. In his free time, Brett enjoys live music, collecting vinyl, and the challenges of a good workout. And, as a self-proclaimed comic book aficionado, he can often be found combing through his local shop for new books to add to his collection.

How to enable one-click unsubscribe email with Amazon Pinpoint

Post Syndicated from Zip Zieper original https://aws.amazon.com/blogs/messaging-and-targeting/how-to-enable-one-click-unsubscribe-email-with-amazon-pinpoint/

Amazon Pinpoint customers who use campaigns, journeys, or the SendMesages API to send more than 5,000 marketing email messages per day are considered “bulk senders”. If your organization meets this criteria, you are now subject to new requirements that were recently established by Google, Yahoo and other large ISPs/ESPs. These providers have mandated these requirements to help protect their user’s inboxes. Detailed information about these requirements is provided in the Amazon Simple Email Service (SES) bulk sender updates blog post.

Per these new requirements, Pinpoint customers that send marketing email messages in bulk must meet all of these criteria:

  • Fully authenticate their email sending domains with SPF, DKIM and DMARC. See this blog.
  • Provide a clearly visible unsubscribe link in the body &/or footer of each message.
  • Enable the “List-Unsubscribe” and “List-Unsubscribe-Post” one-click unsubscribe (the subbect of this blog post). You can learn more about these headers and how they are used in SES in this related blog post.
  • Honor all unsubscribe POST requests within 48 hours, after which time you shouldn’t be sending emails to the now unsubscribed end-user.
  • Actively monitor spam complaint rates, and take the steps needed to ensure these rates remain below acceptable levels as defined by the ESPs.

This blog post provides Pinpoint customers with the steps necessary to enable the one-click unsubscribe button via email headers for “List-Unsubscribe” and “List-Unsubscribe-Post” as defined by RFC 2369 and RFC 8058.

Unsubscribe Process Overview

Pinpoint now supports the inclusion of the “List-Unsubscribe” and “List-Unsubscribe-Post” email headers that enable compatible email client apps to render a one-click unsubscribe button when displaying emails from a subscription list. When you include these headers in the emails you send by Pinpoint, those end-users who want to unsubscribe from your emails can do so by simply clicking the unsubscribe button in their email app (see image). Once pressed, the unsubscribe button fires off a POST request to the URL you have defined in the “List-Unsubscribe” header.

You, the Pinpoint customer, are responsible for defining the “List-Unsubscribe” and “List-Unsubscribe-Post” headers, as well as supplying the system or process invoked by the “List-Unsubscribe” and “List-Unsubscribe-Post” email headers. Your system or process must, when activated by the unsubscribe action, update that end-user’s preferences accordingly so that within 48 hours, any end-user who unsubscribes will no longer receive unwanted emails.

If you only use Pinpoint’s campaigns and journeys, you may elect to use the Pinpoint endpoint’s OptOut attribute to store the user’s unsubscribe preferences. Possible values for OptOut are: ALL, the user has opted out and doesn’t want to receive any messages; and, NONE, the user hasn’t opted out and wants to receive all messages. It is important to note, however, that the SendMessages API ignores the Pinpoint endpoint’s OptOut attribute.

If you do not currently offer your recipients the option to unsubscribe to unwanted emails, you will need to develop & deploy a system or process to receive end-user unsubscribe requests to be in compliance with these new requirements. An example solution with sample code to processes email opt-out requests for Pinpoint can be found here. You can read more about this example in this blog post.

REQUIRED: Update the SES IAM role used by Pinpoint

Because Pinpoint uses SES resources for sending email messages, when using campaigns or journeys you must now create (or update) an IAM Orchestration sending role to grant Pinpoint service access to your SES resources. This allows Pinpoint to send emails via SES. To add or update the IAM role, follow the steps outlined in the Pinpoint documentation.

Note – If you are sending emails directly via the SendMesage, API you do not need an IAM Orchestration sending role, but you must have permissions for ses:SendEmail and ses:SendRawEmail.

Add easy unsubscribe email headers:

The steps you need to take to enable one-click unsubscribe in your Pinpoint emails depends on how you send emails, and whether or not you use templates, as shown below:

Decision tree for adding headers

Use SendMessages with the AWS SDK or CLI

Using the AWS CLI: add headers for the “List-Unsubscribe” and “List-Unsubscribe-post” as shown in the example below:

aws pinpoint send-messages \
--region us-east-1 \
--application-id ce796be37f32f178af652b26eexample \
--message-request '{
    "Addresses": {
        "[email protected]": {"ChannelType": "EMAIL"},
    },
    "MessageConfiguration": {
        "EmailMessage": {
            "SimpleEmail": {
                "Subject": {"Data":"URL with easy unsubscribe headers", "Charset":"UTF-8"},
                "TextPart": {"Data":"with headers list-unsubscribe and list-unsubscribe-post.\n\nUnsubscribe: <https://www.example.com/preferences>", "Charset":"UTF-8"},
                "HtmlPart": {"Data":"<html><body>with headers list-unsubscribe and list-unsubscribe-post<br><br><a ses:tags=\"unsubscribeLinkTag:optout\" href=\"https://example.com/?address=x&topic=x\">Unsubscribe</a></body></html>", "Charset":"UTF-8"},
                "Headers": [
                    {"Name":"List-Unsubscribe", "Value":"<https://example.com/?address=x&topic=x>, <mailto: [email protected]?subject=TopicUnsubscribe>"},
                    {"Name":"List-Unsubscribe-Post", "Value":"List-Unsubscribe=One-Click"}
                ]
            }
        }
    }
}

Send an email message

Below is an example using the SendMessages API from the AWS SDK for Python (Boto3) that includes the List-Unsubscribe headers. This example assumes that you’ve already installed and updated the SDK for Python (Boto3) to the latest version available. For more information, see Quickstart in the AWS SDK for Python (Boto3) API Reference.

import logging  # Logging library to log messages
import boto3  # AWS SDK for Python
from botocore.exceptions import ClientError  # Exception handling for boto3
import hashlib  # Library to generate unique hashes

# Configure logger
logger = logging.getLogger(__name__)

# Define constants
CHARSET = "UTF-8"
REGION = 'us-east-1'

def send_email_message(
    pinpoint_client,
    project_id, 
    sender,
    to_addresses,
    subject,
    html_message,
    text_message,
):
    """
    Sends an email message with HTML and plain text versions.

    :param pinpoint_client: A Boto3 Pinpoint client.
    :param project_id: The Amazon Pinpoint project ID to use when you send this message.
    :param sender: The "From" address. This address must be verified in
                   Amazon Pinpoint in the AWS Region you're using to send email.
    :param to_addresses: The list of addresses on the "To" line. If your Amazon Pinpoint account
                         is in the sandbox, these addresses must be verified.
    :param subject: The subject line of the email.
    :param html_message: The HTML content of the email.
    :param text_message: The plain text content of the email.
    :return: A dict of to_addresses and their message IDs.
    """
    try:
        # Create a dictionary of addresses with unique unsubscribe URLs
        # The addresses are encoded using the SHA256 hashing algorithm from the hashlib library
        # to create a unique and obfuscated unsubscribe URL for each recipient. This ensures
        # that the unsubscribe link is specific to each individual recipient, preventing
        # potential abuse or unauthorized unsubscribes. The hashed value is appended to the
        # base unsubscribe URL, allowing the email service to identify the intended recipient
        # when the unsubscribe link is clicked, while also protecting the recipient's personal
        # email address from being directly exposed in the URL.
        addresses = {
            address: {
                "ChannelType": "EMAIL",
                "Substitutions": {
                    "unsubscribeURL": [f"https://example.com/unsub/{hashlib.sha256(address.encode()).hexdigest()}"],
                }
            }
            for address in to_addresses
        }
        
        # Send email using Amazon Pinpoint
        response = pinpoint_client.send_messages(
            ApplicationId=project_id,
            MessageRequest={
                "Addresses": addresses,
                "MessageConfiguration": {
                    "EmailMessage": {
                        "FromAddress": sender,
                        "SimpleEmail": {
                            "Subject": {"Charset": CHARSET, "Data": subject},
                            "HtmlPart": {"Charset": CHARSET, "Data": html_message},
                            "TextPart": {"Charset": CHARSET, "Data": text_message},
                            "Headers": [
                                {"Name": "List-Unsubscribe", "Value": "{{unsubscribeURL}}"},
                                {"Name": "List-Unsubscribe-Post", "Value": "List-Unsubscribe=One-Click"}
                            ],
                        },
                    }
                }
            }
        )
    except ClientError as e:
        # Log exception if sending email fails
        logger.exception("Couldn't send email: %s", e)
        raise
    else:
        # Return a dictionary of addresses and their respective message IDs
        return {
            address: message["MessageId"] 
        for address, message in response["MessageResponse"]["Result"].items()
        }

def main():
    # Sample data for sending email
    project_id = "ce796be37f32f178af652b26eexample"  # Amazon Pinpoint project ID
    sender = "[email protected]"  # Verified sender email address
    to_addresses = ["[email protected]", "[email protected]", "[email protected]"]  # Recipient email addresses
    subject = "Amazon Pinpoint Unsubscribe Headers Test (SDK for Python (Boto3))"  # Email subject
    text_message = """Amazon Pinpoint Test (SDK for Python)
    -------------------------------------
    This email was sent with Amazon Pinpoint using the AWS SDK for Python (Boto3).
    For more information, see https://aws.amazon.com/sdk-for-python/
                """  # Plain text message
    html_message = """<html>
    <head></head>
    <body>
      <h1>Amazon Pinpoint Test (SDK for Python (Boto3)</h1>
      <p>This email was sent with
        <a href='https://aws.amazon.com/pinpoint/'>Amazon Pinpoint</a> using the
        <a href='https://aws.amazon.com/sdk-for-python/'>
          AWS SDK for Python (Boto3)</a>.</p>
    </body>
    </html>
                """  # HTML message

    # Create a Pinpoint client
    pinpoint_client = boto3.client("pinpoint", region_name=REGION)

    print("Sending email.")
    # Send email and print message IDs
    try:
        message_ids = send_email_message(
            pinpoint_client,
            project_id,
            sender,
            to_addresses,
            subject,
            html_message,
            text_message,
        )
        print(f"Message sent! Message IDs: {message_ids}")
    except ClientError as e:
        print(f"Failed to send messages: {e}")

# Entry point of the script
if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)  # Set logging level to INFO
    main()

Send an email message with an existing email template.

If you use message templates to send email messages via AWS SDK for Python (Boto3), you can add the headers for List-Unsubscribe and List-Unsubscribe-post into the template, and then fill those variables with unique values per recipient, as shown in the code example below. First, you would create the template via the UI and add the Headers in the new fields as shown in the image below.

Or you can create the template, with headers, via the AWS CLI:

aws pinpoint create-email-template --template-name MyEmailTemplate \
--email-template-request '{
    "Subject": "Amazon Pinpoint Unsubscribe Headers Test using email template",
    "TextPart": "Hello, welcome to our service. We are glad to have you with us. If you wish to unsubscribe, click here: {{unsubscribeURL}}",
    "HtmlPart": "<html><body><h1>Hello, welcome to our service</h1><p>We are glad to have you with us.</p><p>If you wish to unsubscribe, click <a href=\"{{unsubscribeURL}}\">here</a>.</p></body></html>",
    "DefaultSubstitutions": "{\"unsubscribeURL\": \"https://example.com/unsubscribe\"}",
    "Headers": [
            {"Name": "List-Unsubscribe","Value": "{{unsubscribeURL}}"},
            {"Name": "List-Unsubscribe-Post","Value": "List-Unsubscribe=One-Click"}
        ]
  }

In this next example, we are including the use of a secret Hash key. By using this format, the unsubscribe URL will include the Pinpoint project ID and a hashed value of the email address combined with the secret key. This provides a more secure and customized unsubscribe experience for the recipients.

import logging  # Logging library to log messages
import boto3  # AWS SDK for Python
from botocore.exceptions import ClientError  # Exception handling for boto3
import hashlib  # Library to generate unique hashes

# Configure logger
logger = logging.getLogger(__name__)

# Define constants
REGION = 'us-east-1'
HASH_SECRET_KEY = "my_secret_key"  # Replace with your secret key

def send_templated_email_message(
    pinpoint_client, 
    project_id, 
    sender, 
    to_addresses, 
    template_name, 
    template_version
):
    """
    Sends an email message with HTML and plain text versions.

    :param pinpoint_client: A Boto3 Pinpoint client.
    :param project_id: The Amazon Pinpoint project ID to use when you send this message.
    :param sender: The "From" address. This address must be verified in
                   Amazon Pinpoint in the AWS Region you're using to send email.
    :param to_addresses: The list of addresses on the "To" line. If your Amazon Pinpoint account
                         is in the sandbox, these addresses must be verified.
    :param template_name: The name of the email template to use when sending the message.
    :param template_version: The version number of the message template.

    :return: A dict of to_addresses and their message IDs.
    """
    try:
        # Create a dictionary of addresses with unique unsubscribe URLs
        # The addresses are encoded using the SHA256 hashing algorithm from the hashlib library
        # to create a unique and obfuscated unsubscribe URL for each recipient. This ensures
        # that the unsubscribe link is specific to each individual recipient, preventing
        # potential abuse or unauthorized unsubscribes. The hashed value is appended to the
        # base unsubscribe URL, allowing the email service to identify the intended recipient
        # when the unsubscribe link is clicked, while also protecting the recipient's personal
        # email address from being directly exposed in the URL.
        addresses = {
            address: {
                "ChannelType": "EMAIL",
                "Substitutions": {
                    "unsubscribeURL": [
                        f"https://www.example.com/preferences/index.html?pid={project_id}&h={hashlib.sha256((address + HASH_SECRET_KEY).encode()).hexdigest()}"
                    ]
                }
            }
            for address in to_addresses
        }
        # Send templated email using Amazon Pinpoint
        response = pinpoint_client.send_messages(
            ApplicationId=project_id,
            MessageRequest={
                "Addresses": addresses,
                "MessageConfiguration": {"EmailMessage": {"FromAddress": sender}},
                "TemplateConfiguration": {
                    "EmailTemplate": {
                        "Name": template_name,
                        "Version": template_version,
                    },
                },
            },
        )
    except ClientError as e:
        # Log exception if sending email fails
        logger.exception("Couldn't send email: %s", e)
        raise
    else:
        # Return a dictionary of addresses and their respective message IDs
        return {
            address: message["MessageId"] 
        for address, message in response["MessageResponse"]["Result"].items()
        }


def main():
    # Sample data for sending email
    project_id = "ce796be37f32f178af652b26eexample"  # Amazon Pinpoint project ID
    sender = "[email protected]"  # Verified sender email address
    to_addresses = ["[email protected]", "[email protected]", "[email protected]"]  # Recipient email addresses
    template_name = "MyEmailTemplate"
    template_version = "1"

    # Create a Pinpoint client
    pinpoint_client = boto3.client("pinpoint", region_name=REGION)
    print("Sending email.")
    # Send email and print message IDs
    try:
        message_ids = send_templated_email_message(
            pinpoint_client,
            project_id,
            sender,
            to_addresses,
            template_name,
            template_version,
        ),
        print(f"Message sent! Message IDs: {message_ids}"),
    except ClientError as e:
        print(f"Failed to send messages: {e}")
        
# Entry point of the script
if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)  # Set logging level to INFO
    main()

Pinpoint Campaigns via API (runtime).

If you send emails using Pinpoint campaigns via the API call (runtime), you can add the headers as described below:

"EmailMessage":{
   "Body": "string", 
   "Title": "string", 
   "HtmlBody": "string", 
    "FromAddress": "string",
   "Headers": [
        {
            "Name": "string", 
            "Value": "string"
        } 
   ]
}

Pinpoint Campaigns & Journeys via AWS Console.

The Pinpoint console enables you to create (or update) your email templates to add support for up to 15 different headers, including the “List-Unsubscribe” and “List-Unsubscribe-Post” headers. Simply open , or create a new, template in the Pinpoint console, scroll to the bottom of the visual message editor, expand the Headers option, and insert the header names and values. Note that if you only use the console UI to send your Campaigns and Journeys, you can store the encoded List-Unsubscribe URL as an attribute in the endpoint, then use that attribute as the value as shown below:

Conclusion.

In this blog, we provide Pinpoint customers with the information and guidance needed to enable a one-click unsubscribe link in their recipients’ compatible email apps via “List-Unsubscribe” and “List-Unsubscribe-Post” email headers. Following this guidance, in conjunction with properly authenticating your email sending domains and monitoring / keeping spam complaints below prescribed thresholds will help ensure high rates of Pinpoint email deliverability.

We welcome your comments on this post below. For additional information, refer to these resources, or contact your AWS account team.

About the Authors

zip

Zip

Zip is an Amazon Pinpoint and Amazon Simple Email Service Sr. Specialist Solutions Architect at AWS. Outside of work he enjoys time with his family, cooking, mountain biking and plogging.

Darren Roback

Darren Roback

Darren is a Senior Solutions Architect with Amazon Web Services based in St. Louis, Missouri. He has a background in Security and Compliance, Serverless Event-Driven Architecture, and Enterprise Architecture. At AWS, Darren partners with customers to help them solve business challenges with AWS technology. Outside of work, Darren enjoys spending time in his shop working on woodworking projects.

Bruno Giorgini

Bruno Giorgini

Bruno Giorgini is a Senior Solutions Architect specializing in Pinpoint and SES. With over two decades of experience in the IT industry, Bruno has been dedicated to assisting customers of all sizes in achieving their objectives. When he is not crafting innovative solutions for clients, Bruno enjoys spending quality time with his wife and son, exploring the scenic hiking trails around the SF Bay Area.

The Four Pillars of Managing Email Reputation

Post Syndicated from Dustin Taylor original https://aws.amazon.com/blogs/messaging-and-targeting/the-four-pillars-of-email-reputation/

Introduction

A sender’s domain and IP reputation strongly indicate email deliverability success. Maintaining a high reputation ensures optimal recipient inboxing. This blog outlines how Amazon SES protects its network reputation to help customers deliver high-quality email consistently. Understanding sender reputation nuances across diverse mailbox providers can be challenging, making issue identification and root cause analysis difficult. We’ll explore SES’ approach to managing domain and IP reputation.

What are Domain and IP Reputation?

Domain and IP reputation are measured by mailbox providers to indicate how reputable a sender is based primarily on how recipients engage with their email. Mailbox providers have their own way of measuring reputation and typically consider indicators such as:

  • A history of the emails received from the domain/IP
  • The authentication technologies used during delivery (SPF, DKIM, DMARC)
  • The rate of user engagement for the messages
  • The rate of complaints generated by the messages
  • The rate of which the mailbox providers’ spam filter determines mail to be spam from a domain/IP

While not an exhaustive list, these are some of the inputs into the reputation of a sender. Of this list, 4 of the 5 have nothing to do with the body, or viewable content, of the email that is received. This illustrates how important it is to have effective processes in place to set up sending from your domain/IPs and the management of your email sending programs.

How does Amazon SES manage Domain and IP Reputation

Management of reputation requires a multi-faceted approach distilled into four distinct pillars: Prevention, Monitoring, Analysis, and Response. Let’s dive deeper into these four pillars to see how Amazon SES operates to protect sending reputation for our service and our customers.

Prevention

Prevention is arguably the most important of the four pillars of reputation management. Abuse, or misuse, is the leading cause of poor reputation. Abuse, or misuse, can be characterized as sending phishing emails, unsolicited emails, or aggressive sending practices ignoring user feedback or lack of engagement, but this is not an exhaustive list. Prevention of abuse is accomplished through customer education (blogs, public documentation, and customer correspondence), service terms, acceptable use policies, and strict rules on setup. These abuse prevention mechanisms aid in educating customers before they use SES on prohibited sending practices as well as providing guidance on email sending best practices. SES implements several mechanisms to mitigate abuse and misuse, including:

  • Production access reviews – Every customer must request access to send email outbound. This step plays two parts: 1\ giving customers an opportunity to test sending from and to verified identities and 2\ preventing malicious senders from being able to open an account and begin uninhibited sending of low-quality mail. Every customer requesting access to send via SES provides information on their sending practices and volume estimates. This information is used in three ways: first to ensure that a customer is following best practices for sending email, second to provide the appropriate limits needed for their business, and third to determine if a customer’s sending practices are a risk to other senders.
  • Restricted sending from only verified identities/domains – Every customer, must verify ownership of an email address or sending domain to send an email on SES. This can be done for email addresses by clicking a verification link or for domains by placing DNS records that SES is able to verify.
  • Daily volume and sending rate limits – SES applies sending limits to an account for the following reasons: prevention of reputational damage and limiting costs should a bug occur within a customer’s application, and limiting the damage an elusive bad actor may cause.

Monitoring

The second pillar of reputation management is accurately monitoring your sending performance. Amazon SES tracks metrics like bounces, complaints, abuse reports, and mailbox provider status codes. Establishing overall sending baselines is crucial to measure the impact of deliverability and reputation changes. Granular monitoring is equally important, including metrics at the account, domain, IP, and blocklist levels.

Having granular data regarding our customer’s sending performance gives SES, and our customers, the opportunity to identify mechanisms in which a customer’s sending can improve, or indicators of when a bad actor may intend to misuse SES. Some of the mechanisms that we use to reduce the risk of reputation degradation include:

  • Monitoring new customer activity closely – The riskiest time for SES is when a new customer begins using SES and we have no historical precedent for the mail they are attempting to send. While the overwhelming majority of our customers send quality email, it’s important to ensure that a customer that is onboarding exhibits good sending practices. A customer may be inexperienced in their sending practices and SES will notify customers early to aid them in improving their sending. This limits the damage that can be done to both SES reputation and that of our customer.
  • Monitor any customers that trend away from the baseline – SES looks to determine what customers are doing well and where they could improve. Should they be given access to send freely, or should there be restrictions?
  • Monitor high-performing customers as well as low-performing customers – For SES, it’s crucial to prevent events that can degrade our reputation, such as sender compromises, uploading purchased lists, or using unsolicited recipient lists. Thoroughly reviewing all customers is essential to avoid reputational degradation from unnoticed compromises or misused recipient lists.
  • Providing our customers with a way to monitor more than bounce and complaint rates – SES provides a feature called the Virtual Deliverability Manager (VDM) which gives customers the added insight into how their messages are received by mailbox providers. These insights are provided in a dashboard that customers can review and dig into problems at the domain level, and broken down by provider.

Analysis

The third of the pillars of reputation management is analysis. Understanding the history of a sender, normal behavior and trends, mailbox provider feedback patterns, and monitoring reputation from a reputation provider enables SES to build a picture of a sender. Lets speak on some specifics about each of these data points further.

  • Sending Behavior – Is this a new sender, or one with an established reputation? Do they have a history of previous bounce/complaint issues? What historical volume is sent?
    • Tip: Understanding the baseline or history of the sender gives you the ability to know when things have changed for the better or worse.
  • Mailbox provider feedback – Amazon SES reviews mailbox provider feedback patterns to analyze responses when sending mail. If normally all SES mail is received successfully and we begin to see a spike in throttles with a negative response message such as this one from Gmail:

    421-4.7.28 Gmail has detected an unusual rate of unsolicited mail originating from your DKIM domain [example.com 36]. To protect our users from spam, mail sent from your domain has been temporarily rate limited. For more information, go to https://support.google.com/mail/?p=UnsolicitedRateLimitError to review our Bulk Email Senders Guidelines. m25-20020ae9e019000000b0078edf1f4c40si26277545qkk.197 – gsmtp

    this could be the first sign of reputation degradation.

    • Tip: Mailbox provider feedback is a good data point for degradation, however this is a late sign as the damage has already occurred. More proactive measures should be in place to ensure this step doesn’t occur.
  • Using reputation providers – External feedback on Amazon SES reputation is critical to validate our processes and identify potential gaps. Selecting a reputation provider has helped SES close this gap.
    • Tip: As a mail provider, you rely on sending metrics and historical data for monitoring senders. However, you may not know how customers acquired their recipient lists – whether through confirmed opt-ins or purchased lists. Purchased lists risk your domain being blocklisted since recipients didn’t sign up for your mail. Lacking visibility into subscription workflows makes it hard to determine why blocklisting occurred. Refer to our FAQ for more on blocklists.

Response

The fourth of the pillars of reputation management is response. Understanding what to do when your reputation begins to show signs of decline is important. Some signals that show reputation declines are: low inbox rates, mail being throttled, mail being blocked, or external reputation tools showing poor reputation for your domain/IP. For Amazon SES, we take action to do the following:

  • Contact customers where metrics breach alarm thresholds.
  • Respond timely to signs of abuse or reputation degradation.
  • Stop sending based on continued, or high-risk, signals of abuse or reputation degradation.
  • Support customers in resolving sending issues to maintain the overall reputation of SES.

It is important to respond quickly to the signals of reputation degradation. The decision to impact a customer’s ability to send mail is not one that Amazon SES takes lightly. A decision to impact a customer’s ability to send mail is made when the quality of mail is abusive in nature (phishing) or if there are signals that the mail being sent is not well received by mailbox providers at scale. In some cases, a customer may not be aware that their sending patterns, practices, or content may be problematic. This can be due to a gap in monitoring, logging, or an issue with credentials being compromised. If the decision to impact a customer’s sending is made, a communication will be sent to that customer so that we can partner with them to resolve the issue.

Amazon SES doesn’t only make the decision to communicate with our customers when there is a problem. SES also communicates with customers, when appropriate, earlier in the reputation management cycle to warn of a negative trend in sending. This can be seen in the review periods that are triggered when increases in bounces, complaints, or mailbox provider feedback is seen. These review periods give SES customers the ability and time to understand the problem, and to work on fixes to avoid serious reputation impact. Being involved early in the discovery phase of a sending event improves the customer experience without the need to negatively impact sending.

Conclusion

Maintaining a positive sending reputation necessitates a diligent approach to prevent abusive emails. The four pillars outlined serve as guidelines to improve email quality: prevention, monitoring, analysis, and response. This is an iterative process that requires moving fluidly between pillars.

About the author:

Dustin Taylor

Dustin Taylor

Dustin is the Manager of anti-abuse and email deliverability for Amazon SES. His focus is both external and internal in helping improve inbox placement for SES customers and finding new ways to fight email abuse. In his off-time he enjoys going bass fishing and is a hobbyist woodworker.

How to Send MMS Using Amazon Pinpoint

Post Syndicated from Tyler Holmes original https://aws.amazon.com/blogs/messaging-and-targeting/how-to-send-mms-using-amazon-pinpoint/

In our digital age, communication has evolved far beyond simple SMS, short for Simple Message Service. The rise of multimedia messaging has transformed the way we share information, express ourselves, and stay connected. MMS, short for Multimedia Messaging Service, is a technology that has become an integral part of our daily lives, enabling us to convey messages that go beyond the confines of plain text. MMS allows senders to transmit a variety of media, including images, videos, audio recordings, and even documents, alongside traditional text. This powerful combination has revolutionized the way we communicate, making it easier to share experiences, ideas, and emotions in a more engaging and visually compelling manner. In a world where visual content is becoming increasingly dominant, MMS has become a crucial tool for companies to communicate with their customers.

What are the differences between SMS and MMS?

MMS is similar to SMS but has some very distinct differences that enable you to send and engage with your customers in different ways.

Message length

  • SMS
    • SMS can send text messages up to 160 characters as long as each character is a part of the GSM 03.38 character set. This means that if you have long links or longer form content that you need to send that these messages will be broken up into multiple message parts. This increases your cost and decreases the recipient reading experience on their device.
  • If your SMS contains any characters that are outside of the GSM 03.38 character set, your message can only contain up to 70 characters. Depending on the language you are using this can be very limiting. Messages that are broken up into multiple message parts can quickly cause you to hit your throughput limits since each part is measured as one message, even if you are only making a single call to the API. A GSM 03.38 message with 161 characters is 2 message parts which means that your throughput is effectively cut in half.
  • MMS
    • MMS can send up to 1,600 characters, regardless of the character set you use. Messages can be much larger, often up to 300 KB depending on the character set you are using. This expanded size allows you to send long form messages and not be so limited if you need to send messages in a character set outside of GSM 03.38 or just have the need to send longer messages.
    • This can reduce your cost and improve the experience for your recipients.
    • The larger message size helps you manage your throughput limitations since, a single MMS message can be sent, rather than an SMS being broken up into multiple message parts.

Message content

  • SMS
    • SMS is text based only. You cannot send media or attachments.
  • MMS
    • MMS can send multimedia content such as images, videos, PDF files, and audio files. This opens up many more options for how it can be used such as attaching a receipt, sending a new product launch gallery, or supporting troubleshooting with a how-to video.
    • A single MMS message can be up to 600KB in size for media files
    • Supported media file formats
      • Portable Network Graphics (PNG)
      • Joint Photographic Experts Group (JPEG)
      • Graphics Interchange Format (GIF)
      • MPEG-4 (MP4)
      • Quicktime File Format (MOV)
      • Portable Document Format (PDF)
      • Virtual Contact File (VCF)
      • More

Message delivery time

  • SMS
    • SMS messages are delivered as a text-only message and are much smaller which may result in it being delivered more promptly.
  • MMS
    • MMS messages may take longer to deliver dependent on the type and size of the multimedia content that is included which may increase the time to be delivered. This makes time sensitive messages such as OTP delivery more costly and potentially slower so it’s not an ideal use case for MMS.

Cost

Network and device support

  • SMS
    • SMS is a widely-adopted and supported protocol across all mobile networks
    • SMS is compatible with all mobile devices, even the most basic models
  • MMS
    • MMS requires specific network infrastructure and compatibility between the sender’s and receiver’s devices and networks
    • MMS requires devices with multimedia capabilities, such as smartphones or feature phones with advanced messaging capabilities. Long text MMS messages sent to an MMS incapable recipient will be redirected to SMS by default

Regional availability and location

What are some examples of MMS messages?

Here are some examples of how MMS messages can be used:

  • Attached video
    • “Introducing our new limited-edition product line! 📽 Check out this exclusive sneak peek video.“
  • Single attached photo
    • “Caught you in the act! 👀 Here’s a fun photo from our recent in-store event. Thanks for stopping by!”
  • Multiple attached photos
    • “Treat yourself to something special this weekend. 💄 Explore our latest makeup collection with this handy lookbook.”
  • Attached receipt
    • “Thanks for your most recent purchase! We hope you love it! Your receipt is attached”

It’s important to note that you do not need to send all of your messages using one or the other. As an example you can use SMS for One Time Password (OTP) sends since they do not require media and generally should be a single message part in any language and use MMS to enable an eye catching marketing release with pictures or video.

How to send MMS with Amazon Pinpoint

You can use the AWS CLI or Amazon Pinpoint SMS and voice v2 API to send MMS messages to your customers. Use the send-media-message AWS CLI command to send an MMS message. There are two ways to send MMS depending on whether you are including media with your message.

How to send MMS with media

The first step to sending MMS is to make sure that you have an Origination Identity (OID) that is enabled for MMS. Amazon Pinpoint supports sending MMS to the US and Canada. Eligible MMS numbers in the US include 10DLC, toll-free, and short codes. Canadian eligible MMS numbers include long codes and short codes.

NOTE: If you procured originators for the US and Canada prior to May 2024, in some cases, existing Pinpoint SMS US and Canadian phone will be automatically updated to begin supporting MMS. To determine if your existing Amazon Pinpoint number(s) are eligible, navigate to the Amazon Pinpoint console Phone Numbers page. If MMS is listed under the “capabilities” column, you can begin using that number for MMS immediately if your number is in an active state. For these existing eligible numbers, there is no additional registration requirement as long as your use case doesn’t change. If the capability listed only shows SMS, you will need to procure a new US or Canadian phone number which will automatically include MMS.

For all newly purchased(Post May 2024) Amazon Pinpoint US and Canadian phone numbers, both SMS and MMS services will automatically be added at time of purchase, simplifying the registration process and providing the flexibility to use either messaging formats upon approval.

You can get started by navigating to the Pinpoint SMS console phone number page and selecting Request Originator. From there, simply follow the self-guided steps for purchasing and registering the phone number If you need help determining what type of OID you need consult this tutorial. Information on registering for MMS enabled OIDs can be found here:

Sending MMS with media included requires you to create a bucket in Simple Storage Service (S3) that your media can be accessed from. The media files have to be stored in a publicly available S3 bucket. Information on creating a bucket and uploading media objects to it can be found here:

You will include the link to this bucket in your “sendMediaMessage” call using the “MediaUrls” attribute seen below. The supported media files formats can be found here.

The only required attributes are:

The CLI command below will send an MMS with the text of “Check out this Photo!” and will have a JPEG image included.

aws pinpoint-sms-voice-v2 —region 'xx-xxxx-xx' send-media-message —destination-phone-number +xxxxxxxxxxx —origination-identity +xxxxxxxxxxx —message-body 'Check out this Photo!' —media-urls 's3://s3-bucket/media_file.jpg'

If Amazon Pinpoint SMS accepts the command you will receive the MessageID. This only means the command was successfully received and not that the destination device has received the message yet.

How to send an MMS with no media

Sending an MMS with no media attached is nearly the same, you just don’t include a link to an S3 bucket in the “MediaUrls” attribute, it’s as easy as that. You might do this to reduce costs if you are sending a message with characters that are outside of the GSM 03.38 character or on a long message with more than 160 GSM 03.38 characters like the below.

The CLI command below will send the following message:

“This message will be broken up into more than 3 message parts if sent via SMS so we are going to use MMS instead to save costs. This is a great way to reduce your costs even if you are not sending any media such as a photo or video attached. Though a single MMS message costs more than a single SMS message there are times where you may want to use MMS simply because it is a cheaper way of communicating with your recipients. Since you can use SMS and MMS for different scenarios you may choose to send an OTP message with SMS and something like this message using MMS.”

aws pinpoint-sms-voice-v2 —region 'xx-xxxx-xx' send-media-message —destination-phone-number +xxxxxxxxxxx —origination-identity +xxxxxxxxxxx —message-body 'This message will be broken up into more than 3 message parts if sent via SMS so we are going to use MMS instead to save costs. This is a great way to reduce your costs even if you are not sending any media such as a photo or video attached. Though a single MMS message costs more than a single SMS message there are times where you may want to use MMS simply because it is a cheaper way of communicating with your recipients. Since you can use SMS and MMS for different scenarios you may choose to send an OTP message with SMS and something like this message using MMS.'

Conclusion

In this post you learned about the differences between SMS and MMS and how each can be used to send different types of messages. MMS is an excellent channel to communicate with your recipients to send highly engaging content that in some scenarios can reduce your cost while improving your customers’ experience. If you are sending content outside of the GSM 03.38 character set, want to send images, videos, or attachments such as receipts then MMS is definitely something you should look into.

A few resources to help you plan for your SMS program:

Take Command Summit: Take Breaches from Inevitable to Preventable on May 21

Post Syndicated from Rapid7 original https://blog.rapid7.com/2024/04/22/take-command-summit-take-breaches-from-inevitable-to-preventable-on-may-21/

Take Command Summit: Take Breaches from Inevitable to Preventable on May 21

Registration is now open for Take Command, a day-long virtual summit in partnership with AWS. You do not want to miss it. You’ll get new attack intelligence, insight into AI disruption, transparent MDR partnerships, and more.

In 2024, adversaries are using AI and new techniques, working in gangs with nation-state budgets. But it’s “inevitable” they’ll succeed? Really?

Before any talk of surrender, please join us at Take Command. We’ve packed the day with information and insights you can take back to your team and use immediately.

You’ll hear from Chief Scientist Raj Samani, our own CISO Jaya Baloo, global security leaders, hands-on practitioners, and Rapid7 Labs leaders like Christiaan Beek and Caitlin Condon. You’ll get a first look at new, emergent research, trends, and intelligence from the curators of Metasploit and our renowned open source communities.

You’ll leave with actionable strategies to safeguard against the newest ransomware, state-sponsored TTPs, and marquee vulnerabilities.

Can’t make the entire day? Check out the agenda, see what fits

The summit kicks off with back-to-back keynotes. First, “Know Your Adversary: Breaking Down the 2024 Attack Intelligence Report” and “The State of Security 2024.”

You’ll get an insider view of Rapid7’s MDR SOC. Sessions range from “Building Defenses Through AI” to “Unlocking Success: Strategies for Measuring Team Performance” to a big favorite “Before, During, & After Ransomware Attacks.” Though no one really talks about it, there’s a lengthy “before” period, and new, good things you can do to frustrate the bad guys.

Take Command will offer strategies on building cybersecurity culture (yes, it’s difficult with humans). And, of course, preparing for the Securities & Exchange Commission’s Cybersecurity Disclosure Rules. You’ll hear from Sabeen Malik, VP, Global Government Affairs and Public Policy, Kyra Ayo Caros Director, Corporate Securities & Compliance and Harley L. Geiger, Venable LLP.

Now, turning the tables on attackers is possible

Adversaries are inflicting $10 trillion in damage to the global economy every year , and the goal posts keep moving. As risks from cloud, IoT, AI and quantum computing proliferate and attacks get more frequent, SecOps have never been more stressed. And more in need of sophisticated guidance.

Mark your calendar for May 21. Get details here. You’ll be saving a lot more than the date.

Serverless IoT email capture, attachment processing, and distribution

Post Syndicated from Stacy Conant original https://aws.amazon.com/blogs/messaging-and-targeting/serverless-iot-email-capture-attachment-processing-and-distribution/

Many customers need to automate email notifications to a broad and diverse set of email recipients, sometimes from a sensor network with a variety of monitoring capabilities. Many sensor monitoring software products include an SMTP client to achieve this goal. However, managing email server infrastructure requires specialty expertise and operating an email server comes with additional cost and inherent risk of breach, spam, and storage management. Organizations also need to manage distribution of attachments, which could be large and potentially contain exploits or viruses. For IoT use cases, diagnostic data relevance quickly expires, necessitating retention policies to regularly delete content.

Solution Overview

This solution uses the Amazon Simple Email Service (SES) SMTP interface to receive SMTP client messages, and processes the message to replace an attachment with a pre-signed URL in the resulting email to its intended recipients. Attachments are stored separately in an Amazon Simple Storage Service (S3) bucket with a lifecycle policy implemented. This reduces the storage requirements of recipient email server receiving notification emails. Additionally, this solution leverages built-in anti-spam and security scanning capabilities to deal with spam and potentially malicious attachments while at the same time providing the mechanism by which pre-signed attachment links can be revoked should the emails be distributed to unintended recipients.

The solution uses:

  • Amazon SES SMTP interface to receive incoming emails.
  • Amazon SES receipt rule on a (sub)domain controlled by administrators, to store raw incoming emails in an Amazon S3 bucket.
  • AWS Lambda function, triggered on S3 ObjectCreated event, to process raw emails, extract attachments, replace each with pre-signed URL with configurable expiry, and send the processed emails to intended recipients.

Solution Flow Details:

  1. SMTP client transmits email content to an email address in a (sub) domain with MX record set to Amazon SES service’s regional endpoint.
  2. Amazon SES SMTP interface receives an email and forwards it to SES Receipt Rule(s) for processing.
  3. A matching Amazon SES Receipt Rule saves incoming email into an Amazon S3 Bucket.
  4. Amazon S3 Bucket emits an S3 ObjectCreated Event, and places the event onto the Amazon Simple Queue Services (SQS) queue.
  5. The AWS Lambda service polls the inbound messages’ SQS queue and feeds events to the Lambda function.
  6. The Lambda function, retrieves email files from the S3 bucket, parses the email sender/subject/body, saves attachments to a separate attachment S3 bucket (7), and replaces attachments with pre-signed URLs in the email body. The Lambda function then extracts intended recipient addresses from the email body. If the body contains properly formatted recipients list, email is then sent using SES API (9), otherwise a notice is posted to a fallback Amazon Simple Notification Service (SNS) Topic (8).
  7. The Lambda function saves extracted attachments, if any, into an attachments bucket.
  8. Malformed email notifications are posted to a fallback Amazon SNS Topic.
  9. The Lambda function invokes Amazon SES API to send the processed email to all intended recipient addresses.
  10. If the Lambda function is unable to process email successfully, the inbound message is placed on to the SQS dead-letter queue (DLQ) queue for later intervention by the operator.
  11. SES delivers an email to each recipients’ mail server.
  12. Intended recipients download emails from their corporate mail servers and retrieve attachments from the S3 pre-signed URL(s) embedded in the email body.
  13. An alarm is triggered and a notification is published to Amazon SNS Alarms Topic whenever:
    • More than 50 failed messages are in the DLQ.
    • Oldest message on incoming SQS queue is older than 3 minutes – unable to keep up with inbound messages (flooding).
    • The incoming SQS queue contains over 180 messages (configurable) over 5 minutes old.

Setting up Amazon SES

For this solution you will need an email account where you can receive emails. You’ll also need a (sub)domain for which you control the mail exchanger (MX) record. You can obtain your (sub)domain either from Amazon Route53 or another domain hosting provider.

Verify the sender email address

You’ll need to follow the instructions to Verify an email address for all identities that you use as “From”, “Source”, ” Sender”, or “Return-Path” addresses. You’ll also need to follow these instructions for any identities you wish to send emails to during initial testing while your SES account is in the “Sandbox” (see next “Moving out of the SES Sandbox” section).

Moving out of the SES Sandbox

Amazon SES accounts are “in the Sandbox” by default, limiting email sending only to verified identities. AWS does this to prevent fraud and abuse as well as protecting your reputation as an email sender. When your account leaves the Sandbox, SES can send email to any recipient, regardless of whether the recipient’s address or domain is verified by SES. However, you still have to verify all identities that you use as “From”, “Source”, “Sender”, or “Return-Path” addresses.
Follow the Moving out of the SES Sandbox instructions in the SES Developer Guide. Approval is usually within 24 hours.

Set up the SES SMTP interface

Follow the workshop lab instructions to set up email sending from your SMTP client using the SES SMTP interface. Once you’ve completed this step, your SMTP client can open authenticated sessions with the SES SMTP interface and send emails. The workshop will guide you through the following steps:

  1. Create SMTP credentials for your SES account.
    • IMPORTANT: Never share SMTP credentials with unauthorized individuals. Anyone with these credentials can send as many SMTP requests and in whatever format/content they choose. This may result in end-users receiving emails with malicious content, administrative/operations overload, and unbounded AWS charges.
  2. Test your connection to ensure you can send emails.
  3. Authenticate using the SMTP credentials generated in step 1 and then send a test email from an SMTP client.

Verify your email domain and bounce notifications with Amazon SES

In order to replace email attachments with a pre-signed URL and other application logic, you’ll need to set up SES to receive emails on a domain or subdomain you control.

  1. Verify the domain that you want to use for receiving emails.
  2. Publish a mail exchanger record (MX record) and include the Amazon SES inbound receiving endpoint for your AWS region ( e.g. inbound-smtp.us-east-1.amazonaws.com for US East Northern Virginia) in the domain DNS configuration.
  3. Amazon SES automatically manages the bounce notifications whenever recipient email is not deliverable. Follow the Set up notifications for bounces and complaints guide to setup bounce notifications.

Deploying the solution

The solution is implemented using AWS CDK with Python. First clone the solution repository to your local machine or Cloud9 development environment. Then deploy the solution by entering the following commands into your terminal:

python -m venv .venv
. ./venv/bin/activate
pip install -r requirements.txt

cdk deploy \
--context SenderEmail=<verified sender email> \
 --context RecipientEmail=<recipient email address> \
 --context ConfigurationSetName=<configuration set name>

Note:

The RecipientEmail CDK context parameter in the cdk deploy command above can be any email address in the domain you verified as part of the Verify the domain step. In other words, if the verified domain is acme-corp.com, then the emails can be [email protected], [email protected], etc.

The ConfigurationSetName CDK context can be obtained by navigating to Identities in Amazon SES console, selecting the verified domain (same as above), switching to “Configuration set” tab and selecting name of the “Default configuration set”

After deploying the solution, please, navigate to Amazon SES Email receiving in AWS console, edit the rule set and set it to Active.

Testing the solution end-to-end

Create a small file and generate a base64 encoding so that you can attach it to an SMTP message:

echo content >> demo.txt
cat demo.txt | base64 > demo64.txt
cat demo64.txt

Install openssl (which includes an SMTP client capability) using the following command:

sudo yum install openssl

Now run the SMTP client (openssl is used for the proof of concept, be sure to complete the steps in the workshop lab instructions first):

openssl s_client -crlf -quiet -starttls smtp -connect email-smtp.<aws-region>.amazonaws.com:587

and feed in the commands (replacing the brackets [] and everything between them) to send the SMTP message with the attachment you created.

EHLO amazonses.com
AUTH LOGIN
[base64 encoded SMTP user name]
[base64 encoded SMTP password]
MAIL FROM:[VERIFIED EMAIL IN SES]
RCPT TO:[VERIFIED EMAIL WITH SES RECEIPT RULE]
DATA
Subject: Demo from openssl
MIME-Version: 1.0
Content-Type: multipart/mixed;
 boundary="XXXXboundary text"

This is a multipart message in MIME format.

--XXXXboundary text
Content-Type: text/plain

Line1:This is a Test email sent to coded list of email addresses using the Amazon SES SMTP interface from openssl SMTP client.
Line2:Email_Rxers_Code:[ANYUSER1@DOMAIN_A,ANYUSER2@DOMAIN_B,ANYUSERX@DOMAIN_Y]:Email_Rxers_Code:
Line3:Last line.

--XXXXboundary text
Content-Type: text/plain;
Content-Transfer-Encoding: Base64
Content-Disposition: attachment; filename="demo64.txt"
Y29udGVudAo=
--XXXXboundary text
.
QUIT

Note: For base64 SMTP username and password above, use values obtained in Set up the SES SMTP interface, step 1. So for example, if the username is AKZB3LJAF5TQQRRPQZO1, then you can obtain base64 encoded value using following command:

echo -n AKZB3LJAF5TQQRRPQZO1 |base64
QUtaQjNMSkFGNVRRUVJSUFFaTzE=

This makes base64 encoded value QUtaQjNMSkFGNVRRUVJSUFFaTzE= Repeat same process for SMTP username and password values in the example above.

The openssl command should result in successful SMTP authentication and send. You should receive an email that looks like this:

Optimizing Security of the Solution

  1. Do not share DNS credentials. Unauthorized access can lead to domain control, potential denial of service, and AWS charges. Restrict access to authorized personnel only.
  2. Do not set the SENDER_EMAIL environment variable to the email address associated with the receipt rule. This address is a closely guarded secret, known only to administrators, and should be changed frequently.
  3. Review access to your code repository regularly to ensure there are no unauthorized changes to your code base.
  4. Utilize Permissions Boundaries to restrict the actions permitted by an IAM user or role.

Cleanup

To cleanup, start by navigating to Amazon SES Email receiving in AWS console, and setting the rule set to Inactive.

Once completed, delete the stack:

cdk destroy

Cleanup AWS SES Access Credentials

In Amazon SES Console, select Manage existing SMTP credentials, select the username for which credentials were created in Set up the SES SMTP interface above, navigate to the Security credentials tab and in the Access keys section, select Action -> Delete to delete AWS SES access credentials.

Troubleshooting

If you are not receiving the email or email is not being sent correctly there are a number of common causes of these errors:

  • HTTP Error 554 Message rejected email address is not verified. The following identities failed the check in region :
    • This means that you have attempted to send an email from address that has not been verified.
    • Please, ensure that the “MAIL FROM:[VERIFIED EMAIL IN SES]” email address sent via openssl matches the SenderEmail=<verified sender email> email address used in cdk deploy.
    • Also make sure this email address was used in Verify the sender email address step.
  • Email is not being delivered/forwarded
    • The incoming S3 bucket under the incoming prefix, contains file called AMAZON_SES_SETUP_NOTIFICATION. This means that MX record of the domain setup is missing. Please, validate that the MX record (step 2) of Verify your email domain with Amazon SES to receive emails section is fully configured.
    • Please ensure after deploying the Amazon SES solution, the created rule set was made active by navigating to Amazon SES Email receiving in AWS console, and set it to Active.
    • This may mean that the destination email address has bounced. Please, navigate to Amazon SES Suppression list in AWS console ensure that recipient’s email is not in the suppression list. If it is listed, you can see the reason in the “Suppression reason” column. There you may either manually remove from the suppression list or if the recipient email is not valid, consider using a different recipient email address.
AWS Legal Disclaimer: Sample code, software libraries, command line tools, proofs of concept, templates, or other related technology are provided as AWS Content or Third-Party Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content or Third-Party Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content or Third-Party Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content or Third-Party Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.

About the Authors

Tarek Soliman

Tarek Soliman

Tarek is a Senior Solutions Architect at AWS. His background is in Software Engineering with a focus on distributed systems. He is passionate about diving into customer problems and solving them. He also enjoys building things using software, woodworking, and hobby electronics.

Dave Spencer

Dave Spencer

Dave is a Senior Solutions Architect at AWS. His background is in cloud solutions architecture, Infrastructure as Code (Iac), systems engineering, and embedded systems programming. Dave’s passion is developing partnerships with Department of Defense customers to maximize technology investments and realize their strategic vision.

Ayman Ishimwe

Ayman Ishimwe

Ayman is a Solutions Architect at AWS based in Seattle, Washington. He holds a Master’s degree in Software Engineering and IT from Oakland University. With prior experience in software development, specifically in building microservices for distributed web applications, he is passionate about helping customers build robust and scalable solutions on AWS cloud services following best practices.

Dmytro Protsiv

Dmytro Protsiv

Dmytro is a Cloud Applications Architect for with Amazon Web Services. He is passionate about helping customers to solve their business challenges around application modernization.

Stacy Conant

Stacy Conant

Stacy is a Solutions Architect working with DoD and US Navy customers. She enjoys helping customers understand how to harness big data and working on data analytics solutions. On the weekends, you can find Stacy crocheting, reading Harry Potter (again), playing with her dogs and cooking with her husband.

How to Implement Self-Managed Opt-Outs for SMS with Amazon Pinpoint

Post Syndicated from Tyler Holmes original https://aws.amazon.com/blogs/messaging-and-targeting/how-to-implement-self-managed-opt-outs-for-sms-with-amazon-pinpoint/

Amazon Pinpoint offers marketers and developers the ability to send SMS to over 240 countries and/or regions around the world; giving users the global reach, scalability, cost effective pricing, and high deliverability that is required to build a successful SMS program. SMS is a flexible communication channel that facilitates different business requirements, including One-Time Password (OTP), reminders, and bulk marketing to name a few. Regardless of the content that you are sending via SMS there is a requirement to manage your recipients’ opt-in/out status. Read on to learn your two options for managing opt-outs and how you can configure them. Amazon Pinpoint offers a fully managed opt-out capability and the ability to self-manage the process with your own tools.

NOTE: If you are sending to US numbers with a toll-free (TFN) number the carriers will automatically manage those numbers and are not eligible for either of these processes.

Managed Opt-Out Process
If you prefer to have Pinpoint manage your opt-out processes you can refer to our blog “How to Manage SMS Opt-Outs with Amazon Pinpoint” to learn how to configure keywords and opt-out lists.

Self-Managed Opt-Out Process
Many customers use Pinpoint’s Managed Opt-Out process to deliver their communications, but some scenarios require the ability to self-manage this process. Self-managing the opt-out process provides more granular control over customer communication preferences and allows customers to centralize those preferences within their own applications.
Common reasons for organizations to implement self-managed opt-out include but aren’t limited to:

  1. Have an existing self-managed opt-out capability with a standing toolset that is already integrated with other aspects of their communication stack.
  2. Need multiple options for their customers to manage the communication preferences such as a web portal, call center, and mobile application to name a few.
  3. Require full control in order to implement custom logic that caters to their business needs.
  4. Want to change their SMS provider to Pinpoint while not changing what they have already built within an existing application.

How to implement Self-Managed Opt-Outs with Pinpoint
Choosing to self-manage your opt-outs requires some configuration within Pinpoint and the use of other AWS services. The solution outlined in this blog will use Amazon Pinpoint in addition to the following services:

  1. AWS Lambda
  2. Amazon DynamoDB
  3. Amazon SNS

NOTE: If you have existing services/applications that allow you to implement similar functionality as explained in this blog, you don’t have to use these services listed above.

What’s in scope?

This blog covers the following scenario:

  1. SMS is being sent with Amazon Pinpoint SMS and Voice V2 API – SendTextMessage
  2. You specify the Origination Identity (OID) to be used (short code, long code, 10DLC, etc) as the parameter to send the SMS to the destination phone number.

While the following scenarios can be self-managed this blog does not cover the following cases:

  1. You use a phone-pool to send SMS.
  2. You do not specify an OID in your call SendTextMessage call and let Amazon Pinpoint figure out and use the appropriate OID.

Keywords in scope

  1. Opt-out keywords – All Opt-out keywords mentioned in this document are included in the code.
  2. Opt-in keywords – This blog considers ‘JOIN’ as a valid keyword that SMS recipients can respond with to opt back in for the SMS communication.

NOTE: The code examples in this blog can be modified to add any additional custom keywords for your use case.

Assumptions/Prerequisites

  1. You have the necessary permissions to configure the following services in the same AWS account and region where Amazon Pinpoint is implemented.
    a. AWS Lambda
    b. Amazon DynamoDB
    c. Amazon SNS
  2. Your instance of Amazon Pinpoint has at least one OID approved and provisioned to send SMS.
    a. If you need help to determine what OID fits your use case(s) use this guide
    b. NOTE: Sender IDs do not have an ability to receive 2-way communication. If you are using Sender IDs, you still must manage opt-outs, but must do so by offering alternative ways of opting out such as a web portal, call center, and/or mobile applications.

Solution Overview

The solution proposed in this blog is fully serverless arhitecture and uses AWS managed services to eliminate a need for you to maintain and manage any of the infrastructure components.

  1. Your application invokes AWS Lambda function ‘InvokeSendTextMessage’ which calls Amazon Pinpoint SMS and Voice V2 API – SendTextMessage.
  2. The AWS Lambda function called ‘InvokeSendTextMessage’ performs the following tasks:
    2a. Fetches the latest item based on the descending order of the timestamp with the destination phone number and OID from Amazon DynamoDB table ‘SMSOptOut‘.
    2b. If an item is found with the customer response as any valid keyword for Opt-out (Refer section Keywords in scope), the process stops and Amazon Pinpoint APIs won’t be called by InvokeSendTextMessage function as the customer chose to opt-out.
    2c. If an item is not found or is found with the customer response as any valid keyword for Opt-in (Refer section Keywords in scope), the function calls Amazon Pinpoint SMS and Voice V2 API – SendTextMessage to deliver it to the customer/destination phone number.

NOTE: Amazon DynamoDB table can also be configured to receive the Opt-out or Opt-in information through various other channels (app, website, customer care etc.) if you have multiple interfaces for customer to do so but that is not in the scope for this blog.

Refer to the section, ‘InvokeSendTextMessage function code’ to understand the sample AWS Lambda function code. The code uses Python 3.12 language.

  1. The message is successfully delivered to a destination phone number.
  2. If the customers responds to the same OID (because you have enabled 2-way SMS feature) with a keyword that is a valid value from all the keywords in scope (Refer section Keywords in scope), Amazon SNS topic is configured with Amazon Pinpoint which captures the customer response.
    Note: The other keywords are not in scope for this blog, but you can specifically add all possible keywords that customers can respond with. There can be an accidental responses by a customer which will be ignored by the AWS Lambda code.
  3. AWS Lambda function ‘AddOptOutInDynamoDB’ is a subscriber to the topic in Amazon SNS and processes customer responses.
  1. AWS Lambda function ‘AddOptOutInDynamoDB’ performs few tasks as described below
  2. 6a. If the customer response is a keyword that is a valid value from all the keywords in scope (Refer section Keywords in scope), Amazon Lambda function ‘AddOptOutInDynamoDB‘ extracts the OID and customer phone number information from the response and adds the entry in the Amazon DynamoDB table ‘SMSOptOut’. This way Amazon DynamoDB table keeps getting latest customer opt-in/opt-out status.
    6b. Once the item is successfully put in dynamodb table ‘SMSOptOut‘, if the customer response was any Opt-out keyword (Refer section Keywords in scope), the function sends a SMS to the customer who has just opted out to confirm the status. “YOU HAVE BEEN UNSUBSCRIBED. IF THIS WAS A MISTAKE PLEASE TEXT “JOIN” TO THIS NUMBER TO BE RESUBSCRIBED”.
    If the customer response was any Opt-in keyword (Refer section Keywords in scope) the function sends a SMS to the customer who has just opted back in to confirm the status. “YOU HAVE BEEN SUBSCRIBED. IF THIS WAS A MISTAKE PLEASE TEXT “STOP” OR “UNSUBSCRIBE” TO THIS NUMBER TO BE UNSUBSCRIBED”. (But SMS recipients can still respond with any valid keyword mentioned in Opt-out keyword list)

NOTE: Refer the section ‘AddOptOutInDynamoDB function code’ to understand the sample code. The code uses Python3.12 language.

  1. The Opt-in/Opt-out status confirmation SMS is successfully delivered to a customer/destination phone number.

Amazon Pinpoint setup

  1. Enable 2-way SMS messaging for the OID that you procured. Refer to the screenshot below for your reference.

2-way SMS setting:

  1. Enable the self-managed opt-out feature for the OID. Once enabled, Amazon Pinpoint does not respond to opt-out messages sent to the SNS topic by your recipients. You can collect the response from the customers in an AWS SNS topic and process it as per your business needs.

Self Managed Opt-Out feature setting:

Amazon SNS setup

  1. On Amazon SNS console, click on ‘Topics’ and then ‘create a topic’ as shown below.

  1. Click ‘Create Subscription‘ to add the Lambda function ‘AddOptOutInDynamoDB‘ as a subscriber by using the Amazon Resource Name (ARN).

Amazon DynamoDB Setup

Table Name: ‘SMSOptOut

The Customer phone number is used as the primary key(PK). The sort key(SK) contains multiple values that include OID, timestamp, and the customer response separated by #. By having generic attribute names as PK and SK, you can expand the usage of this table for accommodating any custom business needs. For example: Customer can use any of the individual phone numbers like short code, long code, or 10DLC to send the SMS and any of these values can be accomodated as a part of the sort key (SK). The sort key can be used for granular retrieval to see the latest customer status (For example: ‘STOP‘). It can then additionally have attributes like OID, Timestamp, Response and others as per your requirements. The table uses On-demand Read/Write capacity mode. Refer to this document to understand On-demand capacity mode in detail.

Sample item in DynamoDB table is below


InvokeSendTextMessage function code

This AWS Lambda function calls Amazon Pinpoint SMS and Voice V2 API – SendTextMessage. It uses Query API for DynamoDB to scan the items for SourcePhoneNumber (customer phone number) and OID (Part of SK) in descending order of the timestamp. If an item exists with customer response value is a valid keyword for Opt-out (Refer section Keywords in scope), it means the customer has opted-out and the SMS can’t be sent. If no item is found or the customer response value is a valid keyword for Opt-in (Refer section Keywords in scope), the customer can be contacted and the funtion calls SendTextMessage API with the same OID and customer phone number.

import boto3
import os
from boto3.dynamodb.conditions import Key, Attr
import json

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('SMSOptOut')
pinpoint = boto3.client('pinpoint-sms-voice-v2')
OptOutKeyword = ['ARRET','CANCEL','END','OPT-OUT','OPTOUT','QUIT','REMOVE','STOP','TD','UNSUBSCRIBE']
OptInKeyword = ['JOIN']

#adds item in the DynamoDB table
def query_table(OID, SourcePhoneNumber):
    try:
        response = table.query(
            KeyConditionExpression=Key('PK').eq(SourcePhoneNumber) & Key('SK').begins_with(OID),
            ScanIndexForward=False,
            Limit=1
        )
    except Exception as e: 
        print("Error when writing an item in DynamoDB table: ",e) 
        raise e
    return response

#send opt-in/opt-out confirmation text using send_text_message.
def send_confirmation_text(SourcePhoneNumber,OID,messageBody,messageType): 
    
    try: 
        response = pinpoint.send_text_message(DestinationPhoneNumber=SourcePhoneNumber, 
        OriginationIdentity=OID,
        MessageBody=messageBody, 
        MessageType=messageType 
        ) 
    except Exception as e: 
        print("Error in sending text message using Pinpoint send_text_message api:", e) 
        raise e

#gets the message from SNS topic
def lambda_handler(event, context):
    OID = event['OID']
    SourcePhoneNumber = event['SourcePhoneNumber']

    response=query_table(OID, SourcePhoneNumber)
    items = response['Items']
    
    #Count number of items. The value will be either 1 or 0.
    count = len(items)

    # If the latest customer response is any OptOutKeyword
    if count == 1 and items[0]['Response'] in OptOutKeyword:
        print("Exit : Customer has opted out, do not send SMS")
    # If the latest customer response is any OptOutKeyword
    elif count == 0 or (count == 1 and items[0]['Response'] in OptInKeyword):
        send_confirmation_text(SourcePhoneNumber,OID,'This is a test message from Amazon Pinpoint','TRANSACTIONAL')
    # Only allowed values for customer response are valid OptOutKeyword or OptInKeyword 
    else: 
        print("The customer response is not one of the allowed keyword")

AddOptOutInDynamoDB Lambda function code

For example: When customer responds with ‘STOP’, the response is captured in the SNS topic that you configured with Amazon Pinpoint. The response json looks as shown below –

{
"originationNumber": "+1224xxxxxxx",
"destinationNumber": "+1844xxxxxxx",
"messageKeyword": "KEYWORD_xxxxxxxxxxxx",
"messageBody": "STOP",
"previousPublishedMessageId": "xxxxxxxxxxxxxx",
"inboundMessageId": "xxxxxxxxxxxxxx"
}

This Lambda function extracts the OID (destinationNumber), Customer phone number (originationNumber), and the customer response (messageBody) from the json payload above and adds an entry in the DynamoDB table (SMSOptOut). Once the item is put successfully in DynamoDB table, the function also sends out a confirmation SMS (either Opt-in or Opt-out) to the customer phone number using SendTextMessage.
For example:

  • If the customer response value is a valid keyword for Opt-out (Refer section Keywords in scope), the confirmation SMS is ‘YOU HAVE BEEN UNSUBSCRIBED. IF THIS WAS A MISTAKE PLEASE TEXT “JOIN” TO THIS NUMBER TO BE RESUBSCRIBED’.
  • If the customer response value is a valid keyword for Opt-in (Refer section Keywords in scope), the confirmation SMS is ‘YOU HAVE BEEN SUBSCRIBED. IF THIS WAS A MISTAKE PLEASE TEXT “STOP” OR “UNSUBSCRIBE” TO THIS NUMBER TO BE UNSUBSCRIBED’.
import json
import boto3
import datetime

dynamodb = boto3.resource('dynamodb')
dynamodb_table = dynamodb.Table('SMSOptOut')
pinpoint = boto3.client('pinpoint-sms-voice-v2')
OptOutKeyword = ['ARRET','CANCEL','END','OPT-OUT','OPTOUT','QUIT','REMOVE','STOP','TD','UNSUBSCRIBE']
OptInKeyword = ['JOIN']

#adds item in the DynamoDB table
def put_item(data,current_timestamp):
    print(dynamodb_table)
    try:
        response = dynamodb_table.put_item(
            Item = {
                'PK': data['originationNumber'],
                'SK': data['destinationNumber']+'#'+current_timestamp+'#'+data['messageBody'], 
                'OID': data['destinationNumber'],
                'Timestamp': current_timestamp,
                'Response': data['messageBody']
            }
        )
    except Exception as e:
        print("Error when writing an item in DynamoDB table: ",e)
        raise e

#send opt-in/opt-out confirmation text using send_text_message.
def send_confirmation_text(data,messageBody,messageType):
    try:
        response = pinpoint.send_text_message(
            DestinationPhoneNumber=data['originationNumber'],
            OriginationIdentity=data['destinationNumber'],
            MessageBody=messageBody,
            MessageType=messageType
            )
    except Exception as e:
        print("Error in sending text message using Pinpoint send_text_message api:", e)
        raise e
        
#gets the message from SNS topic
def lambda_handler(event, context):
    message = event['Records'][0]['Sns']['Message']
    data = json.loads(message)
    current_timestamp = datetime.datetime.now().isoformat()

    if data['messageBody'] in OptOutKeyword:
        put_item(data,current_timestamp)
        send_confirmation_text(data, 'YOU HAVE BEEN UNSUBSCRIBED. IF THIS WAS A MISTAKE PLEASE TEXT "JOIN" TO THIS NUMBER TO BE RESUBSCRIBED', 'TRANSACTIONAL')
    elif data['messageBody'] in OptInKeyword:
        put_item(data,current_timestamp)
        send_confirmation_text(data, 'YOU HAVE BEEN SUBSCRIBED. IF THIS WAS A MISTAKE PLEASE TEXT "STOP" OR "UNSUBSCRIBE" TO THIS NUMBER TO BE UNSUBSCRIBED', 'TRANSACTIONAL')
    else:
        print("The customer response is not one of the allowed keyword")

Clean Up

DynamoDB storage and any Lambda invocation will incur a cost so it is important to delete these resources if you do not plan on using them as shown below.

DynamoDB:

    1. On DynamoDB table, select table ‘SMSOptOut’ and click Delete. Confirm the action.

Lambda:

  1. On Lambda console, find the 2 functions you created and click Actions → Delete. Confirm the actions.

Amazon Pinpoint

  1. After deleting the DynamoDB table and Lambda functions your self-managed Opt-out flow will not work so you will need to disable self-managed opt-outs in Pinpoint for the respective OID as shown below.

Conclusion
In this post, you learned how to implement a self-managed opt-out workflow when using Pinpoint SMS. Keep in mind that when you implement the self-managed Opt-out flow, Pinpoint will not track or maintain any opt-out status for the OID that it was enabled for.

Take the time to plan out your approach, follow the steps outlined in this blog, and take advantage of any resources available to you within your support tier.

Decide what origination IDs you will need here
Review the documentation for the V2 SMS and Voice API here
Check out the support tiers comparison here

Resources:
https://docs.aws.amazon.com/sms-voice/latest/userguide/phone-numbers-sms-by-country.html
https://aws.amazon.com/blogs/messaging-and-targeting/how-to-utilise-amazon-pinpoint-to-retry-unsuccessful-sms-delivery/
https://docs.aws.amazon.com/pinpoint/latest/userguide/channels-sms-limitations-opt-out.html
https://docs.aws.amazon.com/pinpoint/latest/userguide/channels-sms-simulator.html
https://docs.aws.amazon.com/dynamodb/
https://docs.aws.amazon.com/sns/
https://docs.aws.amazon.com/lambda/

An introduction to Amazon WorkMail Audit Logging

Post Syndicated from Zip Zieper original https://aws.amazon.com/blogs/messaging-and-targeting/an-introduction-to-amazon-workmail-audit-logging/

Amazon WorkMail’s new audit logging capability equips email system administrators with powerful visibility into mailbox activities and system events across their organization. As announced in our recent “What’s New” post, this feature enables the comprehensive capture and delivery of critical email data, empowering administrators to monitor, analyze, and maintain compliance.

With audit logging, WorkMail records a wide range of events, including metadata about messages sent, received, and failed login attempts, and configuration changes. Administrators have the option to deliver these audit logs to their preferred AWS services, such as Amazon Simple Storage System (S3) for long-term storage, Amazon Kinesis Data Firehose for real-time data streaming, or Amazon CloudWatch Logs for centralized log management. Additionally, standard CloudWatch metrics on audit logs provide deep insights into the usage and health of WorkMail mailboxes within the organization.

By leveraging Amazon WorkMail’s audit logging capabilities, enterprises have the ability to strengthen their security posture, fulfill regulatory requirements, and gain critical visibility into the email activities that underpin their daily operations. This post will explore the technical details and practical use cases of this powerful new feature.

In this blog, you will learn how to configure your WorkMail organization to send email audit logs to Amazon CloudWatch Logs, Amazon S3, and Amazon Data Firehose . We’ll also provide examples that show how to monitor access to your Amazon WorkMail Organization’s mailboxes by querying the logs via CloudWatch Log Insights.

Email security

Imagine you are the email administrator for a biotech company, and you’ve received a report about spam complaints coming from your company’s email system. When you investigate, you learn these complaints point to unauthorized emails originating from several of your company’s mailboxes. One or more of your company’s email accounts may have been compromised by a hacker. You’ll need to determine the specific mailboxes involved, understand who has access to those mailboxes, and how the mailboxes have been accessed. This will be useful in identifying mailboxes with multiple failed logins or unfamiliar IP access, which can indicate unauthorized attempts or hacking. To identify the cause of the security breach, you require access to detailed audit logs and familiar tools to analyze extensive log data and locate the root of your issues.

Amazon WorkMail Audit Logging

Amazon WorkMail is a secure, managed business email service that hosts millions of mailboxes globally. WorkMail features robust audit logging capabilities, equipping IT administrators and security experts with in-depth analysis of mailbox usage patterns. Audit logging provides detailed insights into user activities within WorkMail. Organizations can detect potential security vulnerabilities by utilizing audit logs. These logs document user logins, access permissions, and other critical activities. WorkMail audit logging facilitates compliance with various regulatory requirements, providing a clear audit trail of data privacy and security. WorkMail’s audit logs are crucial for maintaining the integrity, confidentiality, and reliability of your organization’s email system.

Understanding WorkMail Audit Logging

Amazon WorkMail’s audit logging feature provides you with the data you need to have a thorough understanding of your email mailbox activities. By sending detailed logs to Amazon CloudWatch Logs, Amazon S3, and Amazon Data Firehose, administrators can identify mailbox access issues, track access by IP addresses, and review mailbox data movements or deletions using familiar tools. It is also possible to configure multiple destinations for each log to meet the needs of a variety of use cases, including compliance archiving.

WorkMail offers four audit logs:

  • ACCESS CONTROL LOGS – These logs record evaluations of access control rules, noting whether access to the endpoint was granted or denied in accordance with the configured rules;
  • AUTHENTICATION LOGS – These logs capture details of login activities, chronicling both successful and failed authentication attempts;
  • AVAILABILITY PROVIDER LOGS – These logs document the use of the Availability Providers feature, tracking its operational status and interactions feature;
  • MAILBOX ACCESS LOGS – Logs in this category record each attempt to access mailboxes within the WorkMail Organization, providing a detailed account of credential and protocol access patterns.

Once audit logging is enabled, alerts can be configured to warn of authentication or access anomalies that surpass predetermined thresholds. JSON formatting allows for advanced processing and analysis of audit logs by third party tools. Audit logging stores all interactions with the exception of web mail client authentication metrics.

WorkMail audit logging in action

Below are two examples that show how WorkMail’s audit logging can be used to investigate unauthorized login attempts, and diagnose a misconfigured email client. In both examples, we’ll use WorkMail’s Mailbox Access Control Logs and query the mailbox access control logs in CloudWatch Log Insights.

In our first example, we’re looking for unsuccessful login attempts in a target timeframe. In CloudWatch Log Insights we run this query:

fields user, source_ip, protocol, auth_successful, auth_failed_reason | filter auth_successful = 0

CloudWatch Log Insights returns all records in the timeframe, providing auth_succesful = 0 (false) and auth_failed_reason = Invalid username or password. We also see the source_ip, which we may decide to block in a WorkMail access control rule, or any other network security system.

Log - unsuccessful Login Attempt

Mailbox Access Control Log – an unsuccessful login attempt

In this next example, consider a WorkMail organization that has elected to block the IMAP protocol using a WorkMail access control rule (below):

WorkMail Access Control Rule blocking IMAP

WorkMail Access Control Rule – block IMAP protocol

Because some email clients use IMAP by default, occasionally new users in this example organization are denied access to email due to an incorrectly configured email client. Using WorkMail’s mailbox access control logs in CloudWatch Log Insights we run this query:

fields user_id, source_ip, protocol, rule_id, access_granted | filter access_granted = 0

And we see the user’s attempt to access their email inbox via IMAP has been denied by the access control rule_id (below):

WorkMail Access Control logs - IMAP blocked by access rule

WorkMail Access Control logs – IMAP blocked by access rule

Conclusion

Amazon WorkMail’s audit logging feature offers comprehensive view of your organization’s email activities. Four different logs provide visibility into access controls, authentication attempts, interactions with external systems, and mailbox activities. It provides flexible log delivery through native integration with AWS services and tools. Enabling WorkMail’s audit logging capabilities helps administrators meet compliance requirements and enhances the overall security and reliability of their email system.

To learn more about audit logging on Amazon WorkMail, you may comment on this post (below), view the WorkMail documentation, or reach out to your AWS account team.

To learn more about Amazon WorkMail, or to create a no-cost 30-day test organization, see Amazon WorkMail.

About the Authors

Miguel

Luis Miguel Flores dos Santos

Miguel is a Solutions Architect at AWS, boasting over a decade of expertise in solution architecture, encompassing both on-premises and cloud solutions. His focus lies on resilience, performance, and automation. Currently, he is delving into serverless computing. In his leisure time, he enjoys reading, riding motorcycles, and spending quality time with family and friends.

Andy Wong

Andy Wong

Andy Wong is a Sr. Product Manager with the Amazon WorkMail team. He has 10 years of diverse experience in supporting enterprise customers and scaling start-up companies across different industries. Andy’s favorite activities outside of technology are soccer, tennis and free-diving.

Zip

Zip

Zip is a Sr. Specialist Solutions Architect at AWS, working with Amazon Pinpoint and Simple Email Service and WorkMail. Outside of work he enjoys time with his family, cooking, mountain biking, boating, learning and beach plogging.

How to Send SMS Using a Sender ID with Amazon Pinpoint

Post Syndicated from Tyler Holmes original https://aws.amazon.com/blogs/messaging-and-targeting/how-to-send-sms-using-a-sender-id-with-amazon-pinpoint/

Amazon Pinpoint enables you to send text messages (SMS) to recipients in over 240 regions and countries around the world. Pinpoint supports all types of origination identities including Long Codes, 10DLC (US only), Toll-Free (US only), Short Codes, and Sender IDs.
NOTE: Certain subtypes of Origination Identities (OIDs), such as Free to End User (FTEU) Short Codes might not be supported

Unlike other origination identities, a Sender ID can include letters, enabling your recipients to receive SMS from “EXAMPLECO” rather than a random string of numbers or phone number. A Sender ID can help to create trust and brand awareness with your recipients which can increase your deliverability and conversion rates by improving customer interaction with your messages. In this blog we will discuss countries that allow the use of Sender IDs, the types of Sender IDs that Pinpoint supports, how to configure Pinpoint to use a Sender ID, and best practices for sending SMS with a Sender ID. Refer to this blog post for guidance planning a multi-country rollout of SMS.

What is a Sender ID?

A sender ID is an alphanumeric name that identifies the sender of an SMS message. When you send an SMS message using a sender ID, and the recipient is in an area where sender ID is supported, your sender ID appears on the recipient’s device instead of a phone number, for example, they will see AMAZON instead of a phone number such as “1-206-555-1234”. Sender IDs support a default throughput of 10 messages per second (MPS), which can be raised in certain situations. This MPS is calculated at the country level. As an example, a customer can send 10 MPS with “SENDERX” to Australia (AU) and 10 MPS with “SENDERX” to Germany (DE).

The first step in deciding whether to use a Sender ID is finding out whether the use of a Sender ID is supported by the country(ies) you want to send to. This table lists all of the destinations that Pinpoint can send SMS to and the Origination Identities that they support. Many countries support multiple origination identities so it’s important to know the differences between them. The two main considerations when deciding what originator to use is throughput and whether it can support a two-way use case.

Amazon Pinpoint supports three types of Sender IDs detailed below. Your selection is dependent on the destination country for your messages, consult this table that lists all of the destinations that Pinpoint can send SMS to.

Dynamic Sender ID – A dynamic sender ID allows you to select a 3-11 character alphanumeric string to use as your originator when sending SMS. We suggest using something short that outlines your brand and use case like “[Company]OTP.” Dynamic sender IDs vary slightly by country and we recommended senders review the specific requirements for the countries they plan to send to. Pay special attention to any notes in the registration section. If the country(ies) you want to send to require registration, read on to the next type of Sender ID.

Registered Sender ID – A registered SenderID generally follows the same formatting requirements as a Dynamic Sender ID, alowing you to select a 3-11 character alphanumeric string to use, but has the added step of completing a registration specific to the country you want to use a Sender ID for. Each country will require slightly different information to register, may require specific forms, as well as a potential registration fee. Use this list of supported countries to see what countries support Sender ID as well as which ones require registration.

Generic, or “Shared” Sender ID – In countries where it is supported, when you do not specify a dynamic Sender ID, or you have not set a Default Sender ID in Pinpoint, the service may allow traffic over a Generic or Shared SenderID like NOTICE. Depending on the country, traffic could also be delivered using a service or carrier specific long or short code. When using the shared route your messages will be delivered alongside others also sending in this manner.

As mentioned, Sender IDs support 10 MPS, so if you do not need higher throughput than this may be a good option. However, one of the key differences of using a Sender ID to send SMS is that they do not support two-way use cases, meaning they cannot receive SMS back from your recipients.

IMPORTANT: If you use a sender ID you must provide your recipients with alternative ways to opt-out of your communications as they cannot text back any of the standard opt-out keywords or any custom opt-in keywords you may have configured. Common ways to offer your recipients an alternative way of opting out or changing their communication preferences include web forms or an app preference center.

How to Configure a Sender ID

The country(ies) you plan on sending to using a Sender ID will determine the configuration you will need to complete to be able to use them. We will walk through the configuration of each of the three types of Sender IDs below.

Step 1 – Request a Sender ID(Dependent on Country, Consult this List)

Some countries require a registration process. Each process, dependent on the country can be unique so it is required that a case be opened to complete this process. The countries requiring Sender ID registration are noted in the following list.
When you request a Sender ID, we provide you with an estimate of how long the request will take to complete. This estimate is based on the completion times that we’ve seen from other customers.

NOTE: This time is not an SLA. It is recommended that you check the case regularly and make sure that nothing else is required to complete your registration. If your registration requires edits it will extend this process each time it requires edits. If your registration passes over the estimated time it is recommended that you reply to the case.

Because each country has its own process, completion times for registration vary by destination country. For example, Sender ID registration in India can be completed in one week or less, whereas it can take six weeks or more in Vietnam. These requests can’t be expedited, because they involve the carriers themselves making changes to the ways that their networks are configured and certify the use case onto their network. We suggest that you start your registration process early so that you can start sending messages as soon as you launch your product or service.

IMPORTANT: Make sure that you are checking on your case often as support may need more details to complete your registration and any delay extends the expected timeline for procuring your Sender ID

Generic Sender ID – In countries that support a Generic or Shared ID like NOTICE there is no requirement to register or configure prior to sending we will review how to send with this type of Sender ID in Step 2.

Dynamic Sender ID – A Dynamic Sender ID can be requested via the API or in the console, complete the following steps to configure these Sender IDs in the console.
NOTE: If you are using the API to send it is not required that you request a Sender ID for every country that you intend on sending to. However, it is recommended, because the request process will alert you to any Sender IDs that require registration so you do not attempt to send to countries that you cannot deliver successfully to. All countries requiring registration for Sender IDs can be found here.

  1. Navigate to the SMS Console
    1. Make sure you are in the region you plan on using to send SMS out of as each region needs to be configured independently and any registrations also need to be made in the account and region in which you will be sending from
  2. Select “Sender IDs” from the left rail
    1. Click on “Request Originator”
    2. Choose a country from the drop down that supports Sender ID
    3. Choose “SMS”
      1. Leave “Voice” unchecked if it is an option.
        NOTE: If you choose Voice than you will not be able to select a Sender ID in the next step
      2. Select your estimated SMS volume
      3. Choose whether your company is local or international in relation to the country you are wanting to configure. Some countries, like India, require proof of residency to access local pricing so select accordingly.
      4. Select “No” for two-way messaging or you will not be able to select a Sender ID in the next step
    4. Click next and choose “Sender ID” and provide your preferred Sender ID.
      NOTE: Refer to the following criteria when selecting your Sender ID for configuration (some countries may override these)

      1. Numeric-only Sender IDs are not supported
      2. No special characters except for dashes ( – )
      3. No spaces
      4. Valid characters: a-z, A-Z, 0-9
      5. Minimum of 3 characters
      6. Maximum of 11 characters. NOTE: India is exactly 6 Characters
      7. Must match your company branding and SMS service or use case.
        1. For example, you wouldn’t be able to choose “OTP” or “2FA” even though you might be using SMS for that type of a use case. You could however use “ANYCO-OTP” if your company name was “Any Co.” since it complies with all above criteria.


NOTE: If the console instructs you to open a case as seen below than your Sender ID likely requires some form of registration. Read on to configure a Registered Sender ID.

Registered Sender ID – A registered sender ID follows the same criteria above for a Dynamic Sender ID, although some countries may have minor criteria changes or formatting restrictions. Follow the directions here to complete this process, AWS support will provide the correct forms needed for the country that you are registering. Each Registered Sender ID will need a separate case per country. Follow the link to the “AWS Support Center” and follow these instructions when creating your case

Step 2 – How to Send SMS with a Sender ID

Sender IDs can be used via three different mechanisms

Option 1 – Using the V2 SMS and Voice API and “SendTextMessage
This is the preferred method of sending and this set of APIs is where all new functionality will be built on.

  1. SendTextMessage has many options for configurability but the only required parameters are the following:
    1. DestinationPhoneNumber
    2. MessageBody
  2. “OriginationIdentity” is optional, but it’s important to know what the outcome is dependent on how you use this parameter:
    1. Explicitly stating your SenderId
      1. Use this option if you want to ONLY send with a Sender ID. Setting this has the effect of only sending to recipients in countries that accept SenderIDs and rejecting any recipients whose country does not support Sender IDs. The US for example cannot be sent to with a Sender ID
    2. Explicitly stating your SenderIdArn
      1. Same effect as “SenderID” above
    3. Leaving OriginationIdentity Blank
      1. If left blank Pinpoint will select the right originator based on what you have available in your account in order of decreasing throughput, from a Short Code, 10DLC (US Only), Long Code, Sender ID, or Toll-Free (US Only), depending on what you have available.
        1. Keep in mind that sending this way opens you up to sending to countries you may not have originators for. If you would like to make sure that you are only sending to countries that you have originators for then you need to use Pools.
    4. Explicitly stating a PoolId
      1. A pool is a collection of Origination Identities that can include both phone numbers and Sender IDs. Use this option if you are sending to multiple country codes and want to make sure that you send to them with the originator that their respective country supports.
        1. NOTE: There are various configurations that can be set on a pool. Refer to the documentation here
          1. Make sure to pay particular attention to “Shared Routes” because in some countries, Amazon Pinpoint SMS maintains a pool of shared origination identities. When you activate shared routes, Amazon Pinpoint SMS makes an effort to deliver your message using one of the shared identities. The origination identity could be a sender ID, long code, or short code and could vary within each country. Turn this feature off if you ONLY want to send to countries for which you have an originator.
          2. Make sure to read this blogpost on Pools and Opt – Outs here
    5. Explicitly stating a PoolArn
      1. Same effect as “PoolId” above

Option 2 – Using a journey or a Campaign

  1. If you do not select an “Origination Phone Number” or a Sender ID Pinpoint will select the correct originator based on the country code being attempted to send to and the originators available in your account.
    1. Pinpoint will attempt to send, in order of decreasing throughput, from a Short Code, 10DLC (US Only), Long Code, Sender ID, or Toll-Free (US Only), depending on what you have available. For example, if you want to send from a Sender ID to Germany (DE), but you have a Short Code configured for Germany (DE) as well, the default function is for Pinpoint to send from that Short Code. If you want to override this functionality you must specify a Sender ID to send from.
      1. NOTE: If you are sending to India on local routes you must fill out the “Entity ID and Template ID that you received when you registered your template with the Telecom Regulatory Authority of India (TRAI)
    2. You can set a default Sender ID for your Project in the SMS settings as seen below.
      NOTE: Anything you configure at the Campaign or Journey level overrides this project level setting

Option 3 – Using Messages in the Pinpoint API

  1. Using “Messages“ is the second option for sending via the API. This action allows for multi-channel(SMS, email, push, etc) bulkified sending but is not recommended to standardize on for SMS sending.
    1. NOTE: Using the V2 SMS and Voice API and “SendTextMessage” detailed in Option 1 above is the preferred method of sending SMS via the API and is where new features and functionality will be released. It is recommended that you migrate SMS sending to this set of APIs.

Conclusion:
In this post you learned about Sender IDs and how they can be used in your SMS program. A Sender ID can be a great option for getting your SMS program up and running quickly since they can be free, many countries do not require registration, and you can use the same Sender ID for lots of different countries, which can improve your branding and engagement. Keep in mind that one of the big differences in using a Sender ID vs. a short code or long code is that they don’t support 2-way communication. Common ways to offer your recipients an alternative way of opting out or changing their communication preferences include web forms or an app preference center.

A few resources to help you plan for your SMS program:
Use this spreadsheet to plan for the countries you need to send to Global SMS Planning Sheet
The V2 API for SMS and Voice has many more useful actions not possible with the V1 API so we encourage you to explore how it can further help you simplify and automate your applications.
If you are needing to use pools to access the “shared pools” setting read this blog to review how to configure them
Confirm the origination IDs you will need here
Check out the support tiers comparison here

How large senders can move from sandbox to production using Amazon SES?

Post Syndicated from Medha Karri original https://aws.amazon.com/blogs/messaging-and-targeting/how-large-senders-can-move-from-sandbox-to-production-using-amazon-ses/

Amazon SES: Email marketing has a potential ROI of $42 for every dollar spent (source link) making it a great tool for businesses whether it is for marketing campaigns, transactional notifications, or other communications. Amazon Simple Email Service (Amazon SES) is a cloud email service provider that can integrate into any application for bulk email sending. Amazon SES is an email service that supports a variety of use cases like transactional emails, system alerts, marketing/promotional/bulk emails, streamlined internal communications, and emails triggered by CRM system as a few examples.

Your journey with AWS began with creating an AWS account and your journey with Amazon SES likely began in the sandbox environment. To help prevent fraud and abuse, and to help protect your reputation as a sender, Amazon SES places all new accounts in the Amazon SES sandbox. Sandbox helps protect accounts from unauthorized use, accidental sends, and unexpected charges and is a safe space for testing with limited sending capabilities – up to 200 emails per day and a rate of 1 email per second.

Transitioning from Sandbox to Production: When you are ready to scale up to production, the process involves a few steps:

    1. Verify your email or domain: Prior to requesting production access, you have to verify an email address or sending domain. You can do that by clicking on Configuration > Verified Identities and click on Create identity button
    2. Access the set up page: On the Account dashboard page click on Get started (image 2.1) or go to Get set up page on the navigation frame on the left.
    3. Before requesting for production access, it is important to test throttling, bounce handling, and unsubscribe handling.
    4. Click on Request production access
    5. Production access form: This brings you to the page where you furnish details to get production access
        1. Enter if your mail type is marketing or transactional. Choose the option that best represents the types of messages you plan on sending. A marketing email promotes your products and services, while a transactional email is an immediate, trigger-based communication.
        2. Provide the URL for your website to help us better understand the kind of content you plan on sending.
        3. Use case description: Here is where you mention the following:
          1. Description: What does your company do and what do you plan on communicating with your users/subscribers through email?
          2. Use cases: Describe at a minimum, 1 or 2 of your use cases here and be descriptive of the use-cases you plan to use SES as a sender. You can also paste what a sample email for this use case looks like (please remove sensitive information)
          3. Mailing list: Describe how you plan to build or acquire your mailing list.
          4. Bounces & complaints: Describe how you handle bounces & complaints.
            1. Amazon SES provides you with resources to manage this. This is a guide on how you can set up notifications for bounces and complaints. After you are notified, how do you plan on handling the bounces and complaints?
          5. Unsubscribe: Describe how your email recipients can opt out of receiving email from you. Amazon SES provides subscription management and you can read more about it here. Additionally, you can read more about the latest email sender requirements here.
        4. Best practices:
          1. Success of your email program depends on various metrics such as bounces, complaints and message quality as listed here. Test your setup and your bounce/complaint processing before requesting production access.
          2. Mention if your account was denied earlier and the reasons for denial (any additional information you can provide will help speed up the process).
          3. Provide your daily and weekly email volumes.
          4. Provide your peak volume throughput or TPS (transactions/emails per second).
          5. We consider each request carefully. Therefore, it is important to provide specifics and not vague messages like “Please remove from sandbox and move to production” or “Please increase sending limit to 40 emails/sec”
          6. More best practices here.

Conclusion: Successfully moving from the sandbox to production in Amazon SES marks a significant step in leveraging email communication for your business. It’s not just about scaling your email capabilities; it’s about enhancing your engagement with customers and prospects through reliable, efficient email delivery. Continuously monitor your email performance, stay updated with Amazon SES features, and adapt your strategy to ensure your email campaigns remain effective and compliant. With these steps and insights, you’re well-equipped to make the most out of Amazon SES, turning it into a vital component of your digital communication strategy. Once your request has been approved, you’ll receive a confirmation from Amazon SES, and you’ll be ready to start sending emails to real recipients.

About the authors:

Medha Karri

Medha Karri is a Senior Product Manager at Amazon Simple Email Service at AWS. He is a technology enthusiast having varied experience in product management and software development. He is passionate to simplify complex technical solutions for customers and enjoys playing Xbox in his free time.

Vinay Ujjini

Vinay Ujjini is an Amazon Pinpoint and Amazon Simple Email Service Worldwide Principal Specialist Solutions Architect at AWS. He has been solving customer’s omni-channel challenges for over 15 years. He is an avid sports enthusiast and in his spare time, enjoys playing tennis & cricket.

Upgrade Your Email Tech Stack with Amazon SESv2 API

Post Syndicated from Zip Zieper original https://aws.amazon.com/blogs/messaging-and-targeting/upgrade-your-email-tech-stack-with-amazon-sesv2-api/

Amazon Simple Email Service (SES) is a cloud-based email sending service that helps businesses and developers send marketing and transactional emails. We introduced the SESv1 API in 2011 to provide developers with basic email sending capabilities through Amazon SES using HTTPS. In 2020, we introduced the redesigned Amazon SESv2 API, with new and updated features that make it easier and more efficient for developers to send email at scale.

This post will compare Amazon SESv1 API and Amazon SESv2 API and explain the advantages of transitioning your application code to the SESv2 API. We’ll also provide examples using the AWS Command-Line Interface (AWS CLI) that show the benefits of transitioning to the SESv2 API.

Amazon SESv1 API

The SESv1 API is a relatively simple API that provides basic functionality for sending and receiving emails. For over a decade, thousands of SES customers have used the SESv1 API to send billions of emails. Our customers’ developers routinely use the SESv1 APIs to verify email addresses, create rules, send emails, and customize bounce and complaint notifications. Our customers’ needs have become more advanced as the global email ecosystem has developed and matured. Unsurprisingly, we’ve received customer feedback requesting enhancements and new functionality within SES. To better support an expanding array of use cases and stay at the forefront of innovation, we developed the SESv2 APIs.

While the SESv1 API will continue to be supported, AWS is focused on advancing functionality through the SESv2 API. As new email sending capabilities are introduced, they will only be available through SESv2 API. Migrating to the SESv2 API provides customers with access to these, and future, optimizations and enhancements. Therefore, we encourage SES customers to consider the information in this blog, review their existing codebase, and migrate to SESv2 API in a timely manner.

Amazon SESv2 API

Released in 2020, the SESv2 API and SDK enable customers to build highly scalable and customized email applications with an expanded set of lightweight and easy to use API actions. Leveraging insights from current SES customers, the SESv2 API includes several new actions related to list and subscription management, the creation and management of dedicated IP pools, and updates to unsubscribe that address recent industry requirements.

One example of new functionality in SESv2 API is programmatic support for the SES Virtual Delivery Manager. Previously only addressable via the AWS console, VDM helps customers improve sending reputation and deliverability. SESv2 API includes vdmAttributes such as VdmEnabled and DashboardAttributes as well as vdmOptions. DashboardOptions and GaurdianOptions.

To improve developer efficiency and make the SESv2 API easier to use, we merged several SESv1 APIs into single commands. For example, in the SESv1 API you must make separate calls for createConfigurationSet, setReputationMetrics, setSendingEnabled, setTrackingOptions, and setDeliveryOption. In the SESv2 API, however, developers make a single call to createConfigurationSet and they can include trackingOptions, reputationOptions, sendingOptions, deliveryOptions. This can result in more concise code (see below).

SESv1-vs-SESv2

Another example of SESv2 API command consolidation is the GetIdentity action, which is a composite of SESv1 API’s GetIdentityVerificationAttributes, GetIdentityNotificationAttributes, GetCustomMailFromAttributes, GetDKIMAttributes, and GetIdentityPolicies. See SESv2 documentation for more details.

Why migrate to Amazon SESv2 API?

The SESv2 API offers an enhanced experience compared to the original SESv1 API. Compared to the SESv1 API, the SESv2 API provides a more modern interface and flexible options that make building scalable, high-volume email applications easier and more efficient. SESv2 enables rich email capabilities like template management, list subscription handling, and deliverability reporting. It provides developers with a more powerful and customizable set of tools with improved security measures to build and optimize inbox placement and reputation management. Taken as a whole, the SESv2 APIs provide an even stronger foundation for sending critical communications and campaign email messages effectively at a scale.

Migrating your applications to SESv2 API will benefit your email marketing and communication capabilities with:

  1. New and Enhanced Features: Amazon SESv2 API includes new actions as well as enhancements that provide better functionality and improved email management. By moving to the latest version, you’ll be able to optimize your email sending process. A few examples include:
    • Increase the maximum message size (including attachments) from 10Mb (SESv1) to 40Mb (SESv2) for both sending and receiving.
    • Access key actions for the SES Virtual Deliverability Manager (VDM) which provides insights into your sending and delivery data. VDM provides near-realtime advice on how to fix the issues that are negatively affecting your delivery success rate and reputation.
    • Meet Google & Yahoo’s June 2024 unsubscribe requirements with the SES v2 SendEmail action. For more information, see the “What’s New blog”
  2. Future-proof Your Application: Avoid potential compatibility issues and disruptions by keeping your application up-to-date with the latest version of the Amazon SESv2 API via the AWS SDK.
  3. Improve Usability and Developer Experience: Amazon SESv2 API is designed to be more user-friendly and consistent with other AWS services. It is a more intuitive API with better error handling, making it easier to develop, maintain, and troubleshoot your email sending applications.

Migrating to the latest SESv2 API and SDK positions customers for success in creating reliable and scalable email services for their businesses.

What does migration to the SESv2 API entail?

While SESv2 API builds on the v1 API, the v2 API actions don’t universally map exactly to the v1 API actions. Current SES customers that intend to migrate to SESv2 API will need to identify the SESv1 API actions in their code and plan to refactor for v2. When planning the migration, it is essential to consider several important considerations:

  1. Customers with applications that receive email using SESv1 API’s CreateReceiptFilter, CreateReceiptRule or CreateReceiptRuleSet actions must continue using the SESv1 API client for these actions. SESv1 and SESv2 can be used in the same application, where needed.
  2. We recommend all customers follow the security best practice of “least privilege” with their IAM policies. As such, customers may need to review and update their policies to include the new and modified API actions introduced in SESv2 before migrating. Taking the time to properly configure permissions ensures a seamless transition while maintaining a securely optimized level of access. See documentation.

Below is an example of an IAM policy with a user with limited allow privileges related to several SESv1 Identity actions only:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ses:VerifyEmailIdentity",
                "ses:Deleteldentity",
                "ses:VerifyDomainDkim",
                "ses:ListIdentities",
                "ses:VerifyDomainIdentity"
            ],
            "Resource": "*"
        }
    ]
}

When updating to SESv2, you need to update this user’s permissions with the SESv2 actions shown below:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ses:CreateEmailIdentity",
                "ses:DeleteEmailIdentity",
                "ses:GetEmailIdentity",
                "ses:ListEmailIdentities"
            ],
            "Resource": "*"
        }
    ]
}

Examples of SESv1 vs. SESv2 APIs

Let’s look at a three examples that compare the SESv1 API with the SESv2 API.

LIST APIs

When listing identities in SESv1 list API, you need to specify type which requires multiple calls to API to list all resources:

aws ses list-identities --identity-type Domain
{
    "Identities": [
        "example.com"
    ]
}
aws ses list-identities --identity-type EmailAddress
{
    "Identities": [
        "[email protected]",
        "[email protected]",
        "[email protected]"
    ]
}

With SESv2, you can simply call a single API. Additionally, SESv2 also provides extended feedback:

aws sesv2 list-email-identities
{
    "EmailIdentities": [
        {
            "IdentityType": "DOMAIN",
            "IdentityName": "example.com",
            "SendingEnabled": false,
            "VerificationStatus": "FAILED"
        },
        {
            "IdentityType": "EMAIL_ADDRESS",
            "IdentityName": "[email protected]",
            "SendingEnabled": true,
            "VerificationStatus": "SUCCESS"
        },
        {
            "IdentityType": "EMAIL_ADDRESS",
            "IdentityName": "[email protected]",
            "SendingEnabled": false,
            "VerificationStatus": "FAILED"
        },
        {
            "IdentityType": "EMAIL_ADDRESS",
            "IdentityName": "[email protected]",
            "SendingEnabled": true,
            "VerificationStatus": "SUCCESS"
        }
    ]
}

CREATE APIs

With SESv1, creating email addresses or domains requires calling two different APIs:

aws ses verify-email-identity --email-address [email protected]
aws ses verify-domain-dkim --domain example.com
{
    "DkimTokens": [
        "mwmzhwhcebfh5kvwv7zahdatahimucqi",
        "dmlozjwrdbrjfwothoh26x6izvyts7qx",
        "le5fy6pintdkbxg6gdoetgbrdvyp664v"
    ]
}

With SESv2, we build an abstraction so you can call a single API. Additionally, SESv2 provides more detailed responses and feedback:

aws sesv2 create-email-identity --email-identity [email protected]
{
    "IdentityType": "EMAIL_ADDRESS",
    "VerifiedForSendingStatus": false
}
aws sesv2 create-email-identity --email-identity example.com
{
    "IdentityType": "DOMAIN",
    "VerifiedForSendingStatus": false,
    "DkimAttributes": {
        "SigningEnabled": true,
        "Status": "NOT_STARTED",
        "Tokens": [
            "mwmzhwhcebfh5kvwv7zahdatahimucqi",
            "dmlozjwrdbrjfwothoh26x6izvyts7qx",
            "le5fy6pintdkbxg6gdoetgbrdvyp664v"
        ],
        "SigningAttributesOrigin": "AWS_SES",
        "NextSigningKeyLength": "RSA_2048_BIT",
        "CurrentSigningKeyLength": "RSA_2048_BIT",
        "LastKeyGenerationTimestamp": "2024-02-23T15:01:53.849000+00:00"
    }
}

DELETE APIs

When calling delete- with SESv1, SES returns 200 (or no response), even if the identity was previously deleted or doesn’t exist:

 aws ses delete-identity --identity example.com

SESv2 provides better error handling and responses when calling the delete API:

aws sesv2 delete-email-identity --email-identity example.com

An error occurred (NotFoundException) when calling the DeleteEmailIdentity operation: Email identity example.com does not exist.

Hands-on with SESv1 API vs. SESv2 API

Below are a few examples you can use to explore the differences between SESv1 API and the SESv2 API. To complete these exercises, you’ll need:

  1. AWS Account (setup) with enough permission to interact with the SES service via the CLI
  2. Upgrade to the latest version of the AWS CLI (aws-cli/2.15.27 or greater)
  3. SES enabled, configured and properly sending emails
  4. A recipient email address with which you can check inbound messages (if you’re in the SES Sandbox, this email must be verified email identity). In the following examples, replace [email protected] with the verified email identity.
  5. Your preferred IDE with AWS credentials and necessary permissions (you can also use AWS CloudShell)

Open the AWS CLI (or AWS CloudShell) and:

  1. Create a test directory called v1-v2-test.
  2. Create the following (8) files in the v1-v2-test directory:

destination.json (replace [email protected] with the verified email identity):

{ 
    "ToAddresses": ["[email protected]"] 
}

ses-v1-message.json

{
   "Subject": {
       "Data": "SESv1 API email sent using the AWS CLI",
       "Charset": "UTF-8"
   },
   "Body": {
       "Text": {
           "Data": "This is the message body from SESv1 API in text format.",
           "Charset": "UTF-8"
       },
       "Html": {
           "Data": "This message body from SESv1 API, it contains HTML formatting. For example - you can include links: <a class=\"ulink\" href=\"http://docs.aws.amazon.com/ses/latest/DeveloperGuide\" target=\"_blank\">Amazon SES Developer Guide</a>.",
           "Charset": "UTF-8"
       }
   }
}

ses-v1-raw-message.json (replace [email protected] with the verified email identity):

{
     "Data": "From: [email protected]\nTo: [email protected]\nSubject: Test email sent using the SESv1 API and the AWS CLI \nMIME-Version: 1.0\nContent-Type: text/plain\n\nThis is the message body from the SESv1 API SendRawEmail.\n\n"
}

ses-v1-template.json (replace [email protected] with the verified email identity):

{
  "Source":"SES Developer<[email protected]>",
  "Template": "my-template",
  "Destination": {
    "ToAddresses": [ "[email protected]"
    ]
  },
  "TemplateData": "{ \"name\":\"SESv1 Developer\", \"favoriteanimal\": \"alligator\" }"
}

my-template.json (replace [email protected] with the verified email identity):

{
  "Template": {
    "TemplateName": "my-template",
    "SubjectPart": "Greetings SES Developer, {{name}}!",
    "HtmlPart": "<h1>Hello {{name}},</h1><p>Your favorite animal is {{favoriteanimal}}.</p>",
    "TextPart": "Dear {{name}},\r\nYour favorite animal is {{favoriteanimal}}."
  }
}

ses-v2-simple.json (replace [email protected] with the verified email identity):

{
    "FromEmailAddress": "[email protected]",
    "Destination": {
        "ToAddresses": [
            "[email protected]"
        ]
    },
    "Content": {
        "Simple": {
            "Subject": {
                "Data": "SESv2 API email sent using the AWS CLI",
                "Charset": "utf-8"
            },
            "Body": {
                "Text": {
                    "Data": "SESv2 API email sent using the AWS CLI",
                    "Charset": "utf-8"
                }
            },
            "Headers": [
                {
                    "Name": "List-Unsubscribe",
                    "Value": "insert-list-unsubscribe-here"
                },
				{
                    "Name": "List-Unsubscribe-Post",
                    "Value": "List-Unsubscribe=One-Click"
                }
            ]
        }
    }
}

ses-v2-raw.json (replace [email protected] with the verified email identity):

{
     "FromEmailAddress": "[email protected]",
     "Destination": {
            "ToAddresses": [
                       "[email protected]"
              ]
       },
      "Content": {
             "Raw": {
                     "Data": "Subject: Test email sent using SESv2 API via the AWS CLI \nMIME-Version: 1.0\nContent-Type: text/plain\n\nThis is the message body from SendEmail Raw Content SESv2.\n\n"
              }
      }
}

ses-v2-tempate.json (replace [email protected] with the verified email identity):

{
     "FromEmailAddress": "[email protected]",
     "Destination": {
       "ToAddresses": [
         "[email protected]"
       ]
     },
     "Content": {
        "Template": {
          "TemplateName": "my-template",
          "TemplateData": "{ \"name\":\"SESv2 Developer\",\"favoriteanimal\":\"Dog\" }",
          "Headers": [
                {
                   "Name": "List-Unsubscribe",
                   "Value": "insert-list-unsubscribe-here"
                },
                {
                   "Name": "List-Unsubscribe-Post",
                   "Value": "List-Unsubscribe=One-Click"
                }
             ]
         }
     }
}

Perform the following commands using the SESv1 API:

send-email (simple):

aws ses send-email --from [email protected] --destination file://destination.json --message file://ses-v1-message.json 
  • The response will return a valid MessageID (signaling the action was successful). An email will be received by the verified email identity.
{
    "MessageId": "0100018dc7649400-Xx1x0000x-bcec-483a-b97c-123a4567890d-xxxxx"
}

send-raw-email:

  • In the CLI, run:
aws ses send-raw-email  --cli-binary-format raw-in-base64-out --raw-message file://ses-v1-raw-message.json 
  • The response will return a valid MessageID (signaling the action was successful). An email will be received by the verified email identity.
{
   "MessageId": "0200018dc7649400-Xx1x1234x-bcec-483a-b97c-123a4567890d-
}

send templated mail:

  • In the CLI, run the following to create the template:
aws ses create-template  --cli-input-json file://my-template.json
  • In the CLI, run:

aws ses send-templated-email --cli-input-json file://ses-v1-template.json

  • The response will return a valid MessageID (signaling the action was successful). An email will be received by the verified email identity.
 {
    "MessageId": "0000018dc7649400-Xx1x1234x-bcec-483a-b97c-123a4567890d-xxxxx"
 }

Perform similar commands using the SESv2 API:

As mentioned above, customers who are using least privilege permissions with SESv1 API must first update their IAM policies before running the SESv2 API examples below. See documentation for more info.

As you can see from the .json files we created for SES v2 API (above), you can modify or remove sections from the .json files, based on the type of email content (simple, raw or templated) you want to send.

Please ensure you are using the latest version of the AWS CLI (aws-cli/2.15.27 or greater).

Send simple email

  • In the CLI, run:
aws sesv2 send-email --cli-input-json file://ses-v2-simple.json
  • The response will return a valid MessageID (signaling the action was successful). An email will be received by the verified email identity
{
    "MessageId": "0100018dc83ba7e0-7b3149d7-3616-49c2-92b6-00e7d574f567-000000"
}

Send raw email (note – if the only reason is to set custom headers, you don’t need to send raw email)

  • In the CLI, run:
aws sesv2 send-email --cli-binary-format raw-in-base64-out --cli-input-json file://ses-v2-raw.json
  • The response will return a valid MessageID (signaling the action was successful). An email will be received by the verified email identity.
{
    "MessageId": "0100018dc877bde5-fdff0df3-838e-4f51-8582-a05237daecc7-000000"
}

Send templated email

  • In the CLI, run:
aws sesv2 send-email --cli-input-json file://ses-v2-tempate.json
  • The response will return a valid MessageID (signaling the action was successful). An email will be received by the verified email identity.
{
    "MessageId": "0100018dc87fe72c-f2c547a1-2325-4be4-bf78-b91d6648cd12-000000"
}

Migrating your application code to SESv2 API

As you can see from the examples above, SESv2 API shares much of its syntax and actions with the SESv1 API. As a result, most customers have found they can readily evaluate, identify and migrate their application code base in a relatively short period of time. However, it’s important to note that while the process is generally straightforward, there may be some nuances and differences to consider depending on your specific use case and programming language.

Regardless of the language, you’ll need anywhere from a few hours to a few weeks to:

  • Update your code to use SESv2 Client and change API signature and request parameters
  • Update permissions / policies to reflect SESv2 API requirements
  • Test your migrated code to ensure that it functions correctly with the SESv2 API
  • Stage, test
  • Deploy

Summary

As we’ve described in this post, Amazon SES customers that migrate to the SESv2 API will benefit from updated capabilities, a more user-friendly and intuitive API, better error handling and improved deliverability controls. The SESv2 API also provide for compliance with the industry’s upcoming unsubscribe header requirements, more flexible subscription-list management, and support for larger attachments. Taken collectively, these improvements make it even easier for customers to develop, maintain, and troubleshoot their email sending applications with Amazon Simple Email Service. For these, and future reasons, we recommend SES customers migrate their existing applications to the SESv2 API immediately.

For more information regarding the SESv2 APIs, comment on this post, reach out to your AWS account team, or consult the AWS SESv2 API documentation:

About the Authors

zip

Zip

Zip is an Amazon Pinpoint and Amazon Simple Email Service Sr. Specialist Solutions Architect at AWS. Outside of work he enjoys time with his family, cooking, mountain biking and plogging.

Vinay_Ujjini

Vinay Ujjini

Vinay is an Amazon Pinpoint and Amazon Simple Email Service Worldwide Principal Specialist Solutions Architect at AWS. He has been solving customer’s omni-channel challenges for over 15 years. He is an avid sports enthusiast and in his spare time, enjoys playing tennis and cricket.

Dmitrijs_Lobanovskis

Dmitrijs Lobanovskis

Dmitrijs is a Software Engineer for Amazon Simple Email service. When not working, he enjoys traveling, hiking and going to the gym.

Message delivery status tracking with Amazon Pinpoint

Post Syndicated from Brijesh Pati original https://aws.amazon.com/blogs/messaging-and-targeting/message-delivery-status-tracking-with-amazon-pinpoint/

In the vast landscape of digital communication, reaching your audience effectively is key to building successful customer relationships. Amazon Pinpoint – Amazon Web Services’ (AWS) flexible, user-focused messaging and targeting solution goes beyond mere messaging; it allows businesses to engage customers through email, SMS, push notifications, and more.

What sets Amazon Pinpoint apart is its scalability and deliverability. Amazon Pinpoint supports a multitude of business use cases, from promotional campaigns and transactional messages to customer engagement journeys. It provides insights and analytics that help tailor and measure the effectiveness of communication strategies.

For businesses, the power of this platform extends into areas such as marketing automation, customer retention campaigns, and transactional messaging for updates like order confirmations and shipping alerts. The versatility of Amazon Pinpoint can be a significant asset in crafting personalized user experiences at scale.

Use Case & Solution overview – Tracking SMS & Email Delivery Status

In a business setting, understanding whether a time-sensitive email or SMS was received can greatly impact customer experience as well as operational efficiency. For instance, consider an e-commerce platform sending out shipping notifications. By quickly verifying that the message was delivered, businesses can preemptively address any potential issues, ensuring customer satisfaction.

Amazon Pinpoint tracks email and SMS delivery and engagement events, which can be streamed using Amazon Kinesis Firehose for storage or further processing. However, third party applications don’t have a direct API to query and obtain the latest status of a message.

To address the above challenge, this blog presents a solution that leverages AWS services for data streaming, storage, and retrieval of Amazon Pinpoint events using a simple API call. At the core of the solution is Amazon Pinpoint event stream capability, which utilizes Amazon Kinesis services for data streaming.

The architecture for message delivery status tracking with Amazon Pinpoint is comprised of several AWS services that work in concert. To streamline the deployment of these components, they have been encapsulated into an AWS CloudFormation template. This template allows for automated provisioning and configuration of the necessary AWS resources, ensuring a repeatable and error-free deployment.

The key components of the solution are as follows:

  1. Event Generation: An event is generated within Amazon Pinpoint when a user interacts with an application, or when a message is sent from a campaign, journey, or as a transactional communication. The event name and metadata depends on the channel SMS or Email.
  2. Amazon Pinpoint Event Data Streaming: The generated event data is streamed to Amazon Kinesis Data Firehose. Kinesis Data Firehose is configured to collect the event information in near real-time, enabling the subsequent processing and analysis of the data.
  3. Pinpoint Event Data Processing: Amazon Kinesis Data Firehose is configured to invoke a specified AWS Lambda function to transform the incoming source data. This transformation step is set up during the creation of the Kinesis Data Firehose delivery stream, ensuring that the data is in the correct format before it is stored, enhancing its utility for immediate and downstream analysis. The Lambda function acts as a transformation mechanism for event data ingested through Kinesis Data Firehose. The function decodes the base64-encoded event data, deserializes the JSON payload, and processes the data depending on the event type (email or SMS)- it parses the raw data, extracting relevant attributes before ingesting it into Amazon DynamoDB. The function handles different event types, specifically email and SMS events, discerning their unique attributes and ensuring they are formatted correctly for DynamoDB’s schema.
  4. Data Ingestion into Dynamo DB: Once processed, the data is stored in Amazon DynamoDB. DynamoDB provides a fast and flexible NoSQL database service, which facilitates the efficient storage and retrieval of event data for analysis.
  5. Data Storage: Amazon DynamoDB stores the event data after it’s been processed by AWS Lambda. Amazon DynamoDB is a highly scalable NoSQL database that enables fast queries, which is essential for retrieving the status of messages quickly and efficiently, thereby facilitating timely decision-making based on customer interactions.
  6. Customer application/interface: Users or integrated systems engage with the messaging status through either a frontend customer application or directly via an API. This interface or API acts as the conduit through which message delivery statuses are queried, monitored, and managed, providing a versatile gateway for both user interaction and programmatic access.
  7. API Management: The customer application communicates with the backend systems through Amazon API Gateway. This service acts as a fully managed gateway, handling all the API calls, data transformation, and transfer between the frontend application and backend services.
  8. Event Status Retrieval API: When the API Gateway receives a delivery status request, it invokes another AWS Lambda function that is responsible for querying the DynamoDB table. It retrieves the latest status of the message delivery, which is then presented to the user via the API.

DynamoDB Table Design for Message Tracking:

The tables below outline the DynamoDB schema designed for the efficient storage and retrieval of message statuses, detailing distinct event statuses and attributes for each message type such as email and SMS:

Attributes for Email Events:

Attribute Data type Description
message_id String The unique message ID generated by Amazon Pinpoint.
event_type String The value would be ’email’.
aws_account_id String The AWS account ID used to send the email.
from_address String The sending identity used to send the email.
destination String The recipient’s email address.
client String The client ID if applicable
campaign_id String The campaign ID if part of a campaign
journey_id String The journey ID if part of a journey
send Timestamp The timestamp when Amazon Pinpoint accepted the message and attempted to deliver it to the recipient
delivered Timestamp The timestamp when the email was delivered, or ‘NA’ if not delivered.
rejected Timestamp The timestamp when the email was rejected (Amazon Pinpoint determined that the message contained malware and didn’t attempt to send it.)
hardbounce Timestamp The timestamp when a hard bounce occurred (A permanent issue prevented Amazon Pinpoint from delivering the message. Amazon Pinpoint won’t attempt to deliver the message again)
softbounce Timestamp The timestamp when a soft bounce occurred (A temporary issue prevented Amazon Pinpoint from delivering the message. Amazon Pinpoint will attempt to deliver the message again for a certain amount of time. If the message still can’t be delivered, no more retries will be attempted. The final state of the email will then be SOFTBOUNCE.)
complaint Timestamp The timestamp when a complaint was received (The recipient received the message, and then reported the message to their email provider as spam (for example, by using the “Report Spam” feature of their email client).
open Timestamp The timestamp when the email was opened (The recipient received the message and opened it.)
click Timestamp The timestamp when a link in the email was clicked. (The recipient received the message and clicked a link in it)
unsubscribe Timestamp The timestamp when a link in the email was unsubscribed (The recipient received the message and clicked an unsubscribe link in it.)
rendering_failure Timestamp The timestamp when a link in the email was clicked (The email was not sent due to a rendering failure. This can occur when template data is missing or when there is a mismatch between template parameters and data.)

Attributes for SMS Events:

Attribute Data type Description
message_id String The unique message ID generated by Amazon Pinpoint.
event_type String The value would be ‘sms’.
aws_account_id String The AWS account ID used to send the email.
origination_phone_number String The phone number from which the SMS was sent.
destination_phone_number String The phone number to which the SMS was sent.
record_status String Additional information about the status of the message. Possible values include:
– SUCCESSFUL/DELIVERED – Successfully delivered.
– PENDING – Not yet delivered.
– INVALID – Invalid destination phone number.
– UNREACHABLE – Recipient’s device unreachable.
– UNKNOWN – Error preventing delivery.
– BLOCKED – Device blocking SMS.
– CARRIER_UNREACHABLE – Carrier issue preventing delivery.
– SPAM – Message identified as spam.
– INVALID_MESSAGE – Invalid SMS message body.
– CARRIER_BLOCKED – Carrier blocked message.
– TTL_EXPIRED – Message not delivered in time.
– MAX_PRICE_EXCEEDED – Exceeded SMS spending quota.
– OPTED_OUT – Recipient opted out.
– NO_QUOTA_LEFT_ON_ACCOUNT – Insufficient spending quota.
– NO_ORIGINATION_IDENTITY_AVAILABLE_TO_SEND – No suitable origination identity.
– DESTINATION_COUNTRY_NOT_SUPPORTED – Destination country blocked.
– ACCOUNT_IN_SANDBOX – Account in sandbox mode.
– RATE_EXCEEDED – Message sending rate exceeded.
– INVALID_ORIGINATION_IDENTITY – Invalid origination identity.
– ORIGINATION_IDENTITY_DOES_NOT_EXIST – Non-existent origination identity.
– INVALID_DLT_PARAMETERS – Invalid DLT parameters.
– INVALID_PARAMETERS – Invalid parameters.
– ACCESS_DENIED – Account blocked from sending messages.
– INVALID_KEYWORD – Invalid keyword.
– INVALID_SENDER_ID – Invalid Sender ID.
– INVALID_POOL_ID – Invalid Pool ID.
– SENDER_ID_NOT_SUPPORTED_FOR_DESTINATION – Sender ID not supported.
– INVALID_PHONE_NUMBER – Invalid origination phone number.
iso_country_code String The ISO country code associated with the destination phone number.
message_type String The type of SMS message sent.
campaign_id String The campaign ID if part of a campaign, otherwise N/A.
journey_id String The journey ID if part of a journey, otherwise N/A.
success Timestamp The timestamp when the SMS was successfully accepted by the carrier/delivered to the recipient, or ‘NA’ if not applicable.
buffered Timestamp The timestamp when the SMS is still in the process of being delivered to the recipient, or ‘NA’ if not applicable.
failure Timestamp The timestamp when the SMS delivery failed, or ‘NA’ if not applicable.
complaint Timestamp The timestamp when a complaint was received (The recipient received the message, and then reported the message to their email provider as spam (for example, by using the “Report Spam” feature of their email client).
optout Timestamp The timestamp when the customer received the message and replied by sending the opt-out keyword (usually “STOP”), or ‘NA’ if not applicable.
price_in_millicents_usd Number The amount that was charged to send the message.

Prerequisites

  • AWS Account Access (setup) with admin-level permission.
  • AWS CLI version 2 with named profile setup. If a locally configured IDE is not convenient, you can use the AWS CLI from the AWS CloudShell in your browser.
  • A Pinpoint project that has never been configured with an event stream (PinpointEventStream).“
  • The Pinpoint ID from the project you want to monitor. This ID can be found in the AWS Pinpoint console on the project’s main page (it will look something like “79788ecad55555513b71752a4e3ea1111”). Copy this ID to a text file, as you will need it shortly.
    • Note, you must use the ID from a Pinpoint project that has never been configured with the PinpointEventStream option.

Solution Deployment & Testing

Deploying this solution is a straightforward process, thanks to the AWS CloudFormation template we’ve created. This template automates the creation and configuration of the necessary AWS resources into an AWS stack. The CloudFormation template ensures that the components such as Kinesis Data Firehose, AWS Lambda, Amazon DynamoDB, and Amazon API Gateway are set up consistently and correctly.

Deployment Steps:

  • Download the CloudFormation Template from this GitHub sample repository. The CloudFormation template is authored in JSON and named PinpointAPIBlog.yaml.
  • Access the CloudFormation Console: Sign into the AWS Management Console and open the AWS CloudFormation console.
  • Create a New Stack:
    • Choose Create Stack and select With new resources (standard) to start the stack creation process.
    • Under Prerequisite – Prepare template, select Template is ready.
    • Under ‘Specify template’, choose Upload a template file, and then upload the CloudFormation template file you downloaded in Step 1.
  • Configure the Stack:
    • Provide a stack name, such as “pinpoint-yourprojectname-monitoring” and paste the Pinpoint project (application) ID. Press Next.
    • Review the stack settings, and make any necessary changes based on your specific requirements. Next.
  • Initiate the Stack Creation: Once you’ve configured all options, acknowledge that AWS CloudFormation might create IAM resources with custom names, and then choose Create stack.
    • AWS CloudFormation will now provision and configure the resources as defined in the template This will take about 20 minutes to fully deploy. You can view the status in the AWS CloudFormation console.

Testing the Solution:

After deployment is complete you can test (and use) the solution.

  • Send Test Messages: Utilize the Amazon Pinpoint console to send test email and SMS messages. Documentation for this can be found at:
  • Verify Lambda Execution:
    • Navigate to the AWS CloudWatch console.
    • Locate and review the logs for the Lambda functions specified in the solution (`aws/lambda/{functionName}`) to confirm that the Kinesis Data Firehose records are being processed successfully. In the log events you should see messages including INIT_START, Raw Kinesis Data Firehouse Record, etc.
  • Check Amazon DynamoDB Data:
    • Navigate to Amazon DynamoDB in the AWS Console.
    • Select the table created by the CloudFormation template and choose ‘Explore Table Items‘.
    • Confirm the presence of the event data by checking if the message IDs appear in the table.
    • The table should have one or more message_id entries from the test message(s) you sent above.
    • Click on a message_id to review the data, and copy the message_id to a text editor on your computer. It will look like “0201123456gs3nroo-clv5s8pf-8cq2-he0a-ji96-59nr4tgva0g0-343434
  • API Gateway Testing:
    • In the API Gateway console, find the MessageIdAPI.
    • Navigate to Stages and copy the Invoke URL provided.

    • Open the text editor on your computer and paste the APIGateway invoke URL.
    • Create a curl command with you API Gateway + ?message_id=message_id. It should look like this: “https://txxxxxx0.execute-api.us-west-2.amazonaws.com/call?message_id=020100000xx3xxoo-clvxxxxf-8cq2-he0a-ji96-59nr4tgva0g0-000000”
    • Copy the full curl command in your browser and enter.
    • The results should look like this (MacOS, Chrome):

By following these deployment and testing steps, you’ll have a functioning solution for tracking Pinpoint message delivery status using Amazon Pinpoint, Kinesis Fire Hose, DynamoDB and CloudWatch.

Clean Up

To help prevent unwanted charges to your AWS account, you can delete the AWS resources that you used for this walkthrough.

To delete the stack follow these following instructions:

Open the AWS CloudFormation console.

  • In the AWS CloudFormation console dashboard, select the stack you created (pinpoint-yourprojectname-monitoring).
  • On the Actions menu, choose Delete Stack.
  • When you are prompted to confirm, choose Yes, Delete.
  • Wait for DELETE_COMPLETE to appear in the Status column for the stack.

Next steps

The solution on this blog provides you an API endpoint to query messages’ status. The next step is to store and analyze the raw data based on your business’s requirements. The Amazon Kinesis Firehose used in this blog can stream the Pinpoint events to an AWS database or object storage like Amazon S3. Once the data is stored, you can catalogue them using AWS Glue, query them via SQL using Amazon Athena and create custom dashboards using Amazon QuickSight, which is a cloud-native, serverless, business intelligence (BI) with native machine learning (ML) integrations.

Conclusion

The integration of AWS services such as Kinesis, Lambda, DynamoDB, and API Gateway with Amazon Pinpoint transforms your ability to connect with customers through precise event data retrieval and analysis. This solution provides a stream of real-time data, versatile storage options, and a secure method for accessing detailed information, all of which are critical for optimizing your communication strategies.

By leveraging these insights, you can fine-tune your email and SMS campaigns for maximum impact, ensuring every message counts in the broader narrative of customer engagement and satisfaction. Harness the power of AWS and Amazon Pinpoint to not just reach out but truly connect with your audience, elevating your customer relationships to new heights.

Considerations/Troubleshooting

When implementing a solution involving AWS Lambda, Kinesis Data Streams, Kinesis Data Firehose, and DynamoDB, several key considerations should be considered:

  • Scalability and Performance: Assess the scalability needs of your system. Lambda functions scale automatically, but it’s important to configure concurrency settings and memory allocation based on expected load. Similarly, for Kinesis Streams and Firehose, consider the volume of data and the throughput rate. For DynamoDB, ensure that the table’s read and write capacity settings align with your data processing requirements.
  • Error Handling and Retries: Implement robust error handling within the Lambda functions to manage processing failures. Kinesis Data Streams and Firehose have different retry behaviors and mechanisms. Understand and configure these settings to handle failed data processing attempts effectively. In DynamoDB, consider the use of conditional writes to handle potential data inconsistencies.
  • Security and IAM Permissions: Secure your AWS resources by adhering to the principle of least privilege. Define IAM roles and policies that grant the Lambda function only the necessary permissions to interact with Kinesis and DynamoDB. Ensure that data in transit and at rest is encrypted as required, using AWS KMS or other encryption mechanisms.
  • Monitoring and Logging: Utilize AWS CloudWatch for monitoring and logging the performance and execution of Lambda functions, as well as Kinesis and DynamoDB operations. Set up alerts for any anomalies or thresholds that indicate issues in data processing or performance bottlenecks.

About the Authors

Brijesh Pati

Brijesh Pati

Brijesh Pati is an Enterprise Solutions Architect at AWS. His primary focus is helping enterprise customers adopt cloud technologies for their workloads. He has a background in application development and enterprise architecture and has worked with customers from various industries such as sports, finance, energy and professional services. His interests include serverless architectures and AI/ML.

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis is an Amazon Pinpoint and Amazon Simple Email Service Senior Specialist Solutions Architect at AWS. He enjoys diving deep into customers’ technical issues and help in designing communication solutions. In his spare time, he enjoys playing tennis, watching crime TV series, playing FPS PC games, and coding personal projects.

Anshika Singh

Anshika Singh

Anshika Singh is an Associate Solutions Architect at AWS specializing in building for GenAI applications. She helps enable customers to use the cloud through the use of code samples and starter projects.

Magic Cloud Networking simplifies security, connectivity, and management of public clouds

Post Syndicated from Steve Welham original https://blog.cloudflare.com/introducing-magic-cloud-networking


Today we are excited to announce Magic Cloud Networking, supercharged by Cloudflare’s recent acquisition of Nefeli Networks’ innovative technology. These new capabilities to visualize and automate cloud networks will give our customers secure, easy, and seamless connection to public cloud environments.

Public clouds offer organizations a scalable and on-demand IT infrastructure without the overhead and expense of running their own datacenter. Cloud networking is foundational to applications that have been migrated to the cloud, but is difficult to manage without automation software, especially when operating at scale across multiple cloud accounts. Magic Cloud Networking uses familiar concepts to provide a single interface that controls and unifies multiple cloud providers’ native network capabilities to create reliable, cost-effective, and secure cloud networks.

Nefeli’s approach to multi-cloud networking solves the problem of building and operating end-to-end networks within and across public clouds, allowing organizations to securely leverage applications spanning any combination of internal and external resources. Adding Nefeli’s technology will make it easier than ever for our customers to connect and protect their users, private networks and applications.

Why is cloud networking difficult?

Compared with a traditional on-premises data center network, cloud networking promises simplicity:

  • Much of the complexity of physical networking is abstracted away from users because the physical and ethernet layers are not part of the network service exposed by the cloud provider.
  • There are fewer control plane protocols; instead, the cloud providers deliver a simplified software-defined network (SDN) that is fully programmable via API.
  • There is capacity — from zero up to very large — available instantly and on-demand, only charging for what you use.

However, that promise has not yet been fully realized. Our customers have described several reasons cloud networking is difficult:

  • Poor end-to-end visibility: Cloud network visibility tools are difficult to use and silos exist even within single cloud providers that impede end-to-end monitoring and troubleshooting.
  • Faster pace: Traditional IT management approaches clash with the promise of the cloud: instant deployment available on-demand. Familiar ClickOps and CLI-driven procedures must be replaced by automation to meet the needs of the business.
  • Different technology: Established network architectures in on-premises environments do not seamlessly transition to a public cloud. The missing ethernet layer and advanced control plane protocols were critical in many network designs.
  • New cost models: The dynamic pay-as-you-go usage-based cost models of the public clouds are not compatible with established approaches built around fixed cost circuits and 5-year depreciation. Network solutions are often architected with financial constraints, and accordingly, different architectural approaches are sensible in the cloud.
  • New security risks: Securing public clouds with true zero trust and least-privilege demands mature operating processes and automation, and familiarity with cloud-specific policies and IAM controls.
  • Multi-vendor: Oftentimes enterprise networks have used single-vendor sourcing to facilitate interoperability, operational efficiency, and targeted hiring and training. Operating a network that extends beyond a single cloud, into other clouds or on-premises environments, is a multi-vendor scenario.

Nefeli considered all these problems and the tensions between different customer perspectives to identify where the problem should be solved.

Trains, planes, and automation

Consider a train system. To operate effectively it has three key layers:

  • tracks and trains
  • electronic signals
  • a company to manage the system and sell tickets.

A train system with good tracks, trains, and signals could still be operating below its full potential because its agents are unable to keep up with passenger demand. The result is that passengers cannot plan itineraries or purchase tickets.

The train company eliminates bottlenecks in process flow by simplifying the schedules, simplifying the pricing, providing agents with better booking systems, and installing automated ticket machines. Now the same fast and reliable infrastructure of tracks, trains, and signals can be used to its full potential.

Solve the right problem

In networking, there are an analogous set of three layers, called the networking planes:

  • Data Plane: the network paths that transport data (in the form of packets) from source to destination.
  • Control Plane: protocols and logic that change how packets are steered across the data plane.
  • Management Plane: the configuration and monitoring interfaces for the data plane and control plane.

In public cloud networks, these layers map to:

  • Cloud Data Plane: The underlying cables and devices are exposed to users as the Virtual Private Cloud (VPC) or Virtual Network (VNet) service that includes subnets, routing tables, security groups/ACLs and additional services such as load-balancers and VPN gateways.
  • Cloud Control Plane: In place of distributed protocols, the cloud control plane is a software defined network (SDN) that, for example, programs static route tables. (There is limited use of traditional control plane protocols, such as BGP to interface with external networks and ARP to interface with VMs.)
  • Cloud Management Plane: An administrative interface with a UI and API which allows the admin to fully configure the data and control planes. It also provides a variety of monitoring and logging capabilities that can be enabled and integrated with 3rd party systems.

Like our train example, most of the problems that our customers experience with cloud networking are in the third layer: the management plane.

Nefeli simplifies, unifies, and automates cloud network management and operations.

Avoid cost and complexity

One common approach to tackle management problems in cloud networks is introducing Virtual Network Functions (VNFs), which are virtual machines (VMs) that do packet forwarding, in place of native cloud data plane constructs. Some VNFs are routers, firewalls, or load-balancers ported from a traditional network vendor’s hardware appliances, while others are software-based proxies often built on open-source projects like NGINX or Envoy. Because VNFs mimic their physical counterparts, IT teams could continue using familiar management tooling, but VNFs have downsides:

  • VMs do not have custom network silicon and so instead rely on raw compute power. The VM is sized for the peak anticipated load and then typically runs 24x7x365. This drives a high cost of compute regardless of the actual utilization.
  • High-availability (HA) relies on fragile, costly, and complex network configuration.
  • Service insertion — the configuration to put a VNF into the packet flow — often forces packet paths that incur additional bandwidth charges.
  • VNFs are typically licensed similarly to their on-premises counterparts and are expensive.
  • VNFs lock in the enterprise and potentially exclude them benefitting from improvements in the cloud’s native data plane offerings.

For these reasons, enterprises are turning away from VNF-based solutions and increasingly looking to rely on the native network capabilities of their cloud service providers. The built-in public cloud networking is elastic, performant, robust, and priced on usage, with high-availability options integrated and backed by the cloud provider’s service level agreement.

In our train example, the tracks and trains are good. Likewise, the cloud network data plane is highly capable. Changing the data plane to solve management plane problems is the wrong approach. To make this work at scale, organizations need a solution that works together with the native network capabilities of cloud service providers.

Nefeli leverages native cloud data plane constructs rather than third party VNFs.

Introducing Magic Cloud Networking

The Nefeli team has joined Cloudflare to integrate cloud network management functionality with Cloudflare One. This capability is called Magic Cloud Networking and with it, enterprises can use the Cloudflare dashboard and API to manage their public cloud networks and connect with Cloudflare One.

End-to-end

Just as train providers are focused only on completing train journeys in their own network, cloud service providers deliver network connectivity and tools within a single cloud account. Many large enterprises have hundreds of cloud accounts across multiple cloud providers. In an end-to-end network this creates disconnected networking silos which introduce operational inefficiencies and risk.

Imagine you are trying to organize a train journey across Europe, and no single train company serves both your origin and destination. You know they all offer the same basic service: a seat on a train. However, your trip is difficult to arrange because it involves multiple trains operated by different companies with their own schedules and ticketing rates, all in different languages!

Magic Cloud Networking is like an online travel agent that aggregates multiple transportation options, books multiple tickets, facilitates changes after booking, and then delivers travel status updates.

Through the Cloudflare dashboard, you can discover all of your network resources across accounts and cloud providers and visualize your end-to-end network in a single interface. Once Magic Cloud Networking discovers your networks, you can build a scalable network through a fully automated and simple workflow.

Resource inventory shows all configuration in a single and responsive UI

Taming per-cloud complexity

Public clouds are used to deliver applications and services. Each cloud provider offers a composable stack of modular building blocks (resources) that start with the foundation of a billing account and then add on security controls. The next foundational layer, for server-based applications, is VPC networking. Additional resources are built on the VPC network foundation until you have compute, storage, and network infrastructure to host the enterprise application and data. Even relatively simple architectures can be composed of hundreds of resources.

The trouble is, these resources expose abstractions that are different from the building blocks you would use to build a service on prem, the abstractions differ between cloud providers, and they form a web of dependencies with complex rules about how configuration changes are made (rules which differ between resource types and cloud providers). For example, say I create 100 VMs, and connect them to an IP network. Can I make changes to the IP network while the VMs are using the network? The answer: it depends.

Magic Cloud Networking handles these differences and complexities for you. It configures native cloud constructs such as VPN gateways, routes, and security groups to securely connect your cloud VPC network to Cloudflare One without having to learn each cloud’s incantations for creating VPN connections and hubs.

Continuous, coordinated automation

Returning to our train system example, what if the railway maintenance staff find a dangerous fault on the railroad track? They manually set the signal to a stop light to prevent any oncoming trains using the faulty section of track. Then, what if, by unfortunate coincidence, the scheduling office is changing the signal schedule, and they set the signals remotely which clears the safety measure made by the maintenance crew? Now there is a problem that no one knows about and the root cause is that multiple authorities can change the signals via different interfaces without coordination.

The same problem exists in cloud networks: configuration changes are made by different teams using different automation and configuration interfaces across a spectrum of roles such as billing, support, security, networking, firewalls, database, and application development.

Once your network is deployed, Magic Cloud Networking monitors its configuration and health, enabling you to be confident that the security and connectivity you put in place yesterday is still in place today. It tracks the cloud resources it is responsible for, automatically reverting drift if they are changed out-of-band, while allowing you to manage other resources, like storage buckets and application servers, with other automation tools. And, as you change your network, Cloudflare takes care of route management, injecting and withdrawing routes globally across Cloudflare and all connected cloud provider networks.

Magic Cloud Networking is fully programmable via API, and can be integrated into existing automation toolchains.

The interface warns when cloud network infrastructure drifts from intent

Ready to start conquering cloud networking?

We are thrilled to introduce Magic Cloud Networking as another pivotal step to fulfilling the promise of the Connectivity Cloud. This marks our initial stride in empowering customers to seamlessly integrate Cloudflare with their public clouds to get securely connected, stay securely connected, and gain flexibility and cost savings as they go.

Join us on this journey for early access: learn more and sign up here.

Broadcom VMware Ends Free VMware vSphere Hypervisor Closing an Era

Post Syndicated from Patrick Kennedy original https://www.servethehome.com/broadcom-vmware-ends-free-vmware-vsphere-hypervisor-closing-an-era/

Broadcom ended the free VMware vSphere Hypervisor era relegating VMware admins to a path similar to mainframe admins

The post Broadcom VMware Ends Free VMware vSphere Hypervisor Closing an Era appeared first on ServeTheHome.

Using one-click unsubscribe with Amazon SES

Post Syndicated from Pavlos Ioannou Katidis original https://aws.amazon.com/blogs/messaging-and-targeting/using-one-click-unsubscribe-with-amazon-ses/

Gmail and Yahoo have announced new requirements for bulk senders that take effect in February 2024. The requirements aim to reduce delivery of malicious or unwanted email to the users of these mailbox providers. We recommend that Amazon SES senders who operate outside of the SES sandbox assume these bulk sender requirements apply to them.

Gmail’s FAQ and Yahoo’s FAQ both clarify that the one-click unsubscribe requirement will not be enforced until June 2024 as long as the bulk sender has a functional unsubscribe link clearly visible in the footer of each message.

This blog presents a reference architecture for Amazon SES senders who independently manage email subscriptions outside of Amazon SES. Alternatively, Amazon SES senders can employ our native subscription management capability as part of their compliance with the Gmail and Yahoo bulk sender requirements.  Note that the scope of Gmail and Yahoo’s bulk sender requirements extends beyond enabling an easy unsubscribe method.  Read our blogs on email authentication and managing spam complaints for more information that will help you successfully operate as a bulk sender with Amazon SES.

Email headers contain metadata that describes the content, sender, relay path, destination, and other elements of an email. The bulk sender easy subscription requirement references use of the List-Unsubscribe email header (RFC2369) and List-Unsubscribe-Post email header (RFC8058). The order of the headers should be first the List-Unsubscribe followed by the List-Unsubscribe-Post.

  • List-Unsubscribe: <https://nutrition.co/?address=x&topic=x>, <mailto:unsubscribe@ nutrition.co?subject=TopicUnsubscribe>
  • List-Unsubscribe-Post: List-Unsubscribe=One-Click

These headers enable email clients and inbox providers to display an unsubscribe link at the top of the email if they support it. This could take the form of a menu item, push button, or another user interface element to simplify the user experience – see the Gmail client screenshot below.

gmail-inbox

Unsubscribing can take place from the email footer by clicking on a hyperlink, and/or from an unsubscribe link that mailbox providers render. These different unsubscribe methods can be custom-built or provided by Amazon SES.

  • Unsubscribe method footer: An unsubscribe link in the email footer, which redirects recipients to a landing page, where they can unsubscribe or edit their communication preferences.
  • Unsubscribe method header: A hyperlink that is rendered by the mailbox provider based on the List-Unsubscribe email header. Recipients can use this link to unsubscribe from that sender.
  • Amazon SES unsubscribe method: The Amazon SES subscription management feature, which provides subscription management via the List-Unsubscribe header and ListManagementOptions footer links.
  • Custom-built unsubscribe method: A custom-built unsubscribe link in the email footer and manually added List-Unsubscribe header.

The table below lists all unsubscribe method combinations, indicating if they are custom-built or provided by Amazon SES and whether they comply with the easy unsubscription requirement from Google and Yahoo.

Unsubscribe method Amazon SES or custom-built Complies with Gmail & Yahoo
Footer & Header Amazon SES Yes
Footer & Header Custom Yes
Header Custom Yes
Footer Custom Partial

Failing to comply with the easy unsubscription requirement mailbox providers such as Gmail and Yahoo will start rejecting non-compliant emails.

Note: Gmail might not show the easy unsubscribe link. This might happen because Gmail shows the link if they trust that the sender is honoring the unsubscribe requests and not attempting to track recipients. We recommend senders continue to provide the unsubscribe link in an easy to find location of the body of the message.

Implementing the unsubscribe header has many benefits for you:

  • Reduces spam complaint rate: Email recipients will click on “Report as SPAM” if they find it difficult to unsubscribe. A high spam complaint rate makes mailbox providers more likely to block your sending. Making unsubscribe easier can improve deliverability.
  • It can increase the trust in your brand: The fact that it is easy for recipients to unsubscribe could be seen as evidence that the content is valuable enough that the company believes people will want to stay subscribed.
  • Reduces issues with false suppression: Senders that rely solely on account-level suppression lists could suppress all email sending to an address even though the recipient may wish to receive other types of email from the account. Offering an easy unsubscribe method allows recipients to indicate which type of email they would like to receive and not receive based on topic or category.

There are two types of list-unsubscribe options:

  • Mailto: unsubscribe requests come in the form of an email sent from the mailbox provider to the email address specified on the List-Unsubscribe header. The process of managing unsubscribe emails can be automated with SES inbound.
  • URL unsubscribe link: redirects recipients to an unsubscribe landing page, from where they can edit further their communication preferences. Adding the List-Unsubscribe-Post email header, senders can provide recipients with one-click unsubscribe experience, which doesn’t require them to visit a landing page.

The mailto option is supported by many mailbox providers and it’s recommended to include it in addition to the URL in the List-Unsubscribe email header and the unsubscribe link in the email footer.

One-click unsubscribe for Amazon SES

This section guides you on how to use Amazon SES V2 SendEmail API operation for email sending and describes how to use other AWS services to effectively manage each kind of unsubscribe request.

The architecture covers both easy unsubscribe options, mailto and URL. This is because not all mailbox providers support the List-Unsubscribe-Post header. The architecture, assumes that Amazon SES has email receiving enabled for the unsubscribe email address used in the List-Unsubscribe mailto header and your recipient preferences can be updated via an API.

The reference architecture diagram illustrates the AWS services used and how they interact with each other to process a recipient’s unsubscribe request:

  • AWS KMS: is a managed service that makes it easy for you to create and control the cryptographic keys that are used to protect your data.
  • Amazon API Gateway: Is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.
  • AWS Lambda: Compute service that runs your code in response to events and automatically manages the compute resources.

The first part of the process is described in detail below:

email-sending-flow

  1. Compliant emails should include the List-Unsubscribe and List-Unsubscribe-Post headers. This can be achieved with the Amazon SES SendEmail V2 API operation. Using MIME standard, build a MIME message containing the headers, subject and body. The MIME message will be in the SES V2 SendEmail API request body under Content => Raw field – see code example below. Amazon SES is planning to extend the SendEmail V2 API to natively support unsubscribe email headers. The unsubscribe email address and URL contain the recipient’s email address and email subject parameters, which are encrypted using AWS Key Management Service. These parameters are used later on to identify and unsubscribe the recipient from a specific topic.
    1. The email domain used to send emails needs to be first verified successfully – see here how to create and verify identities in SES.
    2. Gmail uses the Friendly From value to populate the unsubscribe pop-up message. Friendly From is the part of the From header that is displayed to the recipient (not the email address) “To stop getting messages like this one, go to the <Friendly From> website to unsubscribe. Learn more.”. If you see Unknown or experience other issues, ensure that the From header of your messages conforms to RFC5322.
      
      	msg = MIMEMultipart()
      	msg.add_header('List-Unsubscribe','<https://nutrition.co/?address=x&topic=x>, <mailto: [email protected]?subject=TopicUnsubscribe>')
      	msg.add_header('List-Unsubscribe-Post','List-Unsubscribe=One-Click')
      	msg.attach(MIMEText("Welcome to Nutrition.co", 'plain')) 
      	msg['Subject'] = "Welcome to Nutrition.co"
      
      	response = sesv2.send_email(
      	  FromEmailAddress='Nutrition.co <[email protected]>',
      	  Destination={'ToAddresses': ['[email protected]']},
      	  Content={
      		  'Raw': {
      			  'Data': msg.as_string()
      		  },
      	  },
      	  ConfigurationSetName='ConfigSet'
      	)
    3. Amazon Pinpoint senders need to use Custom channel instead of Amazon Pinpoint’s native email channel. Custom channel gives the flexibility to invoke an AWS Lambda function and execute custom code such as calling Amazon Pinpoint’s send_messages API operation. Using Amazon Pinpoint’s send_messages API operation you can specify an endpoint as the recipient and add the email content and the List-Unsubscribe and List-Unsubscribe-Post headers in a MIME message under the RawEmail => Data field – see below a code example:
      	msg = MIMEMultipart()
      	msg.add_header('List-Unsubscribe','<https://nutrition.co/?address=x&topic=x>, <mailto: [email protected]?subject=TopicUnsubscribe>')
      	msg.add_header('List-Unsubscribe-Post','List-Unsubscribe=One-Click')
      	msg.attach(MIMEText("Welcome to Nutrition.co", 'plain')) 
      	msg['Subject'] = "Welcome to Nutrition.co"
      
      	endpoint_id = "endpoint_id"
      	application_id = "application_id"
      
      	response = pinpoint.send_messages(
      	ApplicationId = application_id,
      	MessageRequest = {
      		'Endpoints': {
      			endpoint_id: {}
      		},
      		'MessageConfiguration': {
      		'EmailMessage': {
      			'FromAddress': 'Nutrition.co <[email protected]>',
      			'RawEmail': {
      				'Data': msg.as_string()
      			}
      		}
      	  }
      	})
  2. The email recipients whose mailbox provider supports List-Unsubscribe, such as Gmail & Yahoo, will see an Unsubscribe hyperlink next to the sender details as shown in the screenshot below.

gmail-inbox

So far, we have talked about how to craft and employ the headers for presenting mail recipients with an easy unsubscribe option.  In the following sections, we’ll walk through the two options for sending the unsubscribe request back to the sender.

The first option uses only the List-Unsubscribe header and only specifies the mailto email address to receive unsubscribe requests. The second option uses both the List-Unsubscribe and the List-Unsubscribe-Post headers. The unsubscribe requests are made with a POST API call to an endpoint provided in the List-Unsubscribe header.

When the recipient clicks on the Unsubscribe call to action next to the sender’s information, a pop-up appears asking for final confirmation using either option – see screenshot below.

unsubscribe-pop-up

Scenario – List-Unsubscribe

list-unsubscribe-scenario

  1. The recipient clicks on the Unsubscribe call to action next to the sender’s details and again on Unsubscribe on the pop-up.
  2. The mailbox provider sends an email to the email address specified in the header List-Unsubscribe => mailto. Amazon SES can be configured to receive emails for the unsubscribe email address, the Amazon SES receipt rule Invoke Lambda function action.
  3. An AWS Lambda function gets invoked. The payload contains all email headers and omits the email body as well as any attachments. The AWS Lambda function uses the AWS KMS key to decrypt the email subject, which contains the topic the recipient wants to unsubscribe from. Depending where your recipient preferences are stored, you can expand the AWS Lambda function code to update the recipients’ communication preferences.

Scenario – List-Unsubscribe & List-Unsubscribe-Post

list-unsubscribe-post-scenario

  1. The recipient clicks on the Unsubscribe call to action next to the sender’s details and again on Unsubscribe on the pop-up.
  2. The mailbox provider performs a POST API call to the URL provided in the List-Unsubscribe header. In this architecture, the URL is an Amazon API Gateway endpoint with an AWS Lambda integration.
  3. An AWS Lambda function gets invoked, which uses the AWS KMS key to decrypt the email address and topic stored in the URL parameters. Depending where your recipient preferences are stored, you can expand the AWS Lambda function code to update the recipients’ communication preferences. The code in the AWS Lambda function serves two purposes 1) processing a POST request to unsubscribe the recipient and 2) processing a GET request to redirect the recipient to page on your website (Gmail specific). Use a micro web framework like Flask to process unsubscribe requests and accordingly redirect recipients to a page of your website.

In Gmail, to view the Go to website call to action, recipients need to first Unsubscribe and then and then click on Unsubscribe again – see diagram below.

unsubscribe-flow-gmail

Conclusion

In this blog you learned how to configure Amazon SES to manage One-click unsubscribe requests when not using SES’s subscription management feature. The reference architecture shows how to structure and add the List-Unsubscribe and List-Unsubscribe-Post email headers when sending emails as well as how to manage unsubscribe requests generated from these email headers respectively. In addition to the List-Unsubscribe and List-Unsubscribe-Post email headers, we recommend (continue) using the footer unsubscribe link.

Easy unsubscribe benefits both the sender and recipient. It is one of the Gmail and Yahoo’s bulk sender requirements announced back in October 2023. The one-click unsubscribe requirement will not be enforced until June 2024 as long as the bulk sender has a functional unsubscribe link clearly visible in the footer of each message.

How to Build a Compliant SMS Opt-In Process With Amazon Pinpoint

Post Syndicated from Tyler Holmes original https://aws.amazon.com/blogs/messaging-and-targeting/how-to-build-a-compliant-sms-opt-in-process-with-amazon-pinpoint/

SMS messaging is a great way to stay in touch with your customers and send them timely, relevant messages. However, it’s always required to get their permission before you start sending them texts. This is known as opting in.

There are a few different ways to opt-in users to your SMS program. One common method is to have them sign up on your website or in your app. You can also collect opt-ins at in-person events or via phone call through your customer service team; though it’s not limited to just those options.

No matter which method you choose, it’s important to make sure that your opt-in process is clear, concise, and compliant with all applicable local laws and regulations in the countries that you are sending to. Here are some best practices:

  • Get explicit consent. Explicit consent is the intentional action taken by a end-user to request a specific message from your service.
  • Provide clear instructions. Tell users how to opt-in, what they are opting into, and how to opt-out of your program. Be sure to include your contact information at the opt-in location in case they have any questions or concerns.
  • Give users the option to choose what kind of messages they want to receive. For example, you might allow them to opt-in to OTP/2FA messages, shipping notifications, or both.
  • Respect users’ privacy. Never sell or share users’ phone numbers with third parties without their permission. 3rd party data sharing is generally considered a prohibited practice by mobile carriers and violates privacy regulations in many countries.
  • Make it easy to opt-out. Users should be able to opt-out of your program at any time by replying with a simple text message, such as “STOP.” See additional relevent documentation related to this: Opting out. Self-managed opt-outs.

The above will help you build a strong audience of engaged subscribers who want to hear from you and improve your chances in successfully registering for a dedicated number. By following these best practices, you can ensure that your SMS opt-in process is compliant and effective.

What carriers require for a compliant opt-in workflow and call-to-action

The primary purpose of the opt-in workflow is to demonstrate that the end-user explicitly consents to receive text messages and understands the nature of the program. Your application is being reviewed by a 3rd party reviewer and sometimes multiple 3rd party reviewers for a single registration, so make sure to provide clear and thorough information about how your end-users opt-in to your SMS service and any associated fees or charges. If the reviewer cannot determine how your opt-in process works or if it is not compliant then your application will be denied and returned. It is important to note that Amazon does not review or approve your use cases and that it’s a telecom industry standard in most countries for 3rd parties to review and approve your use case prior to sending.

Note: If you have a use case that is internal to your business, you are still required to demonstrate explicit opt-in consent from the recipients. There are no exceptions to having an opt-in workflow and explicit consent is always required.

If your opt-in process requires a login, is not yet public, is a verbal opt-in, or if it occurs on printed forms or fliers then make sure to thoroughly document how this process is completed by the end-user receiving messages — remember, these are 3rd party reviewers and if they’re unable to access where your end-users opt-in, they will require thorough information via other means like text or screenshots. Provide a screenshot of the Call to Action (CTA) in such cases. If the consent is being asked for and supplied verbally, as in a contact center situation, make sure to provide the verbal scripts to ensure the entire CTA is shown. Host any screenshots on a publicly accessible website (like S3, OneDrive, or Google Drive) and provide the URL when you submit (NOTE: toll-free number registration process supports attachments and do not require a public URL to be included). Regardless of the medium used to collect end-user information (e.g., webform, point of sale, fliers, or verbal opt-ins), the requirements are the same. In the case of online and printed materials, they would be shown as text to the end-users. In the case of verbal opt-ins (i.e., on the phone), the information below would be verbally read to the end-user.

Call-to-action/opt-in requirements

The following items are the minimum that must be presented to an end-user at the time of opt-in to ensure your SMS program is compliant:

  • Program (brand) name
  • Message frequency disclosure. (example: “Message frequency varies” or “One message per login”)
  • Customer care contact information (example: “Text HELP or call 1-800-111-2222 for support.”)
  • Opt-out information (example: “Text STOP to opt-out of future messages.”)
  • Include “Message and data rates may apply” disclosure.
  • Link to a publicly accessible Terms & Conditions page
  • Link to a publicly accessible Privacy Policy page

**Now lets break the above bullet points down into more detail:


Program, service, brand name

All SMS originator types that require registration must disclose the program name, product description, or both in service messages, on the call-to-action, and in the terms and conditions. The program name is the sponsor of the messaging program, often the brand name or company name associated with the sending use case. The product description describes the product advertised by the program.

Publicly accessible terms & conditions page

The terms should be live and publicly accessible. For verbal scripts, a URL must be read off to the end-user enrolling in the SMS program, or the comprehensive terms must be directly included in the script. You should provide a compliant screenshot, link, or mockup of the SMS Terms of Service in the registration submission.

Below is a copy of the boilerplate terms of service that cover minimum requirements from the carriers:

  1. {Program name}
  2. {Insert program description here; this is simply a brief description of the kinds of messages users can expect to receive when they opt-in.}
  3. You can cancel the SMS service at any time. Just text “STOP” to the short code. After you send the SMS message “STOP” to us, we will send you an SMS message to confirm that you have been unsubscribed. After this, you will no longer receive SMS messages from us. If you want to join again, just sign up as you did the first time and we will start sending SMS messages to you again.
  4. If you are experiencing issues with the messaging program you can reply with the keyword HELP for more assistance, or you can get help directly at {support email address or toll-free number}.
  5. Carriers are not liable for delayed or undelivered messages
  6. As always, message and data rates may apply for any messages sent to you from us and to us from you. You will receive {message frequency}. If you have any questions about your text plan or data plan, it is best to contact your wireless provider.
  7. If you have any questions regarding privacy, please read our privacy policy: {link to privacy policy}

Publicly accessible privacy policy page

Message Senders are responsible for protecting the privacy of Consumers’ information and must comply with applicable privacy law. Message Senders should maintain a privacy policy for all programs and make it accessible from the initial call-to-action. The privacy policy should be labeled clearly and all cases, terms and conditions and privacy policy disclosures must provide up-to-date, accurate information about program details and functionality. For verbal scripts, a URL must be read off to the end-user enrolling in the SMS program, or the comprehensive terms must be directly included in the script.

One of the key items carriers look for in a Privacy Policy is the sharing of end-user information with third-parties. If your privacy policy mentions data sharing or selling to non-affiliated third parties, there is a concern that customer data will be shared with third parties for marketing purposes.

Express consent is required for SMS; therefore, sharing data is prohibited. Privacy policies must specify that this data sharing excludes SMS opt-in data and consent. Privacy policies can be updated (or draft versions provided) where the practice of sharing personal data to third parties is expressly omitted from the number registration.

Example: “The above excludes text messaging originator opt-in data and consent; this information will not be shared with any third parties.”

Message frequency disclosure

The message frequency disclosure provides end-users an indication of how often they’ll receive messages from you. For example, a recurring messaging program might say “one message per week.” A one-time password or multi-factor authentication use case might say “message frequency varies” or “one message per login attempt”.

Customer care contact information

Customer care contact information must be clear and readily available to help Consumers understand program details as well as their status with the program. Customer care information should result in Consumers receiving help.

Numbers should always respond to customer care requests, regardless of whether the requestor is subscribed to the program. At a minimum, Message Senders must respond to messages containing the HELP keyword with the program name and further information about how to contact the Message Sender. SMS programs should promote customer care contact instructions at program opt-in and at regular intervals in content or service messages, at least once per month.

Example: “For more information, text ‘HELP’ or call 1-800-123-1234.”

Opt-Out Information

Opt-out mechanisms facilitate Consumer choice to terminate communications from text messaging programs. Message Senders should acknowledge and respect Consumers’ opt-out requests consistent with the following guidelines:

  • Message Senders should ensure that Consumers have the ability to opt-out at any time
  • Message Senders should support multiple mechanisms of opt-out, including: phone call, email, or text
  • Message Senders should acknowledge and honor all Consumer opt-out requests by sending one final opt-out confirmation message to notify the Consumer that they have opted-out successfully. No further messages should be sent following the confirmation message.

Message Senders should include opt-out information in the call-to-action, terms and conditions, and opt-in confirmation.

If a 2FA/OTP program requires end-users to opt-in and request an OTP from the same CTA, and it is compliant with all applicable regulations, then the sender does not need to explicitly opt-out that number if the user texts “STOP” to the business’s number. However, the sender must still respond with a compliant opt-out response.

See the following Amazon Pinpoint blog post on How to Manage SMS Opt-Outs with Amazon Pinpoint

“Message and data rates may apply” disclosure

All SMS programs must display or must be read out loud (if a verbal opt-in) the disclosure verbatim: “Message and data rates may apply”. By requiring the disclosure, US mobile carriers are helping to ensure that consumers are aware of the potential costs of sending and receiving text messages, and that they have consented to receive those messages before they are sent.


SMS Opt-Ins for Independent Software Vendors (ISVs)

Definitions

In this section, we’ll outline the terms we use, to help better explain each party, and the requirements.

ISV: ISVs are positioned between Amazon Pinpoint and the ISV’s end-business customers. While they may operate differently, and/or offer different services, their requirements for SMS program registrations are largely the same.

End Business: The End Business is how we refer to your ISV customers. This is generally the entity that creates the messaging content, distributes it through your platform, and interacts with their end-users (message recipients).

Note: In some rare cases, an ISV platform can be considered the end business if they control content via templates, and collect and manage opt-in in their entirety — meaning the ISV information directly will be used for the registration and will be branded as such in the text messages. If you are unsure, we recommend including the information for the entity (your customer) that is engaging with the opted-in handset with registration. ISVs who don’t include this information (if it is required) risk their verification request being rejected.

End-User: The message recipient is considered the end-user. The person with the handset where messages terminate. As an independent software vendor (ISV), you need to comply with all applicable laws and regulations when it comes to SMS opt-ins. This means that your end business(es) need to get explicit consent from their end-users before text messages start being sent and give end-users the option to opt-out of their program at any time. You also need to provide them with a registered and approved phone number to send their SMS messages to ensure that they are delivered reliably and not flagged as spam.

When does an ISV have to submit each end business?

SMS program registrations requires end-user business information, not ISV information. This means the ISV needs to provide a mechanism for their end businesses to provide their information to be submitted for registration. For ISVs or aggregators who provide messaging services to businesses, it’s expected that the information provided represents the entity (your customer) that is sending messages to the opted-in handset.

NOTE: Amazon uses this information in accordance with all applicable obligations, and only to verify the end-user is a legitimate business. Amazon will not contact the end-business user with the information provided.

Submissions that are missing information or are populated with ISV/aggregator information may be rejected. Exceptions may apply when the use case clearly showcases that the ISV manages opt-in mechanisms, is the sole message content creator, and the messages clearly come from the ISV, not their end businesses. For example, if the ISV owns a web application that requires their end-customers to enroll into OTP.

If you are unsure, we recommend including the information for the entity (your customer) that is engaging with the opted-in handset with registration. ISVs who don’t include this information (if it is required) risk their verification request being rejected.

In conclusion

Getting user consent through a compliant opt-in process is crucial for any SMS messaging program. Key elements include clearly disclosing the program details, providing easy opt-out methods, having accessible terms of service and privacy policies, and adhering to all applicable regulations. For ISVs enabling businesses to send SMS messages, it’s important they provide a way for each end business to submit their own information for registration and comply with the requirements. By following SMS best practices around opt-ins, businesses can build trust with subscribers and ensure deliverability of their text messaging campaigns.

How to Migrate Your SMS Program to Amazon Pinpoint

Post Syndicated from Tyler Holmes original https://aws.amazon.com/blogs/messaging-and-targeting/how-to-migrate-your-sms-program-to-amazon-pinpoint/

How to Migrate Your SMS Program to Amazon Pinpoint

In the fast-paced realm of communication, where every second counts and attention spans are shorter than ever, the choice of channels that you use to deliver your message to your recipients is critical. While we often find ourselves swept away by the allure of flashy social media platforms and sleek email interfaces, it’s the unassuming text message, or SMS, that continually proves to be one of the most effective options. According to Statista, there over 5 billion mobile internet users globally, amounting to over 60% of the earth’s population of ~8 billion. SMS obviously provides an expansive reach that can help businesses connect with a diverse audience but in order to do that at scale, you need to use a service like Amazon Pinpoint that facilitates the ability to send SMS to over 240 countries and/or regions around the world. If you have a current SMS provider and are considering Pinpoint SMS for its global reach, scalability, cost effective pricing, and demonstrably high deliverability, this guide will walk you through how to migrate from your current provider.

There are several common reasons our customers give us when considering a migration. Don’t worry if your situation doesn’t fit into a neat box, we help customers navigate the dynamic landscape of SMS that is constantly evolving. Let’s dive deep into each of the below to highlight some common things we hear from our customers.

  • My current provider doesn’t deliver to countries I want to send to
  • My current provider is more expensive than Pinpoint pricing
    • Our pricing is available on the public pricing page here. Each country has it’s own cost associated with it so enter in the countries you would like to see pricing for. These prices are per message sent so if you are planning on sending to multiple countries factor in the types of messages that you will want to send as well as the countries. If your use case includes 2 way communication make sure to factor the number of inbound messages you expect into your calculations.
      • NOTE: Depending on the language the available characters per message varies, which can affect your calculations on cost. See here for an explanation
  • My current provider doesn’t have features that Pinpoint has
    • Among many other features Pinpoint has the ability to send over multiple channels, including: SMS, Email, Push/In-App, Voice, Over the Top (OTT) services such as WhatsApp, as well as interact with third-party APIs giving you the flexibility to send to many other channels.
  • My current provider is not native to AWS
    • Pinpoint, being native to the AWS Cloud, boasts the capability to seamlessly integrate with a wide array of services, including AI/ML offerings such as Amazon Personalize, Amazon Bedrock, and Amazon SageMaker, among others. This means you can leverage various AWS services to create innovative solutions that enhance and optimize the communications sent through Pinpoint.
  • My current provider does not have good deliverability
    • Price is not the only factor to consider when looking at SMS providers. If you find another provider with lower pricing make sure to ask about their deliverability to the countries you are wanting to send to. There is a big difference between sending an SMS at a low price, and actually delivering that SMS. We are happy to discuss deliverability with you, just reach out to your Account Manager if you have one or contact us to start a conversation about your migration.
  • I’m not happy with the customer support of my current provider
    • The SMS landscape is constantly changing and our SMS experts are here to help guide you through the process. Whether it’s regulatory changes, pricing changes, or creating complex architectures to support your needs. Reach out to your Account Manager if you have one or contact us to start a conversation about your migration and get your questions answered.

Regardless of your reason for considering migrating there are four scenarios that most of our customers find themselves in when beginning to plan for an SMS migration.

I have not sent SMS before but I would like to start sending through Pinpoint
Skip ahead to the section on “Checklist for Planning an SMS Migration” to start planning for sending SMS

I have number(s) (Also known as Originators, Origination Identities (OIDs), Toll-Free, 10DLC, Long Code, Short Code, and/or SenderID) with a different provider and I would like to move those to Pinpoint
The ability to “port” numbers from other providers is dependent on the type of originator, the vendor you procured them from, and the country that they support. You may need to get new originators so factor that into your timeline and reach out to your Account Manager to determine whether your originators are able to be ported over. Once you have done that, pull the reports for how much volume you are sending to each country with your current provider and then skip ahead to the section on “Checklist for Planning an SMS Migration” to start planning for sending SMS

I have a current provider but I would like to procure new numbers from Pinpoint
Pull the reports for how much volume you are sending to each country with your current provider and then skip ahead to the section on “Checklist for Planning an SMS Migration” to start planning for sending SMS

I have a current provider but would like to split traffic between them and Pinpoint
Pull the reports for how much volume you are sending to the countries you plan on migrating to AWS and then skip ahead to the section on “Checklist for Planning an SMS Migration” to start planning for sending SMS. Make sure that you consider how you will be managing opt-outs across two providers. Pinpoint offers centrally managed opt-outs but self-management is also an option. All Delivery Receipts/Reporting (DLRs) and inbound/outbound events can be streamed through Amazon Kinesis, Amazon CloudWatch, and/or Amazon Simple Notification Service (SNS) if you need to send those events to another location inside or outside of the AWS Cloud.

Checklist for Planning an SMS Migration

  • Setup a spreadsheet similar to the one outlined in this post
  • Identify your use case(s)
    • Note whether your use case is one-way or two-way
      • NOTE: Not all countries support 2-way communications, which is the ability to have the recipient send a message back to the OID.
      • NOTE: Sender ID also does not support 2-way communication so if you are planning on using Sender ID you will need to account for how to opt recipients out of future communications.
  • Identify your countries
  • Identify your volume per country
    • If you are already sending SMS with another provider pull a report over a representative time period.
  • Identify your throughput needs (Also referred to as Messages per Second, MPS, Transactions per Second, or TPS) for each country
    • Most origination identities are chosen for their ability to support a certain level of MPS, not volume, so if you have seasonality make sure to account for burst rates. There are quotas for the APIs that govern sending as well as quotas for the different types of originators.
  • Identify which origination identities you will need for each country using this guide
    • Make note of any countries/OIDs that require registration
    • Reach out to your Account Manager if you have one or contact us to start a conversation about your migration.
    • If you have OIDs you would like to migrate make sure you determine whether that is possible ASAP since your timelines could be affected by the outcome.

Make sure you give ample time for your migration. There are many entities involved in delivering SMS, from governments, to mobile carriers, to third-party registrars, and more, which means that timelines are not always within your control. Ask questions, take advantage of the expert resources we have at AWS, and the content we have produced around these topics.

Content to read

  • Review the countries and regions we support here
  • Use the format for aggregating information on your use cases outlined in this post here
  • Decide what origination IDs you will need here
  • Review the documentation for the V2 SMS and Voice API here
  • Review the Pinpoint API and SendMessage here
  • Check out the support tiers comparison here