Tag Archives: messaging

Introducing the Amazon Pinpoint SMS sandbox

Post Syndicated from Brent Meyer original https://aws.amazon.com/blogs/messaging-and-targeting/introducing-the-amazon-pinpoint-sms-sandbox/

Amazon Pinpoint now includes a new feature called the SMS sandbox. If you’ve sent email through Amazon Pinpoint (or if you’ve used Amazon SES), the sandbox might be a familiar concept. This new feature helps protect your Amazon Pinpoint account against unauthorized use, accidental sends, and unexpected charges. In this post, I’ll describe the SMS sandbox feature. You’ll learn what the sandbox is and what the benefits are for you. I’ll also talk about how to use your Amazon Pinpoint account when it’s in the sandbox, and how to have your account removed from the sandbox.

About the sandbox

The SMS sandbox is enabled by default on all new Amazon Pinpoint accounts. Also, if you have an existing account that has never had a spending limit increase (that is, if your account has a monthly spending limit of $1), your account is probably still in the SMS sandbox.

The sandbox applies to both Amazon Pinpoint and Amazon SNS. If your account is removed from the sandbox in Amazon SNS, it’s also out of the sandbox in Amazon Pinpoint, and vice-versa.

While your account is in the sandbox, you can still use all of the features of Amazon Pinpoint. However, there are a few important things to keep in mind. First, your monthly spending limit is fixed at $1. You can increase this amount when you remove your account from the sandbox. Second, you can only send messages to destination phone numbers that you’ve verified. Third, country-specific rules may apply to the registration process. For example, in the United States, you’re required to have a dedicated phone number for sending SMS messages.

SMS sandbox benefits

The SMS sandbox is a valuable tool for ensuring the security of your accounts. It protects against messages accidentally being sent to unintended recipients during your development and testing processes. The sandbox also helps protect the SMS ecosystem by preventing bad actors from sending unsolicited messages to arbitrary phone numbers.

Verifying destination phone numbers

One of the biggest changes with this release is the concept of verified phone numbers. When your account is in the sandbox, you can only send SMS messages to verified phone numbers. A verified phone number is a number that you own, or that is owned by somebody who provided permission to receive messages from you.

Note: You only have to verify recipients’ phone numbers when your account is in the sandbox. When your account is out of the sandbox, you can send messages to any phone number, even if that phone number hasn’t been verified.

While your account is in the sandbox, you can have up to 10 verified phone numbers in each AWS Region. After you verify a phone number, you have to wait 24 hours before you can delete it.

The verification process involves two steps. First, you enter the number that you want to verify in the Amazon Pinpoint console. This step is shown in the following image.

When you do this, Amazon Pinpoint sends a verification code to the phone number that you specified.

Next, you enter the verification code in the same section of the Amazon Pinpoint console where you started the verification process. This step is shown in the following image. It’s important to keep in mind that these verification codes are only valid for 15 minutes.

If the code that you enter matches the code that was sent to the phone number, then the phone number becomes verified. The following image shows an example of a phone number that has been successfully verified.

Step-by-step procedures for verifying phone numbers are available in the Amazon Pinpoint User Guide. For this verification process, we waive the standard messaging fees associated with sending the verification code up to five times per phone number.

Using the sandbox

When you complete the verification process for a phone number, you can use that phone number as a destination for your messages.

If you plan to use campaigns or journeys to send SMS messages, you can create them without restrictions. When you launch your campaigns or journeys, Amazon Pinpoint only sends messages to verified recipients. If you try to send a test message during the process of creating a campaign or journey, Amazon Pinpoint asks you to specify an origination phone number, and to select a verified phone number as the recipient. After your campaigns or journeys are sent, the analytics dashboards will indicate the number of recipients that you targeted, and that messages were only delivered to verified recipients, if there were any.

If you use the SendMessages API to send your messages, pass the verified phone number as a key in the Addresses object. You can see a basic example in the following Python code example:

import boto3
from botocore.exceptions import ClientError
 
client = boto3.client('pinpoint',region_name='us-east-1')
try:
    response = client.send_messages(
        ApplicationId='7353f53e6885409fa32d07cedexample',
        MessageRequest={
            'Addresses': {
                '+14255550142': {           # the verified phone number
                    'ChannelType': 'SMS'
                }
            },
            'MessageConfiguration': {
                'SMSMessage': {
                    'Body': 'This is a test',
                    'MessageType': 'TRANSACTIONAL',
                    'OriginationNumber': '+18445550123'
                }
            }
        }
    )
except ClientError as e:
    print(e.response['Error']['Message'])
else:
    print("Message sent! Message ID: "
            + response['MessageResponse']['Result']['+14255550142]['MessageId'])

Moving out of the sandbox

When you move your account out of the SMS sandbox, you can send messages to any phone number, even if you haven’t verified it. When your account is out of the sandbox, you also gain the ability to increase your spending limit to a value higher than $1 per month.

You can determine whether your account is still in the SMS sandbox on the SMS settings page of the Amazon Pinpoint console. The top of the page (shown in the following image) shows the steps that you must take before you can start sending SMS messages. Step 2 of this section tells you whether your account is still in the SMS sandbox.
To create a request to get out of the sandbox, create a Service Limit Increase case in the AWS Support Center. In your case, you have to provide certain details about your use case and about your consent-gathering practices. You can find complete instructions for creating these requests in the Amazon Pinpoint User Guide.

I highly recommend that you fill in all of the fields in the request form, including those that state that they’re optional. Having this information will help the AWS Support team better understand your use case. Incomplete information could result in your request being delayed or denied.

After you submit your request, the AWS Support team responds to your ticket within 24 business hours. However, the response might include additional questions. Be sure to return to the AWS Support Center periodically after you submit your request so that you can answer questions if they arise.

Wrapping up

We designed the SMS sandbox to be flexible enough to enable low-volume development and testing use cases with minimal disruptions. At the same time, the sandbox provides additional security against unintended sending, and it deters malicious senders.

The sandbox is enabled by default for all new AWS accounts. There are no additional costs for using the sandbox, or for having your account removed from it.

Ready to start sending text messages with Amazon Pinpoint? Go to console.aws.amazon.com/pinpoint today to get started!

Advanced Amazon Pinpoint Templates using Message Template Helpers

Post Syndicated from davelem original https://aws.amazon.com/blogs/messaging-and-targeting/advanced-amazon-pinpoint-templates-using-message-template-helpers/

Personalization of customer messages is a proven way to increase engagement of promotional and transactional communications. In order to make these communications repeatable and scalable, building personalization through templates is often required.

Using the Advanced Template Capabilities feature of Amazon Pinpoint, it’s now possible to create highly customized templates used for email, SMS, and Push Notifications.

Pinpoint templates are personalized using Handlebars.js. The new message template helpers are an expansion on the default Handlebars.js features. Please refer to handlebarsjs.com for more information on the default functionality of Handlebars.js

In this blog we will build an Order Confirmation template that will demonstrate a few helpers from each of the following categories:

Prerequisites

Before creating a template, you need to have an existing Amazon Pinpoint Project with the email channel enabled. The following will walk you through creating a project if you don’t already have one: Create and configure a Pinpoint Project

Architecture Overview

Step 1: Create a CSV file with sample Endpoint and Imported Segment

In Amazon Pinpoint, an endpoint represents a specific method of contacting a customer. This could be their email address (for email messages) or their phone number (for SMS messages) or a custom endpoint type. Endpoints can also contain custom attributes, and you can associate multiple endpoints with a single user. In this step, we create a simple CSV file which we will use to create a Segment in Pinpoint.

The data below contains the sample Order and Product data we will use in our Order Confirmation Email.

  1. Create a .CSV file named AdvancedTemplatesSegment.csv with the following data:
    ChannelType,Address,Id,Attributes.FirstName,Attributes.LastName,Attributes.OrderDate,Attributes.OrderNumber,Attributes.ProductNumber,Attributes.ProductNumber,Attributes.ProductNumber,Attributes.ProductNumber,Attributes.ProductNumber,Attributes.ProductName,Attributes.ProductName,Attributes.ProductName,Attributes.ProductName,Attributes.ProductName,Attributes.Amount,Attributes.Amount,Attributes.Amount,Attributes.Amount,Attributes.Amount,Attributes.ItemCount,Attributes.ItemCount,Attributes.ItemCount,Attributes.ItemCount,Attributes.ItemCount,Attributes.CLVTier,User.UserId,Metrics.Age
    EMAIL,[email protected],287b3858-3097-40e3-9af4-19bd4509a8f2,Mary,Smith,2021-01-15T18:07:13Z,460-ITS-2320,DWG8799598,XTC5517773,XRO7471152,EAT5122843,LNP9056489,non lectus aliquam,sapien placerat ante,semper sapien a libero nam dui,vitae consectetuer eget rutrum,nisl ut volutpat sapien arcu,68.88,32.89,53.19,45.38,47.31,20,76,33,15,53,High,66af7a81-77f2-485f-b115-d8c3a00f7077,84
    EMAIL,[email protected],b42e6c5f-3e15-4fdd-b61c-499508271082,,,2021-01-30T22:33:22Z,296-OZA-6579,VMC0637283,RGM6575767,BTM9430068,XCV9343127,GVU2858284,a libero nam dui,sit amet consectetuer adipiscing,at ipsum,ut dolor morbi,nullam molestie,74.86,83.18,15.42,97.03,37.42,13,94,50,54,84,Low,dadc1be9-daf4-46ce-9069-13565e03eaa0,61

    NOTE: The file above has a few attributes that are the key to personalizing our email and including multiple items in our Order Confirmation table:

    • Attributes.FirstName – This will allow us to personalize with a salutation if available.
    • Attributes.CLVTier – This is an attribute that could be specified from a Machine Learning model to determine the customers CLV Tier. We will be using it to provide coupons specific to a given CLV Tier. See Predictive Segmentation Using Amazon Pinpoint and Amazon SageMaker for an example solution that demonstrates using Machine Learning to analyze information in Pinpoint.
    • Attributes.ProductNumber – Note that we have multiple columns that repeat for the product information in the order.  Pinpoint attributes are actually stored as a list, so if you pass multiple columns with the same name it will add items to the attribute list.This is the key to how we are able to display a table of information, but note that it does require making sure the attributes are aligned in the proper columns. For example, Attributes.ProductNumber[0] needs to align with Attributes.ProductName[0]See Using variables with message template helpers for more details.
  2. Search for [email protected] above and replace with two valid email addresses. Note that if your account is still in the sandbox these will need to be verified email addresses. If you only have access to a single email address you can use labels by adding a plus sign (+) followed by a string of text after the local part of the address and before the at (@) sign. For example: [email protected] and [email protected]
  3. Create a Pinpoint Segment
    1. Open the Amazon Pinpoint console at http://console.aws.amazon.com/pinpoint, and then choose the project that you created as part of the Prerequisites.
    2. In the navigation pane, choose Segments, and then choose Create a segment.
    3. Select Import a Segment.
    4. Browse to or Drag and Drop the .CSV file you created in the previous step.
    5. Use the default Segment Name and select Create Segment.

Step 2: Build The Message Template

  1. Open the Amazon Pinpoint console at http://console.aws.amazon.com/pinpoint.
  2. In the navigation pane, choose Message templates, and then choose Create template.
  3. Select Email as the Channel.
  4. For Template name use: AdvancedTemplateExample.
  5. For Subject use: AdvancedTemplateExample.
  6. Paste the following code into the HTML Editor. We will take some time later on to dig into the specific Handlebars helpers:
    {{#* inline "salutation"}}
        {{#if Attributes.FirstName.[0]}}
            Dear {{Attributes.FirstName.[0]}},<br />
        {{else}}
            Dear Valued Customer,<br />
        {{/if}}
    {{/inline}}
    
    {{#* inline "clvcoupon"}}
        {{#if Attributes.CLVTier.[0]}}
            {{#eq Attributes.CLVTier.[0] "High"}}
                As a thank-you for your continued support, please use this coupon code for <strong>30%</strong> off your next order: <strong>WELOVEYOU30</strong>
            {{/eq}}
        {{/if}}
    {{/inline}}
    
    {{#* inline "footer"}}
        <hr />	
        Accent Athletics - 1234 Anywhere Ave, Anywhere USA, 12345 - <a href="https://www.example.com/preferences/index.html?pid={{ApplicationId}}&uid={{User.UserId}}&h={{sha256 (join User.UserId "d67c37ed538b751d850de18" "+" prefix="" suffix="")}}">Manage Preferences</a>
        <hr />
    {{/inline}}
    
    
    <!DOCTYPE html>
        <html lang="en">
        <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    </head>
    <body>
      {{> salutation}}
      Thank-you for your order! {{> clvcoupon}}<br /><br />
      Order Date: {{now format="d MMM yyyy HH:mm:ss" tz=America/Los_Angeles}}<br /><br />
      <table>
      <thead>
        <tr style="background-color: #f2f2f2;">
          <th style="text-align:left; width:75px">
            Product #
          </th>
          <th style="text-align:left; width:200px;">
            Name
          </th>
          <th style="text-align:center; width:75px;">
            Count
          </th>
          <th style="text-align:center; width:75px;">
            Amount
          </th>
        </tr>
      </thead>
      <tbody>
      {{#each Attributes.ProductNumber}}
        {{#eq (modulo @index 2) "1.0"}}
            <tr style="background-color: #f2f2f2;">
        {{else}}
            <tr>
        {{/eq}}
          <td style="text-align:left;">{{this}}</td>
          <td style="text-align:left;">{{#with (lookup ../Attributes.ProductName @index)}}{{this}}{{/with}}</td>
          <td style="text-align:center;">{{#with (lookup ../Attributes.ItemCount @index)}}{{this}}{{/with}}</td>
          <td style="text-align:center;">${{#with (lookup ../Attributes.Amount @index)}}{{this}}{{/with}}</td>
        </tr>
      {{else}}
        <tr>
          <td style="text-align:left;">{{Attributes.ProductNumber}}</td>
          <td style="text-align:center;">{{Attributes.ProductName}}</td>
          <td style="text-align:center;">{{Attributes.ItemCount}}</td>
          <td style="text-align:center;">{{Attributes.Amount}}</td>
        </tr>
      {{/each}}
      </tbody>
      </table>
      {{> footer}}
    </body>
    </html>

  7. Click Create to finish creating your template

Step 3: Create an Amazon Pinpoint Campaign

By sending a campaign, we can verify that our Amazon Pinpoint project is configured correctly, and that we created the segment and template correctly.

To create the segment and campaign:

  1. Open the Amazon Pinpoint console at http://console.aws.amazon.com/pinpoint, and then choose the project that you created in step 1.
  2. In the navigation pane, choose Campaigns, and then choose Create a campaign.
  3. Name the campaign “AdvancedTemplateTest.” Under Choose a channel for this campaign, choose Email, and then choose Next.
  4. On the Choose a segment page, choose the “AdvancedTemplateExample” segment that you just created, and then choose Next.
  5. In Create your message, choose the template we just created, ‘AdvancedTemplateExample. Note: You will see an Alert with: “This template contains a reference to an attribute from another project…” This is expected as Pinpoint is scanning the template for attributes allowing you to specify default values in case the endpoint doesn’t contain a value for the attribute. In this blog post we are using the {{#if}} conditional helper to handle any missing data, i.e. {{#if Attributes.FirstName.[0]}}
  6. On the Choose when to send the campaign page, keep all of the default values, and then choose Next.
  7. On the Review and launch page, choose Launch campaign.

Within a few seconds, you should receive an email:

So what just happened?

Let’s take a deeper dive at each of helpers we included in the template:

{{#* inline "salutation"}}
    {{#if Attributes.FirstName.[0]}}
        Dear {{Attributes.FirstName.[0]}},<br /><br />
    {{else}}
        Dear Valued Customer,<br /><br />
    {{/if}}
{{/inline}}

First you will notice that we are making use of Inline Partials. Using Inline Partials allows you to build a library of frequently used snippets of content. In this case we frequently use a salutation in our communications. You can build and maintain your own frequently used snippets and include them at the beginning of the template.

Later in the message we can simply include: {{> salutation}} to include a salutation in our email.

In this example we also see the {{#if}} helper which is used to evaluate if a first name is available on the endpoint. If the name is found, a greeting is returned that passes the user’s first name in the response. Otherwise, the else statement returns an alternative greeting.

{{#* inline "clvcoupon"}}
    {{#if Attributes.CLVTier.[0]}}
        {{#eq Attributes.CLVTier.[0] "High"}}
            As a thank-you for your continued support, please use this coupon code for <strong>30%</strong> off your next order: <strong>WELOVEYOU30</strong>
        {{/eq}}
    {{/if}}
{{/inline}}

Again, we are using Inline Partials to organize our code. Additionally we are using {{#if}} to see if the user has a CLVTier attribute and if so, we use the {{#eq}} conditional helper to see if their CLVTier is “High” as we only want this coupon to display for customers that fall into that tier.

Note that CLVTier is an attribute that is populated along with the Endpoint when we created the Segment above. You could also use a solution such as Predictive Segmentation using Amazon Pinpoint and Amazon SageMaker to incorporate Machine Learning to classify your existing users.

{{#* inline "footer"}}
    <hr />
    Accent Athletics - 1234 Anywhere Ave, Anywhere USA, 12345 - <a href="https://www.example.com/preferences/index.html?pid={{ApplicationId}}&uid={{User.UserId}}&h={{sha256 (join User.UserId "d67c37ed538b751d850de18" "+" prefix="" suffix="")}}">Manage Preferences</a>
    <hr />
{{/inline}}

In the example above we are using the {{sha256}} and {{join}} helpers to create a secure link to the Preference Center deployed as part of the Amazon Pinpoint Preference Center solution.

{{> salutation}}
Thank-you for your order! {{> clvcoupon}}<br /><br /><hr />
Order Date: {{now format="d MMM yyyy HH:mm:ss" tz=America/Los_Angeles}}<br /><br />

This is where all of our hard work implementing Inline Partials really starts to pay off. To include our salutation and coupon we simply need to specify: {{> salutation}} and {{> clvcoupon}}

The {{now}} string helper allows us to display the current date and time in the format of our choosing. Please reference the following for more details on the date pattern and available timezones:

<tbody>
{{#each Attributes.ProductNumber}}
{{#eq (modulo @index 2) "1.0"}}
    		<tr style="background-color: #f2f2f2;">
{{else}}
    		<tr>
{{/eq}}
    	<td style="text-align:left;">{{this}}</td>
    	<td style="text-align:left;">{{#with (lookup ../Attributes.ProductName @index)}}{{this}}{{/with}}</td>
    	<td style="text-align:center;">{{#with (lookup ../Attributes.ItemCount @index)}}{{this}}{{/with}}</td>
    	<td style="text-align:center;">${{#with (lookup ../Attributes.Amount @index)}}{{this}}{{/with}}</td>
</tr>
{{else}}
<tr>
    		<td style="text-align:left;">{{Attributes.ProductNumber}}</td>
    		<td style="text-align:center;">{{Attributes.ProductName}}</td>
    		<td style="text-align:center;">{{Attributes.ItemCount}}</td>
    		<td style="text-align:center;">{{Attributes.Amount}}</td>
</tr>
{{/each}}
</tbody>

This particular section has a lot going on. We will break each part down for further explanation:

  • {{#each}} – Allows us to loop through each of the values in our attribute. In this case our ProductNumber attribute will contain: [“order#1”, “order#2”,”order#3, etc.]
    Note that if you only have one item in the attribute array. Pinpoint will simplify that into a single string attribute. That is why we have the {{each}} part of the {{#else}} statement. This allows us to reference the attribute as a single string in case we don’t have a collection of values in the attribute.
  • {{#eq (modulo @index 2) “1.0”}} – In order to alternate our background color for even/odd rows, we are making use of the {{modulo}} operator from the Math and encoding helpers which will return the remainder of two given numbers allowing us to determine if this is an odd or even row.
    @index is a native Handlebars.js property that contains the current index we are on in the loop.
  • {{this}} – When iterating through a collection using {{#each}}, {{this}} allows you to reference the current item in the collection
  • {{#with (lookup ../Attributes.ProductName @index)}}{{this}}{{/with}}Lookup is a built-in handlebars helper that allows us to find values in another collection. We are using the index combined with lookup to find the Product Name that goes along with the Product Number we are currently on. The same pattern is used for the remaining columns of the table.The ability to lookup values in another attribute collection is the key to how we are able to display a table of information, but note that it does require making sure the attributes are aligned in the proper columns. For example, Attributes.ProductNumber[0] needs to align with Attributes.ProductName[0]See Using variables with message template helpers for more details.
{{> footer}}

And just to wrap things up, let’s pull in the footer we defined with Inline Partials.

Next Steps

Using the techniques above, you can create sophisticated and personalized communications using Amazon Pinpoint.

Think about your existing communications to see if you can use personalization to increase customer engagement for your promotional and transactional messages.

Amazon Pinpoint is a flexible and scalable outbound and inbound marketing communications service. Learn more here: https://aws.amazon.com/pinpoint/

Complying with DMARC across multiple accounts using Amazon SES

Post Syndicated from Brendan Paul original https://aws.amazon.com/blogs/messaging-and-targeting/complying-with-dmarc-across-multiple-accounts-using-amazon-ses/

Introduction

For enterprises of all sizes, email is a critical piece of infrastructure that supports large volumes of communication from an organization. As such, companies need a robust solution to deal with the complexities this may introduce. In some cases, companies have multiple domains that support several different business units and need a distributed way of managing email sending for those domains. For example, you might want different business units to have the ability to send emails from subdomains, or give a marketing company the ability to send emails on your behalf. Amazon Simple Email Service (Amazon SES) is a cost-effective, flexible, and scalable email service that enables developers to send mail from any application. One of the benefits of Amazon SES is that you can configure Amazon SES to authorize other users to send emails from addresses or domains that you own (your identities) using their own AWS accounts. When allowing other accounts to send emails from your domain, it is important to ensure this is done securely. Amazon SES allows you to send emails to your users using popular authentication methods such as DMARC. In this blog, we walk you through 1/ how to comply with DMARC when using Amazon SES and 2/ how to enable other AWS accounts to send authenticated emails from your domain.

DMARC: what is it, why is it important?

DMARC stands for “Domain-based Message Authentication, Reporting & Conformance”, and it is an email authentication protocol (DMARC.org). DMARC gives domain owners and email senders a way to protect their domain from being used by malicious actors in phishing or spoofing attacks. Email spoofing can be used as a way to compromise users’ financial or personal information by taking advantage of their trust of well-known brands. DMARC makes it easier for senders and recipients to determine whether or not an email was actually sent by the domain that it claims to have been sent by.

Solution Overview

In this solution, you will learn how to set up DKIM signing on Amazon SES, implement a DMARC Policy, and enable other accounts in your organization to send emails from your domain using Sending Authorization. When you set up DKIM signing, Amazon SES will attach a digital signature to all outgoing messages, allowing recipients to verify that the email came from your domain. You will then set your DMARC Policy, which tells an email receiver what to do if an email is not authenticated. Lastly, you will set up Sending Authorization so that other AWS accounts can send authenticated emails from your domain.

Prerequisites

In order to complete the example illustrated in this blog post, you will need to have:

  1. A domain in an Amazon Route53 Hosted Zone or third-party provider. Note: You will need to add/update records for the domain. For this blog we will be using Route53.
  2. An AWS Organization
  3. A second AWS account to send Amazon SES Emails within a different AWS Organizations OU. If you have not worked with AWS Organizations before, review the Organizations Getting Started Guide

How to comply with DMARC (DKIM and SPF) in Amazon SES

In order to comply with DMARC, you must authenticate your messages with either DKIM (DomainKeys Identified Mail), SPF (Sender Policy Framework), or both. DKIM allows you to send email messages with a cryptographic key, which enables email providers to determine whether or not the email is authentic. SPF defines what servers are allowed to send emails for their domain. To use SPF for DMARC compliance you need to set up a custom MAIL FROM domain in Amazon SES. To authenticate your emails with DKIM in Amazon SES, you have the option of:

In this blog, you will be setting up a sending identity.

Setting up DKIM Signing in Amazon SES

  1. Navigate to the Amazon SES Console 
  2. Select Verify a New Domain and type the name of your domain in
  3. Select Generate DKIM Settings
  4. Choose Verify This Domain
    1. This will generate the DNS records needed to complete domain verification, DKIM signing, and routing incoming mail.
    2. Note: When you initiate domain verification using the Amazon SES console or API, Amazon SES gives you the name and value to use for the TXT record. Add a TXT record to your domain’s DNS server using the specified Name and Value. Amazon SES domain verification is complete when Amazon SES detects the existence of the TXT record in your domain’s DNS settings.
  5. If you are using Route 53 as your DNS provider, choose the Use Route 53 button to update the DNS records automatically
    1. If you are not using Route 53, go to your third-party provider and add the TXT record to verify the domain as well as the three CNAME records to enable DKIM signing. You can also add the MX record at the end to route incoming mail to Amazon SES.
    2. A list of common DNS Providers and instructions on how to update the DNS records can be found in the Amazon SES documentation
  6. Choose Create Record Sets if you are using Route53 as shown below or choose Close after you have added the necessary records to your third-party DNS provider.

 

Note: in the case that you previously verified a domain, but did NOT generate the DKIM settings for your domain, follow the steps below. Skip these steps if this is not the case:

  1. Go to the Amazon SES Console, and select your domain
  2. Select the DKIM dropdown
  3. Choose Generate DKIM Settings and copy the three values in the record set shown
    1. You may also download the record set as a CSV file
  4. Navigate to the Route53 console or your third-party DNS provider. Instructions on how to update the DNS records in your third-party can be found in the Amazon SES documentation
  5. Select the domain you are using
  6. Choose Create Record

  1. Enter the values that Amazon SES has generated for you, and add the three CNAME records to your domain
  2. Wait a few minutes, and go back to your domain in the Amazon SES Console
  3. Check that the DKIM status is verified

You also want to set up a custom MAIL FROM domain that you will use later on. To do so, follow the steps in the documentation.

Setting up a DMARC policy on your domain

DMARC policies are TXT records you place in DNS to define what happens to incoming emails that don’t align with the validations provided when setting up DKIM and SPF. With this policy, you can choose to allow the email to pass through, quarantine the email into a folder like junk or spam, or reject the email.

As a best practice, you should start with a DMARC policy that doesn’t reject all email traffic and collect reports on emails that don’t align to determine if they should be allowed. You can also set a percentage on the DMARC policy to perform filtering on a subset of emails to, for example, quarantine only 50% of the emails that don’t align. Once you are in a state where you can begin to reject non-compliant emails, flip the policy to reject failed authentications. When you set the DMARC policy for your domain, any subdomains that are authorized to send on behalf of your domain will inherit this policy and the same rule will apply. For more information on setting up a DMARC policy, see our documentation.

In a scenario where you have multiple subdomains sending emails, you should be setting the DMARC policy for the organizational domain that you own. For example, if you own the domain example.com and also want to use the sub-domain sender.example.com to send emails you can set the organizational DMARC policy (as a DNS TXT record) to:

Name Type Value
1 _dmarc.example.com TXT “v=DMARC1;p=quarantine;pct=50;rua=mailto:[email protected]

This DMARC policy states that 50% of emails coming from example.com that fail authentication should be quarantined and you want to send a report of those failures to [email protected]. For your sender.example.com sub-domain, this policy will be inherited unless you specify another DMARC policy for our sub-domain. In the case where you want to be stricter on the sub-domain you could add another DMARC policy like you see in the following table.

 

Name Type Value
1 _dmarc.sender.example.com TXT “v=DMARC1;p=reject;pct=100;rua=mailto:[email protected];ruf=mailto:[email protected]

This policy would apply to emails coming from sender.example.com and would reject any email that fails authentication. It would also send aggregate feedback to [email protected] and detailed message-specific failure information to [email protected] for further analysis.

Sending Authorization in Amazon SES – Allowing Other Accounts to Send Authenticated Emails

Now that you have configured Amazon SES to comply with DMARC in the account that owns your identity, you may want to allow other accounts in your organization the ability to send emails in the same way. Using Sending Authorization, you can authorize other users or accounts to send emails from identities that you own and manage. An example of where this could be useful is if you are an organization which has different business units in that organization. Using sending authorization, a business unit’s application could send emails to their customers from the top-level domain. This application would be able to leverage the authentication settings of the identity owner without additional configuration. Another advantage is that if the business unit has its own subdomain, the top-level domain’s DKIM settings can apply to this subdomain, so long as you are using Easy DKIM in Amazon SES and have not set up Easy DKIM for the specific subdomains.

Setting up sending authorization across accounts

Before you set up sending authorization, note that working across multiple accounts can impact bounces, complaints, pricing, and quotas in Amazon SES. Amazon SES documentation provides a good understanding of the impacts when using multiple accounts. Specifically, delegated senders are responsible for bounces and complaints and can set up notifications to monitor such activities. These also count against the delegated senders account quotas. To set up Sending Authorization across accounts:

  1. Navigate to the Amazon SES Console from the account that owns the Domain
  2. Select Domains under Identity Management
  3. Select the domain that you want to set up sending authorization with
  4. Select View Details
  5. Expand Identity Policies and Click Create Policy
  6. You can either create a policy using the policy generator or create a custom policy. For the purposes of this blog, you will create a custom policy.
  7. For the custom policy, you will allow a particular Organization Unit (OU) from our AWS Organization access to our domain. You can also limit access to particular accounts or other IAM principals. Use the following policy to allow a particular OU to access the domain:

{
  “Version”: “2012-10-17”,
  “Id”: “AuthPolicy”,
  “Statement”: [
    {
      “Sid”: “AuthorizeOU”,
      “Effect”: “Allow”,
      “Principal”: “*”,
      “Action”: [
        “SES:SendEmail”,
        “SES:SendRawEmail”
      ],
      “Resource”: “<Arn of Verified Domain>”,
      “Condition”: {
        “ForAnyValue:StringLike”: {
          “aws:PrincipalOrgPaths”: “<Organization Id>/<Root OU Id>/<Organizational Unit Id>”
        }
      }
    }
  ]
}

9. Make sure to replace the escaped values with your Verified Domain ARN and the Org path of the OU you want to limit access to.

 

You can find more policy examples in the documentation. Note that you can configure sending authorization such that all accounts under your AWS Organization are authorized to send via a certain subdomain.

Testing

You can now test the ability to send emails from your domain in a different AWS account. You will do this by creating a Lambda function to send a test email. Before you create the Lambda function, you will need to create an IAM role for the Lambda function to use.

Creating the IAM Role:

  1. Log in to your separate AWS account
  2. Navigate to the IAM Management Console
  3. Select Role and choose Create Role
  4. Under Choose a use case select Lambda
  5. choose Next: Permissions
  6. In the search bar, type SES and select the check box next to AmazonSESFullAccess
  7. Choose Next:Tags and Review
  8. Give the role a name of your choosing, and choose Create Role

Navigate to Lambda Console

  1. Select Create Function
  2. Choose the box marked Author from Scratch
  3. Give the function a name of your choosing (Ex: TestSESfunction)
  4. In this demo, you will be using Python 3.8 runtime, but feel free to modify to your language of choice
  5. Select the Change default execution role dropdown, and choose the Use an existing role radio button
  6. Under Existing Role, choose the role that you created in the previous step, and create the function

Edit the function

  1. Navigate to the Function Code portion of the page and open the function python file
  2. Replace the default code with the code shown below, ensuring that you put your own values in based on your resources
  3. Values needed:
    1. Test Email Address: an email address you have access to
      1. NOTE: If you are still operating in the Amazon SES Sandbox, this will need to be a verified email in Amazon SES. To verify an email in Amazon SES, follow the process here. Alternatively, here is how you can move out of the Amazon SES Sandbox
    2. SourceArn: The arn of your domain. This can be found in Amazon SES Console → Domains → <YourDomain> → Identity ARN
    3. ReturnPathArn: The same as your Source ARN
    4. Source: This should be your Mail FROM Domain @ your domain
      1. Your Mail FROM Domain can be found under Domains → <YourDomain> → Mail FROM Domain dropdown
      2. Ex: [email protected]
    5. Use the following function code for this example

import json
import boto3
from botocore.exceptions import ClientError

client = boto3.client('ses')
def lambda_handler(event, context):
    # Try to send the email.
    try:
        #Provide the contents of the email.
        response = client.send_email(
            Destination={
                'ToAddresses': [
                    '<[email protected]>',
                ],
            },
            Message={
                'Body': {
                    'Html': {
                        'Charset': 'UTF-8',
                        'Data': 'This email was sent with Amazon SES.',
                    },
                },
                'Subject': {
                    'Charset': 'UTF-8',
                    'Data': 'Amazon SES Test',
                },
            },
            SourceArn='<your-ses-identity-ARN>',
            ReturnPathArn='<your-ses-identity-ARN>',
            Source='<[email protected]>',
             )
    # Display an error if something goes wrong.
    except ClientError as e:
        print(e.response['Error']['Message'])
    else:
        print("Email sent! Message ID:"),
        print(response['ResponseMetadata']['RequestId'])

  1. Once you have replaced the appropriate values, choose the Deploy button to deploy your changes

Run a Test invocation

  1. After you have deployed your changes, select the “Test” Panel above your function code

  1. You can leave all of these keys and values as default, as the function does not use any event parameters
  2. Choose the Invoke button in the top right corner
  3. You should see this above the test event window:

Verifying that the Email has been signed properly

Depending on your email provider, you may be able to check the DKIM signature directly in the application. As an example, for Outlook, right click on the message, and choose View Source from the menu. You should see line that shows the Authentication Results and whether or not the DKIM/SPF signature passed. For Gmail, go to your Gmail Inbox on the Gmail web app. Choose the message you wish to inspect, and choose the More Icon. Choose View Original from the drop-down menu. You should then see the SPF and DKIM “PASS” Results.

Cleanup

To clean up the resources in your account,

  1. Navigate to the Route53 Console
  2. Select the Hosted Zone you have been working with
  3. Select the CNAME, TXT, and MX records that you created earlier in this blog and delete them
  4. Navigate to the SES Console
  5. Select Domains
  6. Select the Domain that you have been working with
  7. Click the drop down Identity Policies and delete the one that you created in this blog
  8. If you verified a domain for the sake of this blog: navigate to the Domains tab, select the domain and select Remove
  9. Navigate to the Lambda Console
  10. Select Functions
  11. Select the function that you created in this exercise
  12. Select Actions and delete the function

Conclusion

In this blog post, we demonstrated how to delegate sending and management of your sub-domains to other AWS accounts while also complying with DMARC when using Amazon SES. In order to do this, you set up a sending identity so that Amazon SES automatically adds a DKIM signature to your messages. Additionally, you created a custom MAIL FROM domain to comply with SPF. Lastly, you authorized another AWS account to send emails from a sub-domain managed in a different account, and tested this using a Lambda function. Allowing other accounts the ability to manage and send email from your sub-domains provides flexibility and scalability for your organization without compromising on security.

Now that you have set up DMARC authentication for multiple accounts in your enviornment, head to the AWS Messaging & Targeting Blog to see examples of how you can combine Amazon SES with other AWS Services!

If you have more questions about Amazon Simple Email Service, check out our FAQs or our Developer Guide.

If you have feedback about this post, submit comments in the Comments section below.

Forwarding emails automatically based on content with Amazon Simple Email Service

Post Syndicated from Murat Balkan original https://aws.amazon.com/blogs/messaging-and-targeting/forwarding-emails-automatically-based-on-content-with-amazon-simple-email-service/

Introduction

Email is one of the most popular channels consumers use to interact with support organizations. In its most basic form, consumers will send their email to a catch-all email address where it is further dispatched to the correct support group. Often, this requires a person to inspect content manually. Some IT organizations even have a dedicated support group that handles triaging the incoming emails before assigning them to specialized support teams. Triaging each email can be challenging, and delays in email routing and support processes can reduce customer satisfaction. By utilizing Amazon Simple Email Service’s deep integration with Amazon S3, AWS Lambda, and other AWS services, the task of categorizing and routing emails is automated. This automation results in increased operational efficiencies and reduced costs.

This blog post shows you how a serverless application will receive emails with Amazon SES and deliver them to an Amazon S3 bucket. The application uses Amazon Comprehend to identify the dominant language from the message body.  It then looks it up in an Amazon DynamoDB table to find the support group’s email address specializing in the email subject. As the last step, it forwards the email via Amazon SES to its destination. Archiving incoming emails to Amazon S3 also enables further processing or auditing.

Architecture

By completing the steps in this post, you will create a system that uses the architecture illustrated in the following image:

Architecture showing how to forward emails by content using Amazon SES

The flow of events starts when a customer sends an email to the generic support email address like [email protected]. This email is listened to by Amazon SES via a recipient rule. As per the rule, incoming messages are written to a specified Amazon S3 bucket with a given prefix.

This bucket and prefix are configured with S3 Events to trigger a Lambda function on object creation events. The Lambda function reads the email object, parses the contents, and sends them to Amazon Comprehend for language detection.

Amazon DynamoDB looks up the detected language code from an Amazon DynamoDB table, which includes the mappings between language codes and support group email addresses for these languages. One support group could answer English emails, while another support group answers French emails. The Lambda function determines the destination address and re-sends the same email address by performing an email forward operation. Suppose the lookup does not return any destination address, or the language was not be detected. In that case, the email is forwarded to a catch-all email address specified during the application deployment.

In this example, Amazon SES hosts the destination email addresses used for forwarding, but this is not a requirement. External email servers will also receive the forwarded emails.

Prerequisites

To use Amazon SES for receiving email messages, you need to verify a domain that you own. Refer to the documentation to verify your domain with Amazon SES console. If you do not have a domain name, you will register one from Amazon Route 53.

Deploying the Sample Application

Clone this GitHub repository to your local machine and install and configure AWS SAM with a test AWS Identity and Access Management (IAM) user.

You will use AWS SAM to deploy the remaining parts of this serverless architecture.

The AWS SAM template creates the following resources:

  • An Amazon DynamoDB mapping table (language-lookup) contains information about language codes and associates them with destination email addresses.
  • An AWS Lambda function (BlogEmailForwarder) that reads the email content parses it, detects the language, looks up the forwarding destination email address, and sends it.
  • An Amazon S3 bucket, which will store the incoming emails.
  • IAM roles and policies.

To start the AWS SAM deployment, navigate to the root directory of the repository you downloaded and where the template.yaml AWS SAM template resides. AWS SAM also requires you to specify an Amazon Simple Storage Service (Amazon S3) bucket to hold the deployment artifacts. If you haven’t already created a bucket for this purpose, create one now. You will refer to the documentation to learn how to create an Amazon S3 bucket. The bucket should have read and write access by an AWS Identity and Access Management (IAM) user.

At the command line, enter the following command to package the application:

sam package --template template.yaml --output-template-file output_template.yaml --s3-bucket BUCKET_NAME_HERE

In the preceding command, replace BUCKET_NAME_HERE with the name of the Amazon S3 bucket that should hold the deployment artifacts.

AWS SAM packages the application and copies it into this Amazon S3 bucket.

When the AWS SAM package command finishes running, enter the following command to deploy the package:

sam deploy --template-file output_template.yaml --stack-name blogstack --capabilities CAPABILITY_IAM --parameter-overrides [email protected] YOUR_DOMAIN_NAME_HERE [email protected] YOUR_DOMAIN_NAME_HERE

In the preceding command, change the YOUR_DOMAIN_NAME_HERE with the domain name you validated with Amazon SES. This domain also applies to other commands and configurations that will be introduced later.

This example uses “blogstack” as the stack name, you will change this to any other name you want. When you run this command, AWS SAM shows the progress of the deployment.

Configure the Sample Application

Now that you have deployed the application, you will configure it.

Configuring Receipt Rules

To deliver incoming messages to Amazon S3 bucket, you need to create a Rule Set and a Receipt rule under it.

Note: This blog uses Amazon SES console to create the rule sets. To create the rule sets with AWS CloudFormation, refer to the documentation.

  1. Navigate to the Amazon SES console. From the left navigation choose Rule Sets.
  2. Choose Create a Receipt Rule button at the right pane.
  3. Add [email protected]YOUR_DOMAIN_NAME_HERE as the first recipient addresses by entering it into the text box and choosing Add Recipient.

 

 

Choose the Next Step button to move on to the next step.

  1. On the Actions page, select S3 from the Add action drop-down to reveal S3 action’s details. Select the S3 bucket that was created by the AWS SAM template. It is in the format of your_stack_name-inboxbucket-randomstring. You will find the exact name in the outputs section of the AWS SAM deployment under the key name InboxBucket or by visiting the AWS CloudFormation console. Set the Object key prefix to info/. This tells Amazon SES to add this prefix to all messages destined to this recipient address. This way, you will re-use the same bucket for different recipients.

Choose the Next Step button to move on to the next step.

In the Rule Details page, give this rule a name at the Rule name field. This example uses the name info-recipient-rule. Leave the rest of the fields with their default values.

Choose the Next Step button to move on to the next step.

  1. Review your settings on the Review page and finalize rule creation by choosing Create Rule

  1. In this example, you will be hosting the destination email addresses in Amazon SES rather than forwarding the messages to an external email server. This way, you will be able to see the forwarded messages in your Amazon S3 bucket under different prefixes. To host the destination email addresses, you need to create different rules under the default rule set. Create three additional rules for [email protected]YOUR_DOMAIN_NAME_HERE , [email protected] YOUR_DOMAIN_NAME_HERE and [email protected]YOUR_DOMAIN_NAME_HERE email addresses by repeating the steps 2 to 5. For Amazon S3 prefixes, use catchall/, english/, and french/ respectively.

 

Configuring Amazon DynamoDB Table

To configure the Amazon DynamoDB table that is used by the sample application

  1. Navigate to Amazon DynamoDB console and reach the tables view. Inspect the table created by the AWS SAM application.

language-lookup table is the table where languages and their support group mappings are kept. You need to create an item for each language, and an item that will hold the default destination email address that will be used in case no language match is found. Amazon Comprehend supports more than 60 different languages. You will visit the documentation for the supported languages and add their language codes to this lookup table to enhance this application.

  1. To start inserting items, choose the language-lookup table to open table overview page.
  2. Select the Items tab and choose the Create item From the dropdown, select Text. Add the following JSON content and choose Save to create your first mapping object. While adding the following object, replace Destination attribute’s value with an email address you own. The email messages will be forwarded to that address.

{

  “language”: “en”,

  “destination”: “[email protected]_DOMAIN_NAME_HERE”

}

Lastly, create an item for French language support.

{

  “language”: “fr”,

  “destination”: “[email protected]_DOMAIN_NAME_HERE”

}

Testing

Now that the application is deployed and configured, you will test it.

  1. Use your favorite email client to send the following email to the domain name [email protected] email address.

Subject: I need help

Body:

Hello, I’d like to return the shoes I bought from your online store. How can I do this?

After the email is sent, navigate to the Amazon S3 console to inspect the contents of the Amazon S3 bucket that is backing the Amazon SES Rule Sets. You will also see the AWS Lambda logs from the Amazon CloudWatch console to confirm that the Lambda function is triggered and run successfully. You should receive an email with the same content at the address you defined for the English language.

  1. Next, send another email with the same content, this time in French language.

Subject: j’ai besoin d’aide

Body:

Bonjour, je souhaite retourner les chaussures que j’ai achetées dans votre boutique en ligne. Comment puis-je faire ceci?

 

Suppose a message is not matched to a language in the lookup table. In that case, the Lambda function will forward it to the catchall email address that you provided during the AWS SAM deployment.

You will inspect the new email objects under english/, french/ and catchall/ prefixes to observe the forwarding behavior.

Continue experimenting with the sample application by sending different email contents to [email protected] YOUR_DOMAIN_NAME_HERE address or adding other language codes and email address combinations into the mapping table. You will find the available languages and their codes in the documentation. When adding a new language support, don’t forget to associate a new email address and Amazon S3 bucket prefix by defining a new rule.

Cleanup

To clean up the resources you used in your account,

  1. Navigate to the Amazon S3 console and delete the inbox bucket’s contents. You will find the name of this bucket in the outputs section of the AWS SAM deployment under the key name InboxBucket or by visiting the AWS CloudFormation console.
  2. Navigate to AWS CloudFormation console and delete the stack named “blogstack”.
  3. After the stack is deleted, remove the domain from Amazon SES. To do this, navigate to the Amazon SES Console and choose Domains from the left navigation. Select the domain you want to remove and choose Remove button to remove it from Amazon SES.
  4. From the Amazon SES Console, navigate to the Rule Sets from the left navigation. On the Active Rule Set section, choose View Active Rule Set button and delete all the rules you have created, by selecting the rule and choosing Action, Delete.
  5. On the Rule Sets page choose Disable Active Rule Set button to disable listening for incoming email messages.
  6. On the Rule Sets page, Inactive Rule Sets section, delete the only rule set, by selecting the rule set and choosing Action, Delete.
  7. Navigate to CloudWatch console and from the left navigation choose Logs, Log groups. Find the log group that belongs to the BlogEmailForwarderFunction resource and delete it by selecting it and choosing Actions, Delete log group(s).
  8. You will also delete the Amazon S3 bucket you used for packaging and deploying the AWS SAM application.

 

Conclusion

This solution shows how to use Amazon SES to classify email messages by the dominant content language and forward them to respective support groups. You will use the same techniques to implement similar scenarios. You will forward emails based on custom key entities, like product codes, or you will remove PII information from emails before forwarding with Amazon Comprehend.

With its native integrations with AWS services, Amazon SES allows you to enhance your email applications with different AWS Cloud capabilities easily.

To learn more about email forwarding with Amazon SES, you will visit documentation and AWS blogs.

Create a serverless feedback collector application using Amazon Pinpoint’s two-way SMS functionality

Post Syndicated from Murat Balkan original https://aws.amazon.com/blogs/messaging-and-targeting/create-a-serverless-feedback-collector-application-by-using-amazon-pinpoints-two-way-sms-functionality/

Introduction

Two-way SMS communication is used by many companies to create interactive engagements with their customers. Traditional SMS notifications are one-way. While this is valid for many different use cases like one-time passwords (OTP) notifications and security notifications or reminders, some other use-cases may benefit from collecting information from the same channel. Two-way SMS allows customers to create this feedback mechanism and enhance business interactions and overall customer experience.

SMS is chosen for its simplicity and availability across different sets of devices. By combining the two-way SMS mechanism with the vast breadth of services Amazon Web Services (AWS) offers, companies can create effective architectures to better interact and serve their customers.

This blog post shows you how a serverless online appointment application can use Amazon Pinpoint’s two-way SMS functionality to collect customer feedback for completed appointments. You will learn how Amazon Pinpoint interacts with other AWS serverless services with its out-of-the-box integrations to create a scalable messaging application.

Architecture

By completing the steps in this post, you can create a system that uses the architecture illustrated in the following image:

The architecture of a feedback collector application that is composed of serverless AWS services

The flow of events starts when a Amazon DynamoDB table item, representing an online appointment, changes its status to COMPLETED. An AWS Lambda function which is subscribed to these changes over DynamoDB Streams detects this change and sends an SMS to the customer by using Amazon Pinpoint API’s sendMessages operation.

Amazon Pinpoint delivers the SMS to the recipient and generates a unique message ID to the AWS Lambda function. The Lambda function then adds this message ID to a DynamoDB table called “message-lookup”. This table is used for tracking different feedback requests sent during a multi-step conversation and associate them with the appointment ids. At this stage, the Lambda function also populates another table “feedbacks” which will hold the feedback responses that will be sent as SMS reply messages.

Each time a recipient replies to an SMS, Amazon Pinpoint publishes this reply event to an Amazon SNS topic which is subscribed by an Amazon SQS queue. Amazon Pinpoint will also add a messageId to this event which allows you to bind it to a sendMessages operation call.

A second AWS Lambda function polls these reply events from the Amazon SQS queue. It checks whether the reply is in the correct format (i.e. a number) and also associated with a previous request. If all conditions are met, the AWS Lambda function checks the ConversationStage attribute’s value from its message-lookup table. According to the current stage and the SMS answer received, AWS Lambda function will determine the next step.

For example, if the feedback score received is less than 5, a follow-up SMS is sent to the user asking if they’ll be happy to receive a call from the customer support team.

All SMS replies from the users are reflected to “feedbacks” table for further analysis.

Deploying the Sample Application

  1. Clone this GitHub repository to your local machine and install and configure AWS SAM with a test AWS IAM user.

You will use AWS SAM to deploy the remaining parts of this serverless architecture.

The AWS SAM template creates the following resources:

    • An Amazon DynamoDB table (appointments) that contains information about appointments, customers and their appointment status.
    • An Amazon DynamoDB table (feedbacks) that holds the received feedbacks from customers.
    • An Amazon DynamoDB table (message-lookup) that holds the Amazon Pinpoint message ids and associate them to appointments to track a multi-step conversation.
    • Two AWS Lambda functions (FeedbackSender and FeedbackReceiver)
    • An Amazon SNS topic that collects state change events from Amazon Pinpoint.
    • An Amazon SQS queue that queues the incoming messages.
    • An Amazon Pinpoint Application with an associated SMS channel.

This architecture consists of two Lambda functions, which are represented as two different apps in the AWS SAM template. These functions are named FeedbackSender and FeedbackReceiver. The FeedbackSender function listens the Amazon DynamoDB Stream associated with the appointments table and sends the SMS message requesting a feedback. Second Lambda function, FeedbackReceiver, polls the Amazon SQS queue and updates the feedbacks table in Amazon DynamoDB. (pinpoint-two-way-sms)

          Note: You’ll incur some costs by deploying this stack into your account.

  1. To start the SAM deployment, navigate to the root directory of the repository you downloaded and where the template.yaml AWS SAM template resides. AWS SAM also requires you to specify an Amazon Simple Storage Service (Amazon S3) bucket to hold the deployment artifacts. If you haven’t already created a bucket for this purpose, create one now. The bucket should have read and write access by an AWS Identity and Access Management (IAM) user.

At the command line, enter the following command to package the application:

sam package --template template.yaml --output-template-file output_template.yaml --s3-bucket BUCKET_NAME_HERE

In the preceding command, replace BUCKET_NAME_HERE with the name of the Amazon S3 bucket that should hold the deployment artifacts.

AWS SAM packages the application and copies it into this Amazon S3 bucket.

When the AWS SAM package command finishes running, enter the following command to deploy the package:

sam deploy --template-file output_template.yaml --stack-name BlogStackPinpoint --capabilities CAPABILITY_IAM

When you run this command, AWS SAM shows the progress of the deployment. When the deployment finishes, navigate to the Amazon Pinpoint console and choose the project named “BlogApplication”. This example uses “BlogStackPinpoint” as the stack name, you can change this to any other name you want.

  1. From the left navigation, choose Settings, SMS and voice. On the SMS and voice settings page, choose the Request phone number button under Number settings

Screenshot of request phone number screen

  1. Choose a target country. Set the Default message type as Transactional, and click on the Request long codes button to buy a long code.

Note: In United States, you can also request a Toll Free Number(TFN)

Screenshot showing long code additio

A long code will be added to the Number settings list.

  1. Choose the newly added number to reach the SMS Settings page and enable the option Enable two-way-SMS. At the Incoming messages destination, select Choose an existing SNS topic, and from the drop down select the Amazon SNS topic that was created by the BlogStackPinpoint stack.

Choose Save to save your SMS settings.

 

Testing the Sample Application

Now that the application is deployed and configured, test it by creating sample records in the Amazon DynamoDB table. Navigate to Amazon DynamoDB console and reach the tables view. Inspect the tables that were created by the AWS SAM application.

Here, appointments table is the table where the appointments and their statuses are kept. It tracks the appointment lifecycle events with items identified by unique ids. In this sample scenario, we are assuming that an appointment application creates a record with ‘CREATED’ status when a new appointment is planned. After the appointment is finished, same application updates the status to ‘COMPLETED’ which will trigger the feedback collection process. Feedback results are collected in the feedbacks table. Amazon Pinpoint message id’s, conversation stage and appointment id’s are kept in the message-lookup table.

  1. To start testing the end-to-end flow, choose the appointments table to open table overview page.
  2. Next, select the Items tab and choose the Create item From the dropdown, select Text. Add the following and choose Save to create your first appointment object. While adding the following object, replace CustomerPhone attribute’s value with a phone number you own. The feedback request messages will be delivered to that number. Note: This number should match the country number for the long code you provisioned.

{

"CustomerName": "Customer A",

"CustomerPhone": "+12345678900",

"AppointmentStatus":"CREATED",

"id": "1"

}

  1. To trigger sending the feedback SMS, you need to set an existing item’s status to “COMPLETED” To do this, select the item and click Edit from the Actions menu.

Replace the item’s current JSON with the following.

{

"AppointmentStatus": "COMPLETED",

"CustomerName": "Customer A",

"CustomerPhone": "+12345678900",

"id": "1"

}

  1. Before choosing the Save button, double check that you have set CustomerPhone attribute’s value to a valid phone number.

After the change, you should receive an SMS message asking for a feedback. Provide a numeric reply of that is less than five to this message. This will trigger a follow up question asking for a consent to receive an in-person callback.

 

During your SMS conversation with the application, inspect the feedbacks table. The feedback you have given over this two-way SMS channel should have been reflected into the table.

If you want to repeat the process, make sure to increment the AppointmentId field for any additional appointment records.

Cleanup

To clean up the resources you used in your account, simply navigate to AWS Cloudformation console and delete the stack named “BlogStackPinpoint”.

After the stack is deleted, you also need to delete the Long code from the Pinpoint Console by choosing the number and pressing Remove phone number button. You can also delete the Amazon S3 bucket you used for packaging and deploying the AWS SAM application.

Conclusion

This architecture shows how Amazon Pinpoint can be used to make two-way SMS communication with your customers. You can implement Two-way SMS functionality in other use cases such as appointment reminders, polls, Q&A services, and more.

To learn more about Pinpoint and it’s two-way SMS mechanism, you can visit the Pinpoint documentation.

 

Send SMS at scale to Indian recipients using Amazon Pinpoint

Post Syndicated from Meng Kang original https://aws.amazon.com/blogs/messaging-and-targeting/send-sms-at-scale-to-indian-recipients-using-amazon-pinpoint/

SMS has one of the highest open rates of all customer communications channels, and is popular with application builders for both transactional use cases like appointment reminders or asynchronous use cases like a SMS chatbot. Amazon Pinpoint supports SMS in over 200 countries and territories, but SMS sending requirements can vary by recipient destination. SMS sending requirements, depending on locale, can include restrictions on origination identity used, messages content, or the routes used to deliver to recipients. Amazon Pinpoint is making it easier for you to send application-to-person (A2P) SMS to Indian recipients using domestic routes. Amazon Pinpoint now supports submitting the Principal Entity ID (PEID) and Template ID using the Send Message API.

Introduction to new regulations and DLT platform

In 2018, the Telecom Regulatory Authority of India (TRAI) released The Telecom Commercial Communication Customer Preference Regulation (TCCCPR) to regulate text messaging in India. To implement this, TRAI adopted a block-chain technology called the Distributed Ledger Technology (DLT) platform. Every business entity who wants to send Application-to-Person (A2P) SMS to their end users in India using domestic routes will need to register their business and use case on the DLT platform. DLT registration is a concept that is unique to the SMS industry in India. If you don’t send text messages to recipients in India then DLT registration doesn’t apply to you. You will not be able to send A2P SMS to Indian recipients using domestic routes if you do not register on the DLT platform. Please note that if you are an international enterprise and you would like to send A2P SMS to recipients in India, you can leverage Amazon Pinpoint’s International Long Distance Operator (ILDO) routes. See more information here.

Changes in sending A2P SMS

DLT has brought many changes to the way SMS is sent in India. These include a new registration process, new message categorization, as well as restrictions around how to send messages. See below for an overview of this process.

Registration with TRAI: Prior to DLT, only service providers/telemarketers were required to register with TRAI. With the updated regulations, business owners/principal entity who wants to send SMS to their customers in India have to sign-up and complete the registration process on DLT platform.

Sender ID & Template Registration: Prior to DLT, bulk SMS service providers used to approve Sender IDs and templates. With the updated regulations, business owners/principal entity have to register Sender ID and Templates on the DLT platform and get those approved.

Customer Preference and Consent: Prior to DLT, customers were either on National Do Not Disturb Registry (DND) or not. The new regulation gave control to consumers/mobile subscribers by offering a time-window where they can manage their preferences based on a specific time or day and allow receipt of certain kinds of promotional messages. It means that customers can choose to receive promotional text from a company even if they have activated DND.

Types of Message: With the new regulation, the DLT-defined domestic routes are as follows.

  • Promotional SMS: these include offers, discounts to non-opt in users, and SMS delivered to Non-DND numbers where the customer has not explicitly opted in. These can only be delivered between 10.00AM to 21:00PM IST. Operators have also stopped supporting delivery notification and receipts for promotional SMS. Promotional messages are sent using 6-digit numeric Sender IDs.
  • Transactional SMS: this is reserved for financial organizations to send One-Time-Passwords (OTPs). Transactional messages use 6-digit alpha Sender IDs.
  • Service Implicit: these include service-related informative messages other than OTPs. Amazon Pinpoint classifies these under the “transactional” route type, so customers will continue to select the “transactional” route type to send these messages.
  • Service Explicit: these include promotional messages customers have opted to receive from a particular business. Amazon Pinpoint classifies these under the “transactional” route type, so customers will continue to select the “transactional” route type to send these messages.

Validation (Scrubbing) Functionality: With DLT, customers’ mobile numbers are filtered in real-time to match the desired criteria set by the customer. This means that if a customer gave consent to receive SMS on Monday, but withdraws the permission on Wednesday, then the SMS will not be delivered on Thursday. Customer preferences are updated in real-time and the results are immediately available on DLT. When sending, the DLT platform will validate the PEID customers used to send against the registered Sender ID, and validate the SMS Template against the registered Brand name, so that only approved businesses using approved SMS Sender are able to send SMS to the end recipient.

DLT registration timeline

TRAI is the entity enforcing DLT implementation in India. TRAI has designated these changes to take place over several phases and months, with each phase increasing the level of restriction on message sending:

Phase 1 (Initiated June 2020): The first phase is to complete Principal Entity ID (PEID) & the SMS Header or Sender ID registration on the DLT platform. Only registered SMS Header or Sender ID can be used to send SMS to India using domestic routes.

Phase 2 (Initiated Nov 2020): The second phase is to register Template ID on the DLT platform. For each SMS send, the PEID and Template ID are validated against the registered Sender ID.

Phase 3 (Initiated April 2021): The third phase is to validate that the content of the message template matches exactly what was registered on the DLT platform. Brand Name is a mandatory field to be included in content for template registration.

Using Pinpoint to send SMS to India

To send SMS messages to India from Amazon Pinpoint, you’ll first need to complete DLT registration. To do so, follow the steps described on the Special requirements for sending SMS messages to recipients in India documentation.

Next, to make sure your SMS messages are delivered successfully using local routes, you need to do the following when using Amazon Pinpoint sending the SMS message.

  • Use a Sender ID which has been registered on the DLT platform that matches your message content.
  • In the Pinpoint Send Message API, provide values for the following parameters:
    • EntityId – The entity ID or Principal Entity (PE) ID received from the regulatory body for sending SMS in your country.
    • TemplateId – The template ID received from the regulatory body for sending SMS in your country.
  • Choose one of the following route types:
    • Promotional – Choose this type for promotional messages, which use a numeric sender ID.
    • Transactional – Choose this type for transactional messages, which use a case-sensitive alphanumeric sender ID.

When adding the content to your message, thoroughly review your content to ensure that it exactly matches the content in the DLT registered template. If you include additional character returns, spaces, punctuation, or mismatched sentence case, carriers will block your SMS. Variables in a template – for example, {#var#} – cannot exceed 30 characters for each variable. The following are some common use cases for message rejection:

No template was found that matched the content sent.
Content sent: <#> 12345 is your OTP to verify mobile number. Your OTP is valid for 15 minutes — ABC Pvt. Ltd.
Matched template: None
Issue: There are no DLT templates that include <#> or {#var#} at the beginning of the DLT registered template.

The value of a variable exceeds 30 characters.
Content sent: 12345 is your OTP code for ABC (ABC Company – India Private Limited) – (ABC 123456789). Share with your agent only. – ABC Pvt. Ltd.
Matched template: {#var#} is your OTP code for {#var#} ({#var#}) – ({#var#} {#var#}). Share with your agent only. – ABC Pvt. Ltd.
Issue: The value of “ABC Company – India Private Limited” in the content sent exceeds a single {#var#} character limit of 30.

The message sentence case does not match the sentence case in the template.
Content sent: 12345 is your OTP code for ABC (ABC Company – India Private Limited) – (ABC 123456789). Share with your agent only. – ABC Pvt. Ltd.
Matched template: {#var#} is your OTP code for {#var#} ({#var#}) – ({#var#} {#var#}). Share with your agent only. – ABC PVT. LTD.
Issue: The company name appended to the DLT matched template is capitalized while the content sent has changed parts of the name to lowercase — “ABC Pvt. Ltd.” vs. “ABC PVT. LTD.”

Start using Amazon Pinpoint to send SMS messages to Indian recipients by following the steps described on the Special requirements for sending SMS messages to recipients in India documentation.

Maintain consistency in emails with custom content using Amazon SES templates

Post Syndicated from Seth Theeke original https://aws.amazon.com/blogs/messaging-and-targeting/maintain-consistency-in-emails-with-custom-content-with-amazon-ses-templates/

When sending emails, content creators often want to add custom content such as images or videos while maintaining consistency in their messages. They also want to send those emails automatically once new content is ready. In this blog, we will show you how to create templates for emails with a common theme by combining Amazon Simple Email Service (Amazon SES) templates, AWS Lambda, Amazon Simple Storage Service (Amazon S3), and Amazon SES templates.

Promotional content (such as logos, images, videos, and more) can be stored, managed, and hosted in Amazon S3. You can then embed this content into promotional emails without making any changes to email templates or email processing. You can trigger a Lambda function to send promotional emails with the newly added content through using the Amazon SES SDK.

This post shows readers how to:

  • Create an Amazon SES email template with tags to be replaced by image URLs
  • Upload those templates to Amazon SES
  • Setup an AWS CloudFormation stack using the AWS Cloud Development Kit(AWS CDK)
  • Create a Lambda function and Amazon S3 bucket to send emails using the AWS SDK for Javascript

Solution Architecture

The proceeding image shows the architecture you will build as part of this post. You will use the AWS CDK to provision an Amazon S3 bucket, Lambda, and AWS Identity and Access Management(IAM) permissions. You will also use the AWS Command Line Interface(AWS CLI) to upload and manage our Amazon SES templates.

The architecture allows you to upload content to S3 which will trigger a Lambda function. That Lambda will form an Amazon SES request using the template you have uploaded and embed the S3 content as a parameter which will be sent to the user and render in their email client.

Metadata

Time to Read: ~ 20 minutes

Time to Complete: ~ 15 minutes

Cost to Complete: Free Tier

Learning Level: Intermediate

Services Used: Amazon Simple Email Service, Amazon Simple Storage Service, AWS Lambda

Prerequisites

For this walkthrough, you should have the following prerequisites:

Solution Overview

You will walk through creating Amazon SES templates and then a CloudFormation stack using the AWS CDK. You will then create a template file, a lambda directory, and a CDK application directory which should all be in the same level in your package structure in order to follow these steps explicitly.

Step 1: Create an Amazon SES template in JSON with a tag representing your image URL and upload via the CLI

Step 2: Initialize an AWS CDK package using the AWS CDK CLI and add necessary dependencies

Step 3: Initialize a NodeJS AWS Lambda package

Step 4: Provision an Amazon S3 bucket and Lambda in your AWS CDK app

Step 5: Configure your Lambda to be triggered when objects are added to S3

Step 6: Configure your Lambda’s IAM role to allow sending emails via Amazon SES

Step 7: Write necessary code for AWS Lambda to send emails via Amazon SES

Step 8: Deploy and Test!

Step 1: Create an Amazon SES Email template

Amazon SES Email templates are defined as a JSON object containing:

  • TemplateName – name of the template, must be unique across email templates and will be used by our Lambda to pass to Amazon SES
  • SubjectPart – represents the subject of the email
  • HtmlPart – represents the body of the email
  • TextPart – when email clients cannot render HTML, this is displayed instead of HtmlPart

More detailed information about email templates can be found in the Amazon SES Developer Guide.

1.     Open your text editor and save the empty file as email-template.json

2.     Paste the following into your json file and save your changes

{
  "Template": {
    "TemplateName": "MyTemplate",
    "SubjectPart": "Greetings Customer",
    "HtmlPart": "<img src={{imageURL}} alt=\"logo\" width=\"100\" height=\"100\">",
    "TextPart": "Dear Customer,\r\nCheck out our website for new promotional content."
  }
}

This template has a single tag called imageURL which will be replaced during execution with our content’s S3 URL.

3.     Run the following AWS CLI command to upload your template to Amazon SES

aws ses create-template --cli-input-json file://email-template.json

4.     Once your template has been uploaded, you can confirm its creation by logging into the AWS Console, navigating to Amazon SES, and then selecting Email Templates

Step 2: Initialize an AWS CDK package and add necessary dependencies

In this section, you will be using the AWS CDK CLI to initialize a new code package and add the dependencies for Lambda and Amazon S3.

1.     Create a new directory at the same level as email-template.json called email-infrastructure

2.     Navigate to the promotional-email-infrastructure directory and run the following CDK CLI command to generate the skeleton for your cdk application

cdk init app --language=typescript

3.     Add dependencies for Amazon S3, Lambda, and IAM by adding the following lines to your dependencies section of your package.json and then run npm install

"@aws-cdk/aws-lambda": "1.86.0"
"@aws-cdk/aws-lambda-event-sources": "1.86.0"
"@aws-cdk/aws-s3": "1.86.0"
"@aws-cdk/aws-iam": "1.86.0"

Make sure to install the version of these dependencies that matches the version of the aws-cdk stack so you don’t run into compatibility issues.

Step 3: Create a NodeJS Lambda package

In this step, you will create the barebones for our Lambda function that will call Amazon SES and revisit in step 7 to implement the handler

1.     Create a new directory at the same level as email-template.json called email-lambda

2.     Add a package.json file in the email-lambda directory that looks like the following

{
    "name": "email-lambda",
    "version": "1.0.0",
    "main": "index.js",
    "dependencies": {
        "aws-sdk": "2.831.0"
    }
}

3.     Add a file called index.js, this will be our Lambda handler and will look like the following for now. Make sure to insert your verified email address into the testAddress variable, this will be used later as both your to and from address for testing.

var AWS = require("aws-sdk");
var ses = new AWS.SES({apiVersion: "2010-12-01"});
var testAddress = "INSERT_VERIFIED_EMAIL_HERE";

exports.handler = async function(event) { 
    console.log(JSON.stringify(event));
    return "200";
}

4.     Finish this step by running npm install in the email-lambda directory to install the aws-sdk dependencies you will use in a later step. Your top-level directory structure should look like the following:

  • email-template.json – contains your email template from step 1
  • email-infrastructure – contains your CDK stack from step 2
  • email-lambda – contains your email lambda function code from step 3

Step 4: Provision an Amazon S3 bucket and Lambda function in your CDK app

In this step, you will add an Amazon S3 bucket and a NodeJS Lambda function into our CDK application based on what you setup in previous steps. After this, you will connect all the pieces together.

1.     Import all the services you need into our CDK stack construct. The imports you will need are listed below, copy them into your editor in the imports section.

import * as s3 from '@aws-cdk/aws-s3';
import * as lambda from '@aws-cdk/aws-lambda';
import * as lambdaEventSource from '@aws-cdk/aws-lambda-event-sources';
import * as iam from '@aws-cdk/aws-iam';
import path = require('path');

2.     Add S3 bucket to CDK App by adding an instance of the Bucket construct

const promotionalContentBucket = new s3.Bucket(this, "DOC-EXAMPLE-BUCKET");

3.     Similarly, add your Lambda function by creating an instance of the Function construct referencing your Lambda function package by path

const emailLambda = new lambda.Function(this, "EmailLambda", {
    code: lambda.Code.fromAsset(path.join(__dirname, "../../email-lambda")),
    handler: "index.handler",
    runtime: lambda.Runtime.NODEJS_12_X
});

4.     Execute npm run build in your CDK directory to ensure you’ve setup the package correctly. You can get additional help from the Troubleshooting Guide for CDK

Step 5: Configure Lambda with an S3 Event Source

In this step, you will configure your Lambda function to be triggered when objects are added to your Amazon S3 bucket by using the Lambda event sources module for CDK.

1.     Create an instance of the S3EventSource construct in your stack for OBJECT_CREATED events only because you don’t want to trigger a lambda invocation when an object is removed for this post

const s3EventSource = new lambdaEventSource.S3EventSource(promotionalContentBucket, {
    events: [s3.EventType.OBJECT_CREATED]
});

2.     Now that you have an event source defined, you need to add the event source to your Lambda function

emailLambda.addEventSource(s3EventSource);

3.     Add the Amazon S3 bucket’s domain name as an environment variable so you can reference objects by URL by adding the environment property to your Lambda function like below.

environment: {
    "BUCKET_DOMAIN_NAME": promotionalContentBucket.bucketDomainName
}

Step 6: Configure the email Lambda IAM role

In this step, you will add an IAM policy statement to your email Lambda’s execution role so it can call Amazon SES.

1.     Add an IAM PolicyStatement construct with effect ALLOW on all resources with SendTemplatedEmail action

const emailPolicyStatement = new iam.PolicyStatement({
    effect: iam.Effect.ALLOW,
    actions: ["ses:SendTemplatedEmail"],
    resources: ["*"]
});

2.     Finish this step by adding your newly created policy to the Lambda execution role

emailLambda.addToRolePolicy(emailPolicyStatement)

Step 7: Add Lambda implementation to send templated emails

You will use the AWS NodeJS Amazon SES SDK to send emails using the SendTemplatedEmail API. Our implementation will assume a batch of size 1 for each Lambda invocation for simplicity.

1.     Replace your function handler in the email Lambda function with the code below. This will read the S3Event’s first record, prepare parameters for the Amazon SES SDK call and invoke the sendTemplatedEmail function with the imageURL embedded into your previously created template.

exports.handler =  async function(event) {  
    console.log(JSON.stringify(event));
    let s3Object = event.Records[0];
    let sendEmailParams = {
        Destination: {
            ToAddresses: [testAddress]
        },
        Template: 'MyTemplate',
        TemplateData: JSON.stringify({
            "imageURL": process.env.BUCKET_DOMAIN_NAME + "/" + s3Object.s3.object.key,
        }),
        Source: testAddress
    };
    let response = await ses.sendTemplatedEmail(sendEmailParams).promise();
    return response;
}

2.     Deploy your stack with the CDK CLI by running cdk deploy, this may take a couple minutes. If you run into problems, see the Troubleshooting Guide for CDK.

Step 8: Test your System

At this point, you should have an Amazon SES template uploaded to your account as well as a CloudFormation stack that contains an Amazon S3 bucket and a Lambda function that is triggered when objects are added to that bucket and had permissions to invoke Amazon SES APIs. Now you will test the system by adding an image into our Amazon S3 bucket.

1.     Log in to the AWS Console

2.     Navigate to Amazon S3

3.     Select your promotional content bucket from the list of buckets

4.     Click Upload on the right-hand side of the screen

5.     Add an image from your computer by clicking Add files

6.     Scroll down to the bottom and expand Additional Upload Options

7.     Scroll down to Access Control List

8.     Select the check boxes for Read for Everyone(public access) so the images are accessible when the user opens their email

9.     Scroll down to the bottom and select Upload

10.     Done! You should have an email in your inbox shortly that renders the image you just uploaded. Check the Lambda logs and errors in case you don’t see your email.

Cleaning up

To avoid incurring future charges, delete your CloudFormation stack by running cdk destroy or manually through the AWS Console. Keep in mind, by default Amazon S3 buckets won’t be deleted so you will need to navigate to Amazon S3 in the AWS Console, clear the bucket of any objects and then manually delete the resource.

Conclusion

Congratulations! You now have an understanding of how to combine Amazon SES templates with Amazon S3 and Lambda to inject custom images into emails without the need for any servers and have launched this stack using the AWS Cloud Development Kit.

Author Bio

My name is Seth Theeke, I work as a Software Development Engineer in Amazon Freight. I’ve been working with AWS since 2016 and hold a Developer Associate Certification. I love soccer and I love software engineering, the simplest things in life!

Solving abandoned cart scenarios using Amazon Pinpoint event-triggered journeys

Post Syndicated from Ryan Lowe original https://aws.amazon.com/blogs/messaging-and-targeting/solving-abandon-cart-scenarios-using-amazon-pinpoints-event-triggered-journeys/

In this post, we will walk through building an abandoned shopping cart user journey in Amazon Pinpoint. Journeys are multi-step user engagements with channels sends (SMS, email or push) based on conditional logic with a goal to drive a high value action. This journey will enable customers to identify users who added a product to their shopping cart but have not purchased, and follow up with them via email to encourage them to complete the transaction. Though the example will be specific to this use case, the steps used can be adapted to fit similar user journeys.

Abandoned Shopping Cart Journey

The Baynard Institute states the average cart abandonment rate is 69.57%, which means over half of users add a product to cart but do not check out. Any improvement in this metric has a direct impact on revenue, and is easily measurable. This makes it a no brainer for marketers to support through campaigns. Previously, customers did not have a way to immediately react to a cart abandonment or other important events within a journey. This meant customers would need to create a segment of users who abandoned their cart and do a daily send. By this time, the user might have purchased an item somewhere else, or lost interest in the product or service.

Solution Overview

The solution that we will build relies on Amazon Pinpoint’s events API to report two application events: AddToCartEvent, CartPurchasedEvent. These events should be reported to Amazon Pinpoint from your electronic shopping cart system. The integration with the online shopping cart system is out of scope for this article. Please refer to the Amazon Pinpoint developer guide for more information.

Architecture Diagram

When a user of your e-commerce shopping cart adds items to their shopping cart, you can report the AddToCartEvent to Amazon Pinpoint. At a later time, when the user completes their purchase, you can report the CartPurchasedEvent. If the CartPurchasedEvent does not get reported to Amazon Pinpoint within an hour of receiving the AddToCartEvent, then you can trigger our abandon shopping cart email to encourage the user to return and complete their purchase.

Using these events, you are able to use Amazon Pinpoint’s journey feature to orchestrate our user experience. You will use the first event, AddToCartEvent, to trigger your journey. After an hour wait, you will then use the second event, CartPurchasedEvent, to filter out users who have completed the purchase. The remaining users will receive your abandon shopping cart email message urging them to return to their cart and complete their order.

Step 1: Create Add to Cart and Purchase custom events

The first step in setting up this solution is to create and report the two custom events. There are multiple ways to report events in your application. For demonstration purposes, we have included two example event calls in the proceeding code chunk using the AWS SDK for Python (Boto3) from inside an AWS Lambda Function.

It is important to note that the Amazon Pinpoint events API can also be used to update endpoints at the same time that the event gets registered. In the proceeding example, the first API call will update the endpoint’s attribute Cart with the contents of the shopping cart. In the second example, the API call update the endpoint’s attribute Purchased with the flag Yes.

Sample Event: Item Hat was added to cart with a price of $29.95

import boto3

client = boto3.client('pinpoint')
app_id = '[PINPOINT_PROJECT_ID]'
endpoint_id = '[ENDPOINT_ID]'
address = '[EMAIL_ADDRESS]'

def lambda_handler(event, context):
    
    client.put_events(
        ApplicationId = applicationId,
        EventsRequest={
            'BatchItem': {
                endpoint_id: {
                    'Endpoint': {
                        'ChannelType': 'EMAIL',
                        'Address': address,
                        'Attributes': {
                            'Cart': ['Hat'],
                            'Purchased': ['No']
                        }
                    },
                    'Events':{
                        'cart-event-2': {
                            'Attributes':{
                                'AddedToCart': 'Hat'
                            },
                            'EventType': 'AddToCartEvent',
                            'Metrics': 29.95,
                            'Timestamp': datetime.datetime.fromtimestamp(time.time()).isoformat()
                        }
                    }
                }
            } 
        }
    )

Sample Event: Cart purchased

import boto3

client = boto3.client('pinpoint')
app_id = '[PINPOINT_PROJECT_ID]'
endpoint_id = '[ENDPOINT_ID]'
address = '[EMAIL_ADDRESS]'

def lambda_handler(event, context):
    
    client.put_events(
        ApplicationId = applicationId,
        EventsRequest={
            'BatchItem': {
                endpoint_id: {
                    'Endpoint': {
                        'ChannelType': 'EMAIL',
                        'Address': address,
                        'Attributes': {
                            'Cart': ['Hat'],
                            'Purchased': ['Yes']
                        }
                    },
                    'Events':{
                        'cart-event-2': {
                            'Attributes':{
                                'Purchased': 'Yes'
                            },
                            'EventType': 'CartPurchasedEvent',
                            'Timestamp': datetime.datetime.fromtimestamp(time.time()).isoformat()
                        }
                    }
                }
            } 
        }
    )

Note: Both events above must be reported to Amazon Pinpoint in order to complete the remaining steps in this post.

Step 2: Create a “Made a Purchase” Dynamic Segment

The second step in our solution is to create a dynamic segment to filter out users who have made a purchase. To do this, you will look for users with the endpoint attribute of Purchased to be the value Yes.

  1. Navigate to your project in the Amazon Pinpoint Console, then Segments.
  2. Choose Create a segment.
  3. Select Build a segment.
  4. Provide the name Made a Purchase into the name field.
  5. Configure Segment Group 1 to add segment filters.
    1. Under the Add filters to refine your segment choose Filter by endpoint.
    2. For the Choose an endpoint attribute dropdown choose Purchased
    3. Ensure Is is chosen in the middle dropdown.
    4. In the Choose values dropdown choose Yes.
  6. Click Create Segment to create your first dynamic segment. Note, a pop-up will appear highlighting that this segment targets multiple endpoint channels. Select I Understand.

Step 3: Create our Abandon Cart Journey

The last step is to design out the journey itself.

  1. Navigate to your project in the Amazon Pinpoint Console, then Journeys.
  2. Choose Create journey to create a new journey.
  3. Give the Journey the name Abandon Cart by replacing the Untitled text.
  4. Define a Journey entry criteria
    1. Choose Set entry condition to expand the Journey entry activity.
    2. Choose Add participants when they perform an activity and choose AddToCartEvent in the Events field.
    3. Choose Save
  5. Create a branch to target users who did not make a purchase
    1. Choose Add activity directly under the Journey entry activity
    2. Under Choose a journey activity choose Yes/no split.
    3. Under Select a condition type choose Segment.
    4. Under Segments choose the Made a Purchase dynamic segment created earlier.
    5. Under Condition evaluation choose Evaluate after and then choose 1 hours.
    6. Choose Save
  6. Add an email activity to send our abandon cart message
    1. Choose Add activity directly under the No split path.
    2. Under Choose a journey activity choose Send email.
    3. Choose Choose an email template and select your messaging template and choose Choose template.
    4. Choose Save.

At this point, your journey should look like the screenshot below. You can now choose Review to walkthrough steps to publish your journey.

Next Steps

You can continue to iterate on this experience using the journeys tool to create a custom user-experience for your users without any code changes.

  • Filter the journey entry event to only high dollar cart items by adding Event metrics filters in the Journey entry criteria.
  • Test out different channels by sending message to users over SMS instead of email.
  • Add additional splits to send messages on users’ preferred channels.
  • Add a second wait of 24 hours and send a final reminder with a 10% off coupon code.
  • Add random splits to do A/B testing of different messages and channels.

Cleanup

To stop and remove the journey in order to not incur further charges, please follow the steps below.

  1. Navigate to your project in the Amazon Pinpoint Console, then Journeys.
  2. Select the Abandon Cart journey.
  3. Choose Stop journey then choose Stop journey again in the Stop journey confirmation.
  4. To fully delete the journey choose Delete from the Actions menu.

Conclusion

Cart abandonment is a major issue that has a direct impact on revenue. This solution allows customers to recognize a user has abandoned a critical flow and allows a marketer to re-engage them through a messaging channel before it is too late. Different components of the user journey can also be A/B tested and targeted with different user segments to drive the highest return from different user cohorts. Once set up, the journey can be always-on and independently drive incremental revenue for a business.

Log into the Amazon Pinpoint Console to get started creating your own abandon shopping cart journey.

Send SMS messages at scale using 10DLC and Amazon Pinpoint

Post Syndicated from Brent Meyer original https://aws.amazon.com/blogs/messaging-and-targeting/send-sms-messages-at-scale-using-10dlc-and-amazon-pinpoint/

This week, we’re adding support for 10DLC phone numbers to Amazon Pinpoint. You can use 10DLC phone numbers to send SMS text messages at scale quickly and affordably.

What is 10DLC?

The abbreviation 10DLC stands for Ten-Digit Long Code. 10DLC phone numbers are intended specifically for sending Application-to-Person (A2P) messages—that is, messages that are sent from applications like Amazon Pinpoint to individual recipients. 10DLC is a concept that’s unique to the SMS industry in the United States. If you don’t send text messages to recipients in the US, then 10DLC doesn’t apply to you.

Before the launch of 10DLC, you could purchase unregistered US long codes instantly through the Amazon Pinpoint console. These long codes didn’t require a registration process—anyone could purchase them for $1 per month. However, the mobile carriers never intended for senders to use them to send A2P messages. For these reasons, their capabilities were limited. To prevent bad actors from sending spam and other malicious content, unregistered long codes could only send one message per second, and about 100 messages in a 24-hour period. Carriers applied heavy filtering to these phone numbers and blocked them for sending high volumes of messages, or as a penalty for sending unsolicited messages.

The alternative to using unregistered long codes is to use a short code. Short codes are a premium SMS product. They offer high rates of deliverability and high throughput (starting at 100 messages per second and going up to thousands of messages per second). The mobile carriers apply a rigorous approval process to short code applications. This process takes several weeks to complete. Short codes cost $995 per month, plus a one-time setup fee of $650. We continue to offer and support short codes in Amazon Pinpoint. Short codes are the right solution for many of our customers, and will continue to be part of the US SMS landscape well into the future.

For many customers though, the ideal solution is somewhere in the middle. 10DLC was designed to cover that middle ground. With 10DLC, senders are required to register both their company and their campaign. This registration information is added to The Campaign Registry (TCR), an industry-wide database of companies and use cases that are authorized to send messages using 10DLC phone numbers. Some use cases, such as one-time passwords and other authentication systems, can be approved within a week. Other use cases, such as promotional messaging, are subject to additional scrutiny, but can still be approved in a few weeks. While 10DLC phone numbers don’t offer the high throughput rates that short codes do, they can exceed the one message per second limit of unregistered long codes while offering higher deliverability rates. And importantly for many customers, they don’t come with the price tag associated with short codes. You pay a one-time fee of $4 to register your company, and a $10 monthly fee for each 10DLC campaign that you register. You also pay a $1 monthly charge for each 10DLC long code that you lease.

Note: On March 1, 2021, T-Mobile will begin to charge a one-time, $50 fee for registering your company. This fee will be charged in addition to the $4 company registration fee. No other carriers have announced similar fees.

The following table compares the costs associated with obtaining and using a short code against the costs of obtaining and using a 10DLC phone number. This table assumes that you only register one 10DLC company and campaign. It also assumes that you only use a single long code with your 10DLC campaign.

Short code 10DLC
One-time fees $650 $54 ($4 company registration + $50 T-Mobile registration fee)
Monthly fees $995 $11 ($1 phone number lease + $10 campaign registration fee)

Senders with very low throughput and volume requirements can register a “low-volume” 10DLC campaign for $2 per month, as opposed to the standard campaign fee of $10 per month. This option is a good choice for test and proof-of-concept use cases.

Drawbacks of using 10DLC phone numbers

For users of Amazon Pinpoint, 10DLC phone numbers offer several benefits. However, they also come with a few drawbacks. One drawback is the different ways that the US carriers support 10DLC. As I mentioned earlier, when you apply for a 10DLC phone number, you have to provide information about your company or brand, and information about your specific messaging use case. The carriers use this information to calculate a trust score. They then use this trust score to determine the capabilities of your 10DLC phone number. On T-Mobile and Sprint, your trust score determines the maximum number of messages that you can send each day through your 10DLC phone number. But for AT&T, your trust score determines the number of messages that you can send each minute, with no limit on the daily number of messages that you can send. (As of this writing, Verizon hasn’t announced their throughput plan.) These differences mean that you must carefully manage your messaging program to stay within the daily and per-second limits imposed by the different carriers.

A final drawback to using 10DLC phone numbers is related to throughput. If your use case requires you to send a large number of text messages in a short amount of time (100 messages per second or more), you need a short code.

10DLC Capabilities

10DLC phone numbers typically have higher per-second and daily sending limits than unregistered long codes. The actual performance of your 10DLC phone number is based on the trust score for the company that you registered. The following table shows the trust score tiers and their associated limits.

Tier Message parts per minute (AT&T) Maximum daily messages (T-Mobile & Sprint)
High 1,800 200,000
Medium-High 300 40,000
Medium-Low 30 10,000
Basic 12 2,000

Setting up 10DLC

To set up 10DLC, you have to do three things. First, you must register your company. Second, you must register your use case. And third, you must add a phone number to your 10DLC campaign.

Important: When you complete the steps in this section, you are charged for registering both your company and your use case. These registration charges can’t be reversed. Only complete these steps if you agree to pay these charges.

Step 1: Register your company

When you register your company, you provide your company details to The Campaign Registry (TCR). The mobile carriers use this data to determine the trustworthiness of your use cases. Company approvals are usually granted instantly.

To register your company:

  1. Sign in to the Amazon Pinpoint console at https://console.aws.amazon.com/pinpoint.
  2. In the navigation pane, under Settings, choose SMS and voice.
  3. On the 10DLC campaigns tab, choose Register company, as shown in the following image.
    Shows the location of the Create 10DLC Company button on the SMS and voice settings page of the Amazon Pinpoint console.
  4. On the Register your company page, fill out the form completely. There are a few things to note in this process:
    • The Doing business as (DBA) or brand name field is mandatory. The value that you provide can be the same as your company name.
    • The Support email and Support phone number are the email address and phone number that your customers can use to contact you when they have questions.
  5. When you finish, choose Create.

Step 2: Register a 10DLC campaign

After you register a company, you can begin to register campaigns. In 10DLC terms, a campaign is a use case or set of closely related use cases. Amazon Pinpoint also sends this information to TCR. Carriers use this information to determine whether traffic that they see from a certain phone number is legitimate. Campaigns associated with common, low-risk use cases can typically be approved in about a week.

To register a 10DLC campaign:

  1. On the SMS and voice settings page, on the 10DLC campaigns tab, choose Create 10DLC Campaign, as shown in the following image.
    Shows the location of the Create 10DLC Campaign button on the SMS and voice settings page of the Amazon Pinpoint console.
  2. On the Create 10DLC Campaign page, do the following:
    1. For Company name, choose the company that you registered in the preceding section.
    2. For 10DLC campaign name, enter a name that describes your messaging use case, such as “Example Corp One-Time Passwords.”
    3. For Vertical, choose the category that most accurately describes your company and use case. For example, if you develop software for the healthcare industry, choose Healthcare.
    4. For Help message, enter the response that will be returned to recipients who reply to your messages with the keyword HELP. A good help message describes the purpose of the campaign. It also provides your customers with a method of contacting you for more help (typically an email address or phone number).
    5. For Stop message, enter the response that will be returned to recipients who reply to your messages with the keyword STOP. A typical stop message tells your customer what type of messages they’re unsubscribing from, and lets them know that you won’t send them any more messages.
    6. Under Campaign use case, choose the use case that most accurately describes how you plan to use the 10DLC phone number. Many common use cases—including two-factor authentication (2FA), marketing, security and fraud alerts, and public service announcements—are considered Standard use cases. Use cases that involve a greater degree of risk for carriers—such as political, sweepstakes, and emergency notifications—are considered Special use cases.
  3. When you finish, choose Create.

Step 3: Associate phone numbers with your 10DLC campaign

After your 10DLC company and campaign are approved, you can purchase new long codes. When you purchase a long code, you choose which 10DLC campaign to associate it with.

To purchase a long code:

  1. On the SMS and voice settings page, on the Phone numbers tab, choose Request long code/toll-free.
  2. On the Define your phone numbers page, in the Phone number 1 section, do the following:
    1. For Country, choose United States.
    2. For Number type, choose 10DLC.
    3. For Assign to existing 10DLC campaign, choose the 10DLC campaign that you created in the preceding section.
    4. For Default message type, choose the option that most accurately describes your use case.
    5. In the Summary section, for Quantity, specify how many phone numbers you want to purchase.
  3. Choose Next. Then, on the Review and request page, choose Request.

Cleanup

If you no longer need the long codes that are associated with your 10DLC campaign registration, you can delete them. If you delete a long code, you’re no longer charged the $1 monthly lease charge. However, you’re still charged the recurring 10DLC campaign registration fee, unless you delete your 10DLC campaign as well.

If you want to delete the 10DLC company or campaign registration information in Amazon Pinpoint, you can do so by opening a case in the AWS Support Center. The SMS and voice settings page in the Amazon Pinpoint console contains links that you can use to quickly open these cases.

Conclusion

If you need to start sending SMS messages to your customers quickly, and without the expense of a short code, 10DLC is a great option. With common use cases such as two-factor authentication, your 10DLC campaigns and phone numbers can be ready to use relatively quickly. Messages that you send using 10DLC will have the high deliverability rates that were previously reserved only for short codes.

Opt-in to the new Amazon SES console experience

Post Syndicated from Simon Poile original https://aws.amazon.com/blogs/messaging-and-targeting/amazon-ses-console-opt-in/

Amazon Web Services (AWS) is pleased to announce the launch of the newly redesigned Amazon Simple Email Service (SES) console. With its streamlined look and feel, the new console makes it even easier for customers to leverage the speed, reliability, and flexibility that Amazon SES has to offer. Customers can access the new console experience via an opt-in link on the classic console.

Amazon SES now offers a new, optimized console to provide customers with a simpler, more intuitive way to create and manage their resources, collect sending activity data, and monitor reputation health. It also has a more robust set of configuration options and new features and functionality not previously available in the classic console.

Here are a few of the improvements customers can find in the new Amazon SES console:

Verified identities

Streamlines how customers manage their sender identities in Amazon SES. This is done by replacing the classic console’s identity management section with verified identities. Verified identities are a centralized place in which customers can view, create, and configure both domain and email address identities on one page. Other notable improvements include:

  • DKIM-based verification
    DKIM-based domain verification replaces the previous verification method which was based on TXT records. DomainKeys Identified Mail (DKIM) is an email authentication mechanism that receiving mail servers use to validate email. This new verification method offers customers the added benefit of enhancing their deliverability with DKIM-compliant email providers, and helping them achieve compliance with DMARC (Domain-based Message Authentication, Reporting and Conformance).
  • Amazon SES mailbox simulator
    The new mailbox simulator makes it significantly easier for customers to test how their applications handle different email sending scenarios. From a dropdown, customers select which scenario they’d like to simulate. Scenario options include bounces, complaints, and automatic out-of-office responses. The mailbox simulator provides customers with a safe environment in which to test their email sending capabilities.

Configuration sets

The new console makes it easier for customers to experience the benefits of using configuration sets. Configuration sets enable customers to capture and publish event data for specific segments of their email sending program. It also isolates IP reputation by segment by assigning dedicated IP pools. With a wider range of configuration options, such as reputation tracking and custom suppression options, customers get even more out of this powerful feature.

  • Default configuration set
    One important feature to highlight is the introduction of the default configuration set. By assigning a default configuration set to an identity, customers ensure that the assigned configuration set is always applied to messages sent from that identity at the time of sending. This enables customers to associate a dedicated IP pool or set up event publishing for an identity without having to modify their email headers.

Account dashboard

There is also an account dashboard for the new SES console. This feature provides customers with fast access to key information about their account, including sending limits and restrictions, and overall account health. A visual representation of the customer’s daily email usage helps them ensure that they aren’t approaching their sending limits. Additionally, customers who use the Amazon SES SMTP interface to send emails can visit the account dashboard to obtain or update their SMTP credentials.

Reputation metrics

The new reputation metrics page provides customers with high-level insight into historic bounce and complaint rates. This is viewed at both the account level and the configuration set level. Bounce and complaint rates are two important metrics that Amazon SES considers when assessing a customer’s sender reputation, as well as the overall health of their account.

The redesigned Amazon SES console, with its easy-to-use workflows, will not only enhance the customers’ on-boarding experience, it will also change the paradigms used for their on-going usage. The Amazon SES team remains committed to investing on behalf of our customers and empowering them to be productive anywhere, anytime. We invite you to opt in to the new Amazon SES console experience and let us know what you think.

Updating opt-in status for Amazon Pinpoint channels

Post Syndicated from Varinder Dhanota original https://aws.amazon.com/blogs/messaging-and-targeting/updating-opt-in-status-for-amazon-pinpoint-channels/

In many real-world scenarios, customers are using home-grown or 3rd party systems to manage their campaign related information. This includes user preferences, segmentation, targeting, interactions, and more. To create customer-centric engagement experiences with such existing systems, migrating or integrating into Amazon Pinpoint is needed. Luckily, many AWS services and mechanisms can help to streamline this integration in a resilient and cost-effective way.

In this blog post, we demonstrate a sample solution that captures changes from an on-premises application’s database by utilizing AWS Integration and Transfer Services and updates Amazon Pinpoint in real-time.

If you are looking for a serverless, mobile-optimized preference center allowing end users to manage their Pinpoint communication preferences and attributes, you can also check the Amazon Pinpoint Preference Center.

Architecture

Architecture

In this scenario, users’ SMS opt-in/opt-out preferences are managed by a home-grown customer application. Users interact with the application over its web interface. The application, saves the customer preferences on a MySQL database.

This solution’s flow of events is triggered with a change (insert / update / delete) happening in the database. The change event is then captured by AWS Database Migration Service (DMS) that is configured with an ongoing replication task. This task continuously monitors a specified database and forwards the change event to an Amazon Kinesis Data Streams stream. Raw events that are buffered in this stream are polled by an AWS Lambda function. This function transforms the event, and makes it ready to be passed to Amazon Pinpoint API. This API call will in turn, change the opt-in/opt-out subscription status of the channel for that user.

Ongoing replication tasks are created against multiple types of database engines, including Oracle, MS-SQL, Postgres, and more. In this blog post, we use a MySQL based RDS instance to demonstrate this architecture. The instance will have a database we name pinpoint_demo and one table we name optin_status. In this sample, we assume the table is holding details about a user and their opt-in preference for SMS messages.

userid phone optin lastupdate
user1 +12341111111 1 1593867404
user2 +12341111112 1 1593867404
user2 +12341111113 1 1593867404

Prerequisites

  1. AWS CLI is configured with an active AWS account and appropriate access.
  2. You have an understanding of Amazon Pinpoint concepts. You will be using Amazon Pinpoint to create a segment, populate endpoints, and validate phone numbers. For more details, see the Amazon Pinpoint product page and documentation.

Setup

First, you clone the repository that contains a stack of templates to your local environment. Make sure you have configured your AWS CLI with AWS credentials. Follow the steps below to deploy the CloudFormation stack:

  1. Clone the git repository containing the CloudFormation templates:
    git clone https://github.com/aws-samples/amazon-pinpoint-rds-integration.git
    cd amazon-pinpoint-rds-integration
  2. You need an S3 Bucket to hold the template:
    aws s3 create-bucket –bucket <YOUR-BUCKET-NAME>
  3. Run the following command to package the CloudFormation templates:
    aws cloudformation package --template-file template_stack.yaml --output-template-file template_out.yaml --s3-bucket <YOUR-BUCKET-NAME>
  4. Deploy the stack with the following command:
    aws cloudformation deploy --template-file template_out.yaml --stack-name pinpointblogstack --capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_NAMED_IAM

The AWS CloudFormation stack will create and configure resources for you. Some of the resources it will create are:

  • Amazon RDS instance with MySQL
  • AWS Database Migration Service replication instance
  • AWS Database Migration Service source endpoint for MySQL
  • AWS Database Migration Service target endpoint for Amazon Kinesis Data Streams
  • Amazon Kinesis Data Streams stream
  • AWS Lambda Function
  • Amazon Pinpoint Application
  • A Cloud9 environment as a bastion host

The deployment can take up to 15 minutes. You can track its progress in the CloudFormation console’s Events tab.

Populate RDS data

A CloudFormation stack will output the DNS address of an RDS endpoint and Cloud9 environment upon completion. The Cloud9 environment acts as a bastion host and allows you to reach the RDS instance endpoint deployed into the private subnet by CloudFormation.

  1. Open the AWS Console and navigate to the Cloud9 service.
    Cloud9Console
  2. Click on the Open IDE button to reach your IDE environment.
    Cloud9Env
  3. At the console pane of your IDE, type the following to login to your RDS instance. You can find the RDS Endpoint address at the outputs section of the CloudFormation stack. It is under the key name RDSInstanceEndpoint.
    mysql -h <YOUR_RDS_ENDPOINT> -uadmin -pmypassword
    use blog_db;
  4. Issue the following command to create a table that holds the user’s opt-in status:
    create table optin_status (
      userid varchar(50) not null,
      phone varchar(50) not null,
      optin tinyint default 1,
      lastupdate TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
    );
  5. Next, load sample data into the table. The following inserts nine users for this demo:
    
    INSERT INTO optin_status (userid, phone, optin) VALUES ('user1', '+12341111111', 1);
    INSERT INTO optin_status (userid, phone, optin) VALUES ('user2', '+12341111112', 1);
    INSERT INTO optin_status (userid, phone, optin) VALUES ('user3', '+12341111113', 1);
    INSERT INTO optin_status (userid, phone, optin) VALUES ('user4', '+12341111114', 1);
    INSERT INTO optin_status (userid, phone, optin) VALUES ('user5', '+12341111115', 1);
    INSERT INTO optin_status (userid, phone, optin) VALUES ('user6', '+12341111116', 1);
    INSERT INTO optin_status (userid, phone, optin) VALUES ('user7', '+12341111117', 1);
    INSERT INTO optin_status (userid, phone, optin) VALUES ('user8', '+12341111118', 1);
    INSERT INTO optin_status (userid, phone, optin) VALUES ('user9', '+12341111119', 1);
  6. The table’s opt-in column holds the SMS opt-in status and phone number for a specific user.

Start the DMS Replication Task

Now that the environment is ready, you can start the DMS replication task and start watching the changes in this table.

  1. From the AWS DMS Console, go to the Database Migration Tasks section.
    DMSMigTask
  2. Select the Migration task named blogreplicationtask.
  3. From the Actions menu, click on Restart/Resume to start the migration task. Wait until the task’s Status transitions from Ready to Starting and Replication ongoing.
  4. At this point, all the changes on the source database are replicated into a Kinesis stream. Before introducing the AWS Lambda function that will be polling this stream, configure the Amazon Pinpoint application.

Inspect the AWS Lambda Function

An AWS Lambda function has been created to receive the events. The Lambda function uses Python and Boto3 to read the records delivered by Kinesis Data Streams. It then performs the update_endpoint API calls in order to add, update, or delete endpoints in the Amazon Pinpoint application.

Lambda code and configuration is accessible through the Lambda Functions Console. In order to inspect the Python code, click the Functions item on the left side. Select the function starting with pinpointblogstack-MainStack by clicking on the function name.

Note: The PINPOINT_APPID under the Environment variables section. This variable provides the Lambda function with the Amazon Pinpoint application ID to make the API call.

LambdaPPAPPID

Inspect Amazon Pinpoint Application in Amazon Pinpoint Console

A Pinpoint application is needed by the Lambda Function to update the endpoints. This application has been created with an SMS Channel by the CloudFormation template. Once the data from the RDS database has been imported into Pinpoint as SMS endpoints, you can validate this import by creating a segment in Pinpoint.

PinpointProject

Testing

With the Lambda function ready, you now test the whole solution.

  1. To initiate the end-to-end test, go to the Cloud9 terminal. Perform the following SQL statement on the optin_table:
    UPDATE optin_status SET optin=0 WHERE userid='user1';
    UPDATE optin_status SET optin=0 WHERE userid='user2';
    UPDATE optin_status SET optin=0 WHERE userid='user3';
    UPDATE optin_status SET optin=0 WHERE userid='user4';
  2. This statement will cause four changes in the database which is collected by DMS and passed to Kinesis Data Streams stream.
  3. This triggers the Lambda function that construct an update_endpoint API call to the Amazon Pinpoint application.
  4. The update_endpoint operation is an upsert operation. Therefore, if the endpoint does not exist on the Amazon Pinpoint application, it creates one. Otherwise, it updates the current endpoint.
  5. In the initial dataset, all the opt-in values are 1. Therefore, these endpoints will be created with an OptOut value of NONE in Amazon Pinpoint.
  6. All OptOut=NONE typed endpoints are considered as active endpoints. Therefore, they are available to be used within segments.

Create Amazon Pinpoint Segment

  1. In order to see these changes, go to the Pinpoint console. Click on PinpointBlogApp.
    PinpointConsole
  2. Click on Segments on the left side. Then click Create a segment.
    PinpointSegment
  3. For the segment name, enter US-Segment.
  4. Select Endpoint from the Filter dropdown.
  5. Under the Choose an endpoint attribute dropdown, select Country.
  6. For Choose values enter US.
    Note: As you do this, the right panel Segment estimate will refresh to show the number of endpoints eligible for this segment filter.
  7. Click Create segment at the bottom of the page.
    PinpointSegDetails
  8. Once the new segment is created, you are directed to the newly created segment with configuration details. You should see five eligible endpoints corresponding to database table rows.
    PinpointSegUpdate
  9. Now, change one row by issuing the following SQL statement. This simulates a user opting out from SMS communication for one of their numbers.
    UPDATE optin_status SET optin=0 WHERE userid='user5';
  10. After the update, go to the Amazon Pinpoint console. Check the eligible endpoints again. You should only see four eligible endpoints.

PinpointSegUpdate

Cleanup

If you no longer want to incur further charge, delete the Cloudformation stack named pinpointblogstack. Select it and click Delete.

PinpointCleanup

Conclusion

This solution walks you through how opt-in change events are delivered from Amazon RDS to Amazon Pinpoint. You can use this solution in other use cases as well. Some examples are importing segments from a 3rd party application like Salesforce and importing other types of channels like e-mail, push, and voice. To learn more about Amazon Pinpoint, visit our website.

Amazon SES celebrates 10 years of email sending and deliverability

Post Syndicated from Simon Poile original https://aws.amazon.com/blogs/messaging-and-targeting/amazon-ses-celebrates-10-years-of-email-sending-and-deliverability/

Amazon Simple Email Service (Amazon SES) turns 10 years old today. Back on January 25th 2011, Amazon Web Services (AWS) had only 15 services. Today, AWS has grown to over 180 services. Jeff Barr launched Amazon SES as part of his web evangelist blog. Much of what he wrote about then is still true today. Even 10 years later, email is an important channel for customer communications. Developers still want to rely on a trusted global partner to deliver email at scale. However, mailbox providers are even more protective of their end users’ security. They actively work to ensure that any perceived, unwanted email doesn’t make it to the inbox.

Inbox providers use several factors to determine the legitimacy of email traffic. Over the last decade, we have worked diligently to measure many of those factors in Amazon SES to help our customers achieve great deliverability. The focus for much of that work has been a combination of investments into reputation, engagement, and trust. I want to outline what we’ve accomplished to improve your email sending over the last 10 years.

Reputation

Reputation is the measurement mailbox providers use to determine how closely you follow their sending standards. Amazon SES measures perceived reputation through metrics such as bounce rate or complaint rate in the reputation dashboard. The reputation dashboard also shares overall Amazon SES account sending status like “Healthy” or “Under Review.” Some Inbox providers, or ISPs, also provide feedback to help us measure the effectiveness of a specific IP or domain in sending trustworthy traffic.

You can influence reputation in Amazon SES through:

  • Setting up dedicated IPs: Set up IPs in Amazon SES for your own specific sending with appropriate warm-up plans. Split IPs out by use case such as separating password resets from marketing messages.
  • Customer owned IPs (New in 2020): You can now transition IPs you’ve invested in through your own data center or with another ESP to Amazon SES without interruption.
  • Following sending volume best practices: Nothing can flag your IP addresses faster than non-predictable sending patterns. We help you manage this through sending quotas.
  • Use our SES email simulator: Test your application sending without messages leaving the sandbox.

 

Engagement

Engagement is the rate by which customers are interacting with your content. Amazon SES helps you measure engagement through conversion rates (such as open or click-through) and unsubscribe rates. These are measured in the event publishing click stream. This area is more of an art in our deliverability calculus because success varies by industry and use case.

You can influence engagement in Amazon SES through:

  • Customizing content as much as possible, but follow content best practices to avoid setting off content filters. Mailbox providers often utilize behavioral content filtering using AI to determine if your content is relevant based on engagement behavior.
  • Use consent and list management (New in 2020) with customized topics and opt-out pages. It’s important to offer recipients a way to select what emails they want to receive from you and give them an option to opt-out. This is a great new feature that we’ve added based on customer feedback.
  • Remove emails that are not engaging from your lists. Some customers have a time limit, for example, 60 days, before they are automatically removed from an active email list.

 

Trust

Earning trust on email sending is done through the adherence to proper sending behavior, as measured by both individual ISPs as well as industry watch-groups. Trust is closely related to reputation.  We measure trust through messages in the reputation dashboard based on feedback loops, Real-time Blocklists (RBLs), and spam-traps. You can also see the complaint rate associated to your sending in the complaint area of the reputation dashboard. It has statuses like healthy or under review.

You can influence trust in Amazon SES through:

 

Deliverability is a multi-dimensional part of email sending, beyond just setting up an SMTP (Simple Mail Transfer Protocol) endpoint, with constant complexities. But, we’re here to help. In addition to these investments in deliverability, we’ve also expanded Amazon SES to 18 regions, including the government cloud. It’s been an exciting time at AWS, and we look forward to supporting all of our customers in the years to come with Amazon SES.

 

 

 

 

Send localized messages using Amazon Pinpoint templates and standard demographic attributes

Post Syndicated from Mohit Palriwal original https://aws.amazon.com/blogs/messaging-and-targeting/send-localized-messages-using-pinpoint-templates-and-standard-demographic-attributes/

As your application user base expands into more countries and languages, it’s important to make sure messages are localized for each recipient to improve engagement. Localizing your messages helps you reach your audience with content specific to their language settings. Creating separate messages for each language and managing each template separately can require a lot of duplication effort. It is also challenging to manage and group templates based on all possible locales or specific campaigns.

Amazon Pinpoint‘s messaging template provides a way to build a single message with multiple localizations. You prepare localizations based on locale of your audience registered with Amazon Pinpoint project.

This blog post walks you through a solution that uses the locale of your user endpoints to build a localized messaging template. We provide you with a template that is used with an Amazon Pinpoint campaign or journeys to target your audience across multiple locale with localized message content. This solution is applicable for all supported channels under Amazon Pinpoint, SMS, email, push, voice. This blog explains the solution for a SMS channel-specific scenario.

Solution overview

The solution below describes the workflow to send localized messaging to a group of users across various locales. The first prerequisite is to create an Amazon Pinpoint project in your AWS account and enable corresponding channels for message sending. Next, you will create an Amazon Pinpoint template using locale-specific message variables and register users endpoints with a demographic locale property. Once segment and template resources are generated, you can create a localized message in your campaign or journey.

Prerequisites

Setting up the solution

1. Set up Amazon Pinpoint

First, create a new Amazon Pinpoint project and configure the desired channels from which you want to send localized messages.

2. Create a localized template

  1. Create an Amazon Pinpoint messaging template with supported message variables of your choice. This builds more dynamic and personalized content.
  2. Use Demographic.Locale from supported Endpoint attributes to customize your message content per locale using eq comparison helper.

Below is an example of using an endpoint standard locale attribute in a template.

{{#eq Demographic.Locale "fr-FR"}} Bienvenue dans l'expérience utilisateur Pinpoint! 
{{else eq Demographic.Locale "de-DE"}} Willkommen bei Pinpoint User Experience! 
{{else}} Welcome to Pinpoint User Experience ! {{/eq}}  

3. Register your users with locale property

Register your user endpoint to pinpoint with the demographic locale/timezone standard attribute.

The below is an example for registering an SMS endpoint with de-DE locale.
aws pinpoint update-endpoint –application-id $APP_ID –endpoint-id

$ENDPOINT_ID --endpoint-request '{"Address":"+19999999999","ChannelType":"SMS","Demographic":{"Locale":"de-DE", "Timezone": "Europe/Berlin"}}'

Note: You can also register your user endpoints using the import segment feature. This accepts a .csv file with all endpoints.

4. Create a segment with all locale users

Create an Amazon Pinpoint segment to define the audience you want to target with localized message.

5. Create a journey or campaign

  1. Create an Amazon Pinpoint campaign or journey.
  2. Use the template from earlier in Step 2.
  3. Create a segment with all locale users from Step 4.Note: You can also use Amazon Pinpoint local time and quiet time features to target your audience in their local time zone or at a specific global time (for example 10am GMT). This also respects the quiet hours (for example 23:00 to 8:00) specific to their local time zone based on the EndpointDemographic.Timezone property.

 

6. Execution:

A marketing campaign manager wants to send a localized message to every audience based of their preferred language.

  1. Creates a single journey targeting a segment with 2 endpoints (each with unique locale) from Step 4.
  2. Create a segment with all locale users using the template defined in Step 2.
  3. Create a localized template

Conclusion

The Amazon Pinpoint messaging template provides you the ease of managing a single template for multiple locales.

With a localized messaging template you can simply target your audience across locales and receive targeted analytics. Get started today by visiting Amazon Pinpoint’s webpage.

Other useful links

 

Strategies for list management with Amazon Pinpoint and Amazon Simple Email Service

Post Syndicated from Heidi Gloudemans original https://aws.amazon.com/blogs/messaging-and-targeting/strategies-for-list-management-with-amazon-pinpoint-and-amazon-simple-email-service/

Managing customer lists is a large part of any outbound customer communication program. From customer acquisition to ongoing engagement, locating the best sources for subscribers and respecting their contact preferences is key to maintaining healthy customer lists. This article will discuss recommendations for list building using Amazon Pinpoint and Amazon Simple Email Service (SES). We will provide recommendations for a subscription process, what information to ask for, how to manage opt-outs, and optimize lists over time.

Customer acquisition

Customer acquisition is the first part of any list activity. There are a few guidelines that all outbound marketers should follow during the list building process. First, do not use 3rd party, leased, or purchased lists. The use of 3rd party lists for email risks complaints, impact to sender reputation, and inclusion in monitoring functions like spam traps. Email service providers like Amazon Pinpoint and Amazon SES will discontinue service for accounts with poor sending behavior that results from use of 3rd party lists.

Second, if you plan on contacting customers across channels, make sure you acquire permission to contact users on each channel. There are many places you can acquire customer contacts, ranging from your website, social media presence, or a QR code on a physical sign. Amazon Pinpoint also has a solution called the Amazon Pinpoint Preference Center that you can deploy to gather and manage customer contact preferences across channels.

There are a few items that you want to include in any customer acquisition form. First, tell the customer how often they can expect communication from you. Is it a weekly newsletter? Monthly? Even better if you give them the option to select how often you communicate with them. Next, tell them what value they can expect from registration. For example, special deals, early access to sales, or even just product and industry news. While you can provide some incentives, avoid providing high-value incentives for registration. Over-the-top enticements like free products will always cause low-quality registrations and resulting low-quality lists.

In addition, make your sign-up forms as concise as possible. Only put high-value content behind registration, and minimize the amount of information customers must provide to register. Having a full profile makes your life as a marketer trying to segment your customers easier. However, it potentially adds friction to the sign-up process which can result in lost customers.

If you can, allow the customer to indicate their content preferences later or during onboarding communications. If you use Amazon SES, you can support up to 20 list topics per account in the Amazon SES list management. For example, if you are a sportswear company, interests could include topics like hiking, biking, or running. You should then send customers emails only about the specific topics that the recipient is interested in receiving. Make sure you retain preference data. All countries are different, but some require you to prove that you received permission to contact a customer.

Managing your customer contacts

Onboarding communication is an essential first step once a customer has submitted their initial registration. Some organizations use the first communication as a registration confirmation step, called a “double opt-in.” In addition to driving engagement and initial calls-to-action (“Confirm your email”), double opt-in emails have the added benefit of verifying that a bot did not submit your customer email.

From the first message you send to a new customer to the last, you should always include the unsubscribe option. Every country has different requirements, of which you should research and educate your organization. Amazon SES now supports subscription management for custom URLs in the footers of your emails. Amazon SES also now supports contact preferences in a custom landing page, where customers can adjust their contact list preferences.

Removing unengaged users

Both Amazon Pinpoint and Amazon SES enable visibility to the success of outbound communications through open rates and click-through rates at the account or campaign level. If time passes with limited engagement from an individual customer, (i.e., they do not open your mails or engage with the content) there is a risk the recipient mailbox provider will start marking your messages as spam. Work with your business to determine the period of time after which you should automatically remove unengaged users from your contact list. Many organizations will remove customers from their contact lists after 60 to 90 days of non-engagement.  If you need the ability to quickly query customers that are not engaging with your communications from your data store of choice, enable event stream data in Amazon Pinpoint or Amazon SES using Amazon Kinesis. Amazon Pinpoint also has a solution, the Digital User Engagement Events Stream Database that creates a data store for that purpose.

Amazon SES and Amazon Pinpoint both also have the concept of global and account suppression lists. Global suppression lists are managed across AWS accounts, while account-based suppression lists are associated to your AWS account. If a customer explicitly unsubscribes from your list using Amazon SES list management, or complains through their inbox provider, they will automatically be added to the respective account suppression list. Customers that are part of suppression lists are no longer sent messages from your account. Respecting contact preferences like unsubscribe is an opportunity to earn trust with that specific customer, the market, and the recipient mailbox provider.

Conclusion

There are a number of additional best practices to drive customer engagement across communications channels. They can include message headlines, copy, graphics/images, and adaptive design across various endpoints and clients. However, nothing is as important as maintaining the trust of end customers with their contact information. Sourcing contact information with the customer permission, respecting contact preferences, and maintaining list hygiene is the cornerstone to a successful customer communications program. Learn more today about Amazon Pinpoint and Amazon SES and customer communications.

How Amazon Simple Email Service supported the growth of email in 2020

Post Syndicated from Simon Poile original https://aws.amazon.com/blogs/messaging-and-targeting/how-amazon-simple-email-service-supported-the-growth-of-email-in-2020/

Over the last 12 months, organizations of all types have increasingly needed to stay connected to their customers. With the move to virtual interactions accelerating across industries, email has remained a trusted channel for customer communications. Amazon Simple Email Service (SES) has seen record outbound email traffic in 2020, supporting critical customer communications during COVID and commercial moments like Black Friday and Cyber Monday.

The importance of email during COVID

Unlike real-time communications like voice or live chat, email is asynchronous. It can be read and consumed at the customer’s leisure. In some geographies like North America, email also represents an individual’s unique identity, persisting longer than mobile phone numbers or social networking accounts. Even with the importance of email established before 2020, it was important to most organizations to send only the right messages during the COVID crisis.

Many organizations chose to decrease promotional or marketing emails during the pandemic voluntarily. This decrease in sending was to recognize the increased stress most individuals were facing in their personal lives. However, even with the drop in marketing emails across organizational types, there was an increased need to communicate and maintain customer engagement. Most organizations went through three distinct customer communication phases with email in 2020: React, Respond, and Reimagine.

  • React – These were the initial emails sent to acknowledge the COVID crisis, occurring early in 2020. These emails included messages reinforcing commitment to customer health, employee safety, or communicating new cleaning protocols.
  • Respond – These messages often included communication on the status of the business or event. Most businesses needed to communicate their transition to remote work, temporary closures, and many in-person events canceled.
  • Reimagine – Throughout the crisis, organizations were reimagining how to do business. Healthcare started operating video consultations, and restaurants shifted to pick up/take out only. Email communication was vital to take customers on the journey into this “new normal,” even as some businesses started to reopen.

To send these customer communications at scale, many organizations worked with Amazon SES.

How Amazon SES scaled and supported customers in 2020

Amazon SES saw several sending spikes that aligned with organizations working to communicate with their customers during COVID. Nine times in 2020, transactions per second (TPS) in Amazon SES exceeding 150% of the previous record held by 2019 Black Friday. This over 150% TPS spike also occurred on 2020 Black Friday and Cyber Monday.

In addition to supporting those upsurges in throughput, the Amazon SES team also responded to customer feedback on increasing the global footprint of Amazon SES. Since January, Amazon SES increased the total number of regions supported from 7 to 14, including the US government cloud. These additional regions were deployed during 2020 as the team worked remotely. This regional expansion enabled customers to adhere to local data sovereignty requirements for email sending while also improving performance.

Customers also told us they needed tools to help them manage compliance with important governance laws like CAN-SPAM and GDPR. Amazon SES released list management to help organizations manage their customer’s contact information and preferences.

Looking forward

As we move into 2021, email will remain at the forefront of customer communication channels. Enterprise customers like Netflix and Duolingo rely on Amazon SES to deliver their email at scale. For more information on how you can use Amazon SES, visit our website.

Automate phone number validation with Amazon Pinpoint

Post Syndicated from Ilya Pupko original https://aws.amazon.com/blogs/messaging-and-targeting/automate-phone-number-validation-with-amazon-pinpoint/

Amazon Pinpoint allows you to engage with your customers across multiple messaging channels like SMS text, email, and voice messages. While planning and executing standard text (SMS) and voice-based campaigns, one of the challenges developers often run into is the need to verify if the phone numbers in their internal database are valid and conform to the standard E.164 format. You can attempt to verify the phone numbers manually one at a time, but it’s tedious. To overcome this issue, Amazon Pinpoint provides a phone number validation service that you can use to determine if a phone number is valid, have it automatically formatted, and obtain additional information about the phone number itself. For example, when you use the phone number validation service, it returns the following information:

  • The phone number in E.164 format.
  • The phone number type (such as mobile, landline, or VoIP).
  • The city and country where the phone number is based.
  • The service provider that is associated with the phone number.

This blog post aims to provide a step-by-step implementation guide and the necessary code to enable an integrated solution for number verification.

Process flows and architecture


This solution uses Amazon Simple Storage Service (Amazon S3), Amazon Pinpoint, AWS Step Functions, Amazon Simple Notification Service (SNS) and AWS Lambda. To initiate the process, you upload your source contact file in the CSV format to the dedicated Amazon S3 bucket. When the CSV file is uploaded, S3 triggers the associated tasks. Based on the optional configuration rules, the application code either runs the Phone Validate logic first or imports the contact information as-is into Amazon Pinpoint as a new imported segment and updates overall Amazon Pinpoint audience information. If Phone Validation is enabled, the system will first generate and save the new output file to Amazon S3 with the valid phone number, metadata, etc. and use this updated contact information during import. Additionally, the system will kick-off a scheduled campaign to all imported contacts.

This CloudFormation template will automatically create the following new resources on your first deploy:

  • AWS Lambda function: These functions contains the application code which validate the phone numbers. It also creates the segment for the uploaded contacts.
  • S3 event notification: When the CSV file is uploaded to the S3 bucket, the S3 Event Notification triggers the AWS Lambda function which initiate the AWS Step Functions State Machine. To learn more about the S3 Event Notification, check the documentation.
  • AWS Step Functions: This solution will set up an infrastructure to automatically trigger when a new file is placed in an S3 bucket. The process, managed by an AWS Step Functions state machine, will start a Pinpoint import process, wait for it to complete, and send notifications that the job started, successfully finished, or failed.
  • IAM role: The IAM role is used to make Amazon Pinpoint calls, to access S3, and interact with AWS Step Functions and Amazon SNS. You can check the IAM documentation to learn more about IAM roles.

Prerequisites and deployment steps

Step 1: Set up the Amazon Pinpoint project and the S3 bucket

In Amazon Pinpoint, a project (also sometimes referred to as “application”) is a collection of settings, customer information, segments, and campaigns. Setting up a Pinpoint project is the first step to deploy our solution. It holds the segment we will use in the later steps.

  1. Navigate to the Amazon Pinpoint from the services tab in the AWS Management Console and create a new Amazon Pinpoint project.
  2. Copy the Project ID from the Amazon Pinpoint console and save it in notepad. You will need it later.

In Amazon S3, create a new bucket to upload the files to. Make sure it is setup according to your company’s security practices. If you have an existing bucket you want to use instead, note that this solution will require a source bucket in the same region as the solution itself and it will override any triggers already in place on the bucket.

Step 2: Deploy code and services

AWS CloudFormation is a service that gives developers and businesses an easy way to create a collection of related AWS and third-party resources. You can provision them in an orderly and predictable fashion.

  1. Download the latest version of the solution from https://github.com/aws-samples/digital-user-engagement-reference-architectures/blob/master/cloudformation/S3_triggered_import.yaml
  2. Log in to your AWS account and navigate to the Amazon CloudFormation from the services tab in the AWS Management Console: https://console.aws.amazon.com/cloudformation/home
  3. Click on the Create Stack button and choose to provision New Resources. Then select Upload a template file and choose the file you just downloaded in the first step.
  4. On the Specify stack details screen all the information is pre-populated as shown in the screenshot below. Parameters:
    · Replace the PinpointProjectID field with the value you saved in Step 1
    · ValidatePhone: Choose true if you wish to validate the numbers via the Pinpoint API before importing the segment.
    · AssumeUS: Choose true if you want to assume US (+1) phone number for any phone 10 digits long or false if you want to import as-is.
    · AutoCreateCampaign: Choose true if you want to automatically create a campaign based on the imported file or false if you want to just import into the system without automatically scheduling any campaigns. This setting will be saved as an ImportSegment Lambda environment variable so you can adjust it later.
    · CampaignDelay: Number of minutes from the time of import to start of the campaign (if AutoCreateCampaign is set to true). Allows for the last-minute double check and/or pause as needed. Will be saved as CreateCampaign Lambda environment variable.
    · FileDropS3Bucket: Name of the existing Amazon S3 Bucket where new import files will be placed. Note that it has to be in the same region as you are running this template and the bucket should not have any existing notification configurations or they will be overwritten.
    · FileDropS3Prefix: Prefix (sub-folder name) of the Amazon S3 Bucket where you will be uploading new files to be imported.
  5. Settings on the configure stack options page are optional, click Next.

Select all acknowledgment boxes and click Create Stack. It takes a couple of minutes for the AWS CloudFormation to deploy all the resources.

The solution is now deployed and you can test it by uploading the sample CSV file to the Amazon S3 bucket. You will notice that the output CSV file is created in the “results” folder of the same S3 bucket, if you have validation enabled. You can also navigate to the Amazon Pinpoint console to check the Amazon Pinpoint segment. Once the deployment is complete and the segment is created, you can leverage Amazon Pinpoint campaigns to reach out to your customers.

Conclusion and Next Steps

Enabling solutions such as this provides an efficient and integrated mechanism to validate phone numbers and import customer contacts into Amazon Pinpoint. It saves time so that you can focus on creating effective campaigns to engage with your customers.

As the potential next steps, you can look into further expanding the solution by:

  1. Adjusting the default security of the Amazon S3 bucket by limiting who has access to new files. You can also adjust its encryption and the expiration of the files.
  2. Build out the lookup AWS Lambda to additionally fetch other information about the contact using your other systems of records and/or even 3rd party tools. You can also add business logic such as blocking numbers from certain countries (or vice versa, only allow certain countries).
  3. Add more dynamic segments and new endpoint (or user) attributes to more easily track the contacts based on their upload dates, type of phone number, etc.

Create a nice interface your users can use to interact with when needing to upload instead of using the S3 console directly. This “interface” may even be just a backend flow that simply integrates your system of records. This is so they don’t have to deal with any interface and uploads in the first place.

For this, and some other reference architectures you could consider, see https://github.com/aws-samples/digital-user-engagement-reference-architectures.

References

Amazon Pinpoint

https://aws.amazon.com/pinpoint/

Validating phone numbers in Amazon Pinpoint

https://docs.aws.amazon.com/pinpoint/latest/developerguide/validate-phone-numbers.html

Amazon Pinpoint Campaigns

https://docs.aws.amazon.com/pinpoint/latest/userguide/campaigns.html

Pinpoint Segment

https://docs.aws.amazon.com/pinpoint/latest/userguide/tutorials-create-a-segment.html

 

Send voice appointment reminders using Amazon Pinpoint custom channels and Amazon Connect

Post Syndicated from Ryan Lowe original https://aws.amazon.com/blogs/messaging-and-targeting/send-voice-appointment-reminders-using-amazon-pinpoint-custom-channels-and-amazon-connect/

Introduction

In this post, we will walk through setting up an always-on appointment reminder campaign in Amazon Pinpoint. No-show rates are a constant challenge for service providers. Industries such as hospitality estimate 20% of diners miss reservations in big cities,1 while salons average five missed appointments per week.2 Professional services such as financial institutions and sales teams have similar challenges to ensure clients do not miss meetings. To these businesses, an appointment missed represents lost revenue. As a result, the no-show rate is a key metric to improve. An outbound voice message provides another way to reach customers versus emails or SMS, and voice reminders give customers the choice of channels based on personal preferences.

Overview

Amazon Pinpoint is a multichannel communications service enabling customers to send both promotional and transactional messages across email, SMS, push notifications, voice, and custom channels. Amazon Connect is an easy to use omnichannel cloud contact center that helps companies provide superior customer service at a lower cost.

There are benefits of using these services together. Amazon Pinpoint allows you to build a segment of users which can be used within a campaign. Amazon Connect can enable customers to send outbound voice messages at scale should your user audience be large and require a high number of transactions per second (TPS).

To use these services together, you setup custom channels in Amazon Pinpoint, which can be created via an AWS Lambda function. These functions enable you to call APIs to trigger message sends as part of Amazon Pinpoint campaigns. Amazon Pinpoint has developed a new AWS Lambda function which can be used to send outbound voice messages via Amazon Connect. This configuration allows you to define the voice message to be sent, define the segment of users you would like to target, and send voice messages at scale through Amazon Connect via the Amazon Pinpoint custom channel.

The audience for this solution are technical customers who are used to working with multiple AWS services and are familiar with AWS Lambda functions. The solution built relies on the Amazon Pinpoint custom channel feature and targeting, along with the Amazon Connect outbound voice API called via a prepared AWS Lambda function. Once completed, you will be able to create an evergreen campaign which will send outbound voice messages to your patients who have an appointment the following day.

The costs associated with this solution will be:

  1. Amazon Connect outbound voice calls per minute
  2. Amazon Connect claimed phone number(s)
  3. Amazon Pinpoint Monthly Targeted Audience (MTA) costs.

The costs for a outbound voice reminder system that sends 10k messages per day, with an average length of 20 seconds per call, to an total monthly audience of 300k, in the US are as follows. Note that prices with vary for other countries. Complete Amazon Connect outbound call pricing can be found here.

Solution

Prerequisites:

For this walkthrough the article assumes:

  • An AWS account
  • Basic understanding of IAM and privileges required to create the following; IAM identity provider, roles, policies, and users
  • Basic understanding of Amazon Pinpoint and how to create a project
  • Basic understanding of Amazon Connect and experience in creating contact flows. More information on setup of Amazon Connect can be found here.

Step 1: Create an Appointment Reminder custom event

The first step in setting up this solution is to create and report a custom event to Amazon Pinpoint. There are multiple ways to report events in your application. Ffor demonstration purposes, below are two example event calls using the AWS SDK for Python (Boto3) from inside an AWS Lambda Function.

It is important to note that the Amazon Pinpoint events API can also be used to update endpoints when the event gets registered. In the below example, the first API call will update the endpoint attributes AppointmentDate and AppointmentTime with the details of the upcoming appointment. These attributes will be used in the outgoing message to the end-user

Sample Event: Appointment Coming Up

import boto3

client = boto3.client('pinpoint')
app_id = '[PINPOINT_PROJECT_ID]'
endpoint_id = '[ENDPOINT_ID]'
address = '[PHONE_NUMBER]'

def lambda_handler(event, context):

client.put_events(
ApplicationId = applicationId,
EventsRequest={
'BatchItem': {
endpoint_id: {
'Endpoint': {
'ChannelType': 'CUSTOM',
'Address': address,
'Attributes': {
'AppointmentDate': ['December 15th, 2020'],
'AppointmentTime': ['2:15pm']
}
},
'Events':{
'appointment-event': {
'Attributes':{},
'EventType': 'AppointmentReminder',
'Timestamp': datetime.datetime.fromtimestamp(time.time()).isoformat()
}
}
}
}
}
)

NOTE: The following steps assume that the AppointmentReminder event is being reported to Amazon Pinpoint. If you are unable to integrate the above API call into your application, you can manually create an AWS Lambda function using a Python runtime with the above code to trigger sample events.

Step 2: Create an Amazon Connect contact flow for outbound calls

This article assumes that you have an Amazon Connect contact center already setup and working. In this step, we will set up our Amazon Connect contact flow to dial our recipients and play read the message before hanging up.

  1. Log in to your Amazon Connect instance using your access URL (https://<alias>.awsapps.com/connect/login).
    Note: Replace alias with your instance’s alias.
  2. In the left navigation bar, pause on Routing, and then choose Contact flows.
  3. Under Contact flows, choose a template, or choose Create contact flow to design a contact flow from scratch. For more information, see Create a New Contact Flow.
  4. Download the sample JSON contact flow configuration file Outbound_calling.json.
  5. Choose the dropdown menu under Save and choose Import flow (beta).
  6. Select the Outbound_calling.json file in the Import flow (beta) dialog and choose Save.
  7. Choose Save to open the Save flow dialog. Then choose Save to close the dialog.
  8. Choose Publish to open the Publish dialog. Then choose Publish to close the dialog.
  9. In the contact flow designer, expand Show additional flow information.
  10. Under ARN, copy the Amazon Resource Name (ARN) contact flow. It looks like the following:
    arn:aws:connect:region:123456789012:instance/[ConnectInstanceId]/contact-flow/[ConnectContactFlowId]Note the ConnectInstanceId and ConnectContactFlowId from the ARN, they will be used in the next step.
  11. In the left navigation bar, pause on Routing and then choose Queues.
  12. Choose the queue you wish to use for the outbound calls.
  13. In the Edit queue screen, expand Show additional queue information.
  14. Under ARN, copy the Amazon Resource Name (ARN) for the queue. It looks like the following:
    arn:aws:connect:region:123456789012:instance/[ConnectInstanceId]/contact-flow/[ConnectQueueId]Note the ConnectQueueId from the ARN. It will be used in the next step.

Step 3: Deploy and modify the Amazon Pinpoint to the Amazon Connect custom channel with AWS Lambda function

Next, we will need to deploy an Amazon Pinpoint custom channel. Custom channels in Amazon Pinpoint allow you to send messages through any service with an API, including Amazon Connect. The AWS Serverless Application Repository contains an open-sourced AWS Lambda function that we will use for our custom channel. After deploying the AWS Lambda function, we will customize it to match our requirements.

  1. Navigate to the AWS Lambda Console, then choose Create function.
  2. Under Create function, Choose Browser serverless app repository.
  3. Under Public applications, choose the checkbox next to Show apps that create custom IAM roles or resource policies and enter amazon-pinpoint-connect-channel in the search box.
  4. Choose the amazon-pinpoint-connect-channel card from the list and review the Application details.
  5. Under Application settings enter the details for ConnectContactFlowId, ConnectInstanceId, and ConnectQueueId from the previous step.
  6. After reviewing all the details, choose the checkbox next to I acknowledge that this app creates custom IAM roles and resource policies and choose Deploy.
  7. Wait a couple minutes for the application to deploy two AWS Lambda functions and an AWS Simple Queue Service queue.
  8. Under Resources, choose the PinpointConnectQueuerFunction resource to open the AWS Lambda function configuration. This is the AWS Lambda function that Amazon Pinpoint will call when the message is crafted.
  9. Under Function code, scroll down to line 31 and replace
    message = "Hello World! -Pinpoint Connect Channel"
    with
    message = "This is a reminder of your upcoming appointment on {0} at {1}".format(endpoint_profile["Attributes"]["AppointmentDate"][0], endpoint_profile["Attributes"]["AppointmentTime"][0])
  10. Choose Deploy.

Step 4: (Optional) Modify the custom channel AWS Lambda function to meet change the rate of outgoing calls

By default, the custom channel we deployed in the previous step will place outbound calls through Amazon Connect at a rate of 1 call every 3 seconds. This allows you to configure how many active outbound calls to avoid running into service limits. Review your current service limits in Amazon Connect for more details.

  1. Navigate to the AWS Lambda Console, then choose AmazonPinpointConnectChannel-backgroundprocessor function.
  2. Under Function code, scroll down to line 73 and replace the sleep timer, currently set with 3 seconds, with your requirements.
  3. Choose Deploy.

Step 5: Create a Pinpoint custom campaign with your lambda function and segment

  1. Create a CSV file to import endpoints with the attributes of AppointmentDate and AppointmentTime.
    Example:
    Id,Address,ChannelType,Attributes.AppointmentDate,Attributes.AppointmentTime
    1,+1[PHONE_NUMBER],SMS,November 30 2020,9:00am
    2,+1[PHONE_NUMBER2],SMS,November 30 2020,10:00am
  2. Navigate to the Amazon Pinpoint console.
  3. In the All Projects list, select your project.
  4. In the navigation pane, choose Segments.
  5. Choose Create a Segment.
  6. Choose Import a segment and upload your CSV file and choose Create segment.
  7. In the navigation pane, choose Campaigns.
  8. Choose Create campaign.
  9. In the Create a campaign wizard, enter a name for campaign name.
  10. Under Channel choose Custom.
  11. Choose Next.
  12. On the Choose a segment screen, choose the segment created above, and choose Next.
  13. On the Create your message screen, do the following:
    a) For Lambda function choose AmazonPinpointConnectChannel that we deployed in Step 3 above.
    b) For endpoint Options choose SMS.
    c) Choose Next.
  14. On the Choose when to send the campaign screen, do the following:
    a) Choose When an event occurs.
    b) Under Events, choose the AppointmentReminder event.
    c) Under campaign dates, choose a Start date and time and an End date and time to be used as the campaign’s duration.
  15. Choose Next.
  16. Review the campaign details and choose Launch campaign.

Cleanup:

To remove the two AWS Lambda functions and the Amazon Simple Queue Service queue provisioned in the steps above in order not to incur further charges, please follow these steps below.

  1. Navigate to the Amazon CloudFormation Console.
  2. Choose severlessrepo-amazon-pinpoint-connect-channel and choose Delete.
  3. Choose Delete stack in the delete confirmation window.

 

Next Steps:

You can continue to iterate on this experience using Amazon Pinpoint and Amazon Connect to create a custom user experience.

To learn more about these services, please visit the Amazon Pinpoint or Amazon Connect web pages.

(1) https://www.scisolutions.com/uploads/news/Missed-Appts-Cost-HMT-Article-042617.pdf

(2) https://blog.carbonfreedining.org/the-ultimate-guide-to-restaurant-no-shows

Auto-reply to incoming emails using Amazon Simple Email Service (SES)

Post Syndicated from Ilya Pupko original https://aws.amazon.com/blogs/messaging-and-targeting/auto-reply-to-incoming-emails-using-amazon-simple-email-service-ses/

Both Amazon Pinpoint and Amazon Simple Email Service (SES) are known for their ability to send out transactional and promotional emails at scale and with ease. However, both are often not set up to receive email replies. Owners often assume that the “no-reply” addresses they are using do not require much consideration. This means that if a customer does reply, they would get an unhelpful server rejection indicating that the address is invalid. They would also not be able to unsubscribe via the simple reply, which is an otherwise established common practice. Automated guidance that the address is not monitored and who and how to reach for assistance would never be provided. In summary, a very unprofessional experience.

If you do have full control over the DNS and are not already receiving emails at the subdomain used for these emails, you can follow this short guide. It walks you through all the setup needed to have automated and templated responses to any address at the domain. This includes the address you use to send emails. Follow this post to ensure that your Amazon SES and Amazon Pinpoint are set up in accordance with common configuration and best business practices to have professional auto-reply to emails sent to the configured sending email addresses.

Solution overview

The proposed solution does not rely on any additional services. It does not add any additional charges beyond the cost directly associated with receiving and sending the emails and the minimal AWS Lambda function for the automated logic. It relies on SES built-in capability to receive emails, Amazon Pinpoint native templates, and uses Lambda for basic orchestration.

lambda diagram for response

Note, in this walkthrough and related code, we are using Amazon Pinpoint templates as they can be managed and maintained directly via the console, but you can choose to use SES templates (via the CreateTemplate API) or, if it makes better sense in your scenario, even just hardcode the template into the AWS Lambda function itself.

To complete the setup, all you must do is follow these steps:

      1. Confirm (Sub-) Domain setup in SES (even if you use Amazon Pinpoint to send your emails out, the SES portion of the console should show the validated domain as well). See SES Developer Guide.
      2. Ensure that your SES domain is verified and you are out of the sandbox. If still in the sandbox, you can only send emails to the Amazon SES mailbox simulator addresses and email addresses/domains that you have pre-verified. See Moving out of the Amazon SES sandbox.
      3. Configure SES to receive incoming emails. Please note that this must be done on the whole subdomain you use, not just a single email address. See Setting up Amazon SES email receiving.
      4. Create/add a new template you want to use via Amazon Pinpoint. Simply switch the console over to Amazon Pinpoint, select Message templates, click Create, select Email, and fill out the rest of the self-explanatory field.
        1. Plaintext portion is optional – you can either skip it or fill it out and enable in the Lambda function we are deploying in the next step.
        2. Similarly, if you prefer to use the SES template, you can instead. Just use the associated line in that same code.
        3. Same with a hardcoded template, if you prefer that for some reason.
      5. Have this pre-defined CloudFormation create the required SES receive rule, and Lambda function. This processes the incoming email and sends back the response, all using the code shared in the dedicated portion of our GitHub, AWS Digital User Engagement Reference Architectures repository. Specifically:
          1. Download the YAML from SES_Auto_Reply.yaml.
          2. Go to CloudFormation in AWS Management Console. (Remember to choose the region you want it deployed on)
          3. Click Create Stack and then choose With new resources
          4. Leave the default “Template is ready“, but switch to ”Upload a template file“ and choose the file you just downloaded
          5. Follow the wizard to give the “stack” a new name and enter the name of the template you created in step 4.
          6. Optionally you can also set the default response address, the addresses and/or domains you want to limit the auto-response to, and adjust the incoming email rule-set it should be stored under (the default should be fine, unless you have manually adjusted it in the past)
      6. Once deployed, the behavior is immediately active and you can further adjust any of these elements.

 

Conclusion and what’s next?

This architecture, once deployed, sends out the templated auto-response using the SES/Pinpoint domain/email address it received the original email on.

The new rule is added to the SES email receiving rule set to allow further customization:

  1. The rule can be limited to specific email address, specific domain, or just be set to be across all domains.
  2. It can also have the default response address set or reuse the address that the original rejected email was sent to.
  3. It can be moved down on the priority with other rules taking precedence and possibly even overriding it.
  4. It can have other actions added to it, like notifying SNS for additional tracking.

The Lambda function looks up the chosen Amazon Pinpoint template and uses it to reply. Here are some of the customizations you may want to consider within this function and the template:

  1. When sending the automated reply, by default, the template’s configured subject is appended with the original incoming email subject. You can adjust this to fit your company’s brand better.
  2. By default, the function supports an optional template tag %%NAME%% and %%ID%%. If the first appears in the template, it is automatically replaced with the original email’s FROM address. And if %%ID%% appears in the template, it is replaced with the SES’s original email message id, to help with any required audits.
  3. It is assumed that no additional tracking and actions are needed on such rejected and auto-replied emails, but you can further modify the flow by moving the rule around and adding more actions (as mentioned above), and even specify a particular/different SES Configuration Set for the outgoing emails.

Are you using this flow as a baseline for a more complex business flow or have other questions about it? We want to hear back – please comment here or file an issue in the GitHub repository. If you want to file a pull request to make it even more useful for others, please do so, we do appreciate community participation.

If you liked this article, we are continually expanding our Amazon Pinpoint and SES Architecture References and publish new solutions for these and other services. For most recent SES documentation, please see official SES documentation site, and for Amazon Pinpoint, please see Amazon Pinpoint documentation site.

 

 

 

Application integration patterns for microservices: Running distributed RFQs

Post Syndicated from James Beswick original https://aws.amazon.com/blogs/compute/application-integration-patterns-running-distributed-rfqs/

This post is courtesy of Dirk Fröhner, Principal Solutions Architect.

The first blog in this series introduces asynchronous messaging for building loosely coupled systems that can scale, operate, and evolve individually. It considers messaging as a communications model for microservices architectures. Part 2 dives into fan-out strategies and applies the respective patterns to a concrete use case.

In this post, I look at how to apply messaging patterns to help coordinate distributed requests and responses. Specifically, I focus on a composite pattern called scatter-gather, as presented in the book “Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions” (Hohpe and Woolf, 2004).

I also show how a client can communicate with a backend via synchronous REST API operations while asynchronous messaging is applied internally for processing.

Overview

The use case is for Wild Rydes, a fictional application that replaces traditional taxis with unicorns. It’s used in several hands-on AWS workshops that illustrate serverless development concepts.

Wild Rydes wants to allow customers to initiate requests for quotation (RFQs) for their rides. This allows unicorns to make special offers to potential customers within a defined schedule. A customer can send their ride details and ask for quotations from all unicorns that are within a certain vicinity. The customer can then choose the best offer.

Wild Rydes

The scatter-gather pattern

The scatter-gather pattern can be used to implement this use-case on the server side. This pattern is ideal for requesting responses from multiple parties, then aggregating and processing that data.

As presented by Hohpe and Woolf, the scatter-gather pattern is a composite pattern that illustrates how to “broadcast a message to multiple recipients and re-aggregate the responses back into a single message”. The pattern is illustrated in the following diagram.

Scatter-gather architecture

The flow starts with the Requester to initiate the broadcast to all potential Responders. This can be architected in a loosely coupled manner using pub-sub messaging with Amazon SNS or Amazon MQ, as shown in this blog post.

All responders must send their answers somewhere for aggregation and processing. This can also be architected in a loosely coupled manner using a message queue with Amazon SQS or Amazon MQ, as described in this blog post.

The Aggregator component consumes the individual responses from the response queue. It forwards the aggregate to the Processor component for final processing. Both Aggregator and Processor can be part of the same application or process. If separated, they can be decoupled through messaging. The Requester can also be part of the same application or process as Aggregator and Processor.

Explaining the architecture and API

In this section, I walk through the use-case and explain how it can be architected and implemented. I show how the scatter-gather pattern works in the backend, and the client-to-backend communication.

Submit instant ride RFQ

To initiate such an RFQ, the customer app communicates with the ride booking service on the backend. The ride booking service exposes a REST API. By default, an RFQ runs for five minutes, but Wild Rydes is working on a feature to let a customer individually set that value.

A request to submit an instant-ride RFQ contains start and destination locations for the ride and the customer ID:

POST /<submit-instant-ride-rfq-resource-path> HTTP/1.1
...

{
    "from": "...",
    "to": "...",
    "customer": "..."
}

The RFQ is a lengthy process so the client app should not expect an immediate response. Instead, the API accepts the RFQ, creates an RFQ task resource, and returns to the client. The response contains a URL to request an update for the status. It also provides an estimated time for the end of the RFQ:

HTTP/1.1 202 Accepted
...

{
    "links": {
        "self": "http://.../<rfq-task-resource-path>",
        "...": "..."
    },
    "status": "running",
    "eta": "..."
}

The following architecture shows this interaction, excluding the process after a new RFQ is submitted.

Client app interaction

Processing the RFQ

The backend uses the scatter-gather pattern to publish the RFQ to unicorns and collect responses for aggregation and processing.

Backend architecture

1. The ride booking service acts as the requester in the scatter-gather pattern. Following a new RFQ from the client app, it publishes the details into an SNS topic. This topic is related to the location of the ride’s starting point since customers need quotes from unicorns within the vicinity. These messages are the green request messages.

2. The unicorn management service maintains instances of unicorn management resources and subscribes them to RFQ topics related to their current location. These resources receive the RFQ request messages and handle the interaction with the Wild Rydes unicorn app.

3. The unicorns in the vicinity are notified through the Wild Rydes unicorn app about the new RFQ and can react if they are available. Notification options between the unicorn management service and the Wild Rydes unicorn app include push notifications and web sockets.

4. Every addressed unicorn can now submit their quote. All quotes go back through the unicorn management resources and the unicorn management service into the RFQ response queue. They act as the responders in the sense of the scatter-gather pattern.

5. The ride booking service also acts as aggregator and processor in the sense of the scatter-gather pattern. It uses SQS to consume messages from an RFQ response queue that eventually contains the RFQ responses from the involved unicorns. It starts doing so immediately after it publishes the details of a new RFQ into the RFQ topic. The messages from the RFQ response queue relate to the blue response messages.

The ride booking service consumes all incoming responses from that queue. This continues until the deadline or all participating unicorns have answered, whatever occurs first. The aggregator responsibility can be as simple as persisting the details of each incoming RFQ response into an Amazon DynamoDB table.

To match incoming responses to the right RFQ, it uses a fundamental integration pattern, correlation ID. In this pattern, a requester adds a unique ID to an outgoing message and each responder is asked to forward this ID in their response.

Also, responders must know where to send their responses to. To keep this dynamic, there is another fundamental integration pattern: return address. It suggests that a requester adds meta information into outgoing messages that indicate the address for their responses. In this architecture, this is the ARN of the SQS queue that acts as the RFQ response queue. This supports an option to simplify the response management: the RFQ response queue is a dedicated queue per customer.

Lastly, the processor responsibility in the ride booking service reads the RFQ responses from the DynamoDB table. It converts the data to JSON for the Wild Rydes customer app.

Check RFQ status

During the RFQ processing, a customer may want to know how many responses have already arrived, or if the results are already available. After submitting an instant ride RFQ, the client receives a representation of the running task. It can use the self-link to request an update:

GET /<rfq-task-resource-path> HTTP/1.1

While the task is running, a response from the ride booking service comes back with the respective status value and the count of responses that have already arrived:

HTTP/1.1 200 OK
...

{
    "links": {
        "self": "http://.../<rfq-task-resource-path>",
        "...": "..."
    },
    "status": "running",
    "responses-received": 2,
    "eta": "..."
}

After the RFQ is completed

An RFQ is completed if either the time is up or all unicorns have answered. The result of the RFQ is then available to the customer. If the client requests an update to the task representation, the response indicates this by redirecting to the RFQ result:

HTTP/1.1 303 See Other
Location: <url-of-rfq-result-resource>

Requesting a representation of the results resource, the client receives the quotes of all the participating unicorns. The frontend customer app can visualize these accordingly:

HTTP/1.1 200 OK
...

{
    "links": { ... },
    "from": "...",
    "to": "...",
    "customer": "...",
    "quotes": [ ... ]
}

The ride booking service can also use means of active notifications to make the customer app aware once the RFQ result is ready, including the link to the RFQ result. Examples for this include push notifications and web sockets.

Conclusion

In this blog, I present the scatter-gather pattern, which is a composite pattern based on pub-sub and point-to-point messaging channels. It also employs correlation ID and return address. I show how this is implemented in the Wild Rydes example application. You can use this integration pattern for communication in your microservices.

I cover how synchronous API communication between end user client and backend can work along with asynchronous messaging for request processing internally.

To learn more:

For more serverless learning resources, visit https://serverlessland.com.

Introducing Amazon SNS FIFO – First-In-First-Out Pub/Sub Messaging

Post Syndicated from Danilo Poccia original https://aws.amazon.com/blogs/aws/introducing-amazon-sns-fifo-first-in-first-out-pub-sub-messaging/

When designing a distributed software architecture, it is important to define how services exchange information. For example, the use of asynchronous communication decouples components and simplifies scaling, reducing the impact of changes and making it easier to release new features.

The two most common forms of asynchronous service-to-service communication are message queues and publish/subscribe messaging:

  • With message queues, messages are stored on the queue until they are processed and deleted by a consumer. On AWS, Amazon Simple Queue Service (SQS) provides a fully managed message queuing service with no administrative overhead.
  • With pub/sub messaging, a message published to a topic is delivered to all subscribers to the topic. On AWS, Amazon Simple Notification Service (SNS) is a fully managed pub/sub messaging service that enables message delivery to a large number of subscribers. Each subscriber can also set a filter policy to receive only the messages that it cares about.

You can use topics when you want to fan out messages to multiple applications, and queues when you want to send messages to one application. Using topics and queues together, you can decouple microservices, distributed systems, and serverless applications.

With SQS, you can use FIFO (First-In-First-Out) queues to preserve the order in which messages are sent and received, and to avoid that a message is processed more than once.

Introducing SNS FIFO Topics
Today, we are adding similar capabilities for pub/sub messaging with the introduction of SNS FIFO topics, providing strict message ordering and deduplicated message delivery to one or more subscribers.

FIFO topics manage ordering and deduplication similar to FIFO queues:

Ordering – You configure a message group by including a message group ID when publishing a message to a FIFO topic. For each message group ID, all messages are sent and delivered in order of their arrival. For example, to ensure the delivery of messages related to the same customer in order, you can publish these messages to the topic using the customer’s account number as the message group ID. There is no limit in the number of message groups with FIFO topics and queues. You don’t need to declare in advance the message group ID, any value will work. If you don’t have a logical distinction between messages, you can simply use the same message group ID for all and have a single group of ordered messages. The message group ID is passed to any subscribed FIFO queue.

Deduplication – Distributed systems (like SNS) and client applications sometimes generate duplicate messages. You can avoid duplicated message deliveries from the topic in two ways: either by enabling content-based deduplication on the topic, or by adding a deduplication ID to the messages that you publish. With message content-based deduplication, SNS uses a SHA-256 hash to generate the message deduplication ID using the body of the message. After a message with a specific deduplication ID is published successfully, there is a 5-minute interval during which any message with the same deduplication ID is accepted but not delivered. If you subscribe a FIFO queue to a FIFO topic, the deduplication ID is passed to the queue and it is used by SQS to avoid duplicate messages being received.

You can use FIFO topics and queues together to simplify the implementation of applications where the order of operations and events is critical, or when you cannot tolerate duplicates. For example, to process financial operations and inventory updates, or to asynchronously apply commands that you receive from a client device. FIFO queues can use message filtering in FIFO topics to selectively receive only a subset of messages rather than every message published to the topic.

How to Use SNS FIFO Topics
A common scenario where FIFO topics can help is when you receive updates that need to be processed in order. For example, I can use a FIFO topic to receive updates from an application where my customers edit their account profiles. Then, I subscribe an SQS FIFO queue to the FIFO topic, and use the queue as trigger for a Lambda function that applies the account updates to an Amazon DynamoDB table used by my Customer management system that needs to be kept in sync.

The decoupling introduced by the FIFO topic makes it easier to add new functionality with minimal impact to existing applications. For example, to reward my loyal customers with additional promotions, I add a new Loyalty application that is storing information in a relational database managed by Amazon Aurora. To keep the customer’s information stored in the Loyalty database in sync with my other applications, I can subscribe a new FIFO queue to the same FIFO topic, and add a new Lambda function that receives customer updates in the same order as they are generated, and applies them to the Loyalty database. In this way, I don’t need to change code and configuration of other applications to integrate the new Loyalty app.

First, I create two FIFO queues in the SQS console, leaving all options to their defaults:

  • The customer.fifo queue to process updates in my Customer management system.
  • The loyalty.fifo queue to help me collect and store customer updates for the Loyalty application.

In the SNS console, I create the updates.fifo topic. I select FIFO as type, and enable Content-based message deduplication.

Then,  I subscribe the customer.fifo and loyalty.fifo queues to the topic.

To be able to receive messages, I add a statement to the access policy of both queues granting the updates.fifo topic permissions to send messages to the queues. For example, for the customer.fifo queue the statement is:

{
  "Effect": "Allow",
  "Principal": {
    "Service": "sns.amazonaws.com"
  },
  "Action": "SQS:SendMessage",
  "Resource": "arn:aws:sqs:us-east-2:123412341234:customer.fifo",
  "Condition": {
    "ArnLike": {
      "aws:SourceArn": "arn:aws:sns:us-east-2:123412341234:updates.fifo"
    }
  }
}

Now, I use the SNS console to publish 4 messages in sequence. For all messages, I use the same message group ID. In this way, they are all in the same message group. The only part that is different is the message body, where I use in order:

  • Update One
  • Update Two
  • Update Three
  • Update One

In the SQS console, I see that only 3 messages have been delivered to the FIFO queues:

Why is that? When I created the FIFO topics, I enabled content-based deduplication. The 4 messages were sent within the 5-minute deduplication window. The last message has been recognized as a duplicate of the first one and has not been delivered to the subscribed queues.

Let’s see the actual messages in the queues. I use the AWS Command Line Interface (CLI) to receive the messages from SQS, and the jq command-line JSON processor to format the output and get only the Message in the Body.

Here are the messages in the customer.fifo queue:

$ aws sqs receive-message --queue-url https://sqs.us-east-2.amazonaws.com/123412341234/customer.fifo --max-number-of-messages 10 | jq '.Messages[].Body | fromjson | .Message'

"Update One"
"Update Two"
"Update Three"

And these are the messages in the loyalty.fifo queue:

$ aws sqs receive-message --queue-url https://sqs.us-east-2.amazonaws.com/123412341234/loyalty.fifo --max-number-of-messages 10 | jq '.Messages[].Body | fromjson | .Message'

"Update One"
"Update Two"
"Update Three"

As expected, the 3 messages with unique content have been delivered to both queues in the same order as they were sent.

Available Now
You can use SNS FIFO topics in all commercial regions. You can process up to 300 transactions per second (TPS) per FIFO topic or FIFO queue. With SNS, you pay only for what you use, you can find more information in the pricing page.

To learn more, please see the documentation.

Danilo