All posts by Pavlos Ioannou Katidis

How Amazon SES Mail Manager Elevates Email Security and Efficiency

Post Syndicated from Pavlos Ioannou Katidis original https://aws.amazon.com/blogs/messaging-and-targeting/how-amazon-ses-mail-manager-elevates-email-security-and-efficiency/

In today’s digital landscape, efficient and secure email management is essential for businesses facing the complexities of cyber threats and regulatory compliance. Companies are seeking ways to safeguard against unauthorized access and apply audit rules, while maintaining operational efficiency. Amazon SES Mail Manager is designed to meet these challenges, offering a suite of features that enhance both inbound and outbound email flows.

Mail Manager provides key components such as traffic policies for detailed email filtering, authenticated ingress endpoints that ensure emails are received only from verified senders, and customizable rule sets that enable administrators to precisely manage email traffic. These tools aim to bolster security and streamline the email management process.

The blog explores Mail Manager’s capabilities by demonstrating how each component works and can be utilized in practical business scenarios. Some common use cases include security, where Mail Manager blocks harmful emails based on IP ranges, TLS versions, and authentication checks while leveraging third-party security add-ons. Another use case is email archiving, where you can use Mail Manager to set up multiple archives with customizable retention periods and encryption, ensuring compliance and easy searchability.

Familiarize with some of mail manager’s key components below before proceeding with the customer use cases.

Mail manager components definition:

  • Ingress endpoints:
    • Open ingress endpoint: a SMTP endpoint responsible for accepting connections, and process SMTP conversation key infrastructure. It’s a key component that utilizes traffic polices and rules that you can configure to determine which emails should be allowed into your organization and which ones should be rejected.
    • Authenticated ingress endpoint: Mail sent to your domain has to come from authorized senders whom you’ve shared your SMTP credentials with, such as your on-premise email servers.
  • Traffic policies: let you determine the email you want to allow or block from your ingress endpoint. A traffic policy consists of one or more policy statements where you allow or deny traffic based on a variety of protocols including recipient address, sender IP address and TLS protocol version.
  • Rules sets: A Rule set is a container for an ordered set of rules you create to perform actions on your email. Each rule consists of conditions and rules.
  • Email add-ons: A suite of 3rd party applications that are seamlessly integrated with Amazon SES mail manager. Some of them are Trend Micro Virus Scanning, Abusix Mail intelligence and Spamhaus Domain Block List.

For a deep dive into Mail Manager’s capabilities, ready this blog.

Customer background and use case

Nutrition.co is an online retail business with multiple departments, including marketing, tech, and sales, that send and receive emails. Nutrition.co is looking for a solution to monitor both outbound and inbound emails and apply various controls such as filtering, message processing, and archiving. Nutrition.co uses Outlook as an enterprise mailbox environment for its employees.

Use case 1: Nutrition.co to the world

This use case focuses on the outbound email flow, where Nutrition.co employees are sending emails outside of Nutrition.co. Some of the requirements include the archival of all outbound emails originated by the marketing department, blocking any tech emails exceeding 1mb and scanning the email content of emails originated by sales. These controls should be centrally managed and provide flexibility to edit/create/delete new ones.

Solution: Each department will direct its outbound emails to an authenticated ingress endpoint by configuring an Exchange transport rule. These endpoints ensure that only authorized senders with SMTP credentials can send emails. Each ingress endpoint generates an A record, which is added as an MX record to the DNS provider for each department’s subdomain. Additionally, each ingress endpoint is associated with a specific traffic policy and rule set. According to Nutrition.co’s requirements, all connections between the departments and the ingress endpoints must use TLS 1.3 or higher. Emails that comply with the traffic policies are processed through distinct rule sets. Emails from marketing that comply with DKIM and SPF are first archived and then sent to the recipient via the Send to Internet action. Tech emails have their recipient’s address rewritten to a test email address, while emails from the sales department undergo content scanning before being sent to the final recipients via the Send to Internet action.

SES-Mail-Manager-Outbound

Use case 2: World to Nutrition.co

This use case focuses on the inbound email flow, where third parties send emails to Nutrition.co. Nutrition.co requires inbound emails to align with SPF and DKIM and have TLS 1.3 or higher to be archived. Emails originating from warehouse.com, Nutrition.co’s fulfilment partner, are containing customer order updates. These emails should be processed by Nutrition.co and accordingly update the customers’ order status database. Furthermore, warehouse.com emails should originate from a certain IP range, have TLS 1.3 or higher and align with SPF and DKIM.

Solution: Nutrition.co will use an open ingress endpoint without authentication for all inbound external emails. This is achieved by adding an MX record generated by Mail Manager upon the creation of the ingress endpoint. This ingress endpoint will be associated with a traffic policy that evaluates TLS. If the inbound email conforms to the traffic policy, it will proceed through the rule set condition and actions. The rule set condition is to align with SPF and DKIM and the actions are to be archived and then sent to the final recipient (Nutrition.co employee) via SMTP Relay. Emails containing parcel delivery updates from warehouse.com will be directed to a separate Nutrition.co subdomain, which routes all inbound emails to an authenticated ingress endpoint. Emails from warehouse.com with TLS 1.3 or higher will meet the traffic policy requirements. If they are SPF and DKIM aligned, they will be stored in a Nutrition.co Amazon S3 bucket as part of the rule set. Using Amazon S3 notifications, an AWS Lambda function is invoked upon receiving an email. This function processes the email payload, and performs an API call to update the Nutrition.co customers’ order status database.

SES-Mail-Manager-Inbound

Archiving inbound emails

In the following section, you will use AWS CloudShell and AWS CLI commands to create a traffic policy that rejects emails with TLS versions lower than 1.3, includes an open ingress endpoint, and establishes a ruleset that archives emails that are DKIM aligned.

Prerequisites: Own a domain and have access to its DNS provider, in order to add the MX record.

Navigate to the AWS Management Console and open CloudShell, find CloudShell availability here. Follow the steps below by copying and pasting the AWS CLI commands to the CloudShell terminal. Note that creating and configuring these resources, can also be done from the AWS Console.

# 1. Creating archive

ARCHIVE=$(aws mailmanager create-archive \
  --archive-name NutritionCo \
  --retention RetentionPeriod=THREE_MONTHS \
  --region ${AWS_REGION} \
  --tags Key=Company,Value=NutritionCo | jq -r '.ArchiveId') && echo $ARCHIVE

# 2. Creating traffic policy

TRAFFIC_POLICY=$(aws mailmanager --region ${AWS_REGION} create-traffic-policy \
  --traffic-policy-name ArchiveTrafficPolicy \
  --default-action DENY \
  --policy-statements '[
    {
      "Action": "ALLOW",
      "Conditions": [
        {
          "TlsExpression": {
            "Evaluate": {
              "Attribute": "TLS_PROTOCOL"
            },
            "Operator": "MINIMUM_TLS_VERSION",
            "Value": "TLS1_3"
          }
        }
      ]
    }
  ]'| jq -r '.TrafficPolicyId') && echo $TRAFFIC_POLICY

# 3. Creating Mailmanager RuleSet for archiving

RULE_SET=$(aws mailmanager --region ${AWS_REGION} create-rule-set \
  --rule-set-name ArchiveRuleSet \
  --rules '[
    {
      "Name": "Archive",
      "Actions": [
        {
          "Archive": {
            "TargetArchive": "'"${ARCHIVE}"'"
          }
        }
      ],
      "Conditions": [
        {
          "VerdictExpression": {
            "Evaluate": {
              "Attribute": "DKIM"
            },
            "Operator": "EQUALS",
            "Values": ["PASS"]
          }
        }
      ]
    }
  ]'| jq -r '.RuleSetId') && echo $RULE_SET

# 4. Create ingress endpoint

aws mailmanager --region ${AWS_REGION} create-ingress-point \
--ingress-point-name Archiving \
--type OPEN \
--traffic-policy-id ${TRAFFIC_POLICY} \
--rule-set-id ${RULE_SET}

To view the resources created above, navigate to the Amazon SES console > Mail Manager and view Traffic policies and Rule sets. Below, you can see the rule in edit mode.

Mail-Manager-RulesetNavigate to Amazon SES > Mail Manager > Ingress endpoint, select the ingress endpoint named Archiving and copy the ARecord, which looks like this <unique-id>.fips.wmjb.mail-manager-smtp.amazonaws.com – see screenshot below. Add this value to your MX record.

Mail-Manager-IngressEndpoint

To test if the MX record has been added successfully, open your local terminal and execute the command below:
nslookup -type=MX <your-domain.com>
The response should return the MX preference and mail exchanger containing the A record value.

Testing

To test if the inbound emails are archived successfully, send an email to an address within the domain for which you have added the MX record. Wait for 3-5 minutes to allow for email processing. Then, navigate to the AWS Management Console, go to Amazon SES, and select Mail Manager. Under Email Archiving, select NutritionCo under Archive and click on Search. This should return all the emails you have sent.

MailManager-Archive

Conclusion & Next steps

In this blog, we delved into the essential features of Amazon SES Mail Manager and its application in managing both inbound and outbound email flows. We explored key components such as traffic policies, authenticated ingress endpoints, and customizable rule sets that enhance security and operational efficiency. Through practical use cases, this blog demonstrates how these features can be implemented to meet the specific needs of a business like Nutrition.co. By leveraging Amazon SES Mail Manager, businesses can significantly enhance their email security and management processes, safeguarding against cyber threats while ensuring compliance and efficiency.

Continue exploring Mail Manager’s features such as SMTP relays and Email add-ons.

About the Authors

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis is an Amazon Pinpoint and Amazon Simple Email Service Senior Specialist Solutions Architect at AWS. He enjoys diving deep into customers’ technical issues and help in designing communication solutions. In his spare time, he enjoys playing tennis, watching crime TV series, playing FPS PC games, and coding personal projects.

Alexey Kiselev

Alexey Kiselev

Alexey Kiselev is a Senior SDE working on Amazon Email. Alexey has played a pivotal role in shaping the design, infrastructure, and delivery of MailManager. With years of experience, deep understanding of the industry and a passion for innovation he is enthusiast and a builder with a core area of interest on scalable and cost-effective email management and email security solutions.

Using one-click unsubscribe with Amazon SES

Post Syndicated from Pavlos Ioannou Katidis original https://aws.amazon.com/blogs/messaging-and-targeting/using-one-click-unsubscribe-with-amazon-ses/

Gmail and Yahoo have announced new requirements for bulk senders that take effect in February 2024. The requirements aim to reduce delivery of malicious or unwanted email to the users of these mailbox providers. We recommend that Amazon SES senders who operate outside of the SES sandbox assume these bulk sender requirements apply to them.

Gmail’s FAQ and Yahoo’s FAQ both clarify that the one-click unsubscribe requirement will not be enforced until June 2024 as long as the bulk sender has a functional unsubscribe link clearly visible in the footer of each message.

This blog presents a reference architecture for Amazon SES senders who independently manage email subscriptions outside of Amazon SES. Alternatively, Amazon SES senders can employ our native subscription management capability as part of their compliance with the Gmail and Yahoo bulk sender requirements.  Note that the scope of Gmail and Yahoo’s bulk sender requirements extends beyond enabling an easy unsubscribe method.  Read our blogs on email authentication and managing spam complaints for more information that will help you successfully operate as a bulk sender with Amazon SES.

Email headers contain metadata that describes the content, sender, relay path, destination, and other elements of an email. The bulk sender easy subscription requirement references use of the List-Unsubscribe email header (RFC2369) and List-Unsubscribe-Post email header (RFC8058). The order of the headers should be first the List-Unsubscribe followed by the List-Unsubscribe-Post.

  • List-Unsubscribe: <https://nutrition.co/?address=x&topic=x>, <mailto:unsubscribe@ nutrition.co?subject=TopicUnsubscribe>
  • List-Unsubscribe-Post: List-Unsubscribe=One-Click

These headers enable email clients and inbox providers to display an unsubscribe link at the top of the email if they support it. This could take the form of a menu item, push button, or another user interface element to simplify the user experience – see the Gmail client screenshot below.

gmail-inbox

Unsubscribing can take place from the email footer by clicking on a hyperlink, and/or from an unsubscribe link that mailbox providers render. These different unsubscribe methods can be custom-built or provided by Amazon SES.

  • Unsubscribe method footer: An unsubscribe link in the email footer, which redirects recipients to a landing page, where they can unsubscribe or edit their communication preferences.
  • Unsubscribe method header: A hyperlink that is rendered by the mailbox provider based on the List-Unsubscribe email header. Recipients can use this link to unsubscribe from that sender.
  • Amazon SES unsubscribe method: The Amazon SES subscription management feature, which provides subscription management via the List-Unsubscribe header and ListManagementOptions footer links.
  • Custom-built unsubscribe method: A custom-built unsubscribe link in the email footer and manually added List-Unsubscribe header.

The table below lists all unsubscribe method combinations, indicating if they are custom-built or provided by Amazon SES and whether they comply with the easy unsubscription requirement from Google and Yahoo.

Unsubscribe method Amazon SES or custom-built Complies with Gmail & Yahoo
Footer & Header Amazon SES Yes
Footer & Header Custom Yes
Header Custom Yes
Footer Custom Partial

Failing to comply with the easy unsubscription requirement mailbox providers such as Gmail and Yahoo will start rejecting non-compliant emails.

Note: Gmail might not show the easy unsubscribe link. This might happen because Gmail shows the link if they trust that the sender is honoring the unsubscribe requests and not attempting to track recipients. We recommend senders continue to provide the unsubscribe link in an easy to find location of the body of the message.

Implementing the unsubscribe header has many benefits for you:

  • Reduces spam complaint rate: Email recipients will click on “Report as SPAM” if they find it difficult to unsubscribe. A high spam complaint rate makes mailbox providers more likely to block your sending. Making unsubscribe easier can improve deliverability.
  • It can increase the trust in your brand: The fact that it is easy for recipients to unsubscribe could be seen as evidence that the content is valuable enough that the company believes people will want to stay subscribed.
  • Reduces issues with false suppression: Senders that rely solely on account-level suppression lists could suppress all email sending to an address even though the recipient may wish to receive other types of email from the account. Offering an easy unsubscribe method allows recipients to indicate which type of email they would like to receive and not receive based on topic or category.

There are two types of list-unsubscribe options:

  • Mailto: unsubscribe requests come in the form of an email sent from the mailbox provider to the email address specified on the List-Unsubscribe header. The process of managing unsubscribe emails can be automated with SES inbound.
  • URL unsubscribe link: redirects recipients to an unsubscribe landing page, from where they can edit further their communication preferences. Adding the List-Unsubscribe-Post email header, senders can provide recipients with one-click unsubscribe experience, which doesn’t require them to visit a landing page.

The mailto option is supported by many mailbox providers and it’s recommended to include it in addition to the URL in the List-Unsubscribe email header and the unsubscribe link in the email footer.

One-click unsubscribe for Amazon SES

This section guides you on how to use Amazon SES V2 SendEmail API operation for email sending and describes how to use other AWS services to effectively manage each kind of unsubscribe request.

The architecture covers both easy unsubscribe options, mailto and URL. This is because not all mailbox providers support the List-Unsubscribe-Post header. The architecture, assumes that Amazon SES has email receiving enabled for the unsubscribe email address used in the List-Unsubscribe mailto header and your recipient preferences can be updated via an API.

The reference architecture diagram illustrates the AWS services used and how they interact with each other to process a recipient’s unsubscribe request:

  • AWS KMS: is a managed service that makes it easy for you to create and control the cryptographic keys that are used to protect your data.
  • Amazon API Gateway: Is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.
  • AWS Lambda: Compute service that runs your code in response to events and automatically manages the compute resources.

The first part of the process is described in detail below:

email-sending-flow

  1. Compliant emails should include the List-Unsubscribe and List-Unsubscribe-Post headers. This can be achieved with the Amazon SES SendEmail V2 API operation. Using MIME standard, build a MIME message containing the headers, subject and body. The MIME message will be in the SES V2 SendEmail API request body under Content => Raw field – see code example below. Amazon SES is planning to extend the SendEmail V2 API to natively support unsubscribe email headers. The unsubscribe email address and URL contain the recipient’s email address and email subject parameters, which are encrypted using AWS Key Management Service. These parameters are used later on to identify and unsubscribe the recipient from a specific topic.
    1. The email domain used to send emails needs to be first verified successfully – see here how to create and verify identities in SES.
    2. Gmail uses the Friendly From value to populate the unsubscribe pop-up message. Friendly From is the part of the From header that is displayed to the recipient (not the email address) “To stop getting messages like this one, go to the <Friendly From> website to unsubscribe. Learn more.”. If you see Unknown or experience other issues, ensure that the From header of your messages conforms to RFC5322.
      
      	msg = MIMEMultipart()
      	msg.add_header('List-Unsubscribe','<https://nutrition.co/?address=x&topic=x>, <mailto: [email protected]?subject=TopicUnsubscribe>')
      	msg.add_header('List-Unsubscribe-Post','List-Unsubscribe=One-Click')
      	msg.attach(MIMEText("Welcome to Nutrition.co", 'plain')) 
      	msg['Subject'] = "Welcome to Nutrition.co"
      
      	response = sesv2.send_email(
      	  FromEmailAddress='Nutrition.co <[email protected]>',
      	  Destination={'ToAddresses': ['[email protected]']},
      	  Content={
      		  'Raw': {
      			  'Data': msg.as_string()
      		  },
      	  },
      	  ConfigurationSetName='ConfigSet'
      	)
    3. Amazon Pinpoint senders need to use Custom channel instead of Amazon Pinpoint’s native email channel. Custom channel gives the flexibility to invoke an AWS Lambda function and execute custom code such as calling Amazon Pinpoint’s send_messages API operation. Using Amazon Pinpoint’s send_messages API operation you can specify an endpoint as the recipient and add the email content and the List-Unsubscribe and List-Unsubscribe-Post headers in a MIME message under the RawEmail => Data field – see below a code example:
      	msg = MIMEMultipart()
      	msg.add_header('List-Unsubscribe','<https://nutrition.co/?address=x&topic=x>, <mailto: [email protected]?subject=TopicUnsubscribe>')
      	msg.add_header('List-Unsubscribe-Post','List-Unsubscribe=One-Click')
      	msg.attach(MIMEText("Welcome to Nutrition.co", 'plain')) 
      	msg['Subject'] = "Welcome to Nutrition.co"
      
      	endpoint_id = "endpoint_id"
      	application_id = "application_id"
      
      	response = pinpoint.send_messages(
      	ApplicationId = application_id,
      	MessageRequest = {
      		'Endpoints': {
      			endpoint_id: {}
      		},
      		'MessageConfiguration': {
      		'EmailMessage': {
      			'FromAddress': 'Nutrition.co <[email protected]>',
      			'RawEmail': {
      				'Data': msg.as_string()
      			}
      		}
      	  }
      	})
  2. The email recipients whose mailbox provider supports List-Unsubscribe, such as Gmail & Yahoo, will see an Unsubscribe hyperlink next to the sender details as shown in the screenshot below.

gmail-inbox

So far, we have talked about how to craft and employ the headers for presenting mail recipients with an easy unsubscribe option.  In the following sections, we’ll walk through the two options for sending the unsubscribe request back to the sender.

The first option uses only the List-Unsubscribe header and only specifies the mailto email address to receive unsubscribe requests. The second option uses both the List-Unsubscribe and the List-Unsubscribe-Post headers. The unsubscribe requests are made with a POST API call to an endpoint provided in the List-Unsubscribe header.

When the recipient clicks on the Unsubscribe call to action next to the sender’s information, a pop-up appears asking for final confirmation using either option – see screenshot below.

unsubscribe-pop-up

Scenario – List-Unsubscribe

list-unsubscribe-scenario

  1. The recipient clicks on the Unsubscribe call to action next to the sender’s details and again on Unsubscribe on the pop-up.
  2. The mailbox provider sends an email to the email address specified in the header List-Unsubscribe => mailto. Amazon SES can be configured to receive emails for the unsubscribe email address, the Amazon SES receipt rule Invoke Lambda function action.
  3. An AWS Lambda function gets invoked. The payload contains all email headers and omits the email body as well as any attachments. The AWS Lambda function uses the AWS KMS key to decrypt the email subject, which contains the topic the recipient wants to unsubscribe from. Depending where your recipient preferences are stored, you can expand the AWS Lambda function code to update the recipients’ communication preferences.

Scenario – List-Unsubscribe & List-Unsubscribe-Post

list-unsubscribe-post-scenario

  1. The recipient clicks on the Unsubscribe call to action next to the sender’s details and again on Unsubscribe on the pop-up.
  2. The mailbox provider performs a POST API call to the URL provided in the List-Unsubscribe header. In this architecture, the URL is an Amazon API Gateway endpoint with an AWS Lambda integration.
  3. An AWS Lambda function gets invoked, which uses the AWS KMS key to decrypt the email address and topic stored in the URL parameters. Depending where your recipient preferences are stored, you can expand the AWS Lambda function code to update the recipients’ communication preferences. The code in the AWS Lambda function serves two purposes 1) processing a POST request to unsubscribe the recipient and 2) processing a GET request to redirect the recipient to page on your website (Gmail specific). Use a micro web framework like Flask to process unsubscribe requests and accordingly redirect recipients to a page of your website.

In Gmail, to view the Go to website call to action, recipients need to first Unsubscribe and then and then click on Unsubscribe again – see diagram below.

unsubscribe-flow-gmail

Conclusion

In this blog you learned how to configure Amazon SES to manage One-click unsubscribe requests when not using SES’s subscription management feature. The reference architecture shows how to structure and add the List-Unsubscribe and List-Unsubscribe-Post email headers when sending emails as well as how to manage unsubscribe requests generated from these email headers respectively. In addition to the List-Unsubscribe and List-Unsubscribe-Post email headers, we recommend (continue) using the footer unsubscribe link.

Easy unsubscribe benefits both the sender and recipient. It is one of the Gmail and Yahoo’s bulk sender requirements announced back in October 2023. The one-click unsubscribe requirement will not be enforced until June 2024 as long as the bulk sender has a functional unsubscribe link clearly visible in the footer of each message.

Deploy Amazon QuickSight dashboard for Amazon Pinpoint engagement events.

Post Syndicated from Pavlos Ioannou Katidis original https://aws.amazon.com/blogs/messaging-and-targeting/deploy-amazon-quicksight-dashboard-for-amazon-pinpoint-engagement-events/

Abstract

Business intelligence (BI) dashboards provide a graphical representation of metrics and key performance indicators (KPIs) to monitor the health of your business. By leveraging BI dashboards to analyze the performance of your customer communications, you gain valuable insights into how they are engaging with your messages and can make data-driven decisions to improve your marketing and communication strategies.

In this blog post we introduce a solution that automates the deployment of an Amazon QuickSight dashboard that enables marketers to analyze their long-term Amazon Pinpoint customer engagement data. These dashboards can be customized further depending the use case. This solution alleviates the need to create data pipelines for storage and analysis of Amazon Pinpoint’s engagement data, while offering a greater variety of widgets and views across email, SMS, campaigns, journeys and transactional messages when comparing to Amazon Pinpoint’s native dashboards.

Amazon Pinpoint is a flexible, scalable marketing communications service that connects you with customers over email, SMS, push notifications, or voice. The service offers ready to use dashboards to view key performance indicators (KPIs) for the various messaging channels, It provides 90 days of events for analysis. However, the raw events used to populate Amazon Pinpoint’s dashboards, can be streamed using Amazon Kinesis Data Firehose to a destination of your choice. This blog will walk you through leveraging this feature to create a data lake to store and analyze data beyond the initial 90 days.

Amazon QuickSight is a cloud-scale business intelligence (BI) service that you can use to deliver easy-to-understand insights in an interactive visual environment.

The solutions leverages the Amazon Cloud Development Kit (CDK) to deploy the needed infrastructure and dashboards.

Use Case(s)

The Amazon QuickSight dashboards deployed through this solution are designed to serve several use cases. Here are just a few examples:

  • View email and SMS costs per Campaign and Journey.
  • Deep dive into engagement insights and performance. (eg: SMS events, Email events, Campaign events, Journey events).
  • Schedule reports to various business stakeholders.
  • Track individual email & SMS statuses to specific endpoints.
  • Analyze open and click rates based on the message send time.

These are some of the use cases you can use these dashboards for and with all the data points being available in Amazon QuickSight, you can create your own views and widgets based on your specific requirements.

Solution Overview

This solution builds upon the Digital User Engagement (DUE) Event Database AWS Solution. It creates a long-term Amazon Pinpoint event data lake. This solution also builds a QuickSight dashboard to visualize and analyze this data. It leverages several other AWS services to tie i all together. It uses AWS Lambda for processing AWS CloudTrail data, Amazon Athena to build views using SQL for Amazon QuickSight, AWS CloudTrail to record any new campaign, journey and segment updates and Amazon DynamoDB to store the campaign, journey and segment metadata. This solution can be segmented into three logical portions: 1) Pinpoint campaign/journey/segment lookup tables. 2) Amazon Athena Views. 3) Amazon QuickSight resources.

The AWS Cloud Development Kit (CDK) is used to deploy this solution to your account. AWS CDK is an open-source software development framework for defining cloud infrastructure as code with modern programming languages and deploying it through AWS CloudFormation.

Pinpoint campaign/journey/segment lookup tables

architecture-diagram

  1. A CloudFormation AWS Lambda-backed custom resource function adds current Pinpoint campaign, journey and segment meta data to Amazon DynamoDB lookup tables. An AWS CloudFormation custom resource is managed by a Lambda function that runs only upon the deployment, update and deletion of the AWS CloudFormation stack.
  2. AWS CloudTrail logs record API actions to an S3 bucket every 5 minutes.
  3. When an AWS CloudTrail log is written to the S3 bucket an AWS Lambda function is invoked and checks for Amazon Pinpoint campaigns/journeys/segments management events such as create, update and delete.
  4. For every Amazon Pinpoint action the AWS Lambda function finds, it queries Amazon Pinpoint to get the respective resource details.
  5. The AWS Lambda function will create or update records in the Amazon DynamoDB table to reflect the changes.
  6. This solution also deploys an Amazon Athena DynamoDB connector. Amazon Athena uses this to query the Amazon DynamoDB lookup tables to enrich the data in the Amazon Pinpoint event data lake.
  7. The Amazon Athena to Amazon DynamoDB connector requires an Amazon S3 spill bucket for any data that exceeds the AWS Lambda function limits

Amazon Athena views

Amazon Athena views are crucial for querying and organizing the data. These views allow QuickSight to interact with the Pinpoint event data lake through standard SQL queries and views. Here’s how they’re set up:

The application creates several named queries (called saved queries in the Amazon Athena console). Each named query uses a SQL statement to create a database view containing a subset of the data from the Pinpoint event data lake (or joins data from a previous view with the Amazon DynamoDB tables created above. The views are also created using an AWS Lambda-backed custom resource.

Amazon QuickSight resources

quicksight-resources-diagram

  1. This solution creates several Amazon QuickSight resources to support the deployed dashboard. These include data sources, datasets, refresh schedules, and an analysis. The refresh schedule determines the frequency that Amazon QuickSight queries the Amazon Athena views to update the datasets.
  2. Amazon Athena retrieves live data from the DUE event database data lake and the Athena DynamoDB Connector whenever the Amazon QuickSight refresh schedule runs.

Prerequisites

  • Deploy the Digital User Engagement (DUE) Event Database solution before continuing
    • After you have deployed this solution, gather the following data from the stack’s Resources section.
      • DUES3DataLake: You will need the bucket name
      • PinpointProject: You will need the project Id
      • PinpointEventDatabase: This is the name of the Glue Database. You will only need this if you used something other than the default of due_eventdb

Note: If you are installing the DUE event database for the first time as part of these instructions, your dashboard will not have any data to show until new events start to come in from your Amazon Pinpoint project.

Once you have the DUE event database installed, you are ready to begin your deployment.

Implementation steps

Step 1 – Ensure that Amazon Athena is setup to store query results

Amazon Athena uses workgroups to separate users, teams, applications, or workloads, to set limits on amount of data each query or the entire workgroup can process, and to track costs. There is a default workgroup called “primary” However, before you can use this workgroup, it needs to be configured with an Amazon S3 bucket for storing the query results.

  1. If you do not have an existing S3 bucket you can use for the output, create a new Amazon S3 bucket.
  2. Navigate to the Amazon Athena console and from the menu select workgroups > primary > Edit > Query result configuration
    1. Select the Amazon S3 bucket and any specific directory for the Athena query result location

Note: If you choose to use a workgroup other that the default “primary” workgroup. Please take note of the workgroup name to be used later.

Step 2 – Enable Amazon QuickSight

Amazon QuickSight offers two types of data sets: Direct Query data sets, which provides real-time access to data sources, and SPICE (Super-fast, Parallel, In-memory Calculation Engine) data sets, which are pre-aggregated and cached for faster performance and scalability that can be refreshed on a schedule.

This solution uses SPICE datasets set to incrementally refresh on a cycle of your choice (Daily or Hourly). If you have already setup Amazon QuickSight, please navigate to Amazon QuickSight in the AWS Console and skip to step 3.

  1. Navigate to Amazon QuickSight on the AWS console
  2. Setup Amazon QuickSight account by clicking the “Sign up for QuickSight” button.
    1. You will need to setup an Enterprise account for this solution.
    2. To complete the process for the Amazon QuickSight account setup follow the instructions at this link
  3. Ensure you have the Admin Role
    1. Choose the profile icon in the top right corner, select Manage QuickSight and click on Manage Users
    2. Subscription details should display on the screen.
  4. Ensure you have enough SPICE capacity for the datasets
    1. Choose the profile icon, and then select Manage QuickSight
    2. Click on SPICE Capacity
  5. Make sure you enough SPICE for all three datasets
    1. if you are still in the free tier, you should have enough for initial testing.
    2. You will need about 2GB of capacity for every 1,000,000 Pinpoint events that will be ingested in to SPICE
    3. Note: If you do not have enough SPICE capacity, deployment will fail
  6. Please note the Amazon QuickSight username. You can find this by clicking profile icon. Example username: Admin/user-name

Step 3 – Collect the Amazon QuickSight Service Role name in IAM

For Amazon Athena, Amazon S3, and Athena Query Federation connections, Amazon QuickSight uses the following IAM “consumer” role by default: aws-quicksight-s3-consumers-role-v0

If the “consumer” role is not present, then QuickSight uses the following “service” role instead : aws-quicksight-service-role-v0.

The version number at the end of the role could be different in your account. Please validate your role name with the following steps.

  1. Navigate to the Identity and Access Management (IAM) console
  2. Go to Roles and search QuickSight
  3. If the consumer role exists, please note its full name
  4. If you only find the service role, please note its full name

Note: For more details on these service roles, please see the QuickSight User Guide

Step 4 – Prepare the CDK Application

Deploying this solution requires no previous experience with the AWS CDK toolkit. If you would like to familiarize yourself with CDK, the AWS CDK Workshop is a great place to start.

  1. Setup your integrated development environment (IDE)
    1. Option 1 (recommended for first time CDK users): Use AWS Cloud9 – a cloud-based IDE that lets you write, run, and debug your code with just a browser
      1. Navigate to Cloud9 in the AWS console and click the Create Environment button
      2. Provide a descriptive name to your environment (e.g. PinpointAnalysis)
      3. Leave the rest of the values as their default values and click Create
      4. Open the Cloud9 IDE
        1. Node, TypeScript, and CDK should be come pre-installed. Test this by running the following commands in your terminal.
          1. node --version
          2. tsc --version
          3. cdk --version
          4. If dependencies are not installed, follow the Step 1 instructions from this article
        2. Using AWS Cloud 9 will incur a nominal charge if you are no longer Free Tier eligible. However, using AWS Cloud9 will simply setup if you do not already have a local environment with AWS CDK and the AWS CLI installed
    2. Option 2: local IDE such as VS Code
      1. Setup CDK locally using this documentation
      2. Install Node, TypeScript and the AWS CLI
        1. Once the CLI is installed, configure your AWS credentials
          1. aws configure
  2. Clone the Pinpoint Dashboard Solution from your terminal by running the command below:
    1. git clone https://github.com/aws-samples/digital-user-engagement-events-dashboards.git
  3. Install the required npm packages from package.json by running the commands below:
    1. cd digital-user-engagement-events-dashboards
    2. npm install

Open the file at digital-user-engagement-events-dashboards/bin/pinpoint-bi-analysis.ts for editing in your IDE.

Edit the following code block your your solution with the information you have gathered in the previous steps. Please reference Table 1 for a description of each editable field.

const resourcePrefix = "pinpoint_analytics_";

...

new MainApp(app, "PinpointAnalytics", {
  env: {
    region: "us-east-1",
  }
  
  //Attributes to change
  dueDbBucketName: "{bucket-name}",
  pinpointProjectId: "{pinpoint-project-id}",
  qsUserName: "{quicksight-username}",

  //Default settings
  athenaWorkGroupName: "primary",
  dataLakeDbName: "due_eventdb",
  dateRangeNumberOfMonths: 6,
  qsUserRegion: "us-east-1", 
  qsDefaultServiceRole: "aws-quicksight-service-role-v0", 
  spiceRefreshInterval: "HOURLY",

  //Constants
  athena_util: athena_util,
  qs_util: qs_util,
});
Attribute Definition Example
resourcePrefix The prefix for all created Athena and QuickSight resources pinpoint_analytics_
region Where new resources will be deployed. This must be the same region that the DUE event database solution was deployed us-east-1
dueDbBucketName The name of the DUE event database S3 Bucket due-database-xxxxxxxxxxus-east-1
qsUserName The name of your QuickSight User Admin/my-user
athenaWorkGroupName The Athena workgroup that was previously configured primary
dataLakeDbName The Glue database created during the DUE event database solution. By default the database name is “due_eventdb” due_eventdb
dateRangeNumberOfMonths The number of months of data the Athena views will contain. QuickSight SPICE datasets will contain this many months of data initially and on full refresh. The QuickSight dataset will add new data incrementally without deleting historical data. 6
qsUserRegion The region where your quicksight user exists. By default, new users will be created in us-east-1. You can check your user location with the AWS CLI: aws quicksight list-users --aws-account-id {accout-id} --namespace default and look for the region in the arn us-east-1
qsDefaultServiceRole The service role collected during Step 3. aws-quicksight-service-role-v0
spiceRefreshInterval Options Include HOURLY, DAILY – This is how often the SPICE 7-day incremental window will be refreshed DAILY

Step 5 – Deploy

  1. CDK requires you to bootstrap in each region of an account. This creates a S3 bucket for deployment. You only need to bootstrap once per account/region
    1. cdk bootstrap
  2. Deploy the application
    1. cdk deploy

Step 6 – Explore

Once your solution deploys, look for the Outputs provided by the CDK CLI. You will find a link to your new Amazon Quicksight Analysis, or Dashboard, as well as a few other key resources. Also, explore the resources sections of the deployed stacks in AWS CloudFormation for a complete list of deployed resources. In the AWS CloudFormation, you should have two stacks. The main stack will be called PinpointAnalytics and a nested stack.

Pricing

The total cost to run this solution will depend on several factors. To help explore what the costs might look like for you, please look at the following examples.

All costs outlined below will assume the following:

  • 1 Amazon QuickSight author
  • 100 Amazon QuickSight analysis reader sessions
  • 100k write API actions for all services in AWS account
  • A total of 1k Amazon Pinpoint campaigns, journeys, and segments resulting in 1k Amazon DynamoDB records
  • 5 million monthly Amazon Pinpoint events – email send, email delivered, etc.

Base Costs:

  • 1 Amazon QuickSight author – can edit all Amazon QuickSight resources
    • $24 – There is a a 30 day trial for 4 authors in the free tier
  • 100 Amazon QuickSight analysis reader sessions OR 6 readers with unlimited access – max $5 per month per reader
    • $30
  • Total Monthly Costs: $54 / month

Variable Costs:

Even with the assumptions listed above, the costs will vary depending on the chosen data retention window as well as the the refresh schedule.

  • SPICE data storage costs.
    • Total size of storage will depend on how many months you choose to display in the dashboard
    • For the above assumptions, the SPICE datasets will cost roughly $3.25 for each month stored in the datasets.
  • Amazon Athena data volume costs
    • With Athena you are charged for the total number of bytes scanned in a query. The solution implements incremental data resfreshes in SPICE. Amazon QuickSight will only query and updates the most recent 7 days of data during each refresh cycle. This can be adjusted as needed.

Scenario 1 – 6-month data analysis with daily refresh:

  • Fixed costs: $57
  • SPICE datasets: $19.50
  • Athena Scans: $1.25
  • Total Costs: $77.75 / Month

Scenario 2 – 12-month data analysis with daily refresh:

  • Fixed costs: $57
  • SPICE datasets: $39
  • Athena Scans: $1.25
  • Total Costs: $97.25 / Month

Scenario 3 – 12-month data analysis with hourly refresh:

  • Fixed costs: $57
  • SPICE datasets: $39
  • Athena Scans: $27.50
  • Total Costs: $123.5 / Month

Note: Several services were not mentioned in the above scenarios (e.g., DynamoDB, Cloudtrail, Lambda, etc). The limited usage of these services resulted in a combined cost of less than a few US dollars per month. Even at a greater scale, the costs from these services will not increase in any significant way.

Clean up

  • Delete the CDK stack running the following from your command line
    • cdk destroy
  • Delete QuickSight account
  • Delete Athena views
    • Go to Glue > Data Catalog > Databases > Your Database Name
    • This should delete all Athena views no longer needed. Views created will start with the resourcePrefix specified in the bin/athena-quicksight-cdk.ts file
  • Delete S3 buckets
    • DynamoDB cloud watch log bucket
    • Dynamo Athena Connector Spill bucket
    • Athena workgroup output bucket
  • Delete DynamoDB tables
    • This solution creates two DynamoDB lookup tables prefixed with the Stack name

Conclusion

In this blog, you have deployed a solution that visualizes Amazon Pinpoint’s email and SMS engagement data using Amazon QuickSight. This solution provides you with an Amazon QuickSight functional dashboard as well as a foundation to design and build new Amazon QuickSight dashboards that meet your bespoke requirements. Parts of the solution, such as the Amazon Athena views, can be ingested with other business intelligence tools that your business might already be using.

Next steps

This solution can be expanded to include Amazon Pinpoint engagement events from other channels such as push notifications, Amazon Connect outbound calls, in-app and custom events. This will require certain updates on the Amazon Athena views and consequently on the Amazon QuickSight dashboards. Furthermore, the Amazon DynamoDB tables store only campaign, journey and segment meta-data. You can extend this part of the solution to include message template meta-data, which will help to analyze performance per message template.

Considerations / Troubleshooting

  • Pinpoint Standard account can be upgraded to an Enterprise account. Enterprise accounts cannot be downgraded to a Standard account.
  • SPICE capacity is allocated separately for each AWS Region. Default SPICE capacity is automatically allocated to your home AWS Region. For each AWS account, SPICE capacity is shared by all the people using QuickSight in a single AWS Region. The other AWS Regions have no SPICE capacity unless you choose to purchase some.
  • The QuickSight Analysis Event rates are calculated on Pinpoint message_id and endpoint_id grain – click rate will be the same if a user clicks an email link one or more than one times
  • All timestamps are in UTC. To display data in another timezone edit event_timestamp_timezone calculated field in every dataset
  • Data inside Amazon QuickSight will refresh depending on the schedule set during deployment. Current options include hourly and daily refreshes.
  • AWS CloudTrail has 5 cloudtrail trails per AWS account.

About the Authors

Spencer Harrison

Spencer Harrison

Spencer was a 2023 WWPS Solution Architect intern at Amazon Web Services. He will graduate with his Masters of Information Systems Management from Brigham Young University in the spring of 2024. After graduation he is aspiring to find opportunities as a solution architect, cloud engineer, or DevOps engineer. Outside of work, Spencer loves going outdoors to wake surf, downhill ski, and play pickle ball.

Daniel Wells

Daniel Wells

With over 20 years of IT experience, Daniel has held many architecture and director positions supporting a wide variety of technologies. He currently works as an AWS Solutions Architect supporting Education Technology companies striving to make a difference for learners and educators worldwide. Daniel’s interests outside of work include music, family, health, education and anything that allows him to express himself creatively.

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis is an Amazon Pinpoint and Amazon Simple Email Service Senior Specialist Solutions Architect at AWS. He enjoys diving deep into customers’ technical issues and help in designing communication solutions. In his spare time, he enjoys playing tennis, watching crime TV series, playing FPS PC games, and coding personal projects.

Send WhatsApp messages via Amazon Pinpoint

Post Syndicated from Pavlos Ioannou Katidis original https://aws.amazon.com/blogs/messaging-and-targeting/send-whatsapp-messages-via-amazon-pinpoint/

In this blog you will deploy a solution that integrates Amazon Pinpoint with WhatsApp for outbound and inbound messages.

Amazon Pinpoint is a multichannel customer engagement platform allowing you to engage with your customers across 6 different channels (push notifications, email, SMS, voice, in-app messages and custom channel). Using Amazon Pinpoint’s custom channel you can extend its capabilities via a webhook or AWS Lambda function. Among many other possibilities, you can use custom channels to send messages to your customers through any API-enabled service, for example WhatsApp.

According to statista, WhatsApp is one of the most used apps in the world and the most popular messaging app in over 100 countries. It reached 2.3 billion active users in 2022 while in January 2022, WhatsApp was the most downloaded chat and messaging app worldwide, amassing approximately 40.6 million downloads across the Apple App Store and the Google Play Store.

Note: WhatsApp is a third-party service subject to additional terms and charges. Amazon Web Services isn’t responsible for any third-party service that you use to send messages with custom channels.

Solution & Architecture

An integration between Amazon Pinpoint and WhatsApp can be achieved for both outbound and inbound messages. The next section dives deeper into the architecture for both outbound and inbound messages. The solution uses Amazon Pinpoint custom channel, AWS Lambda, Amazon API Gateway, AWS Cloudformation and AWS Secrets Manager.

Outbound messages

For outbound messages Amazon Pinpoint integrates with WhatsApp via its custom channel allowing users to send WhatsApp messages using Pinpoint campaigns and journeys. Specifically, Pinpoint invokes an AWS Lambda function and performs an API call to WhatsApp. The API call contains the WhatsApp access token, the customer’s mobile number and the WhatsApp message template name.

outbound-message

  1. Amazon Pinpoint campaign or journey using endpoint type CUSTOM invokes an AWS Lambda function. The payload along with the endpoint data should contain the WhatsApp message template name as part of the Custom Data field.
  2. The AWS Lambda obtains the WhatsApp access token from the AWS Secrets Manager and performs a POST API call to the WhatsApp API.
  3. The WhatsApp message gets delivered to the customer.

Inbound messages

For inbound messages WhatsApp requires a Callback URL. This solution utilizes Amazon API Gateway to create the Callback URL and AWS Lambda to authorize and process inbound messages.

inbound-message

  1. Customer sends a message to your WhatsApp number.
  2. WhatsApp makes a GET API call to the Amazon API Gateway endpoint for verification purposes. All subsequent calls containing the customers’ messages are POST.
  3. If the API call method is GET, the AWS Lambda checks if the verify token matches the one stored as an AWS Lambda Environment Variable. If it’s TRUE, it returns a code called HubChallenge that WhatsApp is expecting in order to verify the connection. For POST API calls, the AWS Lambda loops through the customer messages and retrieves the customer’s phone number, timestamp, message_id and message_body. For each message processed, the AWS Lambda function performs an API call to WhatsApp to mark the message as read.

Considerations

  • Message delivery/engagement events aren’t being recorded.
  • Messages sent aren’t personalized and they are currently using message templates hosted by WhatsApp.
  • It is recommended to use endpoint type CUSTOM and not SMS for the following reasons:
    • WhatsApp’s phone number format doesn’t contain + comparing to Pinpoint SMS address format. If you decide to use the endpoint type SMS you will need to process the endpoint Address by removing the +.
    • Using the endpoint type SMS forces you to send WhatsApp messages with the same throughput (messages per second) as your Pinpoint SMS channel.

Prerequisites

  1. AWS account.
  2. An Amazon Pinpoint project – How to create an Amazon Pinpoint project.
  3. An Amazon Pinpoint CUSTOM endpoint with address a mobile number which is associated to a WhatsApp account. See example CUSTOM endpoint in a CSV here.
  4. A Meta (Facebook) developer account, for more details please go to the Meta for Developers console.

Implementation

Meta for Developers console

  1. Navigate and login into the Meta for Developers console, click My Apps and select Create App (or use an existing app of type Business).
  2. Select Business as an app type, which supports WhatsApp and click Next.
  3. Provide a display name, contact email, choose whether or not to attach Business Account (optional) and select Create App.
  4. Navigate to the Dashboard and select Set Up in the WhatsApp service in the Add product to your app section.
  5. Create or select an existing Meta Business Account and select Continue.
  6. Navigate to WhatsApp/Getting Started and take a note of the Phone number ID, which will be needed in AWS CloudFormation template later on. WhatsAppPhoneNumberId
  7. On the WhatsApp/Getting Started page, add your customer phone number you are going to use for testing in the Select a recipient phone number dropdown. Follow the instructions to add and verify your phone number. Note: You must have WhatsApp registered with the number and the WhatsApp client installed on your mobile device. Verification message could appear in the Archived list in your WhatsApp client and not in the main list of messages.

Create a new user to access WhatsApp via API

  1. Open Meta’s Business Manager and select business you created or associated your app with earlier.
  2. Below Users, select System Users and choose Add to create a new system user.
  3. Give a name to the system user and set their role as Admin and click Create System User.
  4. Use the Add Assets button to associate the new user with your WhatsApp app. From the Select asset type list, select Apps, then in the Select assets, select your WhatsApp app’s name. Enable the Test app Partial access for the user, select Save Changes and Done.
  5. Click on the Generate new token button, select the WhatsApp app created earlier and choose Permanent as Token expiration.
  6. Select whatsapp_business_messaging and whatsapp_business_management from the list of Available Permissions and click Generate token at the bottom.
  7. Copy and save your access token. This will be needed in AWS CloudFormation template later on. Make sure you copied the token before clicking on OK.

For more details on creating the access token, you can navigate to WhatsApp/Configuration and click on Learn how to create a permanent token.

Solution deployment

  1. Download the AWS CloudFormation template and navigate to the AWS CloudFormation console under the AWS region you want to deploy the solution.
  2. Select Create stack and With new resources. Choose Template is ready as Prerequisite – Prepare template and Upload a template file as Specify template. Upload the template downloaded in step 1.
  3. Fill the AWS CloudFormation parameters as shown below:
    1. ApiGatewayName: This is the name of the Amazon API Gateway resource.
    2. PhoneNumberId: This is the WhatsApp phone number Id you obtained from the Meta for Developers console under WhatsApp/Getting Started.
    3. PinpointProjectId: Paste your Amazon Pinpoint’s project Id. This allows Amazon Pinpoint to invoke the AWS Lambda, which sends WhatsApp messages as part of a campaign or journey.
    4. VerifyToken: The verify token is an alphanumeric token that you provide to WhatsApp when setting up the Webhook Callback URL for inbound messages and notifications. You can decide the value of this token e.g. 123abc.
    5. WhatsAppAccessToken: The access token should start with Bearer EEAEAE… and you should have obtained it from the section of this blog Create a new user to access WhatsApp via API.
  4. Once the AWS CloudFormation stack is deployed, copy the Amazon API GateWay endpoint from the AWS CloudFormation outputs tab. Navigate to the Meta for Developers App dashboard, choose Webhooks, select Whatsapp Business Account and subscribe to messages. SubscribeToMessages
  5. Paste the Amazon API Gateway endpoint as a Callback URL. For the Verify token, provide the same value as the AWS CloudFormation template parameter VerfiyToken and select Verify and save. VerifyAndSave

Testing

  • Sending messages: To test sending a message to WhatsApp using Amazon Pinpoint:
    • Navigate to the Amazon Pinpoint Campaigns
    • Create a new Campaign with WhatsAppCampaign as the Campaign name, select Standard campaign as the Campaign type, choose Custom as Channel and select Next.
    • Select a segment that includes the CUSTOM endpoint that you will send the message to
    • Choose the AWS Lambda Function containing the name WhatsAppSendMessageLambda. Under Custom data type hello_world, for Endpoint Options choose Custom and select Next. Note that the hello_world is the WhatsApp default message template.
    • In Step 4 leave everything with the default values, scroll to the bottom of the page and select Next.
    • Choose Launch campaign.
  • Receiving messages: Text or reply to the WhatsApp number. The inbound messages are being printed in the Amazon CloudWatch logs of the AWS Lambda function containing the name WhatsAppWebHookLambda. ReceivedMessage

Next steps

There are several ways to extend this solution’s functionality, see some of them below:

  • Instead of specifying the WhatsApp message template name, provide directly the text you want to send using the Pinpoint’s custom channel Custom data field. To do this, update the AWS Lambda function code responsible for sending messages with the one below:
    import os
    import json
    import boto3
    from urllib import request, parse
    from botocore.exceptions import ClientError
    phone_number_id = os.environ['PHONE_NUMBER_ID']
    secret_name = os.environ['SECRET_NAME']
    def handler(event, context):
        print("Received event: {}".format(event))
        session = boto3.session.Session()
        client = session.client(service_name='secretsmanager')
        try:
            get_secret_value_response = client.get_secret_value(SecretId=secret_name)
        except ClientError as e:
            raise e
        else:
            secret = get_secret_value_response['SecretString']
            url = 'https://graph.facebook.com/v15.0/'+ phone_number_id + '/messages'
            message = event['Data'] # Obtaining the message from the Custom Data field
            for key in event['Endpoints'].keys(): 
                to_number = str(event['Endpoints'][key]['Address'])
                send_message(secret, to_number, url, message_template)
    def send_message(secret, to_number, url, message_template):
        headers = {
            'content-type': 'application/json',
            'Authorization': secret
        }
        # Building the request body and insted of type = template, it's replaced with type = text
        data = parse.urlencode({
            'messaging_product': 'whatsapp',
            'to': to_number,
            'type': 'text',
            'text': {
                'body': message
            }
        }).encode()
        req =  request.Request(url, data=data, headers=headers)
        resp = request.urlopen(req)
  • Use WhatsApp’s message template components to populated dynamically variables. This requires an update on the respective WhatsApp message template and API request body to WhatsApp’s API. The message template should look like this:

PersonalizedMessageTemplate

And the API request body should look like this. Note that the value for each variable should be obtained from the Pinpoint endpoint or user attributes.

{
  "from": from_number,
  "to": to_number,
  "channel": "whatsapp",
  "content": {   
    "contentType": "template",
    "template": {
        "templateId" : "first_pinpoint_message",
        "templateLanguage" : "en",
        "components" : {
            "body" : [
                    {
                        "type": "text",
                        "text": "Pavlos"
                    }
            ]           
        }
   }
  }
}

Clean-up

To delete the solution, navigate to the AWS CloudFormation console and delete the stack deployed.

About the Authors

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis is an Amazon Pinpoint and Amazon Simple Email Service Senior Specialist Solutions Architect at AWS. He enjoys diving deep into customers’ technical issues and help in designing communication solutions. In his spare time, he enjoys playing tennis, watching crime TV series, playing FPS PC games, and coding personal projects.

Push notification engagement metrics tracking

Post Syndicated from Pavlos Ioannou Katidis original https://aws.amazon.com/blogs/messaging-and-targeting/push-notification-engagement-metrics-tracking/

In this blog you will learn how to track and attribute Amazon Pinpoint push notification events for Campaigns and Journeys via API.

Amazon Pinpoint is a multichannel customer engagement platform allowing you to engage with your customers across 6 different channels. Amazon Pinpoint’s push notification channel, can send messages to your mobile app users via Firebase Cloud Messaging (FCM), Apple Push Notification service (APNs), Baidu Cloud Push, Amazon Device Messaging (ADM).

Push notifications is a preferable channel of communication as it notifies your app users even when they are not on your app. This increases app engagement and probability of customers to convert. Additionally, users who download your app but don’t register, can still be targeted and receive your messages.

Using Amazon Pinpoint’s push notification channel you can engage users with highly curated content. The messages can be personalized with customer data stored in Amazon Pinpoint, images, deep links and custom alert sounds – read more here. Amazon Pinpoint Campaigns and Journeys enable marketers to schedule communications, build multichannel experiences and for developers it offers a rich API to send messages. By default, all Amazon Pinpoint accounts are configured to send 25,000 messages per second, which can be increased by requesting a quota increase.

Measuring success of your communications is paramount for optimizing future customer engagements. Amazon Pinpoint push notifications offer the following three events:

  • _opened_notification – This event type indicates that the recipient tapped the notification to open it.
  • _received_foreground – This event type indicates that the recipient received the message as a foreground notification.
  • _received_background – This event type indicates that the recipient received the message as a background notification.

To track the above events from your mobile application, it is recommended using AWS Amplify’s push notification library which is currently available only in React Native.

Solution description

This blog provides an alternative for AWS Amplify for Amazon Pinpoint push notification tracking. Specifically, it utilizes Amazon Pinpoint’s Events API operation, which can be used to record events your customers generate on your mobile or web application. The same API operation can be used to record push notification engagement events.

The Events API operation request body is populated with the Campaign or Journey attributes received via the push notification payload metadata. These attributes help Amazon Pinpoint to attribute the events back to the correct Campaign or Journey

This blog provides examples of campaign, journey & transactional push notification payloads and how to correctly populate the Events API operation. Furthermore it shares an architecture to securely call Amazon Pinpoint’s API from your application’s frontend.

Prerequisites

This post assumes that you already have an Amazon Pinpoint project that is correctly configured to send push notification to your various endpoints using Campaigns or Journeys. Refer to the getting started guide and setting up Amazon Pinpoint mobile push channels for information on how to set up your Amazon Pinpoint project.

You will also need the AWS Mobile SDKs for the respective platform of your apps. The following are the repositories that can be used:

Implementation

The push notification payload received from the application differs between campaign, journey and transactional messages. This blog provides examples for campaign, journey and transactional message payloads as well as how to populate the Amazon Pinpoint Events API request body correctly to report push notification tracking data to Amazon Pinpoint.

Push notification message payload examples:

Campaign payload example:

{
   "pinpoint.openApp":"true",
   "pinpoint.campaign.treatment_id":"0",
   "pinpoint.notification.title":"Message title",
   "pinpoint.notification.body":"Message body",
   "data":"{\"pinpoint\":{\"endpointId\":\"endpoint_id1\",\"userId\":\"user_id1\"}}",
   "pinpoint.campaign.campaign_id":"5befa9dc28b1430cb0469554789e3f99",
   "pinpoint.notification.silentPush":"0",
   "pinpoint.campaign.campaign_activity_id":"613f918c7a4440b69b09c4806d1a9357",
   "receivedAt":"1671009494989",
   "sentAt":"1671009495484"
}

Journey payload example:

{
   "pinpoint.openApp":"true",
   "pinpoint.notification.title":"Message title",
   "pinpoint":{
      "journey":{
         "journey_activity_id":"ibcF4z9lsp",
         "journey_run_id":"5df6dd97f9154cb688afc0b41ab221c3",
         "journey_id":"dc893692ea9848faa76cceef197c5305"
      }
   },
   "pinpoint.notification.body":"Message body",
   "data":"{\"pinpoint\":{\"endpointId\":\"endpoint_id1\",\"userId\":\"user_id1\"}}",
   "pinpoint.notification.silentPush":"0"
}

Transactional payload example:

Note the transactional payload is the same for both messages sent to a push notification token and endpoint-id. Additionally the pinpoint.campaign.campaign_id is always set to _DIRECT.

{
   "pinpoint.openApp":"true",
   "pinpoint.notification.title":"Message title",
   "pinpoint.notification.body":"Message body",
   "pinpoint.campaign.campaign_id":"_DIRECT",
   "pinpoint.notification.silentPush":"0",
   "receivedAt":"1671731433375",
   "sentAt":"1671731433565"
}

Recording push notification events

To record push notification events from your mobile or web application, we will leverage the AWS Mobile SDKs or the Amazon Pinpoint Events API. To prevent inaccurate metrics such as double counting” it is recommended using the appropriate endpoint_id as Pinpoint uses this for de-duplication. Below you can find examples for both Events REST API and put_events AWS Python SDK – Boto3. Visit this page for more information on how to create a signed AWS API request.

Campaign event example – REST API:

Required fields: endpoint_id1, EventType, Timestamp, campaign_id and campaign_activity_id

POST https://pinpoint.us-east-1.amazonaws.com/v1/apps/<Pinpoint-App-id>/events

{
   "BatchItem":{
      "<endpoint_id1>":{
         "Endpoint":{}
       },
      "Events":{
         "<event_id>":{
            "EventType":"_campaign.opened_notification",
            "Timestamp":"2022-12-14T09:50:00.000Z",
            "Attributes":{
               "treatment_id":"0",
               "campaign_id":"5befa9dc28b1430cb0469554789e3f99",
               "campaign_activity_id":"613f918c7a4440b69b09c4806d1a9357"
            }
         }
      }
   }
}

Campaign event example – Python SDK:

Required fields: ApplicationId, endpoint_id, EventType, Timestamp, campaign_id and campaign_activity_id

import boto3 
client = boto3.client("pinpoint")
response = client.put_events(
  ApplicationId = <Pinpoint-App-id>,
  EventsRequest = { 
    "BatchItem": {
      "<event_id>": {
        "Endpoint": {},
        "Events": { 
          "<endpoint_id1>": { 
            "EventType":"_campaign.opened_notification",
            "Timestamp": "2022-12-14T09:50:00.000Z",
            "Attributes": {
              "treatment_id":"0",
              "campaign_id":"5befa9dc28b1430cb0469554789e3f99",
              "campaign_activity_id":"613f918c7a4440b69b09c4806d1a9357"
            }
          }
        }
      }
    }
  }
)
print(response)

Journey event example – REST API:

Required fields: endpoint_id, EventType, Timestamp, journey_id and journey_activity_id

POST https://pinpoint.us-east-1.amazonaws.com/v1/apps/<Pinpoint-App-id>/events

{
   "BatchItem":{
      "<endpoint_id1>":{
         "Endpoint":{}
      },
      "Events":{
         "<event_id>":{
            "EventType":"_journey.opened_notification",
            "Timestamp":"2022-12-14T09:50:00.000Z",
            "Attributes":{
               "journey_id":"5befa9dc28b1430cb0469554789e3f99",
               "journey_activity_id":"613f918c7a4440b69b09c4806d1a9357"
            }
         }
      }
   }
}

Journey event example – Python SDK:

Required fields: ApplicationId, endpoint_id1, EventType, Timestamp, journey_id and journey_activity_id

import boto3 
client = boto3.client("pinpoint")
response = client.put_events(
  ApplicationId = <Pinpoint-App-id>,
  EventsRequest = { 
    "BatchItem": {
      "<endpoint_id1>": {
        "Endpoint": {},
        "Events": { 
          "<event_id>": { 
            "EventType":"_journey.opened_notification",
            "Timestamp": "2022-12-14T09:50:00.000Z",
            "Attributes": {
              "journey_id":"5befa9dc28b1430cb0469554789e3f99",
              "journey_activity_id":"613f918c7a4440b69b09c4806d1a9357"
            }
          }
        }
      }
    }
  }
)
print(response)

Transactional event:

Amazon Pinpoint doesn’t support push notification metrics for transactional messages. Specifically, transactional messages don’t offer a field that can be used to attribute engagement events. These engagement events can still be recorded using the Amazon Pinpoint’s Events API. However, unlike Campaign & Journey events, the transactional push notification message payload doesn’t provide an identifier such as Campaign id or Journey Id that can be used as an Amazon Pinpoint event attribute for data reconciliation purposes.

Next steps

Requests to the Amazon Pinpoint Events API must be signed using AWS Signature version 4. We recommend using the AWS Mobile SDKs which handle request signing on your behalf. You can use the AWS Mobile SDKs with temporary limited-privilege Amazon Cognito credentials. For more information and examples, see Getting credentials.

 

About the Authors

Franklin Ochieng

Franklin Ochieng

Franklin Ochieng is a senior software engineer at the Amazon Pinpoint team. He has attained over 7 years experience at AWS building highly scalable system that solve complex problems for our customers. Outside of work, Frank enjoys getting out in nature and playing basketball or pool.

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis is an Amazon Pinpoint and Amazon Simple Email Service Senior Specialist Solutions Architect at AWS. He enjoys diving deep into customers’ technical issues and help in designing communication solutions. In his spare time, he enjoys playing tennis, watching crime TV series, playing FPS PC games, and coding personal projects.

Shift management using Amazon Pinpoint’s 2-way-SMS

Post Syndicated from Pavlos Ioannou Katidis original https://aws.amazon.com/blogs/messaging-and-targeting/shift-management-using-amazon-pinpoints-2-way-sms/

Businesses with physical presence such as retail, restaurants and airlines need reliable solutions for shift management when demand fluctuates, employees call in sick, or other unforeseen circumstances arise. Their linear dependence on staff forces them to create overtime shifts as a way to cope with demand.

The overtime shift communication between the business and employees needs to be immediate and scalable. Employees in such industries might not have access to internet, which rules out communication channels like email and push notification. Furthermore once the employees receive the available shifts, they need an easy way to book them and if required request further support.

In these situations, businesses need communication methods that are accessible by any mobile device, available without internet connection, and allow for replies. SMS fills all of these requirements and more. This blog showcases how with Amazon Pinpoint SMS channel and other AWS services you can develop an application that communicates available shifts to employees who are interested while allowing them to book by replying.

The solution presented in this blog is for shift management but with small changes it can also communicate available appointments to customers / patients. An example of such use case can be found in health care, where there is a waiting list to see a doctor and patients might cancel the very last moment. The solution can communicate these available slots to all patients in the waiting list and allow them to book via SMS.

Architecture

The solution uses Amazon Pinpoint two way SMS, Amazon DynamoDB, AWS Lambda, Amazon Simple Notification Service (SNS) and Amazon Connect (optional). The next section dives deeper into the architecture diagram and logic flow.

shift_management_architecture

  1. The operator adds an item to the Shift’s campaigns Amazon DynamoDB table. The item consists of a unique key that is used as the Amazon Pinpoint Campaign Id and an attribute Campaign Message, which is used as the SMS message text that the campaign recipients will receive. The Campaign Message needs to include all available shifts that the operator wants to notify the employees about.
    1. Note: This can be done either from the AWS console or programmatically via API.
  2. Amazon DynamoDB streams invokes an AWS Lambda function upon creation of a new Amazon DynamoDB item. The AWS Lambda function uses the Amazon DynamoDB data to create and execute an Amazon Pinpoint SMS Campaign based on an existing customer segment of employees who are interested in overtime shifts. The customer segment is a prerequisite and it includes SMS endpoints of the employees who are interested in receiving shift updates.
  3. Employees who belong in that segment receive an SMS with the available shifts and they can reply to book the ones they are interested in.
  4. Any inbound SMS is published on an Amazon SNS topic.
  5. The 2 way SMS AWS Lambda function subscribes to the Amazon Simple Notification Service and processes all inbound SMS based on their message body.
  6. The Shift’s status Amazon DynamoDB table stores the status of the shifts, which gets updated depending on the inbound SMS.
  7. If the employee requires further assistance, they can trigger an Amazon Connect outbound call via SMS.

The diagram below illustrates the four possible messages an employee can send to the application. To safeguard the application from outsiders and bad actors, the 2 way SMS AWS Lambda function looks up if the senders mobile number is in an allow list. In this solution, the allow list is hardcoded as an AWS Lambda environment variable but it can be stored in a data base like Amazon DynamoDB.

shift management inbound-sms-business-logic

Solution implementation

Prerequisites

To deploy this solution, you must have the following:

  1. An originating identity that supports 2 way SMS in the country you are planning to send SMS to – Supported countries and regions (SMS channel).
  2. A mobile phone to send and receive SMS.
  3. An AWS account.
  4. An Amazon Pinpoint project – How to create an Amazon Pinpoint project.
  5. An SMS customer segment – Download the example CSV, that contains one SMS endpoint. Replace the phone number (column C) with yours and import it to Amazon Pinpoint – How to import an Amazon Pinpoint segment.
  6. Add your mobile number in the Amazon Pinpoint SMS sandbox – Amazon Pinpoint SMS sandbox.
  7. An Amazon Connect instance, number & contact flow if you want your employees to be able to request an agent call back. Download the example Connect contact flow that you can import to your Amazon Connect instance.

Note: UK numbers with a +447 prefix are not allowed by default. Before you can dial these UK mobile numbers, you must submit a service quota increase request. For more information, see Amazon Connect Service Quotas in the Amazon Connect Administrator Guide.

Deploy the solution

  1. Download the CloudFormation template and navigate to the AWS CloudFormation console in the AWS region you want to deploy the solution.
  2. Select Create stack and With new resources. Choose Template is ready as Prerequisite – Prepare template and Upload a template file as Specify template. Upload the template downloaded in step 1.
  3. Fill the AWS CloudFormation parameters as shown below:
    1. ApprovedNumbers: The mobile numbers that are allowed to use this service. The format should be E164 and if there is more than one number separate them by comma e.g. +4457434243,+432434324.
    2. OriginationNumber: The mobile number that you have in your Amazon Pinpoint account in E164 format e.g. +44384238975.
    3. PinpointProjectId: The existing Amazon Pinpoint project Id.
    4. SegmentId: The Amazon Pinpoint existing segment Id that you want to send the SMS notifications to.
    5. ConnectEnable: Select YES if you already have an Amazon Connect instance with a Contact Flow and Queue. If you select NO ignore all the fields below, the solution will still be deployed but employees won’t be able to request a call back.
    6. InstanceId: The Amazon Connect InstanceId. Follow this link to learn how to find your Amazon Connect InstanceId.
    7. ContactFlowID: The Amazon Connect Contact Flow Id that you want this solution to use. Follow this link to learn how to find your Amazon Connect ContactFlow id.
    8. QueueID: The Amazon Connect Queue Id. To obtain the Amazon Connect Queue Id navigate to your Amazon Connect instance > Routing > Queues and it should appear on the browser URL, see example: https://your-instance.awsapps.com/connect/queues/edit?id=0c7fed63-815b-4040-8dbc-255800fca6d7.
    9. SourcePhoneNumber: The Amazon Connect number in E164 format that is connected to the Contact Flow provided in step 7.
  4. Once the solution has been successfully deployed, navigate to the Amazon DynamoDB console and access the ShiftsStatus DynamoDB table. Each item created represents a shift and should have a unique shift_id that employees use to book the shifts, a column shift_status with value = available and a column shift_info where you can put additional information about the shift – see example below:
    {
       "shift_id":{
          "S":"XYZ1234"
       },
       "shift_info":{
          "S":"15/08 5h nightshift"
       },
       "shift_status":{
          "S":"available"
       }
    }

    Pinpoint_shift_status_dynamoDB

  5. Navigate to Amazon Pinpoint console > SMS and voice > Phone numbers, select the phone number that you used as OriginationNumber for this solution and enable Two-way SMS. Under the Incoming messages destination section, select Choose an existing SNS topic and select the one containing the name TwoWaySMSSNSTopic.
  6. Navigate to the Amazon DynamoDB console and access the ShiftsCampaignDynamoDB table. Each item you create represents an Amazon Pinpoint SMS campaign. Create an Amazon DynamoDB item and provide a unique campaign_id, which will be used as the Amazon Pinpoint Campaign name. Create a new attribute (string) with the name campaign_message and type all available shifts that you want to communicate via this campaign. It is important to include the shift id(s) for each of shifts you want your employees to be able to request – see example below.
    • Note: By completing this step, you will trigger an Amazon Pinpoint SMS Campaign. You can access the campaign information and analytics from the Amazon Pinpoint console.
{
  "campaign_id": {
    "S": "campaign_id1"
  },
  "campaign_message": {
    "S": "15/08 5h nightshift XYZ123, 18/08 3h dayshift XYZ124"
  }
}

shift_campaign_dynamoDB

Testing the solution

  • Make sure you have created the shifts in the ShiftsStatusDynamoDB Amazon DynamoDB table.
  • To test the SMS Campaign, replicate step 6 under Deploy the solution section.
  • Reply to the SMS received with the options below:
    1. Send a shift_id that doesn’t exist to receive an automatic response “This is not a valid shift code, please reply by typing REQUEST to view the available shifts”.
    2. Send a valid & available shift_id to book the shift and then check the ShiftsStatusDynamoDB Amazon DynamoDB table, where the shift_status should change to taken and there should be a new column employee with the mobile number of the employee who has requested it.
    3. Send REQUEST to receive all shifts with shift_status = available.
    4. If you have deployed the solution along with Amazon Connect, send AGENT and await for the call.

Next steps

This solution can be extended to support SMS sending to multiple countries by acquiring the respective originating identities. Using Amazon Pinpoint phone number validate service API the application can identify the country for each recipient and choose the correct originating identity accordingly.

Depending your business requirements you might want to change the agent call back option to chat via SMS. You can extend this solution to connect the agent via SMS chat by following the steps in this blog.

Clean-up

To delete the solution, navigate to the AWS CloudFormation console and delete the stack deployed.

About the Authors

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis is an Amazon Pinpoint and Amazon Simple Email Service Senior Specialist Solutions Architect at AWS. He loves diving deep into his customer’s technical issues and help them design communication solutions. In his spare time, he enjoys playing tennis, watching crime TV series, playing FPS PC games, and coding personal projects.

Target your customers with ML based on their interest in a product or product attribute.

Post Syndicated from Pavlos Ioannou Katidis original https://aws.amazon.com/blogs/messaging-and-targeting/use-machine-learning-to-target-your-customers-based-on-their-interest-in-a-product-or-product-attribute/

Customer segmentation allows marketers to better tailor their efforts to specific subgroups of their audience. Businesses who employ customer segmentation can create and communicate targeted marketing messages that resonate with specific customer groups. Segmentation increases the likelihood that customers will engage with the brand, and reduces the potential for communications fatigue—that is, the disengagement of customers who feel like they’re receiving too many messages that don’t apply to them. For example, if your business wants to launch an email campaign about business suits, the target audience should only include people who wear suits.

This blog presents a solution that uses Amazon Personalize to generate highly personalized Amazon Pinpoint customer segments. Using Amazon Pinpoint, you can send messages to those customer segments via campaigns and journeys.

Personalizing Pinpoint segments

Marketers first need to understand their customers by collecting customer data such as key characteristics, transactional data, and behavioral data. This data helps to form buyer personas, understand how they spend their money, and what type of information they’re interested in receiving.

You can create two types of customer segments in Amazon Pinpoint: imported and dynamic. With both types of segments, you need to perform customer data analysis and identify behavioral patterns. After you identify the segment characteristics, you can build a dynamic segment that includes the appropriate criteria. You can learn more about dynamic and imported segments in the Amazon Pinpoint User Guide.

Businesses selling their products and services online could benefit from segments based on known customer preferences, such as product category, color, or delivery options. Marketers who want to promote a new product or inform customers about a sale on a product category can use these segments to launch Amazon Pinpoint campaigns and journeys, increasing the probability that customers will complete a purchase.

Building targeted segments requires you to obtain historical customer transactional data, and then invest time and resources to analyze it. This is where the use of machine learning can save time and improve the accuracy.

Amazon Personalize is a fully managed machine learning service, which requires no prior ML knowledge to operate. It offers ready to use models for segment creation as well as product recommendations, called recipes. Using Amazon Personalize USER_SEGMENTATION recipes, you can generate segments based on a product ID or a product attribute.

About this solution

The solution is based on the following reference architectures:

Both of these architectures are deployed as nested stacks along the main application to showcase how contextual segmentation can be implemented by integrating Amazon Personalize with Amazon Pinpoint.

High level architecture

Architecture Diagram

Once training data and training configuration are uploaded to the Personalize data bucket (1) an AWS Step Function state machine is executed (2). This state machine implements a training workflow to provision all required resources within Amazon Personalize. It trains a recommendation model (3a) based on the Item-Attribute-Affinity recipe. Once the solution is created, the workflow creates a batch segment job to get user segments (3b). The job configuration focuses on providing segments of users that are interested in action genre movies

{ "itemAttributes": "ITEMS.genres = \"Action\"" }

When the batch segment job finishes, the result is uploaded to Amazon S3 (3c). The training workflow state machine publishes Amazon Personalize state changes on a custom event bus (4). An Amazon Event Bridge rule listens on events describing that a batch segment job has finished (5). Once this event is put on the event bus, a batch segment postprocessing workflow is executed as AWS Step Function state machine (6). This workflow reads and transforms the segment job output from Amazon Personalize (7) into a CSV file that can be imported as static segment into Amazon Pinpoint (8). The CSV file contains only the Amazon Pinpoint endpoint-ids that refer to the corresponding users from the Amazon Personalize recommendation segment, in the following format:

Id
hcbmnng30rbzf7wiqn48zhzzcu4
tujqyruqu2pdfqgmcgkv4ux7qlu
keul5pov8xggc0nh9sxorldmlxc
lcuxhxpqh/ytkitku2zynrqb2ce

The mechanism to resolve an Amazon Pinpoint endpoint id relies on the user id that is set in Amazon Personalize to be also referenced in each endpoint within Amazon Pinpoint using the user ID attribute.

State machine for getting Amazon Pinpoint endpoints

The workflow ensures that the segment file has a unique filename so that the segments within Amazon Pinpoint can be identified independently. Once the segment CSV file is uploaded to S3 (7), the segment import workflow creates a new imported segment within Amazon Pinpoint (8).

Datasets

The solution uses an artificially generated movies’ dataset called Bingewatch for demonstration purposes. The data is pre-processed to make it usable in the context of Amazon Personalize and Amazon Pinpoint. The pre-processed data consists of the following:

  • Interactions’ metadata created out of the Bingewatch ratings.csv
  • Items’ metadata created out of the Bingewatch movies.csv
  • users’ metadata created out of the Bingewatch ratings.csv, enriched with invented data about e-mail address and age
  • Amazon Pinpoint endpoint data

Interactions’ dataset

The interaction dataset describes movie ratings from Bingewatch users. Each row describes a single rating by a user identified by a user id.

The EVENT_VALUE describes the actual rating from 1.0 to 5.0 and the EVENT_TYPE specifies that the rating resulted because a user watched this movie at the given TIMESTAMP, as shown in the following example:

USER_ID,ITEM_ID,EVENT_VALUE,EVENT_TYPE,TIMESTAMP
1,1,4.0,Watch,964982703 
2,3,4.0,Watch,964981247
3,6,4.0,Watch,964982224
...

Items’ dataset

The item dataset describes each available movie using a TITLE, RELEASE_YEAR, CREATION_TIMESTAMP and a pipe concatenated list of GENRES, as shown in the following example:

ITEM_ID,TITLE,RELEASE_YEAR,CREATION_TIMESTAMP,GENRES
1,Toy Story,1995,788918400,Adventure|Animation|Children|Comedy|Fantasy
2,Jumanji,1995,788918400,Adventure|Children|Fantasy
3,Grumpier Old Men,1995,788918400,Comedy|Romance
...

Users’ dataset

The users dataset contains all known users identified by a USER_ID. This dataset contains artificially generated metadata that describe the users’ GENDER and AGE, as shown in the following example:

USER_ID,GENDER,E_MAIL,AGE
1,Female,[email protected],21
2,Female,[email protected],35
3,Male,[email protected],37
4,Female,[email protected],47
5,Agender,[email protected],50
...

Amazon Pinpoint endpoints

To map Amazon Pinpoint endpoints to users in Amazon Personalize, it is important to have a consisted user identifier. The mechanism to resolve an Amazon Pinpoint endpoint id relies that the user id in Amazon Personalize is also referenced in each endpoint within Amazon Pinpoint using the userId attribute, as shown in the following example:

User.UserId,ChannelType,User.UserAttributes.Gender,Address,User.UserAttributes.Age
1,EMAIL,Female,[email protected],21
2,EMAIL,Female,[email protected],35
3,EMAIL,Male,[email protected],37
4,EMAIL,Female,[email protected],47
5,EMAIL,Agender,[email protected],50
...

Solution implementation

Prerequisites

To deploy this solution, you must have the following:

Note: This solution creates an Amazon Pinpoint project with the name personalize. If you want to deploy this solution on an existing Amazon Pinpoint project, you will need to perform changes in the YAML template.

Deploy the solution

Step 1: Deploy the SAM solution

Clone the GitHub repository to your local machine (how to clone a GitHub repository). Navigate to the GitHub repository location in your local machine using SAM CLI and execute the command below:

sam deploy --stack-name contextual-targeting --guided

Fill the fields below as displayed. Change the AWS Region to the AWS Region of your preference, where Amazon Pinpoint and Amazon Personalize are available. The Parameter Email is used from Amazon Simple Notification Service (SNS) to send you an email notification when the Amazon Personalize job is completed.

Configuring SAM deploy
======================
        Looking for config file [samconfig.toml] :  Not found
        Setting default arguments for 'sam deploy'     =========================================
        Stack Name [sam-app]: contextual-targeting
        AWS Region [us-east-1]: eu-west-1
        Parameter Email []: [email protected]
        Parameter PEVersion [v1.2.0]:
        Parameter SegmentImportPrefix [pinpoint/]:
        #Shows you resources changes to be deployed and require a 'Y' to initiate deploy
        Confirm changes before deploy [y/N]:
        #SAM needs permission to be able to create roles to connect to the resources in your template
        Allow SAM CLI IAM role creation [Y/n]:
        #Preserves the state of previously provisioned resources when an operation fails
        Disable rollback [y/N]:
        Save arguments to configuration file [Y/n]:
        SAM configuration file [samconfig.toml]:
        SAM configuration environment [default]:
        Looking for resources needed for deployment:
        Creating the required resources...
        [...]
        Successfully created/updated stack - contextual-targeting in eu-west-1
======================

Step 2: Import the initial segment to Amazon Pinpoint

We will import some initial and artificially generated endpoints into Amazon Pinpoint.

Execute the command below to your AWS CLI in your local machine.

The command below is compatible with Linux:

SEGMENT_IMPORT_BUCKET=$(aws cloudformation describe-stacks --stack-name contextual-targeting --query 'Stacks[0].Outputs[?OutputKey==`SegmentImportBucket`].OutputValue' --output text)
aws s3 sync ./data/pinpoint s3://$SEGMENT_IMPORT_BUCKET/pinpoint

For Windows PowerShell use the command below:

$SEGMENT_IMPORT_BUCKET = (aws cloudformation describe-stacks --stack-name contextual-targeting --query 'Stacks[0].Outputs[?OutputKey==`SegmentImportBucket`].OutputValue' --output text)
aws s3 sync ./data/pinpoint s3://$SEGMENT_IMPORT_BUCKET/pinpoint

Step 3: Upload training data and configuration for Amazon Personalize

Now we are ready to train our initial recommendation model. This solution provides you with dummy training data as well as a training and inference configuration, which needs to be uploaded into the Amazon Personalize S3 bucket. Training the model can take between 45 and 60 minutes.

Execute the command below to your AWS CLI in your local machine.

The command below is compatible with Linux:

PERSONALIZE_BUCKET=$(aws cloudformation describe-stacks --stack-name contextual-targeting --query 'Stacks[0].Outputs[?OutputKey==`PersonalizeBucketName`].OutputValue' --output text)
aws s3 sync ./data/personalize s3://$PERSONALIZE_BUCKET

For Windows PowerShell use the command below:

$PERSONALIZE_BUCKET = (aws cloudformation describe-stacks --stack-name contextual-targeting --query 'Stacks[0].Outputs[?OutputKey==`PersonalizeBucketName`].OutputValue' --output text)
aws s3 sync ./data/personalize s3://$PERSONALIZE_BUCKET

Step 4: Review the inferred segments from Amazon Personalize

Once the training workflow is completed, you should receive an email on the email address you provided when deploying the stack. The email should look like the one in the screenshot below:

SNS notification for Amazon Personalize job

Navigate to the Amazon Pinpoint Console > Your Project > Segments and you should see two imported segments. One named endpoints.csv that contains all imported endpoints from Step 2. And then a segment named ITEMSgenresAction_<date>-<time>.csv that contains the ids of endpoints that are interested in action movies inferred by Amazon Personalize

Amazon Pinpoint segments created by the solution

You can engage with Amazon Pinpoint customer segments via Campaigns and Journeys. For more information on how to create and execute Amazon Pinpoint Campaigns and Journeys visit the workshop Building Customer Experiences with Amazon Pinpoint.

Next steps

Contextual targeting is not bound to a single channel, like in this solution email. You can extend the batch-segmentation-postprocessing workflow to fit your engagement and targeting requirements.

For example, you could implement several branches based on the referenced endpoint channel types and create Amazon Pinpoint customer segments that can be engaged via Push Notifications, SMS, Voice Outbound and In-App.

Clean-up

To delete the solution, run the following command in the AWS CLI.

The command below is compatible with Linux:

SEGMENT_IMPORT_BUCKET=$(aws cloudformation describe-stacks --stack-name contextual-targeting --query 'Stacks[0].Outputs[?OutputKey==`SegmentImportBucket`].OutputValue' --output text)
PERSONALIZE_BUCKET=$(aws cloudformation describe-stacks --stack-name contextual-targeting --query 'Stacks[0].Outputs[?OutputKey==`PersonalizeBucketName`].OutputValue' --output text)
aws s3 rm s3://$SEGMENT_IMPORT_BUCKET/ --recursive
aws s3 rm s3://$PERSONALIZE_BUCKET/ --recursive
sam delete

For Windows PowerShell use the command below:

$SEGMENT_IMPORT_BUCKET=$(aws cloudformation describe-stacks --stack-name contextual-targeting --query 'Stacks[0].Outputs[?OutputKey==`SegmentImportBucket`].OutputValue' --output text)
$PERSONALIZE_BUCKET=$(aws cloudformation describe-stacks --stack-name contextual-targeting --query 'Stacks[0].Outputs[?OutputKey==`PersonalizeBucketName`].OutputValue' --output text)
aws s3 rm s3://$SEGMENT_IMPORT_BUCKET/ --recursive
aws s3 rm s3://$PERSONALIZE_BUCKET/ --recursive
sam delete

Amazon Personalize resources like Dataset groups, datasets, etc. are not created via AWS Cloudformation, thus you have to delete them manually. Please follow the instructions in the official AWS documentation on how to clean up the created resources.

About the Authors

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis is an Amazon Pinpoint and Amazon Simple Email Service Specialist Solutions Architect at AWS. He loves to dive deep into his customer’s technical issues and help them design communication solutions. In his spare time, he enjoys playing tennis, watching crime TV series, playing FPS PC games, and coding personal projects.

Christian Bonzelet

Christian Bonzelet

Christian Bonzelet is an AWS Solutions Architect at DFL Digital Sports. He loves those challenges to provide high scalable systems for millions of users. And to collaborate with lots of people to design systems in front of a whiteboard. He uses AWS since 2013 where he built a voting system for a big live TV show in Germany. Since then, he became a big fan on cloud, AWS and domain driven design.

Incident notification mechanism using Amazon Pinpoint two-way SMS

Post Syndicated from Pavlos Ioannou Katidis original https://aws.amazon.com/blogs/messaging-and-targeting/incident-notification-mechanism-using-amazon-pinpoint-two-way-sms/

Unexpected situations that require immediate attention can occur in any industry. Part of resolving these incidents is the notifications’ delivery. For example, utility companies that have installed gas sensors need to notify immediately the available engineer if a leak occurs.

The goal of an incident management process is to restore a normal service operation as quickly as possible and to minimize the impact on business operations, thus ensuring that the best possible levels of service quality and availability are maintained. A key element of incident management is sending timely notifications to the assigned or available resource(s) who can rectify the issue.

An incident can take place at any time and the resource(s) assigned to it might not have internet access and even if they receive the message they might not be equipped to work on it. This creates five key requirements for an incident notifications mechanism:

  1. Notify the resources via a communication channel that ensures message delivery even without internet access
  2. Enable assigned resources to respond to a request via a communication channel that doesn’t require internet access
  3. Send reminder(s) in case there is no response from the assigned resource(s)
  4. Escalate to another resource in case the first one doesn’t reply or declines the incident
  5. Store the incident details & status for reporting and data analysis

In this blog post, I share a solution on how you can automate the delivery of incident notifications. This solution utilizes Amazon Pinpoint SMS channel to contact the designated resources who might not have access to the internet. Furthermore, the recipient of the SMS is able to reply with an acknowledgement. AWS Step Functions orchestrates the user journey using AWS Lambda functions to evaluate the recipients’ response and trigger the next best action. You will use AWS CloudFormation to deploy this solution.

Use Cases

An incident notification mechanism can vary depending the organization’s requirements and 3rd party system integrations. In this blog the solution covers all five points listed above but it might require further modifications depending your use case.

With minor modifications this solution can also be used in the following use cases:

  1. Medicine intake notification: It will notify the patient via SMS that it is their time to take their medicine. If the patient doesn’t acknowledge the SMS by replying then this can be escalated to their assigned doctor
  2. Assignment submission: It will notify the student that their assignment is due. If the student doesn’t acknowledge the SMS by replying then this can be escalated to their teacher

High-level Architecture

The solution requires the country of your SMS recipients to support two-way SMS. To check which countries, support two-way SMS visit this page.  If two-way SMS is supported then you will need to request a dedicated originating identity. You can also use Toll Free Number or 10DLC if your recipients are in the US.

Note: Sender ID doesn’t support two-way SMS.

A new incident is represented as an item in an Amazon DynamoDB table containing information such as description, URL, incident_id as well as the contact numbers for two resources. A resource is someone who has been assigned to work on this incident. The second resource is for escalation purposes in case the first one doesn’t acknowledge or decline the incident notification.

The Amazon DynamoDB table covers three functions for this solution:

  1. A way to add new incidents using either the AWS console or programmatically
  2. As a storage for variables that indicate the incident’s status and can be used from the solution to determine the next action(s)
  3. As a historical data storage for all incidents that have been created for data analysis purposes

The solution utilizes Amazon DynamoDB Streams to invoke an AWS Lambda function every time a new incident is created. The AWS Lambda function triggers an AWS Step Function State machine, which orchestrates three AWS Lambda functions:

  1. Send_First_SMS: Sends the first SMS
  2. Reminder_SMS: Sends a reminder SMS if the resource does not acknowledge the first SMS
  3. Incident_State_Review: Assesses the status of the incident and either goes back to the first AWS Lambda function or finishes the AWS Step Function State machine execution

The AWS Step Functions State machine uses the Choice state, which evaluates the response of the previous AWS Lambda function and decides on the next state. This is a very useful feature that can reduce custom code and potentially AWS Lambda invocations resulting to cost savings.

Additionally, the waiting between steps is also managed from AWS Step Functions State machine using the Wait state. This can be configured to wait seconds, days or till a specific point in the future.

To be able to receive SMS, this solution uses Amazon Pinpoint’s two-way SMS feature. When receiving an SMS Amazon Pinpoint sends a payload to an Amazon SNS topic, which needs to be created separately. An AWS Lambda function that is subscribed to the Amazon SNS topic processes the SMS content and performs one or both of the following actions:

  1. Update the incident status in the DynamoDB table
  2. Create a new Step Function State machine execution

In this solution SMS recipients can reply by typing either yes or no. The SMS response is not case sensitive.

An inbound SMS payload contains the originationNumber, destinationNumber, messageKeyword, messageBody, inboundMessageId and previousPublishedMessageId. Noticeably there isn’t a direct way to associate an inbound SMS with an incident. To overcome this challenge this solution uses a second DynamoDB table, which stores the message_id and incident_id every time an SMS is send to any of the two resources. This allows the solution to use the previousPublishedMessageId from the inbound SMS payload to fetch the respective incident_id from the second DynamoDB table.

The code in this solution uses AWS SDK for Python (Boto3).

Prerequisites

  1. An Amazon Pinpoint project with the SMS channel enabled – Guide on how to enable Amazon Pinpoint SMS channel
  2. Check if the country you want to send SMS to, supports two-way SMS – List with countries that support two-way SMS
  3. An originating identity that supports two-way SMS – Guide on how to request a phone number
  4. Increase your monthly SMS spending quota for Amazon Pinpoint – Guide on how to increase the monthly SMS spending quota

Deploy the solution

Step 1: Create an S3 bucket

  1. Navigate to the Amazon S3 console
  2. Select Create bucket
  3. Enter a unique name for Bucket name
  4. Select the AWS Region to be the same as the one of your Amazon Pinpoint project
  5. Scroll to the bottom of the page and select Create bucket
  6. Follow this link to download the GitHub repository. Once the repository is downloaded, unzip it and navigate to  \amazon-pinpoint-incident-notifications-mechanism-main\src
  7. Access the S3 bucket created above and upload the five .zip files

Step 2: Create a stack

  1. The application is deployed using an AWS CloudFormation template.
  2. Navigate to the AWS CloudFormation console select Create stack > With new resources (standard)
  3. Select Template is ready as Prerequisite – Prepare template and choose Upload a template file as Template source
  4. Select Choose file and from the GitHub repository downloaded in step 1.6 navigate to amazon-pinpoint-incident-notifications-mechanism-main\cfn upload CloudFormation_template.yaml and select Next
  5. Type Pinpoint-Incident-Notifications-Mechanism as Stack name, paste the S3 bucket name created in step 1.5 as the LambdaCodeS3BucketName, type the Amazon Pinpoint Originating Number in E.164 format as OriginatingIdenity, paste the Amazon Pinpoint project ID as PinpointProjectId and type 40 for WaitingBetweenSteps
  6. Select Next, till you reach to Step 4 Review where you will need to check the box I acknowledge that AWS CloudFormation might create IAM resources and then select Create Stack
  7. The stack creation process takes approximately 2 minutes. Click on the refresh button to get the latest event regarding the deployment status. Once the stack has been deployed successfully you should see the last Event with Logical ID Pinpoint-Incident-Notifications-Mechanism and with Status CREATE_COMPLETE

Step 3: Configure two-way SMS SNS topic

  1. Navigate to the Amazon Pinpoint console > SMS and voice > Phone numbers. Select the originating identity that supports two-way SMS. Scroll to the bottom of the page and click to expand the  and check the box to enable it.

    For SNS topic select Choose an existing SNS topic then using the drop down choose the one that contains the name of the AWS CloudFormation stack from Step 2.4 as well as the name TwoWaySMSSNSTopic and click Save.

Step 4: Create a new incident

To create a new incident, navigate to Amazon DynamoDB console > Tables and select the table containing the name of the AWS CloudFormation stack from Step 2.4 as well as the name IncidentInfoDynamoDB. Select View items and then Create item.

On the Create item page choose JSON, copy and paste the JSON below into the text box and replace the values for the first_contact and second_contact with a valid mobile number that you have access to.

Note: If you don’t have two different mobile numbers, enter the same for both first_contact and second_contact fields. The mobile numbers must follow E.164 format +<country code><number>.

{
   "incident_id":{
      "S":"123"
   },
   "incident_stat":{
      "S":"not_acknowledged"
   },
   "double_escalation":{
      "S":"no"
   },
   "description":{
      "S":"Error 111, unit 1 malfunctioned. Urgent assistance is required."
   },
   "url":{
      "S":"https://example.com/incident/111/overview"
   },
   "first_contact":{
      "S":"+4479083---"
   },
   "second_contact":{
      "S":"+4479083---"
   }
}

Incident fields description:

  • incident_id: Needs to be unique
  • incident_stat: This is used from the application to store the incident status. When creating the incident, this value should always be not_acknowledged
  • double_escalation: This is used from the application as a flag for recipients who try to escalate an incident that is already escalated. When creating the incident, this value should always be no
  • description: You can type a description that best describes the incident. Be aware that depending the number of characters the SMS parts will increase. For more information on SMS character limits visit this page
  • url: You can add a URL that resources can access to resolve the issue. If this field is not pertinent to your use case then type no url
  • first_contact: This should contain the mobile number in E.164 format for the first resource
  • second_contact: This should contain the mobile number in E.164 format for the second resource. The second resource will be contacted only if the first one does not acknowledge the SMS or declines the incident

Once the above is ready you can select Create item. This will execute the AWS Step Functions State machine and you should receive an SMS. You can reply with yes to acknowledge the incident or with no to decline it. Depending your response, the incident status in the DynamoDB table will be updated and if you reply no then the incident will be escalated sending a SMS to the second_contact.

Note: The SMS response is not case sensitive.

Clean-up

To remove the solution:

  1. Delete the AWS CloudFormation stack by following the steps listed in this guide
  2. Delete the dedicated originating identity that you used to send the SMS by following the steps listed in this guide
  3. Delete the Amazon Pinpoint project by navigating the Amazon Pinpoint console, select your Amazon Pinpoint Project, choose Settings > General settings > Delete Project

Next Steps

This solution currently works only if your SMS recipients are in one country. If your use case requires to send SMS to multiple countries you will need to:

  • Check this page to ensure that these countries support two-way SMS
  • Follow the instructions in this page to obtain a number that supports two-way SMS for each country
  • Expand the solution to identify the country of the SMS recipient and to choose the correct number accordingly. To identify the country of the SMS recipient you can use Amazon Pinpoint’s phone number validate service via Amazon Pinpoint API or SDKs. The phone validate service returns a list of data points per mobile number with one of them being the Country

Incidents that are not being acknowledged by any of the assigned resources, have their status updated to unacknowledged but they don’t escalate further. Depending your requirements, you can expand the solution to send an email using Amazon Pinpoint APIs or perform an outbound call using Amazon Connect APIs.

Conclusion

In this blog post, I have demonstrated how your organization can use Amazon Pinpoint two-way SMS and Step Functions to automate incident notifications. Furthermore, the solution highlights the synergy of AWS services and how you can build a custom solution with little effort that meets your requirements.

About the Author

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis

Pavlos Ioannou Katidis is an Amazon Pinpoint and Amazon Simple Email Service Specialist Solutions Architect at AWS. He loves to dive deep into his customer’s technical issues and help them design communication solutions. In his spare time, he enjoys playing tennis, watching crime TV series, playing FPS PC games, and coding personal projects.

Dynamically personalize your in-product user experience using Amazon Pinpoint in-app messaging

Post Syndicated from Pavlos Ioannou Katidis original https://aws.amazon.com/blogs/messaging-and-targeting/dynamically-personalize-your-in-product-user-experience-using-amazon-pinpoint-in-app-messaging/

Many businesses today struggle to align out-of-product messaging through channels such as email and SMS, with in-product messaging shown when a users is within a mobile or web application. Customers will present one message to a user through a targeted email, but once a user visits the application they are presented with different messaging. This creates confusion for the user, and reduces the chances of them performing a high-value action such as a purchasing a discounted product. Customers can get around this by hard coding certain messages into their application, however this is time consuming for development teams, and slower to implement as it requires a new release of a mobile or web client.

Amazon Pinpoint in-app messaging allows customers to create, target and display in-product messages to users dynamically without the need to update client-side code after initial implementation. This allows a non-technical persona such as a marketer to modify the application experience and target user messaging independently of a development team. This also allows the in-product messaging to share the user targeting as the out-of-app messaging. This creates consistent user messaging, and increases the chance a user performs a high value action.

The blog outlines how to create in-app endpoints, segments, and campaigns. Then how to fetch in-app messages, implement simple logic to control message prioritization, message caps, and to listen for events in order to show the message at the desired moment.

Solution Overview

Assume you are a retailer and want to display a banner with a promotion to all customers with a recent purchase over $500 when they launch the application. To deliver the above experience using the in-app messaging channel, you will need to create a dynamic customer segment where User.UserAttribute.LastPurchaseValue > $500, design an in-app message template with a call-to-action to claim the promotion, and create an in-app campaign. The in-app campaign will be triggered based on the customer event app_launch and only for customers who belong to the dynamic segment created above. To render the message and send in-app message engagement events back to Amazon Pinpoint, you will need to go through an one time setup that is explained in a later section of this blog. Monitor your in-app campaign performance across different metrics, using the Amazon Pinpoint campaign analytics dashboard.

In-app channel implementation can differ depending the use case and requirements. The creation of customer segments, message templates and campaigns can be done either via the Amazon Pinpoint console or programmatically using Amazon Pinpoint APIs. The in-app messages retrieval, rendering and recording of engagement events can either be build and managed from you or use AWS Amplify.

In the following sections you will be introduced to the seven components of the in-app channel and how they operate together:

  • Step 1: Creating in-app endpoints & segment
  • Step 2: Creating an in-app message template
  • Step 3: Creating an in-app campaign
  • Step 4: Querying available in-app messages for an Amazon Pinpoint customer
  • Step 5: Rendering in-app messages
  • Step 6: Recording in-app events
  • Step 7: In-app message display logic using SessionCap, DailyCap, TotalCap

Prerequisites

For this blog post, you should have the following prerequisites:

Step 1: Creating an Amazon Pinpoint customer segment

In Amazon Pinpoint, users can have one or more endpoints. An endpoint describes a unique address, such as an email or mobile number. Similar to other Amazon Pinpoint channels, you need to create or import in-app endpoints with Channel = IN_APP. To retrieve in-app messages for a user, you have to use their IN_APP endpoint id. Note that the Address is not a required field for in-app and can be left blank.

  1. Copy the text below and save it as CSV in your computer
    Id,ChannelType,Address,User.UserId,OptOut
    111,IN_APP,123,userid1,NONE
  2. Navigate to the Amazon Pinpoint console
  3. Select the Amazon Pinpoint project that you want to set up the in-app channel
  4. Navigate to the Segments’ section
  5. Choose Import a segment
  6. Select Upload files from your computer as Import method
  7. Select Choose files and find the CSV file you created in step 1
  8. Choose Create segment
  9. Navigate to AWS Cloudshell console and wait till the terminal loads
  10. Replace <Application id> with your Amazon Pinpoint application id in the following command aws pinpoint get-endpoint –application-id <Application id> —endpoint-id 111
  11. Execute the command in step 10 by pasting it in the AWS CloudShell terminal and press Enter. You should be able to see a response similar to the one below


Step 2: Creating an in-app Message Template

In-app message templates contain a variety of fields with some of them offering the option to choose from pre-defined values such as Header alignment and others in a form of free text such as Message. The end result is a banner that includes a Header, Message, Image, Button(s) and Custom data with all of them being fully customizable. While building an in-app template, you can preview the banner across Phone, Tablet and Browser. This preview is for reference purposes only as the rendering can vary according to the end user’s device as well as your preference on how to render it.

Note: The message template for in-app currently doesn’t support message helpers for personalization but it is a feature the Amazon Pinpoint product team is exploring.

  1. Navigate to Message templates
  2. Select Create template and choose In-app messaging as Channel
  3. Type my_first_in-app_message_template as Template name
  4. Complete the  section, as per your message requirements
  5. Select Create

Step 3: Creating an in-app Campaign

A campaign is a messaging initiative that engages a specific audience segment. A campaign sends tailored messages according to a schedule or customer event that you define.

  1. Navigate to your Amazon Pinpoint project and select Campaigns and Create a campaign
  2. Type my_first_in-app_campaign as Campaign name
  3. Select Standard campaign as Campaign type and In-app messaging as Channel
  4. Select Very important for Set prioritization. This configuration is specific to the in-app channel and it helps you identify the most important in-app message for an endpoint
  5. Select Next and choose the segment in-app-segment from the dropdown. This should be an imported segment that you created in Step 1: Creating an Amazon Pinpoint customer segment. The Segment estimate should show 1 endpoints
  6. Select Next and choose the in-app message template with the name my_first_in-app_message_template, then select Next
  7. An in-app campaign needs to have a Trigger event, which will determine when the in-app message will be displayed. You can add event Attributes and/or Metrics to make it more specific. To learn how to record events with Amazon Pinpoint visit Reporting events in your application. If you currently do not record any events in your Amazon Pinpoint project type test_event as Trigger events
  8. Select Start and End date and time for Campaign dates. Note that in-app campaigns need to start at least 15 minutes later from the time of publishing
  9. In the Edit campaign settings section you will find the fields, which specify the Maximum number of session messages viewed per endpoint (SessionCap), Maximum number of daily messages viewed per endpoint (DailyCap) and Maximum number of messages viewed per endpoint (TotalCap). These values indicate how many times the in-app message can be displayed to the customer for that in-app campaign within a session, day and in total respectively. In all three campaign setting fields enter the number 10 and select Override project-level setting where applicable Set prioritization, Trigger events and Caps are part of the in-app message payload that you receive when calling Amazon Pinpoint’s In-app messages REST API operation. You will use this information to decide whether to render or not that in-app message.
  10. Select Next scroll down and select Launch campaign

Step 4: Querying available in-app messages for an Amazon Pinpoint customer

To retrieve in-app messages for an Amazon Pinpoint customer, you will need to have their IN_APP endpoint id and either use the In-app messages REST API operation, one of the AWS SDKs that support Amazon Pinpoint, AWS Command Line Interface or AWS Amplify.

Note: AWS Amplify manages on your behalf the in-app messages request, rendering and tracking, thus if you are using AWS Amplify for Amazon Pinpoint in-app channel the steps below are not required.

In the request body you need to specify the IN_APP endpoint id. If there are any available in-app messages for that endpoint id, the response will contain a JSON object with the top ten active in-app messages based on their priority (the ten in app message response is a hard limit). Loop through the in-app messages and identify the one that meets the criteria based on the Trigger event and Prioritization.

  1. Navigate to the AWS CloudShell console
  2. Replace <Application id> with your Amazon Pinpoint application id in the following command aws pinpoint get-in-app-messages –application-id <Application id> —endpoint-id 111
  3. Execute the command in step 2 by pasting it in the AWS CloudShell terminal and press Enter. You should be able to see a response similar to the one below

The response should contain only one in-app campaign. You can see all the in-app message template data and campaign configuration are present in the response.

Note: Campaigns that have passed their end date, or have reached their daily or total cap limit won’t show in the response. In case the response contains more than one in app message with the same priority and they both haven’t exceeded their caps, you can use the in-app campaign start date to evaluate which one to display.

It is recommended to retrieve the in-app messages once per session and store them locally. That way in every event the customer triggers in your mobile / web app you would check against the in-app messages stored locally instead of performing additional calls to Amazon Pinpoint. This approach decreases the in-app channel cost as you pay per request.

You can perform the operation of retrieving in-app messages for an Amazon Pinpoint customer either client side or server side. Server side can be implemented using the architecture illustrated below, which utilizes Amazon API Gateway and AWS Lambda creating a development framework agnostic approach. Furthermore Amazon API Gateway is offering a great variety of authentication and authorization mechanisms.

The server side architecture depicted below doesn’t cover the use case for offline customers. If this is a requirement then it is recommend to store in-app messages and fetch them locally when the device doesn’t have internet connectivity. Once the device is connected back to the internet you can retrospectively send any in-app related events.

Note: If you are using AWS Amplify, it will retry to publish customer offline events that occurred once the device gets back online.

Step 5: Rendering in-app messages

Render the in-app messages yourself based on the in-app message API response or use AWS Amplify which will render it on your behalf. AWS Amplify allows you to provide your own In-App Messaging UI component to override the default Amplify provided UI.

Step 6: Recording in-app events

Measuring in-app campaigns’ performance is based on four metrics:

  • Message displayed: a message has been displayed to an end user
  • Message dismissed: a user has dismissed a message
  • Message clicked: a user has clicked through a message
  • Any event type: Any event that a user can trigger on the mobile or web app

Fire the above events either from client or server side as Amazon Pinpoint custom events. Amazon Pinpoint custom events can be recorded using put_events REST API operation or AWS SDKs that support Amazon Pinpoint.

Note: If you are using AWS Amplify, the in-app events will be recorded automatically

To have these events recorded under Amazon Pinpoint Campaign deliver metrics dashboard, you have to use the following EventType names:

  • Message displayed: _inapp.message_displayed
  • Message dismissed: _inapp.message_dismissed
  • Message clicked: _inapp.message_clicked
  • Any event type: No specific name is required

In addition to the EventType, a few other fields are required in order to attribute these events to the correct in-app campaign. Within the event attributes’ object of the request payload, the fields campaign_id and delivery_type must be provided. Campaign_id should match the InApp campaign_id, while the delivery_type should be IN_APP_MESSAGE. Additionally, the treatment_id is necessary if you are running an A/B test.

Note: If you do not use the above event names and attributes, you won’t see any events under Campaign delivery metrics and Campaign engagement rates on the Amazon Pinpoint console.

Step 7: In-app message display logic using SessionCap, DailyCap and TotalCap

Message display logic refers to the logic that stores and assesses the number of times a user has seen / interacted with the in-app message. Amazon Pinpoint calculates the DailyCap & TotalCap as long as you record the _inapp.message_displayed event or using AWS Amplify. For the SessionCap event you need to count the _inapp.message_displayed locally on your mobile / web application unless you are using AWS Amplify.

Note: When retrieving the in-app messages from Amazon Pinpoint, the payload contains the remaining number of times you can display the in-app message daily & total.

Conclusion

This post walks you through how to configure Amazon Pinpoint to send in-app messages to your customers when browsing your mobile / web application. Using this Amazon Pinpoint channel, you can now:

  • Create in-app segments, message templates and campaigns
  • Retrieve in-app messages per user
  • Render in-app messages
  • Record customer engagement data with the in-app message

Related links

To learn more about the technologies or features used to create this solution, explore the following pages: