All posts by Bruno Giorgini

Cómo Enviar SMS Internacionales con Números Gratuitos de EE.UU. Usando AWS End User Messaging

Post Syndicated from Bruno Giorgini original https://aws.amazon.com/blogs/messaging-and-targeting/como-enviar-sms-internacionales-con-numeros-gratuitos-de-ee-uu-usando-aws-end-user-messaging/

AWS End User Messaging ahora admite capacidades de SMS internacional para Números Gratuitos de EE.UU. (TFN). Esta nueva función permite a las empresas usar un solo TFN de EE.UU. para enviar mensajes SMS a más de 150 países, simplificando el alcance global. Beneficia principalmente a clientes que necesitan enviar alertas transaccionales unidireccionales—como contraseñas de un solo uso (OTP) o notificaciones de envío—y empresas que quieren crear prototipos rápidamente y probar su estrategia de mensajería en nuevos mercados internacionales sin la complejidad de adquirir números específicos por país.

Esta guía te mostrará los pros y contras de esta función y cómo habilitarla y cuándo usarla versus métodos tradicionales de envío específicos por país.

¿Qué Son los Números Gratuitos Internacionales de EE.UU.?

Un número gratuito internacional de EE.UU. es un TFN (toll-free number) estándar de EE.UU. que ha sido habilitado con la capacidad de enviar mensajes SMS a destinos fuera de Estados Unidos. Esta función es compatible con versiones anteriores, lo que significa que puedes habilitarla en cualquier TFN de EE.UU. nuevo o existente en tu cuenta.

Cómo Habilitar el Envío Internacional

Hay tres formas principales de habilitar esta función para tus Números Gratuitos de EE.UU.:

  • Habilitar el envío internacional al registrar un nuevo número en la consola.
  • Habilitar el envío internacional para un número existente en la consola.
  • Habilitar el envío internacional para un número existente a través del AWS CLI.

1. Habilitar Al Registrar un Nuevo Número Gratuito de EE.UU. (Consola)

  • Desde la consola de AWS End User Messaging, navega a Administrar SMS
  • Desde la consola de AWS End User Messaging, navega a Configuraciones > Números de teléfono > y selecciona Originador de la solicitud
  • Paso 1: Seleccione un país, selecciona Estados Unidos (US) como tu país de destino
  • En Paso 2: Defina el caso de uso, configura las diversas opciones listadas para tu Caso de uso de mensajería previsto, y selecciona para habilitar el envío Internacional, antes de hacer clic en Siguiente
  • Para Paso 3: Seleccionar tipo de originador, selecciona Gratuito, valida tus opciones de Política de recursos, selecciona Siguiente
  • En Paso 4: Revisar y solicitar: Verifica que la información que ingresaste sea correcta y selecciona Solicitar. Nota: Las solicitudes de registro de números gratuitos de EE.UU. pueden tomar aproximadamente 15 días hábiles para ser aprobadas.

Para más información, consulta Solicitar un número de teléfono en AWS End User Messaging SMS

2. Habilitar para un Número Gratuito de EE.UU. Existente (Consola o CLI)

Si ya has adquirido un TFN, puedes habilitar la función de envío internacional en cualquier momento.

Usando la Consola de Administración de AWS:

  • Navega a Configuraciones > Números de teléfono > y selecciona un número Gratuito existente
  • Localiza la pestaña Envío internacional y elige Editar configuración
  • Marca la casilla Habilitar envío internacional en los detalles de tu número de teléfono
    • Guardar Cambios

Usando el AWS CLI

El comando update-phone-number te permite modificar las capacidades de un número de teléfono, mientras que el comando describe-phone-numbers te permite verificar su estado.

1. Para Habilitar el Envío Internacional:

Usa el parámetro --international-sending-enabled

aws pinpoint-sms-voice-v2 update-phone-number \
    --phone-number-id "phone-a1b2c3d4e5f67890" \
    --international-sending-enabled \
    --region us-east-1

Nota: Reemplaza "phone-a1b2c3d4e5f67890" con el ID real de tu número de teléfono

2. Para Deshabilitar el Envío Internacional:

Usa el parámetro --no-international-sending-enabled

aws pinpoint-sms-voice-v2 update-phone-number \
    --phone-number-id "phone-a1b2c3d4e5f67890" \
    --no-international-sending-enabled \
    --region us-east-1

Respuesta Esperada (para update-phone-number):

Un comando exitoso devuelve el objeto JSON completo para el número de teléfono. Confirma el cambio verificando que el valor InternationalSendingEnabled sea true

{
    "PhoneNumberArn": "arn:aws:sms-voice:us-east-1:111122223333:phone-number/phone-a1b2c3d4e5f67890",
    "PhoneNumberId": "phone-a1b2c3d4e5f67890",
    "PhoneNumber": "+18005550199",
    "Status": "ACTIVE",
    "IsoCountryCode": "US",
    "MessageType": "TRANSACTIONAL",
    "NumberCapabilities": [
        "SMS"
    ],
    "NumberType": "TOLL_FREE",
    "MonthlyLeasingPrice": "2.00",
    "TwoWayEnabled": true,
    "InternationalSendingEnabled": true,
    "CreatedTimestamp": "2025-08-15T10:30:00.123Z"
}

3. Para Verificar el Estado Actual:

Usa el comando describe-phone-numbers con tu ID de Número de Teléfono para verificar su configuración actual en cualquier momento.

aws pinpoint-sms-voice-v2 describe-phone-numbers \
    --phone-number-ids "phone-a1b2c3d4e5f67890" \
    --region us-east-1

Beneficios y Limitaciones

Esta función ofrece una nueva forma poderosa de llegar a una audiencia global, pero es importante entender dónde destaca y cuáles son sus limitaciones.

Beneficios (Ventajas)

  • Alcance Global con un Solo Número: Envía SMS a más de 150 países usando un solo TFN de EE.UU. existente.
  • Gestión Simplificada: Evita la complejidad operacional y el costo de comprar y gestionar una flota de números de teléfono específicos por país.
  • Prototipado y Pruebas Rápidas: Prueba rápidamente campañas de mensajería en nuevos mercados internacionales antes de comprometerte con el enfoque de mejores prácticas de adquirir números dedicados en el país.
  • Optimización de Costos para Alertas Unidireccionales: Proporciona un método rentable para enviar mensajes transaccionales unidireccionales de alto volumen como OTP, recordatorios de citas y notificaciones de envío globalmente.

Limitaciones y Consideraciones Técnicas

  • SMS Bidireccional Limitado a EE.UU. y Canadá: Las conversaciones SMS bidireccionales confiables solo son compatibles para destinatarios en Estados Unidos y Canadá.
  • Solo Unidireccional para Todos los Otros Países: Para todos los otros destinos, esto es solo unidireccional.
  • Entrega con máximo esfuerzo (no garantizado): El envío fuera de EE.UU. y Canadá es con máximo esfuerzo (no garantizado). El número de teléfono que aparece en el dispositivo del destinatario puede ser reemplazado con un número local o ID de Remitente, por lo que la mensajería bidireccional no funcionará para estos destinos. Para más detalles sobre maximizar la entrega, lee Una Guía para Optimizar la Entrega de SMS y Mejores Prácticas (Inglés).
  • La Exclusión Gestionada No Está Garantizada Internacionalmente: La funcionalidad automática de respuesta STOP no funciona para destinos fuera de EE.UU. y Canadá. Para destinatarios internacionales, debes proporcionar un método alternativo de exclusión.
  • Rendimiento Estándar (3 MPS): Los TFN internacionales tienen un rendimiento predeterminado de 3 Partes de Mensaje Por Segundo (MPS). Para campañas de alto volumen y alto rendimiento, los números específicos por país dedicados (como códigos cortos) son la mejor práctica recomendada.

Entendiendo el Costo

El precio para esta función es directo:

  • Sin Tarifas Mensuales Adicionales: No hay cargo extra por habilitar la capacidad de envío internacional en tu TFN de EE.UU. Solo pagas el arrendamiento mensual estándar por el número mismo.
  • Mensajería de Pago por Uso: Se te factura por cada mensaje SMS saliente a la tarifa estándar por mensaje para el país de destino.

Para una lista completa y actualizada de precios por país, visita la página de Precios de AWS End User Messaging.

Cuándo Usar TFN Internacional vs. Números Específicos por País

Elegir la herramienta correcta depende de tu caso de uso. Aquí hay una comparación simple:

Caso de Uso ¿Usar TFN Internacional? ¿Usar Número Específico por País (Mejor Práctica)?
Probar rápidamente un nuevo mercado . Es la forma más rápida de comenzar. No, este enfoque toma más tiempo para configurar.
Enviar alertas unidireccionales (OTP, notificaciones) . Es una solución simple y rentable. , pero es más complejo si necesitas enviar a muchos países.
Requerir conversaciones bidireccionales Solo para EE.UU. y Canadá. . Este es el enfoque requerido para SMS bidireccional confiable en un país específico.
Garantizar que tu marca/número aparezca consistentemente No, planifica que el ID del Remitente no se preserve, ya que esto no está garantizado internacionalmente. . Esta es la razón principal para usar un número dedicado en el país.
Maximizar la entregabilidad para campañas críticas No, la entrega es “con máximo esfuerzo”. . Un número local proporciona la mayor probabilidad de entrega exitosa.

Consideraciones y Próximos Pasos

Una vez que hayas habilitado tu envío internacional sobre Números Gratuitos de EE.UU., puedes mejorar tu estrategia de mensajería considerando resistencia, monitoreo y escalabilidad. Los siguientes recursos proporcionan mejores prácticas para mejorar tu envío.

Conclusión

El SMS Internacional para Números Gratuitos de EE.UU. es una herramienta estratégica poderosa para empresas que buscan simplificar su mensajería global. Destaca en permitir pruebas rápidas en nuevos mercados y entregar eficientemente alertas transaccionales unidireccionales en todo el mundo desde un solo número.

Sin embargo, no es un reemplazo para la mejor práctica de usar números de teléfono dedicados en el país cuando conversaciones bidireccionales confiables y marca garantizada son críticas para el éxito de tu campaña. Al entender sus beneficios y limitaciones, puedes usar estratégicamente esta función para comenzar rápidamente mientras planificas un movimiento a largo plazo hacia códigos específicos por país para tus mercados más importantes.

Creando experiencias de cliente con IA mediante un hub de comunicaciones moderno

Post Syndicated from Bruno Giorgini original https://aws.amazon.com/blogs/messaging-and-targeting/creando-experiencias-de-cliente-con-ia-mediante-un-hub-de-comunicaciones-moderno/

Los clientes de hoy esperan que las organizaciones satisfagan proactivamente sus necesidades con contenido personalizado, entregado en el momento, lugar y forma de su elección. Buscan interacciones dinámicas y conscientes del contexto con conversaciones sofisticadas a través de todos los canales de comunicación. Esta creciente demanda ejerce presión sobre las organizaciones para transformar sus flujos de trabajo de experiencia del cliente para mejorar la lealtad y aumentar la eficiencia operativa. Si bien los avances recientes en Generative AI (GenAI), incluida la hiperpersonalización y Agentic AI, ofrecen posibilidades interesantes, también presentan nuevos desafíos. Las organizaciones necesitan una arquitectura flexible y reutilizable que les permita incorporar GenAI en sus sistemas existentes de participación del cliente sin requerir una revisión completa de sus soluciones dispares actuales.

Esta publicación de blog explora cómo construir un centro de comunicaciones moderno impulsado por IA utilizando ejemplos de GitHub de código abierto que integran servicios de SMS/MMS y WhatsApp con capacidades de GenAI. Las organizaciones pueden crear experiencias innovadoras de cliente impulsadas por IA con una rápida prueba de concepto sin interrumpir los sistemas existentes.

En combinación con Vector Databases y Retrieval Augmented Generation (RAG), GenAI hace posible reorganizar el conocimiento en un solo sistema y consultar desde una única interfaz de usuario a través de conversación en lenguaje natural con un chatbot o asistente virtual. Canalizar las comunicaciones de los clientes a través de un centro de comunicaciones multicanal vinculado con capacidades de GenAI ayuda a unificar los mecanismos de participación del cliente y agiliza la creación de experiencias ricas para el cliente. Los clientes interactúan con agentes de IA y bots de preguntas y respuestas en el canal de comunicación que les resulta conveniente para autogestionar sus necesidades. Las organizaciones pueden construir experiencias de cliente agnósticas al canal de comunicación mientras recopilan eventos de participación del canal y datos conversacionales en un almacén de datos centralizado para obtener información en tiempo real, consultas ad-hoc, análisis y entrenamiento de ML.

Descripción general de la solución

En el núcleo de la solución se encuentra el Centro de Comunicaciones Moderno que conecta los canales de comunicación digital con servicios clave de GenAI, como Amazon Bedrock y Amazon Q, junto con servicios de AWS ML, bases de datos, almacenamiento y computación sin servidor.

Este diagrama muestra la arquitectura de la solución en Nivel 300

AWS End User Messaging y Amazon SES proporcionan acceso a nivel de API a canales de comunicación digital, ofreciendo servicios seguros, escalables, de alto rendimiento y rentables para que las aplicaciones empresariales intercambien SMS/MMS, WhatsApp, notificaciones push y de voz, y correo electrónico con los clientes.

Una colección de código de muestra de código abierto, publicada en el repositorio AWS-samples de GitHub, ilustra cómo facilitar conversaciones generativas en canales SMS/MMS y WhatsApp. Esto se extenderá para incluir servicios de correo electrónico. Dos componentes clave forman la base de las Muestras de Integración de GenAI: el Orquestrador de chat Multicanal con Agentes de IA, y la Base de Datos de Participación y Análisis para End User Messaging y SES. Nos referiremos a estos simplemente como el Procesador de Conversaciones y la Base de Datos de Participación en el diagrama de la solución.

El Procesador de Conversaciones recibe mensajes de clientes a través de AWS End User Messaging y Amazon Simple Email Service (SES), almacena los detalles de la conversación e invoca al Agente de Amazon Bedrock relevante. Los Agentes de Amazon Bedrock utilizan Modelos de Lenguaje Grandes (LLMs) y bases de conocimiento para analizar tareas, dividirlas en pasos accionables, ejecutar esos pasos o buscar en la base de conocimiento, observar resultados y refinar iterativamente su enfoque hasta completar la tarea junto con una respuesta. Alternativamente, el Procesador de Conversaciones puede funcionar como un bot de preguntas y respuestas, en cuyo caso utiliza Amazon Bedrock Knowledge Bases junto con su función RAG para generar una respuesta LLM y enviarla por el mismo canal que el mensaje del cliente.

La Base de Datos de Participación recopila y combina datos de participación del cliente y registros conversacionales de todos los canales de comunicación, almacenando la información en un data lake centralizado en Amazon S3. Al convertir los datos a un formato común y canónico, la solución simplifica la consulta y el análisis de estos eventos entrantes. Una función Lambda Transformer aprovecha las Plantillas Apache Velocity para transformar los datos JSON entrantes, permitiendo obtener información en tiempo real.

Los datos de eventos sin procesar almacenados en el data lake de Amazon S3 pueden luego alimentar otros servicios de AWS para su procesamiento posterior. Por ejemplo, los datos pueden fluir hacia Amazon Connect Customer Data Profiles o Amazon SageMaker para apoyar el entrenamiento de modelos de machine learning. Los analistas de datos pueden usar Amazon Athena para realizar consultas directas para informes detallados ad-hoc, o enviar los datos a Amazon QuickSight para visualizaciones avanzadas y capacidades de consulta en lenguaje natural a través de Amazon Q en QuickSight.

NOTA: Existe la posibilidad de que los usuarios finales envíen Información Personal Identificable (PII) en los mensajes. Para proteger la privacidad del cliente, considere usar Amazon Comprehend para ayudar a redactar PII antes de almacenar mensajes en S3. La siguiente publicación de blog proporciona una buena descripción general de cómo usar Comprehend para redactar PII: Redact sensitive data from streaming data in near-real time using Amazon Comprehend and Amazon Kinesis Data Firehose.

Amazon Bedrock proporciona capacidades centrales de GenAI como LLMs, Knowledge Bases, Retrieval Augmented Generation (RAG), agentes de IA y Guardrails, para comprender las solicitudes de los clientes, determinar qué acción tomar y qué comunicar de vuelta. Amazon Bedrock Knowledge Bases proporciona conocimiento y razonamiento específico de la organización, mientras que los Agentes de Amazon Bedrock automatizan tareas de múltiples pasos conectándose perfectamente con los sistemas, APIs y fuentes de datos de la empresa.

Requisitos previos

Los siguientes requisitos previos son necesarios para construir su centro de comunicaciones moderno:

  • Una cuenta de AWS. Regístrese para obtener una cuenta de AWS en el sitio web de AWS si no tiene una.
  • Roles y permisos apropiados de AWS Identity and Access Management (IAM) para Amazon Bedrock, AWS End User Messaging y Amazon S3. Para más información, consulte Create a service role for model import.
  • Configuración de AWS End User Messaging: Necesitará configurar la identidad de origen necesaria en el servicio AWS End User Messaging para entregar mensajes a través de SMS o WhatsApp. Si configura SMS, se debe aprovisionar un Número de Teléfono de Origen SMS registrado y activo en AWS End User Messaging SMS. (Dentro de Estados Unidos, use 10DLC o Números Gratuitos (TFNs)). Si configura WhatsApp, se debe aprovisionar un número activo que haya sido registrado con Meta/WhatsApp en AWS End User Messaging Social.
  • Modelos de Amazon Bedrock: Bedrock Anthropic Claude 3.0 Sonnet y Titan Text Embeddings V2 habilitados en su región. Tenga en cuenta que estos son los modelos predeterminados utilizados por la solución; sin embargo, puede experimentar con diferentes modelos.
  • Docker instalado y en ejecución – Se utiliza localmente para empaquetar recursos para el despliegue.
  • Node (> v18) y NPM (> v8.19) instalados y configurados en su computadora
  • AWS Command Line Interface (AWS CLI) instalado y configurado
  • AWS CDK (v2) instalado y configurado en su computadora.

Implementación del Procesador de Conversaciones y Base de Datos de Participación

Implemente las siguientes dos soluciones. Si bien no es obligatorio, es mejor implementarlas en este orden, ya que las salidas de la Base de Datos de Participación pueden utilizarse en el ejemplo de Chat Multicanal:

    1. Engagement Database and Analytics for End User Messaging and SES
    2. Orquestrador de chat Multicanal con Agentes de IA

Cada solución contiene instrucciones detalladas para implementar los servicios requeridos usando AWS Cloud Development Kit (CDK). La primera solución de Base de Datos de Participación creará un flujo de Amazon Data Firehose que puede utilizarse como entrada para la segunda aplicación de Chat Multicanal, de modo que los datos puedan almacenarse y consultarse en la Base de Datos de Participación.

Orquestrador de chat Multicanal con Agentes de IA

Esta solución demuestra cómo los usuarios pueden interactuar con tres diferentes fuentes de conocimiento. Puede que no necesite las tres, sin embargo, esto debería servir como un buen ejemplo para construir la fuente de conocimiento adecuada para su caso de uso particular:

Construya sus Bases de Conocimiento en Amazon Bedrock usando Amazon S3. Por defecto, la solución creará Bases de Conocimiento usando un Bucket de Amazon S3 como fuente de datos. Esta solución le permite cargar documentos a un bucket de Amazon S3 para poblar la base de conocimiento.

NOTA: El proyecto inicial crea un bucket S3 para almacenar los documentos utilizados para la Base de Conocimiento de Bedrock. Por favor, considere usar Amazon Macie para ayudar en el descubrimiento de datos potencialmente sensibles en buckets S3. Amazon Macie puede habilitarse en una prueba gratuita durante 30 días, hasta 150GB por cuenta.

Construya su Base de Conocimiento en Amazon Bedrock usando un Web Crawler. Opcionalmente configure su base de conocimiento para escanear o rastrear sitio(s) web para poblar su base de conocimiento.

Agentes de Amazon Bedrock: Opcionalmente permita que sus usuarios chateen con Agentes de Amazon Bedrock. Los agentes tienen el beneficio adicional de soportar bases de conocimiento para responder preguntas y guiar a los usuarios a través de la recopilación de información necesaria para automatizar una tarea como hacer una reserva. Hay agentes de ejemplo disponibles en el repositorio Amazon Bedrock Agent Samples. Tenga en cuenta que necesitará tener un Agente de Amazon Bedrock creado en su región antes de implementar la solución.

Conclusión

Un Centro de Comunicaciones Moderno, acoplado de manera flexible con servicios centrales de Generative AI, establecerá una base componible para construir experiencias de cliente agnósticas al canal de comunicación. Construya uno aprovechando las Muestras de Integración de GenAI, el Procesador de Conversaciones y la Base de Datos de Participación, combinándolos con los servicios de comunicación digital seguros, escalables, de alto rendimiento y rentables de AWS End User Messaging y Amazon SES. Esto proporcionará un único punto de acceso conversacional a bases de conocimiento y capacidades de IA agéntica en Amazon Bedrock. Comience a experimentar con innovaciones de experiencia del cliente impulsadas por IA con una rápida prueba de concepto que no interferirá con su configuración actual de participación del cliente.

Acerca de los Autores

Automatización de flujos de trabajo con WhatsApp usando AWS End User Messaging Social

Post Syndicated from Bruno Giorgini original https://aws.amazon.com/blogs/messaging-and-targeting/automatizacion-de-flujos-de-trabajo-con-whatsapp-usando-aws-end-user-messaging-social/

En el mundo actual, WhatsApp se ha convertido en una aplicación de comunicación ampliamente utilizada, con más de 2.700 millones de usuarios globalmente. Como propietario de un negocio, probablemente tenga clientes que usan WhatsApp regularmente. Esto presenta una oportunidad para usar WhatsApp como un canal adicional para aumentar su alcance e interactuar con sus clientes de manera más efectiva. La mensajería de WhatsApp no necesita limitarse a interacciones uno a uno con sus clientes. También puede usarla para automatizar algunos de sus flujos de trabajo empresariales. Estos flujos de trabajo pueden ser parte de sus procesos corriendo en AWS, o puede construir nuevas automatizaciones usando servicios de AWS.

AWS End User Messaging Social vincula de forma nativa su WhatsApp Business Account (WABA) y su cuenta de AWS. Esto ayuda a simplificar la construcción de integraciones entre la aplicación de mensajería WhatsApp de su cliente y sus cargas de trabajo ejecutándose en AWS. Además, también proporciona una experiencia de facturación unificada dentro de AWS.

Esta publicación explica una solución para establecer esta integración y automatizar sus flujos de trabajo empresariales usando WhatsApp y AWS End User Messaging Social.

Visión general de la solución

La solución utiliza AWS End User Messaging Social para integrar su carga de trabajo implementada en AWS con su portafolio de negocios de Meta. Esto le permite recibir mensajes de WhatsApp que sus clientes finales envían, directamente en su entorno AWS. Un mensaje entrante de WhatsApp que AWS End User Messaging Social recibe activa una notificación de Amazon Simple Notification Service (SNS), que luego activa una función AWS Lambda. Esta función Lambda procesa la solicitud y luego envía un mensaje de WhatsApp como respuesta al usuario. Podrá construir flujos de trabajo empresariales complejos y marketing automatizado de WhatsApp expandiendo esta solución.

La arquitectura de la solución de la publicación muestra cómo el cliente envía un mensaje de WhatsApp a una carga de trabajo alojada en AWS. AWS End User Messaging Social recibe el mensaje de WhatsApp y, a continuación, lo reenvía a un tema de redes sociales. Esto activa una función Lambda. A continuación, la función Lambda envía una respuesta de WhatsApp al teléfono móvil del cliente a través de AWS End User Messaging Social.

Figura 1: Arquitectura de la Solución

 

IMPORTANTE: Tenga en cuenta que cualquier usuario de WhatsApp puede enviar un mensaje a su número WABA y activar el flujo de trabajo. Para evitar incurrir en cargos continuos, complete los pasos en la sección “Limpieza“.

Requisitos previos

Antes de comenzar esta guía, asegúrese de tener:

  1. Una cuenta de AWS
  2. Un portafolio de negocios de Meta. Cree uno siguiendo estas instrucciones
  3. Un número de teléfono para crear una WABA
  4. Un teléfono móvil con la aplicación WhatsApp Messenger instalada para probar la solución. Tenga en cuenta que la aplicación móvil usa un número de teléfono diferente al que está asociado con su WABA.
  5. AWS CLI instalado y configurado para acceder a su cuenta AWS
  6. Serverless Application Model CLI para implementar la aplicación de ejemplo
  7. node.js >= 22.x

Implementación de la solución

Paso 1: Crear un tema SNS

  1. Navegue a la página de SNS.
  2. Use la región de AWS donde planea usar AWS End User Messaging Social.
  3. Seleccione “Standard” como “Type” del tema
  4. Ingrese “WhatsAppIncomingMessages” como “Name” del tema
  5. Haga clic en el botón “Create topic“.
  6. En los detalles del tema que aparecen, tome nota del “ARN” del tema

Paso 2: Agregar WABA a AWS End User Messaging Social

  1. Navegue a la página WABA.
  2. Haga clic en el botón “Add WhatsApp phone number
  3. Haga clic en “Launch Facebook portal” para conectar su cuenta AWS y su portafolio de negocios de Meta.
  4. Siga las instrucciones en la ventana emergente de Facebook para completar el registro. A continuación se muestran capturas de pantalla de los pasos clave en el proceso:
Captura de pantalla de los pasos que debes seguir para conectar tu cuenta de AWS y tu cartera de negocios Meta.

Figura 2: Conectar AWS y el Portafolio de Negocios de Meta

  1. Después de que se cierre la ventana emergente de registro de Facebook, verá este mensaje “Link established” en su consola AWS. Ingrese el ARN del tema SNS creado en el Paso 1 en la sección “Message and event destination”.
Captura de pantalla de la consola de AWS que muestra la conexión establecida entre AWS y la cuenta WABA y el tema de SNS

Figura 3: Conexión exitosa entre la cuenta WABA y AWS

  1. Haga clic en “Add phone number” para completar la vinculación de su cuenta comercial de WhatsApp con su cuenta AWS.
  2. Navegue a la página WABA y verifique que la cuenta agregada esté en estado “Active”.
  3. Haga clic en el ID de cuenta comercial que agregó para navegar a la página de detalles.
  4. En la sección “Phone Numbers“, tome nota del “Phone number ID” del número que agregó.

Paso 3: Implementar la función Lambda para manejar los mensajes de WhatsApp de los clientes

  1. Clone el repositorio de ejemplo e implemente la función Lambda siguiendo los pasos a continuación:
$ git clone https://github.com/aws-samples/aws-end-user-messaging-social-automation.git

$ cd aws-end-user-messaging-social-automation/
  1. Construya el proyecto ejecutando este comando:
$ sam build
  1. Implemente la aplicación ejecutando el comando a continuación y siga las indicaciones:
$ sam deploy --guided
...
# Enter the details for questions asked
Stack Name [sam-app]: EndUserMessagingWhatsApp
AWS Region [us-east-1]: <AWS region where you have created the WABA account>
Parameter SNSTopicArn []: <ARN of the SNS topic created in Step 1>
Parameter PhoneNumberID []: <Phone number ID from Step 2>

Acepte los valores predeterminados del resto de las preguntas para completar la implementación. Esto creará una función Lambda que se activa cuando se envía un mensaje de WhatsApp a su número WABA.

Paso 4: Prueba de la solución

  1. Desde un teléfono que tenga WhatsApp instalado, envíe este mensaje a su número WABA:
Hello
  1. Valide que ve lo siguiente en respuesta al mensaje anterior:
    • Marca azul, indicando que el mensaje ha sido recibido y leído
    • Una reacción 👋 a su mensaje
    • Un mensaje de respuesta diciendo “Hello {nombre del perfil}, how can we help you?”
    • Una lista de opciones para elegir
  2. Seleccione una de las opciones
  3. Recibirá un mensaje estático que lo dirigirá a encontrar más información sobre AWS End User Messaging Social en la página de documentación. La interacción se verá como en la imagen siguiente:
Captura de pantalla de la interacción del usuario final en WhatsApp. El usuario envía un mensaje de inicio "Hello!", al que la empresa responde con 2 opciones para elegir. El cliente selecciona la opción de consultar el estado del pedido, a lo que la empresa responde con un enlace a la documentación sobre el servicio de mensajería para el usuario final.

Figura 4: Interacción del usuario final con WhatsApp con el negocio

Solución de problemas

Acceda a los registros de su función lambda navegando a la página de registros de CloudWatch y buscando “WhatsAppMessageHandler” en el cuadro de búsqueda. Seleccione el grupo de registros de su función Lambda y haga clic en el flujo de registro correspondiente al momento en que ejecutó la prueba. La función Lambda registra información cuando se recibe y procesa un mensaje. Sin embargo, los eventos SNS recibidos por Lambda y los mensajes de usuario extraídos se excluyen deliberadamente de los registros de CloudWatch. El evento contiene información como el número de teléfono del remitente y el nombre del perfil de WhatsApp que su empresa puede clasificar como sensible. Verifique las políticas de manejo de datos de su organización antes de registrar esta información. Encuentre más información sobre la estructura de mensajes de los diferentes tipos de mensajes en la documentación para desarrolladores de Meta.

También puede usar AWS CLI para enviar mensajes a los clientes. Puede enviar mensajes de tipo “text” si su cliente ya ha iniciado una conversación con usted. Ejecute el siguiente comando para enviar una respuesta a un mensaje de WhatsApp:

aws socialmessaging send-whatsapp-message \
--cli-binary-format raw-in-base64-out \
--message '{"messaging_product": "whatsapp", "to": "+15554567890", "text": { "preview_url": false, "body": "Hello! How can we help you?"}}' \
--origination-phone-number-id phone-number-id-beac123456789abcdefgh \
--meta-api-version v20.0

Tenga en cuenta que solo puede enviar mensajes de tipo “text” al cliente si han iniciado una conversación con usted y han pasado menos de 24 horas desde la última vez que le enviaron un mensaje. Use mensajes con plantillas para iniciar conversaciones con sus clientes, por ejemplo, para enviar una OTP (contraseña de un solo uso) o mensajes de marketing. Meta necesita aprobar las plantillas de mensajes antes de que pueda enviarlas a los clientes.

Monitoreo

AWS End User Messaging Social puede publicar datos relacionados con el costo de usar el servicio en las métricas de CloudWatch. Para habilitarlo, ingrese el comando a continuación para crear un rol vinculado al servicio IAM llamado “AWSServiceRoleForSocialMessaging“:

aws iam create-service-linked-role --aws-service-name social-messaging.amazonaws.com

Puede monitorear AWS End User Messaging Social usando CloudWatch para ver métricas:

  • WhatsAppMessageFeeCount
  • WhatsAppConversationFeeCount

También puede recibir notificaciones cuando se alcance un umbral específico configurando una alarma de CloudWatch en estas métricas. Esto puede ayudarlo a realizar un seguimiento de los costos asociados con AWS End User Messaging Social.

Limpieza

Tenga en cuenta que cualquier usuario de WhatsApp puede enviar un mensaje a su número WABA y activar el flujo de trabajo. Para evitar incurrir en cargos continuos, complete los siguientes pasos de limpieza:

  1. Para eliminar la función Lambda y los recursos asociados, ejecute el siguiente comando:
sam delete
  1. Para eliminar los recursos de AWS End User Messaging Social, navegue a la página WABA. Seleccione el ID de cuenta comercial de su WABA y haga clic en el botón “Unlink”.

Conclusión

Esta publicación muestra cómo puede simplificar la integración entre WhatsApp y sus cargas de trabajo ejecutándose en AWS, permitiéndole automatizar las interacciones con sus clientes finales. Muestra cómo puede activar una función Lambda desde un mensaje de WhatsApp iniciado por su cliente final. Puede extender esta solución para construir flujos de trabajo más complejos, por ejemplo, activar AWS Step Functions, invocar sus cargas de trabajo basadas en contenedores ejecutándose en AWS, entre otros.

Para más información, consulte los siguientes recursos:

Automate workflows with WhatsApp using AWS End User Messaging Social

Post Syndicated from Bruno Giorgini original https://aws.amazon.com/blogs/messaging-and-targeting/whatsapp-aws-end-user-messaging-social/

In today’s world, WhatsApp has become a widely used communication app, with over 2.7 billion users globally. As a business owner, you likely have customers who use WhatsApp regularly. This presents an opportunity to use WhatsApp as an additional channel to increase your reach and engage with your customers more effectively. WhatsApp messaging need not be limited to one-on-one interactions with your customers. You can also use it to automate some of your business workflows. These workflows can be part of your existing workloads running on AWS, or you can build new automations using AWS services.

AWS End User Messaging Social natively links your WhatsApp Business Account (WABA) and your AWS account. This helps simplify building integrations between your customer’s WhatsApp messenger app and your workloads running on AWS. Additionally, it also provides unified billing experience within AWS.

This post walks through a solution to establish this integration and automate your business workflows using WhatsApp and End User Messaging Social.

Overview of solution

The solution uses End User Messaging Social to integrate your workload deployed on AWS with your Meta business portfolio. This allows you to receive WhatsApp messages that your end customers send, directly in your AWS environment. An incoming WhatsApp message that End User Messaging Social receives triggers an Amazon Simple Notification Service (SNS) notification, which then activates an AWS Lambda function. This Lambda function processes the request and then sends a WhatsApp message as a response to the user. You can build complex business workflows and automated WhatsApp marketing expanding this solution.

Solution architecture of the post shows customer sending a WhatsApp message to a workload that is hosted on AWS. AWS End User Messaging Social receives the WhatsApp message. It then forwards the message to an SNS topic. This triggers a Lambda function. The Lambda function then sends a WhatsApp reply back to the customer’s mobile phone through End User Messaging Social

Figure 1: Solution Architecture

 

IMPORTANT: Note that any WhatsApp user can send a message to your WABA number and trigger the workflow. To avoid incurring ongoing charges, complete the steps in the “Cleaning up” section.

Prerequisites

Before starting this walkthrough, ensure you have:

  1. An AWS account
  2. A Meta business portfolio. Create one by following these instructions
  3. A phone number to create a WABA
  4. A phone with WhatsApp Messenger app installed to test the solution. Note that the mobile app uses a different phone number than the one that is associated with your WABA.
  5. AWS CLI installed and configured to access to your AWS account
  6. Serverless Application Model CLI to deploy the sample application
  7. node.js >= 22.x

Implementing the solution

Step 1: Create an SNS topic

  1. Navigate to the SNS page.
  2. Use the AWS region where you plan to use End User Messaging Social.
  3. Select “Standard” as the topic “Type”
  4. Enter “WhatsAppIncomingMessages” as the “Name” of the topic
  5. Click on the “Create topic” button.
  6. In the topics details that appear, note the “ARN” of the topic

Step 2: Add WABA to AWS End User Messaging

  1. Navigate to the WABA page.
  2. Click on the “Add WhatsApp phone number” button
  3. Click on “Launch Facebook portal” to connect your AWS account and your Meta business portfolio.
  4. Follow the instructions in the Facebook pop-up to complete the registration. Below are screenshots of the key steps in the process:
Screenshot of the steps you need to follow to connect your AWS account and your Meta business portfolio.

Figure 2: Connect AWS and Meta Business Portfolio

  1. After the Facebook signup popup closes, you will see this “Link established” message on your AWS console. Enter the ARN of the SNS topic created in Step 1 in the “Message and event destination” section.
Screenshot of the AWS console that shows established connection between AWS and WABA account and the SNS topic

Figure 3: Successfully linked WABA and AWS account

[Alt text: Screenshot of the AWS console that shows an established connection between AWS and WABA account and the SNS topic]

  1. Click “Add phone number” to complete linking your WhatsApp business account with your AWS account.
  1. Navigate to the WABA page and verify that the added account is in “Active” status.
  2. Click on the Business account ID you added to navigate to the details page.
  3. From the “Phone Numbers” section, note the “Phone number ID” of the number that you added.

Step 3: Deploy the Lambda function to handle customer WhatsApp messages

  1. Clone the example repository and deploy the Lambda function by following the steps below:
$ git clone https://github.com/aws-samples/aws-end-user-messaging-social-automation.git

$ cd aws-end-user-messaging-social-automation/
  1. Build the project by running this command:
$ sam build
  1. Deploy the application by running the command below and follow the prompts:
$ sam deploy --guided
...
# Enter the details for questions asked
Stack Name [sam-app]: EndUserMessagingWhatsApp
AWS Region [us-east-1]: <AWS region where you have created the WABA account>
Parameter SNSTopicArn []: <ARN of the SNS topic created in Step 1>
Parameter PhoneNumberID []: <Phone number ID from Step 2>

Accept the defaults of the rest of the questions to complete the deployment. This will create a Lambda function that is triggered when a WhatsApp message is sent to your WABA number

Step 4: Testing the solution

  1. From a phone that has WhatsApp installed, send this message to your WABA phone number
Hello
  1. Validate that you see the following in response to the previous message:
    1. Blue check mark, indicating the message has been received and read
    2. A 👋 reaction to your message
    3. A reply message saying “Hello {profile name}, how can we help you?”
    4. A list of options to choose from
  2. Select one of the options
  3. You will receive a static message pointing you to find more information about the End User Social in the documentation page. The interaction will look like the picture below:
Screenshot of the end user’s WhatsApp interaction. User sends a Hello! Message, to which the business responds with 2 options to choose from. Customer selects check order status option, to which the business responds with a link to documentation about End user messaging service.

Figure 4: End user’s WhatsApp interaction with the business

Troubleshooting

Access the logs of your lambda function by navigating to the CloudWatch logs page and searching for “WhatsAppMessageHandler” in the search box. Select the log group of your Lambda function and click on the log stream corresponding to the time you ran the test. The Lambda function logs information when a message is received and processed. However, Lambda’s received SNS events and extracted user messages are deliberately excluded from CloudWatch logs. The event contains information such as sender’s phone number and WhatsApp profile name that can be classified as sensitive by your business. Check your organization’s data handling policies before logging this information. Find more information on the message structure of the different message types in Meta developer documentation.

You can also use AWS CLI to send messages to customers. You can send messages of “text” type if your customer has already initiated a conversation with you. Run the command below to send a reply to a WhatsApp message:

aws socialmessaging send-whatsapp-message \
--cli-binary-format raw-in-base64-out \
--message '{"messaging_product": "whatsapp", "to": "+15554567890", "text": { "preview_url": false, "body": "Hello! How can we help you?"}}' \
--origination-phone-number-id phone-number-id-beac123456789abcdefgh \
--meta-api-version v20.0

Note that you can only send messages of type “text” to customer if they have started a conversation with you and it has been less than 24 hours since the last time they messaged you. Use templated messages to start conversations with your customers, for example, to send an OTP (one-time password) or marketing messages. Meta needs to approve message templates before you can send them to customers.

Monitoring

End User Messaging Social can publish data related to cost of using the service to CloudWatch metrics. To enable this, enter the command below to create a IAM service linked role called “AWSServiceRoleForSocialMessaging”:

aws iam create-service-linked-role --aws-service-name social-messaging.amazonaws.com

You can monitor AWS End User Messaging Social using CloudWatch to view metrics:

  • WhatsAppMessageFeeCount
  • WhatsAppConversationFeeCount

You can get also get notified when a specific threshold is reached by setting up a CloudWatch alarm on these metrics. This can help you keep track of costs associated with End User Messaging Social.

Cleaning up

Note that any WhatsApp user can send a message to your WABA number and trigger the workflow. To avoid incurring ongoing charges, complete the following cleanup steps:

  1. To delete the Lambda function and the associated resources, run the following command:
sam delete
  1. To delete the End User Messaging Social resources, navigate to the WABA page. Select the Business account ID of your WABA and click on the “Unlink” button.

Conclusion

This post shows how you can simplify the integration between WhatsApp and your workloads running on AWS, allowing you to automate interactions with your end customers. It shows how you can trigger a Lambda function from a WhatsApp message initiated by your end customer. You can extend this solution to build more complex workflows, for example triggering an AWS Step Functions , invoking your container based workloads running on AWS among others.

For more information, see the following resources:

Build a Secure One-Time Password Architecture with AWS

Post Syndicated from Bruno Giorgini original https://aws.amazon.com/blogs/messaging-and-targeting/build-a-secure-one-time-password-architecture-with-aws/

In today’s digital landscape, where cyberattacks continue to grow more sophisticated, the need for robust security measures has never been more paramount. One-Time Passwords (OTPs) have long been a crucial component of multi-factor authentication. They provide an additional layer of security to protect user accounts from unauthorized access.

The landscape of OTP delivery is evolving rapidly. While organizations increasingly favor more secure, phishing-resistant methods like passwordless solutions and hardware security keys, many still rely on SMS-based OTPs or require time to transition to newer technologies.

For organizations already leveraging Okta as their identity provider, AWS offers a comprehensive guidance on implementing phone-based multi-factor authentication. The “AWS Guidance for Okta Phone-Based Multi-Factor Authentication on AWS” provides a detailed reference architecture and implementation steps for integrating Okta with AWS services to deliver OTPs via SMS or voice calls.

This blog post offers a comprehensive guide for implementing a reliable, multi-channel OTP solution using AWS services including Amazon DynamoDB, Amazon Simple Email Service (SES), and AWS End User Messaging.

By the end of this blog, you’ll understand how to generate, store, and deliver OTPs via email, SMS, and voice. You’ll also learn best practices for secure OTP implementation. This solution serves organizations that need to maintain SMS-based OTP capabilities.

Let’s explore how to build a secure, multi-channel OTP solution on AWS.

AWS End User Messaging is the new name for Amazon Pinpoint’s SMS, MMS, push, WhatsApp, and text to voice messaging capabilities.

The authentication flow

Let’s imagine a hypothetical scenario where a bank customer want’s to access his online account:

Alex, a customer of the XYZ financial institution, needed to access their online account. They initiated the login process and requested an OTP from the mobile or web application provided by the bank. Upon receiving the request, the bank’s server created a user-specific session to handle the OTP generation and verification. A unique one-time password was then generated and sent to Alex’s registered mobile number via SMS. Alex received the OTP on their phone and had three attempts to enter the correct code within a 10-minute timeframe. This security measure prevented unauthorized access to their account. If Alex couldn’t receive the SMS, they had another option. They could request the bank to send the same OTP to their registered email address, if they had one on file. If Alex entered the correct OTP, the login process would be successful, and they would be granted access to their online banking services. However, if they exceeded the three attempts in the 10-minute time limit, their ability to login to the account would be temporarily suspended for security reasons and Alex would have to call the bank to lift the suspension or wait 2 hours to retry again. The bank implements this multi-factor authentication process with an alternative email-based OTP delivery. This approach safeguards Alex’s sensitive financial information and enhances the security of digital banking services. It also provides a backup option if the primary SMS channel is unavailable.

OTP user flow

Prerequisites

To use the code examples provided in this blog post, you’ll need to have the following AWS resources in place:

  1. AWS Account: Sign up for an AWS account at AWS website if you don’t have one.
  2. Verified Email Address in Amazon SES: Enable email delivery of OTPs by verifying an email address in Amazon SES service.
  3. AWS End User Messaging Configuration: You’ll need to configure the necessary origination identity in the AWS End User Messaging service to deliver the OTPs via SMS or voice.

With these prerequisites in place, you’ll be ready to use the code examples provided in the following sections to implement your secure OTP solution.

Architecture:

OTP architecture

Flow Explained:

  1. The user initiates the process by requesting an OTP.
  2. The request is sent through Amazon API Gateway.
  3. AWS Lambda receives the request and processes it.
  4. AWS KMS is used to encrypt the OTP for secure storage.
  5. The encrypted OTP and related information are stored in Amazon DynamoDB.
  6. AWS End User Messaging is used to send the OTP to the user via SMS, email, or voice, Amazon SES is used to send the OTP over email.
  7. When the user receives the OTP, they enter it in the portal for verification. The verification process encrypts the value with the key from AWS KMS and goes through the same flow (API Gateway -> Lambda)
  8. The Lambda decrypts the OTP for verification using KMS and compares it with the stored value in DynamoDB, which is also decrypted using the same KMS key.

Typical architecture for a secure one-time password (OTP) solution would involve the following components:

  1. Front-end Application: The OTP functionality is typically exposed through a web or mobile application, which serves as the user-facing interface.
  2. API Gateway: The front-end application interacts with the OTP solution through an API Gateway. This gateway serves as the entry point, providing scalable and secure access to underlying services.
  3. AWS Lambda: The business logic for generating, storing, and verifying the OTPs is handled by one or more AWS Lambda functions. These serverless functions are responsible for the core OTP-related operations.
  4. AWS KMS: Encrypts the OTP submitted for verification by the customer on the client side. AWS Lambda then decrypts it before verifying it against the OTP stored in Amazon DynamoDB.
  5. Amazon DynamoDB: The generated OTP encrypted and associated metadata, such as creation timestamp and expiration, are securely stored in an Amazon DynamoDB table.
  6. AWS End User Messaging: Used to deliver the OTPs to the users through various communication channels, such as SMS, and voice.
  7. Amazon SES: Deliver the OTPs to the users via email.

In a production environment, it’s also important to consider the following security measures:

  • AWS WAF (Web Application Firewall): To protect the API Gateway from common web-based attacks, such as SQL injection and cross-site scripting (XSS).
  • Authentication and Authorization Services: Ensuring that the front-end application and users are properly authenticated and authorized before accessing the OTP-related functionality. Visit Control and manage access to REST APIs in API Gateway to view the available methods of managing access to Amazon API Gateway.

This architecture enables organizations to build a comprehensive and secure one-time password solution. It protects users’ sensitive information and offers a seamless authentication experience.

Generating OTPs

To generate the OTPs, the server used the pyotp (link) library in Python. This library provides a secure random number generator to create unique, hexadecimal-encoded tokens. The server-side generation ensures that the OTPs are truly random and unpredictable, a crucial requirement for effective one-time password authentication.

The server generates a 6-character hexadecimal OTP, creating approximately 16.8 million possible unique combinations. This approach keeps codes short and easy for users to enter while maintaining security. After generation, the server securely stores the OTP and sends it to the user through the chosen delivery channel (SMS, email, or voice).

Sample Code:

import secrets
import pyotp

def generate_otp():
    """
    Generates a secure one-time password using the pyotp library.
    
    Returns:
        str: The generated one-time password.
    """
    # Generate a random base32 secret - https://pyauth.github.io/pyotp/
    totp = pyotp.TOTP(pyotp.random_base32())
    
    # Use the Time-based One-Time Password (TOTP) algorithm to generate a 6-digit OTP
    return totp.now()

It’s important to note that the generated OTP values should be encrypted on the client-side before being sent to the server for storage. This can be achieved by using AWS Key Management Service (KMS) to securely encrypt the OTP values.

By encrypting the OTP values before storing them in the DynamoDB table, you can further enhance the security of the solution and protect against potential data breaches. The encrypted values ensure that even if the database is compromised, the raw OTP values are not directly accessible.

Next, the encrypted OTP values are stored in the DynamoDB table, along with necessary metadata to manage the OTP lifecycle. This metadata includes creation timestamp, expiration, and verification attempts. The specifics of this storage process are covered in the ‘Securely Storing OTPs’ section.

Securely Storing OTPs

Once generated, the OTPs will be stored in an Amazon DynamoDB table. DynamoDB is a fully managed NoSQL database service that provides reliable, high-performance data storage and retrieval, making it an ideal choice for our secure OTP solution. To store the OTPs, you’ll create a DynamoDB table with the user_id as the primary key. This approach ensures that the same user can’t generate multiple OTPs. The put_item() operation will fail if it encounters a duplicate user_id value. Depending your use case, you can change this to be a random id or a concatenation of the user id and a random id.

Once generated, the OTPs are stored in an Amazon DynamoDB table. DynamoDB, a fully managed NoSQL database service, provides reliable, high-performance data storage and retrieval, making it ideal for our secure OTP solution.

To store the OTPs, create a DynamoDB table with the user_id as the primary key. This approach allows for efficient retrieval of a user’s current OTP. When storing a new OTP for a user:

  1. If no existing entry is found for the user_id, a new item is created.
  2. If an entry already exists, it’s updated with the new OTP, effectively overwriting the old one.

This method ensures that each user has only one active OTP at a time, while still allowing users to request new OTPs when needed (for example, if the previous one expired).

Depending on your use case, you can modify the primary key to be a random id or a concatenation of the user id and a random id for additional security.

In addition to the user_id and otp_code, we’ll also include the following attributes:

    • creation_timestamp: The timestamp indicating when the OTP was generated. This is compared with the timestamp of each attempt to ensure all attempts fall within the allowed time window.
    • ttl: The Unix timestamp representing the time-to-live (TTL) for the OTP, after which the DynamoDB item will be automatically deleted. Set this value to 24 hours from the creation time. This allows for a reasonable cleanup period while ensuring expired OTPs are removed from the database.
    • attempts: The number of remaining verification attempts for the OTP.
    • verified: A boolean flag indicating whether the OTP has been successfully verified.
    • locked: A boolean flag indicating whether the user’s account has been locked due to exhausted verification attempts.

Sample Code:

import time 
import boto3 
from datetime import datetime, timedelta 

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('otp_main')
def store_otp(user_id, otp_code):
    """
    Stores the generated one-time password in an Amazon DynamoDB table with a creation timestamp, TTL, and remaining attempts.
    
    Args:
        user_id (str): The unique identifier for the user.
        otp_code (str): The generated one-time password.
    
    Returns:
        dict: The response from the DynamoDB put_item operation.
    """
    # Get the current timestamp
    creation_timestamp = datetime.now().isoformat()
    
    # Calculate the expiration time for the OTP (10 minutes from now)
    expiration_time = datetime.now() + timedelta(minutes=10)
    
    # Convert the expiration time to a Unix timestamp for the DynamoDB TTL
    ttl_value = int(time.mktime(expiration_time.timetuple()))
    
    # Store the OTP, creation timestamp, TTL, remaining attempts, and verification status in the DynamoDB table
    response = table.put_item(
        Item={
            'user_id': user_id,
            'otp_code': otp_code,
            'creation_timestamp': creation_timestamp,
            'ttl': ttl_value,
            'attempts': 3,
            'verified': False,
            'locked': False
        }
    )
    
    return response

We use the user_id as the primary key and store creation timestamp, TTL, remaining attempts, verification status, and account lock status. This approach ensures a secure and efficient OTP storage and retrieval process. This approach also allows for precise management of OTP expiration and account locking, as demonstrated in the Verifying OTPs section.

The encrypted OTP values are stored in the otp_code attribute. Encryption is performed on the client-side using a secure key management solution, like the AWS KMS client-side library. This ensures that the raw OTP values are never transmitted or stored in plain text, further enhancing the security of the solution.

Note: As an optional enhancement, you could use Amazon SQS with a visibility timeout set to the OTP validity period. A payload containing the user_id is sent to a Lambda function. After the visibility timeout, the function processes the SQS message and deletes the corresponding DynamoDB item. This approach provides greater precision compared to relying solely on DynamoDB TTL, though it adds complexity to the implementation. The current solution compares each verification attempt’s timestamp with the creation timestamp, ensuring that no attempts occur after the OTP has expired.

Delivering OTPs via Multiple Channels

Now that we have a secure way to generate and store the OTPs, it’s time to focus on delivering them to your users. Our solution leverages the AWS End User Messaging capabilities to provide a seamless and redundant OTP delivery experience across multiple communication channels.

Sending OTPs via SMS and Voice

AWS End User Messaging offers a versatile platform for OTP delivery across multiple channels, including email, SMS, voice calls, push notifications, and WhatsApp. This provides a redundant and convenient authentication experience for your users, ensuring they can receive their one-time passwords via their preferred method.

Sample Code:

import boto3

def send_otp_sms(mobile_number, otp_code, user_id, region_name):
    """
    Sends an OTP code to the user's mobile number using AWS End User Messaging SMS.
    
    Args:
        mobile_number (str): The phone number to send the OTP to.
        otp_code (str): The one-time password to be sent.
        user_id (str): The unique identifier for the user.
        region_name (str): The AWS region to use for the SESv2 client.
    
    Returns:
        dict: The response from the End User Messaging SMS send_text_message operation.
    """
    # Construct the SMS message with the OTP code
    message = f"""
    This is an AWS End User Messaging OTP message.

    Your one-time password is: {otp_code}.
    """
    
    try:
        # Create a new SMS-Voice v2 client
        aws_sms = boto3.client('pinpoint-sms-voice-v2')
        # Use the End User Messaging SMS client to send the SMS message
        response = aws_sms.send_text_message(
            DestinationPhoneNumber=mobile_number,
            MessageBody=message,
            MessageType='TRANSACTIONAL'
        )
        return {'StatusCode': 200, 'Response': response['MessageId']}
    except ClientError as e:
        error_message = e.response['Error']['Message']
        return {'StatusCode': 500, 'Response': error_message}

Sending OTPs via Email

To deliver OTPs via email, we’ll use the Amazon SES (Simple Email Service) SendEmail API. SES is a highly scalable and cost-effective email service. It can send notifications, alerts, and in our case, one-time passwords to users.

Sample Code:

import boto3
from botocore.exceptions import ClientError

def send_otp_email(user_id, email_address, otp_code, region_name):
    """
    Sends an OTP code to the user's email address using Amazon SESv2.
    
    Args:
        user_id (str): The unique identifier for the user.
        email_address (str): The email address to send the OTP to.
        otp_code (str): The one-time password to be sent.
        region_name (str): The AWS region to use for the SESv2 client.
    
    Returns:
        dict: The response from the SESv2 send_email operation.
    """
    try:
        # Create a new SESv2 client
        ses = boto3.client('sesv2', region_name=region_name)

        # Construct the email message with the OTP code
        message = "<p>Your one-time password is: </p> {otp_code}"
        html_body = message.format(otp_code=otp_code)

        # Use the SESv2 client to send the email
        response = aws_email.send_email(
            FromEmailAddress='[email protected]',
            Destination={
                'ToAddresses': [
                    email_address,
                ]
            },
            Content={
                'Simple': {
                    'Subject': {
                        'Charset': 'UTF-8',
                        'Data': 'Your AWS OTP code'
                    },
                    'Body': {
                        'Html': {
                            'Charset': 'UTF-8',
                            'Data': html_body
                        }
                    }
                }
            }
        )
        return {'StatusCode': 200, 'Response': response['MessageId']}
    except ClientError as e:
        error_message = e.response['Error']['Message']
        return {'StatusCode': 500, 'Response': error_message}

Verifying OTPs

The final piece of our secure OTP solution is the process of verifying the one-time passwords entered by your users. This is a crucial step in the authentication flow, as it ensures that only legitimate users are granted access to your applications or services.

The OTP verification logic is handled by a Lambda function that interacts directly with the DynamoDB table where the OTPs are stored. This Lambda function performs the following steps:

  1. Retrieve the stored OTP and its associated metadata from the DynamoDB table, using the user_id as the primary key. This metadata includes the creation timestamp and the number of remaining attempts.
  2. Decrypt the retrieved OTP value using the KMS client-side library, as the OTP was encrypted on the client side before being stored.
  3. Compare the unencrypted OTP value with the one entered by the user.
  4. Verify that the OTP has not expired by comparing the creation timestamp with the current time.
  5. If the OTP is valid and not expired, update the verification status in the DynamoDB table and delete the corresponding item.
  6. If the OTP is invalid or expired, deduct an attempt from the remaining attempts count stored in the DynamoDB table.
  7. If the remaining attempts count reaches zero, lock the user’s account and return an appropriate response.

Sample Code:

import boto3
from boto3.dynamodb.conditions import Key
from datetime import datetime, timedelta

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('otp_main')

def verify_otp(otp_entered, user_id):
    """
    Verifies the one-time password entered by the user against the stored OTP in DynamoDB.
    
    Args:
        otp_entered (str): The one-time password entered by the user.
        user_id (str): The unique identifier for the user.
    
    Returns:
        dict: The result of the OTP verification, containing the verification status and the OTP code.
    """
    try:
        # Query the DynamoDB table to find the stored OTP for the given user
        response = table.query(
            KeyConditionExpression=Key('user_id').eq(user_id)
        )
        
        if 'Items' in response and response['Items']:
            for item in response['Items']:
                # decrypt the retrieved OTP value using the KMS client-side library
                if str(otp_entered) == decrypt_otp(item['otp_code'], user_id):
                    # Check if the OTP has expired
                    creation_timestamp = datetime.fromisoformat(item['creation_timestamp'])
                    if datetime.now() - creation_timestamp < timedelta(minutes=10):
                        # Update the verification status and delete the DynamoDB item
                        update_item = table.update_item(
                            Key={'user_id': user_id},
                            UpdateExpression='SET verified = :verified',
                            ExpressionAttributeValues={':verified': True}
                        )
                        table.delete_item(Key={'user_id': user_id})
                        return {'result': True, 'otp': item['otp_code']}
                    else:
                        # Deduct an attempt from the remaining attempts count
                        update_item = table.update_item(
                            Key={'user_id': user_id},
                            UpdateExpression='SET attempts = attempts - :1',
                            ExpressionAttributeValues={':1': 1}
                        )
                        if item['attempts'] <= 0:
                            # Lock the account if the attempts are exhausted
                            update_item = table.update_item(
                                Key={'user_id': user_id},
                                UpdateExpression='SET locked = :locked',
                                ExpressionAttributeValues={':locked': True}
                            )
                        return {'result': False, 'otp': item['otp_code']}
                else:
                    # Handle invalid OTPs
                    pass
        
        # If the OTP is not found or does not match, return a failure result
        return {'result': False, 'otp': None}
    except Exception as e:
        error_message = str(e)
        return {'result': False, 'error': error_message}

def decrypt_otp(encrypted_otp, user_id):
    """
    decryptes the OTP value using the KMS client-side library.
    
    Args:
        encrypted_otp (str): The encrypted OTP value stored in DynamoDB.
        user_id (str): The unique identifier for the user.
    
    Returns:
        str: The unencrypted OTP value.
    """
    # decrypt the OTP value using the KMS client-side library
    return decrypt_using_kms(encrypted_otp, user_id)

 

In this implementation, a Lambda function handles the OTP verification logic. This ensures sensitive operations like OTP decryption and managing expiration and attempt counts occur in a secure, serverless environment.

Best Practices

As you implement a secure one-time password solution, it’s important to consider the following best practices:

OTP Message Best Practices

When delivering OTPs via email, SMS, or voice, clearly specify the sender and the content of the message. For example, the email subject and body, as well as the SMS or voice message, should include information like:

“This is a one-time password from [Company Name] for payment confirmation of your flight ABC123.”

Security Reminder in OTP Messages

Include a security reminder in the OTP message to encourage users to report any unauthorized access attempts. For example:

“If you did not request this OTP, please call [phone number] to report it.”

This helps raise user awareness and provides a clear course of action if they suspect their account has been compromised.

Configuration Set / Originating Identity

Include appropriate configuration sets and Context or EmailTags when using AWS End User Messaging services to deliver OTPs. This records message delivery events and traces them to your organization. Read more about Amazon SES and AWS End User Messaging configuration sets.
For example, in the send_otp_sms() and send_otp_email() functions, you should include the following parameters:

response = aws_sms.send_text_message(
    DestinationPhoneNumber=mobile_number,
    MessageBody=message,
    MessageType='TRANSACTIONAL',
    ConfigurationSetName='otp-config-set',
    OriginationNumber='+12345678901',
    Context={
        'user_id': user_id
    }
)
response = ses.send_email(
    FromEmailAddress='[email protected]',
    Destination={'ToAddresses': [email_address]},
    Content={
        # ...
    },
    ConfigurationSetName='otp-config-set',
    EmailTags=[
        {
            'Name': 'user_id',
            'Value': user_id
        }
    ]
)

Deleting OTPs After Verification

After a successful OTP verification, it’s recommended to delete the corresponding DynamoDB item. This helps maintain a clean and up-to-date database, reducing the risk of unauthorized access or potential data breaches.

Tracking Verification Attempts

Consider adding a column in the DynamoDB table to track the number of verification attempts for each OTP. This can help you implement rate-limiting and other security measures to prevent brute-force attacks.

Encrypting OTPs on the Client-side

As mentioned earlier, the OTP values should be encrypted on the client-side using a secure key management solution, such as the AWS KMS client-side library. This ensures that the raw OTP values are never transmitted or stored in plain text, further enhancing the security of the solution.

Following these best practices ensures your one-time password solution is secure and user-friendly. It also maintains necessary controls and traceability for production use cases.

Conclusion

In this guide, we’ve demonstrated a secure, multi-channel One-Time Password (OTP) solution using AWS services. You can now generate, store, and deliver OTPs via email, SMS, and voice channels using Amazon DynamoDB, Amazon SES, and AWS End User Messaging.

We’ve covered several important points throughout this process. We discussed using a secure random number generator and encrypting algorithms to generate and store OTPs. This ensures strong protection for your users’ sensitive information. By integrating with Amazon SES and AWS End User Messaging, you provide users with a convenient, redundant authentication experience through multiple channels.

This guide equips you with tools to maintain SMS-based OTP capabilities. However, it’s important to note the industry’s shift towards more secure, phishing-resistant authentication methods. These include passwordless solutions and hardware security keys. We encourage you to explore and implement these newer technologies as you develop your OTP solution.

Looking ahead, consider potential enhancements to this solution. Integrating support for standards like FIDO2 WebAuthn and Passkeys could allow seamless authentication without traditional OTPs. Keep these options as backup or alternative methods. Also, consider incorporating a mechanism to escalate users to live support for authentication issues.

Implement the secure OTP solution outlined in this guide and continuously update your authentication strategies. This approach ensures your organization remains equipped to protect users and assets from evolving digital threats.

Automate AWS End User Messaging US toll-free Number Registrations

Post Syndicated from Bruno Giorgini original https://aws.amazon.com/blogs/messaging-and-targeting/automate-us-tfn-registrations/

Introduction:

AWS End User Messaging enables customers to send SMS messages to recipients located across the globe.
If you are planning on rolling out SMS across multiple countries, we recommend reviewing this blog. When you send SMS or MMS messages using AWS End User Messaging SMS, you must use a specific origination identity that supports the sending of SMS. In the US the originators that are supported are Toll-Free(TFN), 10DLC, and short codes and each of these has its own registration process.

This blog discusses the process of programmatically registering Toll-Free numbers (TFNs), which can be used exclusively within the United States. TFNs have a max throughput of 3 Messages Per Second(MPS) and are typically used for low-volume/throughput use cases. You can learn more about TFNs here.

TFNs have a relatively simple registration process, however, for Independent Software Vendors (ISVs), SaaS providers, or large organizations with many teams or use cases, completing dozens, or hundreds of TFN registrations manually can be time-consuming, tedious, and error-prone. AWS End User Messaging APIs can be used to streamline the TFN registration process by enabling AWS customers to programmatically register TFNs in the USA. You can learn more about the TFN registration requirements and process here.

In this post, we will discuss the AWS End User Messaging APIs required for programmatically registering TFNs. We’ll explore a simple Bash script you can use to automate the registration of a single TFN, as well as a Python script that you can use to bulk register multiple TFNs. Both of these scripts help simplify the TFN registration process and can save both time and effort for businesses looking to leverage AWS End User Messaging for their communication needs.

Outline of the APIs used in the scripts:

There are seven actions that need to be taken in order for a TFN registration to be created and submitted. The two scripts in this post automate these actions:

  1. create-registration
    • Creates a new registration
    • The “RegistrationType” field controls whether this is a registration for a Toll-Free, or 10DLC or a SenderID.
  2. describe-registration-field-definitions
    • Retrieves the specified RegistrationType field definitions.
    • You can use DescribeRegistrationFieldDefinitions to view the requirements for creating, filling out, and submitting each registration type. In this post we have chosen Toll-Free and provided the values.
  3. create-registration-attachment
    • The carriers require you to provide a mockup that closely resemble the opt-in experience that your customers will complete to ensure end-users have consented to receiving SMS messages from this toll-free number. This should be a screenshot of your website or mobile application’s opt-in workflow.
    • The maximum file size is 500kb, and valid file extensions are PDF, JPEG, or PNG.
    • You need to wait until the attachment is uploaded before moving to step 4.
  4. put-registration-field-value
    • This action needs to be repeated for all required fields (retrieved in step 2)
  5. request-phone-number
    • Request an origination phone number for use in your account (needed for step 6)
  6. create-registration-association
    • This associates the registration with the origination identity (phone number) you requested in step 5.
    • You will need the PhoneNumberId for the origination identity.
  7. submit-registration-version
    1. This will submit the specified registration for review and approval
    2. IMPORTANT:
      • Make sure that all of your data is correct, especially before attempting to submit multiple registrations using the second script option below.
      • Once your script has submitted the registration, it’s initial status will be “CREATED” and should change to “REVIEWING” within 24 hours.
      • Once submitted, you will be unable to edit or delete this registration until it is approved or rejected by the third-party registrar that controls the registration process for the type of registration you are submitting (in this blog we’re registering TFNs).
      • You begin incurring costs for the TFNs as soon as you successfully submit the registration.

No matter which script you decide to use, it’s important to keep a these things in mind:

  1. The registration information you provide will be reviewed by a third-party company. This is standard, no matter which provider you use for SMS
  2. While AWS submits the registration on your behalf, we do not participate in the actual review process, nor can we influence the third-party.
  3. The review process for TFNs can take up to 15 business days and if the information provided in the registration process is incomplete, inaccurate, or the use case falls into a forbidden category (Guidelines for using toll-free numbers) , the application can be rejected.
  4. If your toll-free number registration is rejected, the status of your registration will change to “Requires Updates“. We recommend monitoring your registration status frequently to ensure that you do not miss an update and extend the time it takes to complete.

Prerequisites for either approach:

Before running either script, you’ll need:

  1. An AWS account with permission to use/provision the AWS End User Messaging service (link) in the target region.

The following information for each TFN you wish to register/request (you can retrieve this list with describe-registration-field-definitions API):

Company Info
  • Company Name
  • Website
  • Address 1
  • Address 2 (optional)
  • City
  • State / Province
  • Zip Code / Postal Code
  • Country Code
Contact Info
  • First Name
  • Last Name
  • Support Email
  • Support Phone Number
Messaging Use Case
  • Monthly Message Volume (you must use one of the options below)
    • “10”
    • “100”
    • “1,000”
    • “10,000”
    • “100,000”
    • “250,000”
    • “500,000”
    • “750,000”
    • “1,000,000”
    • “5,000,000”
    • “10,000,000+”
  • Use Case Category (you must use one of the options below)
    • “Two-factor authentication”
    • “One-time passcodes”
    • “Notifications”
    • “Polling and surveys”
    • “Info on demand”
    • “Promotions & marketing”
    • “Other”
  • Use Case Details
  • Opt-In Description – The primary purpose of the Opt-in Description is to demonstrate that the end user explicitly consents to receive text messages and understands the nature of the program. Your application is being reviewed by a 3rd party reviewer, so make sure to provide clear and thorough information about how your end-users opt-in to your SMS service and any associated fees or charges. If the reviewer cannot determine how your opt-in process works then your application will be denied and returned. If your Opt-in process requires a log-in, is not yet published publicly, is a verbal opt-in, or if it occurs on printed sources such as fliers and paper forms then make sure to thoroughly document how this process is completed by the end-user receiving messages. Provide a screenshot and host the screenshot on a publicly accessible website (like OneDrive or Google Drive) and provide the URL in the text as well as in the Opt-In Image below.
  • Opt-In Image – This is optional, but highly recommended, even if you provide a description above:
    • If your experience is not publicly available, the carriers will require you to provide one to ensure end-users have consented to receiving SMS messages from this toll-free number.
    • You are encouraged to provide a screenshot (PDF, JPEG, or PNG) of your website or mobile application’s opt-in workflow. This could also be a screenshot of a paper form, a verbal opt-in script, or an as yet published website experience in development.
    • The maximum file size is 500Kb.
Message Samples
  • Message Sample 1
  • Message Sample 2 (optional)
  • Message Sample 3 (optional)

Option 1 – Execute a script to register a single TFN

The script requires the additional prerequisites (if you use AWS CloudShell to run the script, the AWS CLI and jq are already installed).

  • AWS CLI installed and configured with appropriate AWS account credentials and region (link)
  • jq (a lightweight and flexible command-line JSON processor) installed (link)
  1. Set environment variables for each of the required fields.

Before running the registration script, you must first set the necessary variables, replacing the placeholder values with your company actual data (these values are listed as a prerequisite above). Ensure that the opt-in screenshot image file optInImage.png is in the same directory as the script, or provide the correct path to the image file in the attachment_body variable. Note – the allowed file types for the opt-in screenshot are PNG, JPG, and PDF with a max file size of 500 kb.

Run the environment variable exports:

export AWS_REGION="us-west-2"
export COMPANY_NAME="Your Company Name"
export COMPANY_WEBSITE="www.yourcompany.com"
export COMPANY_ADDRESS1="123 Main Street"
export COMPANY_ADDRESS2="Suite 200"
export COMPANY_CITY="Your City"
export COMPANY_STATE="Your State"
export COMPANY_ZIPCODE="12345"
export COMPANY_COUNTRY_CODE="US"
export CONTACT_FIRSTNAME="John"
export CONTACT_LASTNAME="Doe"
export CONTACT_EMAIL="[email protected]"
export CONTACT_PHONE="+1234567890"
export USECASE_DETAILS="Details about your use case"
export OPTIN_DESCRIPTION="Description of opt-in process"
export MONTHLY_MESSAGE_VOLUME="10"
export USECASE_CATEGORY="Other"
export MESSAGE_SAMPLE1="Sample message"
export ATTACHMENT_BODY="fileb://optInImage.png"
  1. Copy and save the below script as register_us_toll_free_number.sh into your current directory. Adjust the region variable in the script to the one you’re using to register the US TFN.
#!/bin/bash
set -euo pipefail

# Check dependencies
command -v aws >/dev/null 2>&1 || { echo "AWS CLI is required but it's not installed. Aborting."; exit 1; }
command -v jq >/dev/null 2>&1 || { echo "jq is required but it's not installed. Aborting."; exit 1; }

# Initialize variables from environment or use default values
region="${AWS_REGION:-'us-west-2'}"
companyInfo_companyName="${COMPANY_NAME:-"Default Company Name"}"
companyInfo_website="${COMPANY_WEBSITE:-"https://defaultcompany.com"}"
companyInfo_address1="${COMPANY_ADDRESS1:-"123 Default St."}"
companyInfo_address2="${COMPANY_ADDRESS2:-""}"  # Optional
companyInfo_city="${COMPANY_CITY:-"Default City"}"
companyInfo_state="${COMPANY_STATE:-"Default State"}"
companyInfo_zipCode="${COMPANY_ZIPCODE:-"00000"}"
companyInfo_isoCountryCode="${COMPANY_COUNTRY_CODE:-"US"}"
contactInfo_firstName="${CONTACT_FIRSTNAME:-"John"}"
contactInfo_lastName="${CONTACT_LASTNAME:-"Doe"}"
contactInfo_supportEmail="${CONTACT_EMAIL:-"[email protected]"}"
contactInfo_supportPhoneNumber="${CONTACT_PHONE:-"+10000000000"}"
messagingUseCase_useCaseDetails="${USECASE_DETAILS:-"Default use case details"}"
messagingUseCase_optInDescription="${OPTIN_DESCRIPTION:-"Default opt-in process description"}"
messagingUseCase_monthlyMessageVolume="${MONTHLY_MESSAGE_VOLUME:-"10"}"
messagingUseCase_useCaseCategory="${USECASE_CATEGORY:-"Other"}"
messageSamples_messageSample1="${MESSAGE_SAMPLE1:-"This is a sample message."}"
attachment_body="${ATTACHMENT_BODY:-"fileb://optInImage.png"}"  # Path to the image, can be overridden

# Log to file
log_file="registration.log"
echo "Logging to $log_file"
exec > >(tee -a "$log_file") 2>&1

# Retry configuration
max_retries=10
retry_interval=2
retry_count=0
upload_complete=false

# Helper function to put registration field values
put_registration_field_value() {
    local field_path="$1"
    local value="$2"
    aws pinpoint-sms-voice-v2 --region "$region" put-registration-field-value \
         --registration-id "$registration_id" \
         --field-path "$field_path" \
         --text-value "$value" || { echo "Failed to update $field_path"; exit 1; }
}

# Helper function to validate registration field values
validate_input() {
    local required_vars=(
        "AWS_REGION"
        "COMPANY_NAME"
        "COMPANY_WEBSITE"
        "COMPANY_ADDRESS1"
        "COMPANY_CITY"
        "COMPANY_STATE"
        "COMPANY_ZIPCODE"
        "COMPANY_COUNTRY_CODE"
        "CONTACT_FIRSTNAME"
        "CONTACT_LASTNAME"
        "CONTACT_EMAIL"
        "CONTACT_PHONE"
        "USECASE_DETAILS"
        "OPTIN_DESCRIPTION"
        "MONTHLY_MESSAGE_VOLUME"
        "USECASE_CATEGORY"
        "MESSAGE_SAMPLE1"
        "ATTACHMENT_BODY"
    )

    for var in "${required_vars[@]}"; do
        if [ -z "${!var}" ]; then
            echo "Error: $var is missing or empty" >&2
            return 1
        fi
    done

    # Validate zip code
    if [[ ! "$COMPANY_ZIPCODE" =~ ^[0-9]+$ ]]; then
        echo "Error: Invalid or missing zip code" >&2
        return 1
    fi

    # Validate monthly message volume
    local valid_volumes=("10" "100" "1,000" "10,000" "100,000" "250,000" "500,000" "750,000" "1,000,000" "5,000,000" "10,000,000+")
    local volume_valid=false
    for volume in "${valid_volumes[@]}"; do
        if [ "$MONTHLY_MESSAGE_VOLUME" = "$volume" ]; then
            volume_valid=true
            break
        fi
    done
    if [ "$volume_valid" = false ]; then
        echo "Error: Invalid or missing monthly_message_volume" >&2
        return 1
    fi

    # Validate use case category
    local valid_categories=("Two-factor authentication" "One-time passcodes" "Notifications" "Polling and surveys" "Info on demand" "Promotions & marketing" "Other")
    local category_valid=false
    for category in "${valid_categories[@]}"; do
        if [ "$USECASE_CATEGORY" = "$category" ]; then
            category_valid=true
            break
        fi
    done
    if [ "$category_valid" = false ]; then
        echo "Error: Invalid or missing use_case_category" >&2
        return 1
    fi

    # Additional validations (you can add more as needed)
    if [[ ! "$COMPANY_WEBSITE" =~ ^https?:// ]]; then
        echo "Error: COMPANY_WEBSITE must start with http:// or https://" >&2
        return 1
    fi
    
    if [[ ! "$CONTACT_EMAIL" =~ ^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$ ]]; then
        echo "Error: CONTACT_EMAIL is not a valid email address" >&2
        return 1
    fi
    
    if [[ ! "$CONTACT_PHONE" =~ ^\+[1-9][0-9]{10,14}$ ]]; then
        echo "Error: CONTACT_PHONE must be in E.164 format (e.g., +10000000000)" >&2
        return 1
    fi
    
    echo "All required variables are set and valid"
    return 0
}


# Step 1: Create registration
echo "Creating registration..."
create_registration_output=$(aws pinpoint-sms-voice-v2 --region "$region" create-registration \
      --registration-type 'US_TOLL_FREE_REGISTRATION' \
      --tags Key=Name,Value='Registration friendly name') || { echo "Failed to create registration"; exit 1; }

# Extract RegistrationId
registration_id=$(echo "$create_registration_output" | jq -r '.RegistrationId')
if [ -z "$registration_id" ]; then
    echo "Error: Registration ID not found!"
    exit 1
fi
echo "Registration ID: $registration_id"

# Step 2: Create registration attachment
echo "Creating registration attachment..."
create_attachment_output=$(aws pinpoint-sms-voice-v2 --region "$region" create-registration-attachment \
      --attachment-body "$attachment_body") || { echo "Failed to create registration attachment"; exit 1; }

# Extract RegistrationAttachmentId
attachment_id=$(echo "$create_attachment_output" | jq -r '.RegistrationAttachmentId')
if [ -z "$attachment_id" ]; then
    echo "Error: Attachment ID not found!"
    exit 1
fi
echo "Attachment ID: $attachment_id"

# Step 3: Wait for the attachment to be fully uploaded
echo "Waiting for attachment upload to complete..."
while [ "$upload_complete" = false ] && [ $retry_count -lt $max_retries ]; do
    # Describe the registration attachment
    describe_attachment_output=$(aws pinpoint-sms-voice-v2 --region "$region" describe-registration-attachments \
        --registration-attachment-ids "$attachment_id")
    
    # Extract the attachment status
    attachment_status=$(echo "$describe_attachment_output" | jq -r '.RegistrationAttachments[0].AttachmentStatus')

    # Check if the status is 'UPLOAD_COMPLETE'
    if [ "$attachment_status" == "UPLOAD_COMPLETE" ]; then
        echo "Attachment upload complete."
        upload_complete=true
    else
        echo "Attachment status: $attachment_status. Retrying in $retry_interval seconds... (attempt $((retry_count + 1))/$max_retries)"
        sleep "$retry_interval"
        retry_count=$((retry_count + 1))
    fi
done
if [ "$upload_complete" = false ]; then
    echo "Attachment upload did not complete within the retry limit."
    exit 1
fi
echo "Attachment upload complete."

# Step 4a: Put registration field values from environment variables
text_field_paths=(
    "companyInfo_companyName"
    "companyInfo_website"
    "companyInfo_address1"
    "companyInfo_address2"
    "companyInfo_city"
    "companyInfo_state"
    "companyInfo_zipCode"
    "companyInfo_isoCountryCode"
    "contactInfo_firstName"
    "contactInfo_lastName"
    "contactInfo_supportEmail"
    "contactInfo_supportPhoneNumber"
    "messagingUseCase_useCaseDetails"
    "messagingUseCase_optInDescription"
    "messageSamples_messageSample1"
)

for text_field_path in "${text_field_paths[@]}"; do
    echo "Putting registration field value for: $text_field_path"
    value=$(eval "echo \$$text_field_path")
    if [ -z "$value" ]; then
        echo "Error: $text_field_path is empty or not set. Skipping."
        continue
    fi
    echo "Value for $text_field_path is: '$value'"
    put_registration_field_value "${text_field_path//_/.}" "$value"
done

# Step 4b: Put other registration field values (Choice fields)
choice_field_paths=(
    "messagingUseCase_monthlyMessageVolume"
    "messagingUseCase_useCaseCategory"
)

for choice_field_path in "${choice_field_paths[@]}"; do
    echo "Putting registration field value for: $choice_field_path"
    value=$(eval "echo \$$choice_field_path")
    if [ -z "$value" ]; then
        echo "Error: $choice_field_path is empty or not set. Skipping."
        continue
    fi
    echo "Value for $choice_field_path is: '$value'"
    aws pinpoint-sms-voice-v2 --region "$region" put-registration-field-value \
        --registration-id "$registration_id" \
        --field-path "${choice_field_path//_/.}" \
        --select-choices "$value" || { echo "Failed to update $choice_field_path"; exit 1; }
done

# Step 5: Associate registration attachment with registration
echo "Associating registration attachment with registration..."
aws pinpoint-sms-voice-v2 --region "$region" put-registration-field-value \
      --registration-id "$registration_id" \
      --field-path 'messagingUseCase.optInImage' \
      --registration-attachment-id "$attachment_id" || { echo "Failed to associate registration attachment"; exit 1; }

# Step 6: Request phone number
echo "Requesting phone number..."
request_phone_number_output=$(aws pinpoint-sms-voice-v2 --region "$region" request-phone-number \
      --iso-country-code 'US' \
      --number-type 'TOLL_FREE' \
      --number-capabilities 'SMS' 'MMS' 'VOICE' \
      --message-type 'TRANSACTIONAL') || { echo "Failed to request phone number"; exit 1; }

# Extract PhoneNumberId
phone_number_id=$(echo "$request_phone_number_output" | jq -r '.PhoneNumberId')
if [ -z "$phone_number_id" ]; then
    echo "Error: Phone number ID not found!"
    exit 1
fi
echo "Phone Number ID: $phone_number_id"

# Step 7: Associate phone number with registration
echo "Associating phone number with registration..."
aws pinpoint-sms-voice-v2 --region "$region" create-registration-association \
      --registration-id "$registration_id" \
      --resource-id "$phone_number_id" || { echo "Failed to associate phone number with registration"; exit 1; }

# Step 8: Submit registration
echo "Submitting registration..."
aws pinpoint-sms-voice-v2 --region "$region" submit-registration-version \
     --registration-id "$registration_id" || { echo "Failed to submit registration"; exit 1; }

echo "Registration submitted successfully!"
  1. Run chmod command to make it executable. Run the script with the environment variable values you provided in Step 1 above.

chmod u+x register_us_toll_free_number.sh
./register_us_toll_free_number.sh
  1. The script completes with the following lines:

Submitting registration...
{
    "RegistrationArn": "arn:aws:sms-voice:us-west-2:637400000000:registration/registration-244aa3f91fcc49b598bdf1234569",
    "RegistrationId": "registration-244aa3f91fcc49b598bdf1234569",
    "VersionNumber": 1,
    "RegistrationVersionStatus": "SUBMITTED",
    "RegistrationVersionStatusHistory": {
        "DraftTimestamp": "2024-09-11T19:44:41+00:00",
        "SubmittedTimestamp": "2024-09-11T19:45:16+00:00"
    }
}
Registration submitted successfully!

Option 2 – Execute a Python script to register more than one Toll-Free number

For ISVs, SaaS providers, or large organizations registering multiple phone numbers on behalf of customers or business units, automating the process can save time and reduce errors. We’ve chosen to use Python because it’s CSV library handles file parsing complexities, and its boto3 library enables seamless interaction with AWS services, making it well-suited for programmatically managing these registrations.

  1. Below is a sample CSV file with the variables for 5 registrations, copy it and replace the sample data with your actual data (these data were listed in the prerequisites above). Save your file as data.csv:
companyInfo_companyName,companyInfo_website,companyInfo_address1,companyInfo_address2,companyInfo_city,companyInfo_state,companyInfo_zipCode,companyInfo_isoCountryCode,contactInfo_firstName,contactInfo_lastName,contactInfo_supportEmail,contactInfo_supportPhoneNumber,messagingUseCase_monthlyMessageVolume,messagingUseCase_useCaseCategory,messagingUseCase_useCaseDetails,messagingUseCase_optInDescription,messageSamples_messageSample1,attachmentFilePath
AnyCompany,https://example.com/example1,123 Any Street,Suite 200,Anytown,WA,98109,US,FirstName,LastName,[email protected],15553331234,10,One-time passcodes,Dev/Demo - Internal testing only,Internal testing only,Your AWS End User Messaging internal testing one time passcode is 123456,./companyA.png
Example Corp,https://example.net/example2,123 Main Street,,Anywhere,WA,98765,US,FirstName2,LastName2,[email protected],15553332222,10,Notifications,Dev/Demo - Internal testing only,Internal testing only,Your AWS End User Messaging internal testing one time passcode is 654321,./companyB.png
AnyDepartment,https://example.org/example3,123 Oak Avenue,Apt B,Nowhere,WA,54321,US,FirstName3,LastName3,[email protected],15553333333,10,Two-factor authentication,Dev/Demo - Internal testing only,Internal testing only,Your AWS End User Messaging internal testing one time passcode is 987654,./companyC.png
AnyOrganization,https://example.com/example4,123 Pine Blvd,,Anywhere Else,WA,12345,US,FirstName4,LastName4,[email protected],15553334444,100,One-time passcodes,Dev/Demo - Internal testing only,Internal testing only,Your AWS End User Messaging internal testing one time passcode is 654789,./companyD.png
AnyGovernment,https://example.org/example5,123 Oak St,,Nowhere Town,WA,67890,US,FirstName5,LastName5,[email protected],15553335555,"100",One-time passcodes,Dev/Demo - Internal testing only,Internal testing only,Your AWS End User Messaging internal testing one time passcode is 987654,./companyE.png
  1. Prepare opt-in screenshot image files for each registration, and upload them in the same directory as you will use for the data.csv and script files. Note: the allowed file types for the opt-in screenshot are PNG, JPG, and PDF with a max file size of 500 kb.
  2. Below is a Python script to create registrations using the data.csv file (above). Save this file as register_phone_numbers_bulk.py into your current directory.
import csv
import boto3
import sys
import argparse
import time

# Parse command line arguments
parser = argparse.ArgumentParser(description="AWS End User Messaging SMS Registration Script")
parser.add_argument("-r", "--region", default="us-west-2", help="AWS region")
parser.add_argument("-c", "--csv_file", default="data.csv", help="Path to the Comma-Separated Values file")
args = parser.parse_args()

# Initialize AWS End User Messaging SMS client
client = boto3.client('pinpoint-sms-voice-v2', region_name=args.region)

def process_csv(csv_file):
    with open(csv_file, newline='', encoding='utf-8') as f:
        reader = csv.DictReader(f)
        
        # Strip BOM from the first key (header) if it exists
        if '\ufeff' in reader.fieldnames[0]:
            reader.fieldnames[0] = reader.fieldnames[0].replace('\ufeff', '')
        
        for row in reader:
            process_row(row)

def process_row(row):
    print(f"================================================================================================")
    
    if 'companyInfo_companyName' not in row:
        print(f"Error: 'companyInfo_companyName' field is missing in the CSV file.")
        return
    
    print(f"Processing entry for: {row['companyInfo_companyName']}")
    print(f"================================================================================================")

    # Validate required fields
    required_fields = [
        'companyInfo_companyName', 'companyInfo_website', 'companyInfo_isoCountryCode', 
        'attachmentFilePath'
    ]
    for field in required_fields:
        if field not in row or not row[field]:
            print(f"Error: Required field '{field}' is missing or empty for entry: {row.get('companyInfo_companyName', 'Unknown Company')}")
            return

    # Validate field values
    if 'companyInfo_zipCode' not in row or not row['companyInfo_zipCode'].isdigit():
        print(f"Error: Invalid or missing zip code for entry: {row['companyInfo_companyName']}")
        return

    valid_volumes = ["10", "100", "1,000", "10,000", "100,000", "250,000", "500,000", "750,000", "1,000,000", "5,000,000", "10,000,000+"]
    if 'messagingUseCase_monthlyMessageVolume' not in row or row['messagingUseCase_monthlyMessageVolume'] not in valid_volumes:
        print(f"Error: Invalid or missing monthly_message_volume for entry: {row['companyInfo_companyName']}")
        return

    valid_categories = ["Two-factor authentication", "One-time passcodes", "Notifications", "Polling and surveys", "Info on demand", "Promotions & marketing", "Other"]
    if 'messagingUseCase_useCaseCategory' not in row or row['messagingUseCase_useCaseCategory'] not in valid_categories:
        print(f"Error: Invalid or missing use_case_category for entry: {row['companyInfo_companyName']}")
        return

    try:
        # Step 1: Create registration
        print("Creating registration...")
        create_registration_output = client.create_registration(
            RegistrationType='US_TOLL_FREE_REGISTRATION'
        )
        registration_id = create_registration_output['RegistrationId']

        # Step 2: Put registration field values from CSV
        print("Putting registration field values...")
        field_mappings = {
            "companyInfo.companyName": {"value": row['companyInfo_companyName'], "type": "text"},
            "companyInfo.website": {"value": row['companyInfo_website'], "type": "text"},
            "companyInfo.address1": {"value": row['companyInfo_address1'], "type": "text"},
            "companyInfo.address2": {"value": row['companyInfo_address2'], "type": "text"},            
            "companyInfo.city": {"value": row['companyInfo_city'], "type": "text"},
            "companyInfo.state": {"value": row['companyInfo_state'], "type": "text"},
            "companyInfo.zipCode": {"value": row['companyInfo_zipCode'], "type": "text"},
            "companyInfo.isoCountryCode": {"value": row['companyInfo_isoCountryCode'], "type": "text"},
            "contactInfo.firstName": {"value": row['contactInfo_firstName'], "type": "text"},
            "contactInfo.lastName": {"value": row['contactInfo_lastName'], "type": "text"},
            "contactInfo.supportEmail": {"value": row['contactInfo_supportEmail'], "type": "text"},
            "contactInfo.supportPhoneNumber": {"value": row['contactInfo_supportPhoneNumber'], "type": "text"},
            "messagingUseCase.useCaseDetails": {"value": row['messagingUseCase_useCaseDetails'], "type": "text"},
            "messagingUseCase.optInDescription": {"value": row['messagingUseCase_optInDescription'], "type": "text"},
            "messageSamples.messageSample1": {"value": row['messageSamples_messageSample1'], "type": "text"},
            "messagingUseCase.monthlyMessageVolume": {"value": row['messagingUseCase_monthlyMessageVolume'], "type": "select"},
            "messagingUseCase.useCaseCategory": {"value": row['messagingUseCase_useCaseCategory'], "type": "select"},
        }

        for field, field_data in field_mappings.items():
            if field_data['type'] == 'text' and field_data['value']:
                client.put_registration_field_value(
                    RegistrationId=registration_id,
                    FieldPath=field,
                    TextValue=field_data['value']
                )
            elif field_data['type'] == 'select' and field_data['value']:
                client.put_registration_field_value(
                    RegistrationId=registration_id,
                    FieldPath=field,
                    SelectChoices=[field_data['value']]
                )

        # Step 3: Create registration attachment
        print("Creating registration attachment...")
        with open(row['attachmentFilePath'], 'rb') as attachment_file:
            create_attachment_output = client.create_registration_attachment(
                AttachmentBody=attachment_file.read()
            )
            attachment_id = create_attachment_output['RegistrationAttachmentId']

        # Step 4: Wait for the attachment upload to finish
        print("Verifying attachment upload complete...")
        attachment_id = create_attachment_output['RegistrationAttachmentId']
        upload_complete = False
        max_retries = 10
        retry_count = 0

        while not upload_complete and retry_count < max_retries:
            try:
                describe_attachment_output = client.describe_registration_attachments(
                    RegistrationAttachmentIds=[attachment_id]
                )
                attachment_status = describe_attachment_output['RegistrationAttachments'][0]['AttachmentStatus']
                if attachment_status == 'UPLOAD_COMPLETE':
                    upload_complete = True
                else:
                    print(f"Attachment status: {attachment_status}. Waiting 2 seconds before retrying... (max {max_retries} retries)")
                    time.sleep(2)
                    retry_count += 1
            except Exception as e:
                print(f"Error occurred while checking attachment status: {str(e)}")
                time.sleep(10)
                retry_count += 1

        if not upload_complete:
            print(f"Error: Attachment upload did not complete for {row['companyInfo_companyName']}")
            return

        # Step 5: Associate registration attachment with registration
        print("Associating registration attachment with registration...")
        client.put_registration_field_value(
            RegistrationId=registration_id,
            FieldPath='messagingUseCase.optInImage',
            RegistrationAttachmentId=attachment_id
        )
        
        # Step 6: Request phone number
        print("Requesting phone number...")
        request_phone_number_output = client.request_phone_number(
            IsoCountryCode='US',
            NumberType='TOLL_FREE',
            NumberCapabilities=['SMS', 'MMS', 'VOICE'],
            MessageType='TRANSACTIONAL'
        )
        phone_number_id = request_phone_number_output['PhoneNumberId']

        # Step 7: Associate phone number with registration
        print("Associating phone number with registration...")
        client.create_registration_association(
            RegistrationId=registration_id,
            ResourceId=phone_number_id
        )
                
        # Step 8: Submit registration
        print("Submitting registration...")
        client.submit_registration_version(
            RegistrationId=registration_id
        )
        print(f"Processing completed for: {row['companyInfo_companyName']}\n")

    except Exception as e:
        print(f"Error occurred while processing {row['companyInfo_companyName']}: {str(e)}")

if __name__ == "__main__":
    process_csv(args.csv_file)
  1. Run the script: python register_phone_numbers_bulk.py -c data.csv -r us-west-2
    The 2 parameters are the name of the CSV file containing your data (shown here as data.csv) and specify the target AWS region (shown here as us-west-2) as parameters. Adjust these parameters as needed to match your environment and data location.

Check status:

Once you have successfully ran the script, you can check the status of your registration requests via AWS End User Messaging Console or with the API using the DescribeRegistrations API. If you want to use the console log in to the AWS End User Messaging Console in the target region and choose Configurations > Registrations from the left navigation. The registration status for each request will initially display “CREATED” and it will change to “REVIEWING” within 24 hours. As noted in the introduction, the actual registration review process is conducted by a third-party company and it can take up to 15 days to receive an acceptance (“COMPLETE”) or request for additional information or corrections (“Requires Update“) for each request.

Registration Status:

Registrations

Phone Number Status:

Phone Number status

Conclusion:

In this post you have learned how to automate the registration process for Toll-Free numbers with a Bash script or Python script and the AWS End User Messaging APIs. Using the API can significantly improve efficiency and reduce manual errors, but keep in mind that you will still need to wait up to 15 business days for the registrations to be approved (or not), and for the numbers to be set to “ACTIVE” status before you can begin sending messages with your TFN(s).

The AWS End User Messaging V2 API for SMS and Voice has other useful actions, and we encourage you to explore how it can further help you simplify and automate your applications.

Resources to help you plan for your SMS program:

Use this spreadsheet to plan for the countries you need to send SMS/MMS

To restrict your sending to only certain countries read this blog

Confirm the origination IDs you will need here

Check out the support tiers comparison

Understanding Google Postmaster Tools (spam complaints) for Amazon SES email senders

Post Syndicated from Bruno Giorgini original https://aws.amazon.com/blogs/messaging-and-targeting/understanding-google-postmaster-tools-spam-complaints-for-amazon-ses-email-senders/

Introduction

Amazon Simple Email Service (SES) includes a robust set of built-in tools, such as the Virtual Deliverability Manager (VDM), to help senders ensure optimal email deliverability. Additionally, deliverability data from email service providers like Postmaster Tools by Google can provide invaluable insights for all sending domain owners, including those using SES for bulk or transactional email. Postmaster Tools offers detailed metrics on factors like delivery errors, spam rates, domain reputation, and recipient feedback for Gmail-hosted inboxes. Combining this external data with SES email sending events is critical for maintaining a healthy sender reputation. By leveraging both SES-native tools and resources like Postmaster Tools, senders can identify and address deliverability issues, ensuring their SES-powered emails reach intended recipients across providers.

Many, but not all, mailbox providers will send recipient feedback in the form of “complaints” that can each be attributed directly to the message that the recipient found to be objectionable. These complaints are available in the SES email sending event type “Complaint”. Gmail does not send spam complaint events because their priority is to protect the privacy of their users from the tracking techniques employed by spammers and data brokers. Gmail requires bulk senders to adopt “easy unsubscribe” mechanisms to reduce the need for their users to report messages as spam, and they will show spam complaint metrics in Postmaster Tools. This blog will show you how to maximize value in the spam complaint metric provided by Postmaster Tools.

Amazon SES now supports custom values in the Feedback-ID header in messages sent through SES. This feature provides additional details to help customers identify deliverability trends. Together with Postmaster Tools, customers can group complaints by identifiers of their choice, such as sender business unit or campaign ID. This makes it easier to track deliverability performance associated with independent workloads and campaigns, and accelerates troubleshooting when diagnosing complaint rates.

This image describes the flow for spam complaints to the email sender

Figure 1: Email Feedback Loop

This blog will guide you through implementing and using Feedback Loops within Postmaster Tools to identify email campaigns receiving high complaint volumes from Gmail users. It covers the history and background of feedback loops, the specific requirements for implementing them with Postmaster Tools, and practical examples using AWS CLI and Boto3 to send SES emails with the necessary Feedback-ID header. By the end, you’ll understand how to effectively set up and use Postmaster Tools to monitor and improve your SES email deliverability.

History and Background of FBLs

Traditional Feedback Loops (herein “FBLs”) have been a cornerstone of email deliverability for many years. Initially developed by Internet Service Providers (ISPs), FBLs serve as a mechanism for recipients to report spam complaints to the sender. This feedback is crucial for email service providers and senders to identify problematic email campaigns, take corrective actions, and maintain a healthy sender reputation.

FBLs operate by allowing recipients to mark emails as spam, which then sends a report to the sender’s email service provider. This report typically includes details about the email that triggered the complaint, enabling the sender to investigate and address any issues. By analyzing these reports, senders can refine their email lists, improve content, and ensure that their emails comply with best practices and regulatory requirements. Senders who receive a higher volume of spam complaints are more likely to be blocked or have their emails routed to the spam folder. While high spam complaints are not the sole reason for deliverability issues, they are often the underlying cause.

Postmaster Tools by Gmail is not a traditional FBL. Postmaster Tools will show complaint feedback metrics, but the complaints are not attributable to any individual recipient.

Requirements for using Postmaster Tools FBL with SES

The FBL helps identify campaigns with high complaint rates from Gmail users, specifically useful for email service providers to detect potential abuse of their services.

Note: Data in Postmaster Tools only applies to messages sent to personal Gmail accounts. A personal Gmail account is an account that ends in @gmail.comor @googlemail.com.

  • Implementation of FBL:
    • Feedback-ID Header: SES embeds a header called Feedback-ID containing parameters (Identifiers) uniquely identifying the account and SenderID (AmazonSES)
    • Header Format: The Feedback-ID header consists of four parameters, separated by colons:
      a:b:c:SenderId
      Where:
      • SenderId is a mandatory parameter that uniquely identifies the sender.
      • In the case of Amazon SES (Simple Email Service), the SenderId is always “AmazonSES” and cannot be overridden.
Header Parameter  Description
a  First parameter in the Feedback-ID header. SES users can customize through ses:feedback-id-a EmailTag
b  Second parameter in the Feedback-ID header. SES users can customize through ses:feedback-id-b EmailTag.
c  Third parameter in the Feedback-ID header. SES uses this to identify the sender account
SenderID  Fourth parameter in the Feedback-ID header. Mandatory parameter that uniquely identifies the sender. For Amazon SES, this is always “AmazonSES” and cannot be overridden.
  • Sender Data Handling:
    • DKIM signing by a sender-owned domain is required to prevent spoofing.
    • The domain must be added and verified in Postmaster Tools.
    • Complaint data is aggregated by distinct values on each of the 4 fields of Feedback-ID.
  • Feedback-ID header Requirements:
    • When sending emails through Amazon SES, users are limited to a single verified header value per traffic stream.
      • This means that the Feedback-ID header cannot contain an individualized value for each destination email address.
      • Instead, the Feedback-ID header needs to contain an identifier that can be used to match a larger campaign or batch of emails, rather than a unique value per recipient.
      • This constraint helps maintain a consistent sender reputation, improves deliverability monitoring and troubleshooting within tools like Postmaster Tools. The Feedback-ID acts as a grouping mechanism, rather than a per-message identifier
    • Identifiers must be unique and non-repetitive across fields.
  • Feedback-ID Example:
    • CampaignIDX:CustomerID2:1.us-west-2.TDQeKqHkSNfQztk25wIeVIGTuNmGDud4r1l7dUlxOio=:AmazonSES
      • Each Identifier is used to report spam percentages independently if unusual rates occur.
      • Amazon SES lets customers set the part a and part b of the Feedback-ID header using the EmailTag ses:feedback-id-a and ses:feedback-id-b
      • Amazon SES will combine these tags into a single Feedback-ID header with the format: Feedback-ID=a:b:region.accountId:AmazonSES

The next steps will cover what’s needed to leverage FBLs with SES.

Step 1 – Add Your Domain(s) To Google’s Postmaster Tools

  • In order to verify with Postmaster Tools that you’re authorized to track the feedback from your domain, you first need to register your ownership of the domain with Postmaster Tools by visiting https://gmail.com/postmaster/.
Verify a new domain with Google Postmaster Tools

Figure 2: Step 1 to verify a domain in Google Postmaster Tools

  • After entering in your domain, you’d be prompted to add a TXT record into your DNS configuration.
Step 2 to verify a domain in Google Postmaster Tools

Figure 3: Step 2 to verify a domain in Google Postmaster Tools

  • Update your sending domain(s) DNS records accordingly.
    • The example below specifies how to create the TXT record in Route53. If you’re using another DNS service provider, please refer to their documentation.
Create a new record in Route53

Figure 4: Create a new record in Route53

    • Navigate to the Route53 Console and click on Hosted zones , specify the hosted zone that contains the domain you want to verify and then Create record.
This image describe the creation of a TXT record including the value provided by Google Postmaster to verify the domain

Figure 5: Add a TXT record with the provided value for verification

    • Following the screenshot, create a TXT record type and paste the value assigned by Google for verification in step 2 here.
  • Go to Postmaster Tools and click on Verify. After successful verification of your domain in Postmaster Tools, you should see the Status column changed from Not Verified to Verified. You can verify your compliance status with the requirements in the Dashboard (2) link.
In this picture we show an example of how the domain would appear once verified in Google Postmaster Tools

Figure 6: Domain verified

  • Follow the recommendations provided in the Postmaster Tools dashboard to fully comply with the requirements (example below):
Email sender requirements

Figure 7: Email sender requirements compliance status recommendations

  • Once you have completed all the verification and configuration steps, you should see compliant checkmarks next to all available requirements (see example below):
Email sender requirements

Figure 8: Email sender requirements compliant status

Step 2 – Add Feedback-ID headers to your SES emails

  • Use this command line to send an email with Feedback-ID using the AWS CLI:
aws sesv2 send-email --from-email-address [email protected] \
   --destination '{"ToAddresses":["[email protected]"]}' \
   --content '{"Simple":{"Subject":{"Data":"Test Subject","Charset":"UTF-8"},"Body":{"Text":{"Data":"Test Data","Charset":"UTF-8"}}}}' \
   --email-tags '[{"Name": "ses:feedback-id-a","Value":"feedback-id-part-a-value"}]'

The values of ses:feedback-id-a and ses:feedback-id-b are specified using the --email-tags option.

  • Alternatively, use Boto3 to send an email with Feedback-ID with the following Python script:
import boto3
from botocore.exceptions import ClientError

def send_email(region_name):
    # Create a new SES client
    ses = boto3.client('sesv2', region_name=region_name)

    # Replace sender and recipient values
    SENDER = "Sender Name <[email protected]>"
    RECIPIENT = "[email protected]"
    CONFIGURATION_SET = "SES_Config_Set"
    SUBJECT = "Amazon SES Test (SDK for Python)"
    BODY_TEXT = "Amazon SES Test (Python)\r\nThis email was sent with Amazon SES using the AWS SDK for Python (Boto)."
    BODY_HTML = """<html>
    <head></head>
    <body>
      <h1>Amazon SES Test (SDK for Python)</h1>
      <p>This email was sent with
        <a href='https://aws.amazon.com/ses/'>Amazon SES</a> using the
        <a href='https://aws.amazon.com/sdk-for-python/'>
          AWS SDK for Python (Boto)</a>.</p>
    </body>
    </html>"""
    CHARSET = "UTF-8"

    try:
        # Send email
        response = ses.send_email(
            FromEmailAddress=SENDER,
            Destination={'ToAddresses': [RECIPIENT]},
            ConfigurationSetName=CONFIGURATION_SET,
            Content={
                "Simple": {
                    "Subject": {
                        "Charset": CHARSET,
                        "Data": SUBJECT
                    },
                    "Body": {
                        "Text": {
                            "Charset": CHARSET,
                            "Data": BODY_TEXT
                        },
                        "Html": {
                            "Charset": CHARSET,
                            "Data": BODY_HTML
                        }
                    },
                    "Headers": [
                        {
                            "Name": "List-Unsubscribe",
                            "Value": "<https://unsubscribe.example.email/[email protected]&topic=topic1>"
                        },
                        {
                            "Name": "List-Unsubscribe-Post",
                            "Value": "One-Click"
                        }
                    ]
                }
            },
            EmailTags=[
                {
                    'Name': 'ses:feedback-id-a',
                    'Value': 'campaign1'
                },
                {
                    'Name': 'ses:feedback-id-b',
                    'Value': 'line-of-business'
                }
            ] #the ses:feedback-id-a and ses:feedback-id-b are specified as a list using EmailTags
        )
        print("Email sent! Response:", response)
        print("Message ID:", response['MessageId'])

    except ClientError as e:
        print(e.response['Error']['Message'])

# Call the function to send the email
send_email(region_name='us-west-2')  # Specify the region here

Step 3 – Viewing FBL results in Postmaster Tools

In order to see any results in the Postmaster Tool dashboard (see examples below), you must send a substantial daily volume of email through the domain(s) you’ve registered. If you see the message “No Data to Display”, your reputation may already be too low, more likely the volume of email traffic sent since you configured the Postmaster tool is insufficient (return to the dashboard in later, after you’ve sent 1,000s of emails).

Figure 9: Feedback loop example image

Figure 9: Feedback loop example image

The image shows a section of the Postmaster Tools dashboard, specifically the Feedback Loop section. This dashboard provides insights into the spam complaint rates and the number of feedback loop identifiers flagged across a given time period, in this case, the last 120 days.

Conclusion

High-volume email senders should look to the combination of Amazon SES’ powerful framework for monitoring in concert with Postmaster Tools to improve and ensure email deliverability. Implementing the Feedback-ID header in your SES emails can significantly enhance your ability to track and troubleshoot deliverability issues. Use Postmaster Tools and the Feedback Loop via Feedback-ID headers in SES emails to gain detailed insights into complaint rates and other key metrics, enabling you to maintain a healthy sender reputation and ensure their emails reach the intended recipients.

Call to Action:

  1. Set Up Postmaster Tools for your sending domain(s)
  2. Verify Your Domain: Register and verify your domain with Postmaster Tools to access valuable insights and track your compliance status.
  3. Set Up Feedback-ID: Start embedding the Feedback-ID header in your emails sent via Amazon SES to take advantage of detailed complaint data and improve your email campaigns.
  4. Monitor and Adjust: Regularly check the Postmaster Tools dashboard to monitor your spam rates and feedback loop identifiers. Use this data to refine your email content and sending practices.
  5. Leverage AWS CLI and Boto3: Utilize the provided AWS CLI commands and Boto3 scripts to automate the process of sending emails with Feedback-ID headers, ensuring consistent and accurate tracking.

By following these steps, you can enhance your email deliverability, reduce spam complaints, and maintain a strong sender reputation. For more information on using Amazon SES and Google’s Postmaster Tools, refer to the Amazon SES Documentation and the Postmaster Tools Guide.

Amazon SES: Email Authentication and Getting Value out of Your DMARC Policy

Post Syndicated from Bruno Giorgini original https://aws.amazon.com/blogs/messaging-and-targeting/email-authenctication-dmarc-policy/

Amazon SES: Email Authentication and Getting Value out of Your DMARC Policy

Introduction

For enterprises of all sizes, email is a critical piece of infrastructure that supports large volumes of communication. To enhance the security and trustworthiness of email communication, many organizations turn to email sending providers (ESPs) like Amazon Simple Email Service (Amazon SES). These ESPs allow users to send authenticated emails from their domains, employing industry-standard protocols such as the Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM). Messages authenticated with SPF or DKIM will successfully pass your domain’s Domain-based Message Authentication, Reporting, and Conformance (DMARC) policy. This blog post will focus on the DMARC policy enforcement mechanism. The blog will explore some of the reasons why email may fail DMARC policy evaluation and propose solutions to fix any failures that you identify. For an introduction to DMARC and how to carefully choose your email sending domain identity, you can refer to Choosing the Right Domain for Optimal Deliverability with Amazon SES The relationship between DMARC compliance and email deliverability rates is crucial for organizations aiming to maintain a positive sender reputation and ensure successful email delivery. There are many advantages when organizations have this correctly setup, these include:

  • Improved Email Deliverability
  • Reduction in Email Spoofing and Phishing
  • Positive Sender Reputation
  • Reduced Risk of Email Marked as Spam
  • Better Email Engagement Metrics
  • Enhanced Brand Reputation

With this foundation, let’s explore the intricacies of DMARC and how it can benefit your organization’s email communication.

What is DMARC?

DMARC is a mechanism for domain owners to advertise SPF and DKIM protection and to tell receivers how to act if those authentication methods fail. The domain’s DMARC policy protects your domain from third parties attempting to spoof the domain in the “From” header of emails. Malicious email messages that aim to send phishing attempts using your domain will be subject to DMARC policy evaluation, which may result in their quarantine or rejection by the email receiving organization. This stringent policy ensures that emails received by email recipients are genuinely from the claimed sending domain, thereby minimizing the risk of people falling victim to email-based scams. Domain owners publish DMARC policies as a TXT record in the domain’s _dmarc.<domain> DNS record. For example, if the domain used in the “From” header is example.com, then the domain’s DMARC policy would be located in a DNS TXT record named _dmarc.example.com. The DMARC policy can have one of three policy modes:

  • A typical DMARC deployment of an existing domain will start with publishing "p=none". A none policy means that the domain owner is in a monitoring phase; the domain owner is monitoring for messages that aren’t authenticated with SPF and DKIM and seeks to ensure all email is properly authenticated
  • When the domain owner is comfortable that all legitimate use cases are properly authenticated with SPF and/or DKIM, they may change the DMARC policy to "p=quarantine". A quarantine policy means that messages which fail to produce a domain-aligned authenticated identifier via SPF or DKIM will be quarantined by the mail receiving organization. The mail receiving organization may filter these messages into Junk folders, or take another action that they feel best protects their recipients.
  • Finally, domain owners who are confident that all of the legitimate messages using their domain are authenticated with SPF or DKIM, may change the DMARC policy to "p=reject". A reject policy means that messages which fail to produce a domain-aligned authenticated identifier via SPF or DKIM will be rejected by the mail receiving organization.

The following are examples of a TXT record that contains a DMARC policy, depending on the desired policy (the ‘p’ tag):

  Name Type Value
1 _dmarc.example.com TXT “v=DMARC1;p=reject;rua=mailto:[email protected]
2 _dmarc.example.com TXT “v=DMARC1;p=quarantine;rua=mailto:[email protected]
3 _dmarc.example.com TXT “v=DMARC1;p=none;rua=mailto:[email protected]
Table 1 – Example DMARC policy

This policy tells email providers to apply the DMARC policy to messages that fail to produce a DKIM or SPF authenticated identifier that is aligned to the domain in the “From” header. Alignment means that one or both of the following occurs:

  • The messages pass the SPF policy for the MAIL FROM domain and the MAIL FROM domain is the same as the domain in the “From” header, or a subdomain. Reference Using a custom MAIL FROM domain to learn more about how to send SPF aligned messages with SES.
  • The messages have a DKIM signature signed by a public key in DNS at a location within the domain of the “From” header. Reference Authenticating Email with DKIM in Amazon SES to learn more about how to send DKIM aligned messages with SES.

DMARC reporting

The rua tag in the domain’s DMARC policy indicates the location to which mail receiving organizations should send aggregate reports about messages that pass or fail SPF and DKIM alignment. Domain owners analyze these reports to discover messages which are using the domain in the “From” header but are not properly authenticated with SPF or DKIM. The domain owner will attempt to ensure that all legitimate messages are authenticated through analysis of the DMARC aggregate reports over time. Mail receiving organizations which support sending DMARC reports typically send these aggregated reports once per day, although these practices differ from provider to provider.

What does a typical DMARC deployment look like?

A DMARC deployment is the process of:

  1. Ensuring that all emails using the domain in the “From” header are authenticated with DKIM and SPF domain-aligned identifiers. Focus on DKIM as the primary means of authentication.
  2. Publishing a DMARC policy (none, quarantine, or reject) for the domain that reflects how the domain owner would like mail receiving organizations to handle unauthenticated email claiming to be from their domain.

New domains and subdomains

Deploying a DMARC policy is easy for organizations that have created a new domain or subdomain for the purpose of a new email sending use case on SES; for example email marketing, transaction emails, or one-time pass codes (OTP). These domains can start with the "p=reject" DMARC enforcement policy because the policy will not affect existing email sending programs. This strict enforcement is to ensure that there is no unauthenticated use of the domain and its subdomains.

Existing domains

For existing domains, a DMARC deployment is an iterative process because the domain may have a history of email sending by one or multiple email sending programs. It is important to gain a complete understanding of how the domain and its subdomains are being used for email sending before publishing a restrictive DMARC policy (p=quarantine or p=reject) because doing so would affect any unauthenticated email sending programs using the domain in the “From” header of messages. To get started with the DMARC implementation, these are a few actions to take:

  • Publish a p=none DMARC policy (sometimes referred to as monitoring mode), and set the rua tag to the location in which you would like to receive aggregate reports.
  • Analyze the aggregate reports. Mail receiving organizations will send reports which contain information to determine if the domain, and its subdomains, are being used for sending email, and how the messages are (or are not) being authenticated with a DKIM or SPF domain-aligned identifier. An easy to use analysis tool is the Dmarcian XML to Human Converter.
  • Avoid prematurely publishing a “p=quarantine” or “p=reject” policy. Doing so may result in blocked or reduced delivery of legitimate messages of existing email sending programs.

The image below illustrates how DMARC will be applied to an email received by the email receiving server and actions taken based on the enforcement policy:

DMARC flow Figure 1 – DMARC Flow

How do SPF and DKIM cause DMARC policies to pass

When you start sending emails using Amazon SES, messages that you send through Amazon SES automatically use a subdomain of amazonses.com as the default MAIL FROM domain. SPF evaluators will see that these messages pass the SPF policy evaluation because the default MAIL FROM domain has a SPF policy which includes the IP addresses of the SES infrastructure that sent the message. SPF authentication will result in an “SPF=PASS” and the authenticated identifier is the domain of the MAIL FROM address. The published SPF record applies to every message that is sent using SES regardless of whether you are using a shared or dedicated IP address. The amazonses.com SPF record lists all shared and dedicated IP addresses, so it is inclusive of all potential IP addresses that may be involved with sending email as the MAIL FROM domain. You can use ‘dig’ to look up the IP addresses that SES will use to send email:

dig txt amazonses.com | grep "v=spf1" amazonses.com. 850 IN TXT "v=spf1 ip4:199.255.192.0/22 ip4:199.127.232.0/22 ip4:54.240.0.0/18 ip4:69.169.224.0/20 ip4:23.249.208.0/20 ip4:23.251.224.0/19 ip4:76.223.176.0/20 ip4:54.240.64.0/19 ip4:54.240.96.0/19 ip4:52.82.172.0/22 ip4:76.223.128.0/19 -all"

Custom MAIL FROM domains

It is best practice for customers to configure a custom MAIL FROM domain, and not use the default amazonses.com MAIL FROM domain. The custom MAIL FROM domain will always be a subdomain of the customer’s verified domain identity. Once you configure the MAIL FROM domain, messages sent using SES will continue to result in an “SPF=PASS” as it does with the default MAIL FROM domain. Additionally, DMARC authentication will result in “DMARC=PASS” because the MAIL FROM domain and the domain in the “From” header are in alignment. It’s important to understand that customers must use a custom MAIL FROM domain if they want “SPF=PASS” to result in a “DMARC=PASS”.

For example, an Amazon SES-verified example.com domain will have the custom MAIL FROM domain “bounce.example.com”. The configured SPF record will be:

dig txt bounce.example.com | grep "v=spf1" "v=spf1 include:amazonses.com ~all"

Note: The chosen MAIL FROM domain could be any sub-domain of your choice. If you have the same domain identity configured in multiple regions, then you should create region-specific custom MAIL FROM domains for each region. e.g. bounce-us-east-1.example.com and bounce-eu-west-2.example.com so that asynchronously bounced messages are delivered directly to the region from which the messages were sent.

DKIM results in DMARC pass

For customers that establish Amazon SES Domain verification using DKIM signatures, DKIM authentication will result in a DKIM=PASS, and DMARC authentication will result in “DMARC=PASS” because the domain that publishes the DKIM signature is aligned to the domain in the “From” header (the SES domain identity).

DKIM and SPF together

Email messages are fully authenticated when the messages pass both DKIM and SPF, and both DKIM and SPF authenticated identifiers are domain-aligned. If only DKIM is domain-aligned, then the messages will still pass the DMARC policy, even if the SPF “pass” is unaligned. Mail receivers will consider the full context of SPF and DKIM when determining how they will handle the disposition of the messages you send, so it is best to fully authenticate your messages whenever possible. Amazon SES has taken care of the heavy lifting of the email authentication process away from our customers, and so, establishing SPF, DKIM and DMARC authentication has been reduced to a few clicks which allows SES customers to get started easily and scale fast.

Why is DMARC failing?

There are scenarios when you may notice that messages fail DMARC, whether your messages are fully authenticated, or partially authenticated. The following are things that you should look out for:

Email Content Modification

Sometimes email content is modified during the delivery to the recipients’ mail servers. This modification could be as a result of a security device or anti-spam agent along the delivery path (for example: the message Subject may be modified with an “[EXTERNAL]” warning to recipients). The modified message invalidates the DKIM signature which causes a DKIM failure. Remember, the purpose of DKIM is to ensure that the content of an email has not been tampered with during the delivery process. If this happens, the DKIM authentication will fail with an authentication error similar to “DKIM-signature body hash not verified“.

Solutions:

  • If you control the full path that the email message will traverse from sender to recipient, ensure that no intermediary mail servers modify the email content in transit.
  • Ensure that you configure a custom MAIL FROM domain so that the messages have a domain-aligned SPF identifier.
  • Keep the DMARC policy in monitoring mode (p=none) until these issues are identified/solved.

Email Forwarding

Email Forwarding There are multiple scenarios in which a message may be forwarded, and they may result in both/either SPF and DKIM failing to produce a domain-aligned authenticated identifier. For SPF, it means that the forwarding mail server is not listed in the MAIL FROM domain’s SPF policy. It is best practice for a forwarding mail server to avoid SPF failures and assume responsibility of mail handling for the messages it forwards by rewriting the MAIL FROM address to be in the domain controlled by the forwarding server. Forwarding servers that do not rewrite the MAIL FROM address pose a risk of impersonation attacks and phishing. Do not add the IP addresses of forwarding servers to your MAIL FROM domain’s SPF policy unless you are in complete control of all sources of mail being forwarded through this infrastructure. For DKIM, it means that the messages are being modified in some way that causes DKIM signature validation failure (see Email Content Modification section above). A responsible forwarding server will rewrite the MAIL FROM domain so that the messages pass SPF with a non-aligned authenticated identifier. These servers will attempt to forward the message without alteration in order to preserve DKIM signatures, but that is sometimes challenging to do in practice. In this scenario, since the messages carry no domain-aligned authenticated identifier, the messages will fail the DMARC policy.

Solution:

  • Email forwarding is an expected type of failure of which you will see in the DMARC aggregate reports. The domain owner must weigh the risk of causing forwarded messages to be rejected against the risk of not publishing a reject DMARC policy. Reference 8.6. Interoperability Considerations. Forwarding servers that wish to forward messages that they know will result in a DMARC failure will commonly rewrite the “From” header address of messages it forwards so that the messages pass a DMARC policy for a domain that the forwarding server is responsible for. The way to identify forwarding servers that rewrite the “From” header in this situation is to publish “p=quarantine pct=0 t=y” in your domain’s DMARC policy before publishing “p=reject”.

Multiple email sending providers are sending using the same domain

Multiple email sending providers: There are situations where an organization will have multiple business units sending email using the same domain, and these business units may be using an email sending provider other than SES. If neither SPF nor DKIM is configured with domain-alignment for these email sending providers, you will see DMARC failures in the DMARC aggregate report.

Solution:

  • Analyze the DMARC aggregate reports to identify other email sending providers, track down the business units responsible for each email sending program, and follow the instructions offered by the email sending provider about how to configure SPF and DKIM to produce a domain-aligned authenticated identifier.

What does a DMARC aggregate report look like?

The following XML example shows the general format of a DMARC aggregate report that you will receive from participating email service providers.

<?xml version="1.0" encoding="UTF-8" ?> 
<feedback> 
  <report_metadata> 
    <org_name>email-service-provider-domain.com</org_name> 
    <email>[email protected]</email> 
    <extra_contact_info>https://email-service-provider-domain.com/> 
    <report_id>620501112281841510</report_id> 
    <date_range> 
      <begin>1685404800</begin> 
      <end>1685491199</end> 
    </date_range> 
  </report_metadata> 
  <policy_published> 
    <domain>example.com</domain>
    <adkim>r</adkim> 
    <aspf>r</aspf> 
    <p>none</p> 
    <sp>none</sp> 
    <pct>100</pct> 
  </policy_published> 
  <record> 
    <row> 
      <source_ip>192.0.2.10</source_ip>
      <count>1</count> 
      <policy_evaluated> 
        <disposition>none</disposition> 
        <dkim>pass</dkim> 
        <spf>fail</spf> 
      </policy_evaluated> 
    </row> 
    <identifiers> 
      <header_from>example.com</header_from>
    </identifiers> 
    <auth_results> 
      <dkim> 
        <domain>example.com</domain> 
        <result>pass</result> 
        <selector>gm5h7da67oqhnr3ccji35fdskt</selector> 
      </dkim> 
      <dkim> 
        <domain>amazonses.com</domain> 
        <result>pass</result> 
        <selector>224i4yxa5dv7c2xz3womw6peua</selector> 
      </dkim> 
      <spf> 
        <domain>amazonses.com</domain> 
        <result>pass</result> 
      </spf> 
    </auth_results> 
  </record> 
</feedback> 

 

How to address DMARC deployment for domains confirmed to be unused for email (dangling or otherwise)

Deploying DMARC for unused or dangling domains is a proactive step to prevent abuse or unauthorized use of your domain. Once you have confirmed that all subdomains being used for sending email have the desired DMARC policies, you can publish a ‘p=reject’ tag on the organizational domain, which will prevent unauthorized usage of unused subdomains without the need to publish DMARC policies for every conceivable subdomain. For more advanced subdomain policy scenarios, read the “tree walk” definitions in https://datatracker.ietf.org/doc/draft-ietf-dmarc-dmarcbis/

Conclusion:

In conclusion, DMARC is not only a technology but also a commitment to email security, integrity, and trust. By embracing DMARC best practices, organizations can protect their users, maintain a positive brand reputation, and ensure seamless email deliverability. Every message from SES passes SPF and DKIM for “amazonses.com”, but the authenticated identifiers are not always in alignment with the domain in the “From” header which carries the DMARC policy. If email authentication is not fully configured, your messages are susceptible to delivery issues like spam filtering, or being rejected or blocked by the recipient ESP. As a best practice, you can configure both DKIM and SPF to attain optimum deliverability while sending email with SES.

 

About the Authors

Bruno Giorgini Bruno Giorgini is a Senior Solutions Architect specializing in Pinpoint and SES. With over two decades of experience in the IT industry, Bruno has been dedicated to assisting customers of all sizes in achieving their objectives. When he is not crafting innovative solutions for clients, Bruno enjoys spending quality time with his wife and son, exploring the scenic hiking trails around the SF Bay Area.
Jesse Thompson Jesse Thompson is an Email Deliverability Manager with the Amazon Simple Email Service team. His background is in enterprise IT development and operations, with a focus on email abuse mitigation and encouragement of authenticity practices with open standard protocols. Jesse’s favorite activity outside of technology is recreational curling.
Sesan Komaiya Sesan Komaiya is a Solutions Architect at Amazon Web Services. He works with a variety of customers, helping them with cloud adoption, cost optimization and emerging technologies. Sesan has over 15 year’s experience in Enterprise IT and has been at AWS for 5 years. In his free time, Sesan enjoys watching various sporting activities like Soccer, Tennis and Moto sport. He has 2 kids that also keeps him busy at home.
Mudassar Bashir Mudassar Bashir is a Solutions Architect at Amazon Web Services. He has over ten years of experience in enterprise software engineering. His interests include web applications, containerization, and serverless technologies. He works with different customers, helping them with cloud adoption strategies.
Priya Priya Singh is a Cloud Support Engineer at AWS and subject matter expert in Amazon Simple Email Service. She has a 6 years of diverse experience in supporting enterprise customers across different industries. Along with Amazon SES, she is a Cloudfront enthusiast. She loves helping customers in solving issues related to Cloudfront and SES in their environment.

 

Manage Incoming Emails at Scale with Amazon SES

Post Syndicated from Bruno Giorgini original https://aws.amazon.com/blogs/messaging-and-targeting/manage-incoming-emails-with-ses/

Introduction

Are you looking for an efficient way to handle incoming emails and streamline your email processing workflows? In this blog post, we’ll guide you through setting up Amazon Simple Email Service (SES) for incoming email, focusing on the setup, monitoring, and use of receipt rules to optimize your email handling.

Amazon SES is a powerful and flexible cloud-based email service that enables you to send and receive emails at scale, while ensuring high deliverability and maintaining compliance with email best practices. By using Amazon SES for incoming email, you can customize your email processing pipeline and seamlessly integrate with other AWS services such as Amazon S3, AWS Lambda, and Amazon SNS.

We’ll start by walking you through the process of verifying your domain and setting up DomainKeys Identified Mail (DKIM) to ensure your emails are secure and authenticated. Next, we’ll explain how to create and manage receipt rule sets and add receipt rules with various actions for different processing scenarios. We’ll also cover monitoring your email processing using Amazon CloudWatch metrics.

As we progress, we’ll dive into advanced topics such as conditional receipt rules and chaining receipt rules, which can help you build complex and tailored email processing workflows, including multi-tenant scenarios. By the end of this post, you’ll have a comprehensive understanding of how to harness the power of Amazon SES for your incoming email needs.

So, let’s get started on simplifying your incoming email processing with Amazon SES!

Setting up Amazon SES for email receiving

Identifying the AWS region

For new users of the Amazon Simple Email Service (SES) inbound feature, it’s important to understand that all AWS resources used for receiving email with Amazon SES, except for Amazon S3 buckets, need to be in the same AWS Region as the Amazon SES endpoint. This means that if you are using Amazon SES in a specific region, such as US West (Oregon), any additional resources like Amazon SNS topics, AWS KMS keys, and Lambda functions also need to be created in the same US West (Oregon) Region. Additionally, to successfully receive email with Amazon SES within a particular Region, you must create an active receipt rule set specifically in that Region. By adhering to these guidelines, new users can effectively configure and utilize the inbound feature of Amazon SES, ensuring seamless email reception and efficient management of related resources. Amazon SES only supports email receiving in certain AWS Regions. For a complete list of Regions where email receiving is supported, see Amazon Simple Email Service endpoints and quotas in the AWS General Reference.

Verifying your domain

Before you can start receiving emails with Amazon SES, you must verify your domain. Domain verification is a crucial step in the setup process, as it confirms your ownership of the domain and helps prevent unauthorized use. In this section, we’ll walk you through the process of verifying your domain in the Amazon SES console.

  1. Sign in to the AWS Management Console and open the Amazon SES console.
  2. In the navigation pane, under Configuration, choose Verified identities.
  3. In the list of Identities section, choose Create identity.
  4. Under Identity details, choose Domain as the Identity type field. You must have access to the domain’s DNS settings to complete the domain verification process.
  5. Enter the name of the domain or subdomain in the Domain field.
  6. You must configure DKIM as part of the domain verification process. For Advanced DKIM settings, ensure that the Enabled box is checked in the DKIM signatures field.
  7. Choose Create identity. 
  8. This will generate a list of DNS records that you need to add to your domain’s DNS configuration. These can be found in the DomainKeys Identified Mail (DKIM) container, under Publish DNS records.

    SES DomainKeys Identified Mail (DKIM)

    Publish DNS records

  9. Add the generated DNS records to your domain’s DNS configuration. These records include a Legacy TXT record for domain verification and CNAME records for DKIM authentication. You may need to consult your domain registrar’s documentation for instructions on adding DNS records.
  10. Once the DNS records have been added, return to the Amazon SES console and wait for your domain’s verification status to change from “Verification pending” to “Verified.” This process may take up to 72 hours, depending on your domain registrar’s DNS propagation time.

Publishing an MX record for Amazon SES email receiving

To enable email receiving with Amazon SES, you need to publish an MX (Mail Exchange) record in your domain’s DNS configuration. The MX record directs incoming emails to Amazon SES for processing. Follow these steps to publish the MX record:

  1. Log in to your domain registrar or DNS management console.
  2. Locate the DNS management section for your domain.
  3. Create a new MX record by specifying the following details:
    • Host/Name/Record: Leave this field blank or enter “@” to represent the root domain.
    • Value/Points to/Target: Enter the value “10 inbound-smtp.[AWS Region].amazonaws.com“, replacing [AWS Region] with the AWS region where you are using Amazon SES for email receiving. For example, if you are using US West (Oregon) region, the value should be “10 inbound-smtp.us-west-2.amazonaws.com“.
    • TTL (Time to Live): Set a TTL value according to your preference or leave it as the default.
  4. Save the MX record.

Once the MX record is published with the correct value, incoming emails addressed to your domain will be routed to Amazon SES for processing. Remember to ensure that any other email-related resources, such as SNS topics or Lambda functions, are also created in the same AWS region as your Amazon SES endpoint.

For more detailed information on publishing MX records for Amazon SES email receiving, you can refer to the official documentation.

Creating a Receipt Rule set

A receipt rule set is a collection of rules that define how Amazon SES processes incoming emails for your domain. Each rule contains one or more actions that determine the processing flow of incoming emails. In this section, we’ll guide you through the process of creating a new receipt rule set in the Amazon SES console and activating it for your domain.

  1. Sign in to the AWS Management Console and open the Amazon SES console.
  2. In the navigation pane, under Configuration, choose Email receiving.
    • Note: if you don’t see the Email receiving option in the menu, check again that you’re in fact in a region supporting this feature.
  3. Under the Receipt rule sets tab in the Email receiving pane, choose Create rule setimage-20230523131953561.png
  4. Enter a name for your new rule set in the Rule set name field. This name should be descriptive and easy to identify, such as “MyApp-IncomingEmail.”
  5. After entering a unique name, choose Create rule setimage-20230523132526096.png
  6. To activate the newly created rule set, choose Set as active next to your rule set’s name. This action will ensure that Amazon SES uses this rule set for processing incoming emails to your domain. Your new rule set will now be listed in the Active rule set section.

For more information on creating and managing receipt rule sets, you can refer to the official documentation.

In the next section, we’ll explore adding receipt rules to your rule set, which define the specific actions to be taken for incoming emails.

Adding Receipt Rules

Receipt rules define the specific actions that Amazon SES should take when processing incoming emails for your domain. Common actions include saving the email to an Amazon S3 bucket, invoking an AWS Lambda function, or publishing a notification to an Amazon SNS topic. In this section, we’ll guide you through the process of adding receipt rules to your rule set in the Amazon SES console and provide examples of when to use each action.

  1. Sign in to the AWS Management Console and open the Amazon SES console.
  2. In the navigation pane, under Configuration, choose Email receiving.
  3. Under the Email receiving pane, in the Receipt rule sets tab, select the name of your active rule set from the All rule sets section. This will navigate to the details page for that rule set.
  4. Choose Create rule to begin creating a new receipt rule.
  5. On the Define rule settings page, under Receipt rule details, enter a unique Rule name.
    • For Status, only clear the Enabled checkbox if you don’t want to run this rule after creation.
    • (Optional) For Transport Layer Security (TLS), by selecting Required you can enforce a specific TLS policy for incoming emails that match this rule. By default, Amazon SES will use the Optional policy, which means it will attempt to use TLS but will not require it.
    • For Spam and virus scanning, only clear the Enabled checkbox if you don’t want Amazon SES to scan incoming messages for spam and viruses.
  6. After entering a unique rule name, choose Next.
  7. On the Add recipients conditions page, under Recipients conditions, use the following procedure to specify one or more recipient conditions. You can have a maximum of 100 recipient conditions per receipt rule.
    • Under Recipient condition, specify the email addresses or domains that this rule should apply to. You can use wildcards to match multiple addresses or domains. For example, you can enter example.com and .example.com to apply the rule to all email addresses within the example.com domain and within all of its subdomains.
    • Repeat this step for each recipient condition you want to add. When you finish adding recipient conditions, choose Next.
  8. On the Add actions page, open the Add new action menu and select the desired action from the list, such as Deliver to S3 bucket, Invoke AWS Lambda function, or Publish to Amazon SNS topic. Configure the selected action’s settings as required.
    • Deliver to S3 bucket: Choose this action if you’re expecting emails with large attachments, need to store emails for archival purposes, or plan to process emails using other AWS services that integrate with Amazon S3. You’ll need to specify the Amazon S3 bucket where the incoming emails should be stored.
    • Invoke AWS Lambda function: Choose this action if you want to process incoming emails using custom logic, such as filtering, parsing, or modifying the email content. You’ll need to specify the AWS Lambda function that should be invoked when an incoming email matches this rule.
    • Publish to Amazon SNS topic: Choose this action if you’re processing smaller emails or want to receive real-time notifications when an email arrives. You’ll need to specify the Amazon SNS topic where notifications should be published.
    • For more information and additional actions, see the Action options section of the Developer Guide.
  9. Once configured, choose Next to proceed to the Review page.
  10. On the Review page, review the settings and actions of the rule. If you need to make changes, choose the Edit option.
  11. When finished, choose Create rule to add the new receipt rule to your rule set. The rule will now be applied to incoming emails that match the specified recipient conditions.
image.png

You can create multiple receipt rules within a rule set, each with different actions and conditions. Amazon SES will apply the rules in the order they appear in the rule set. For more information on creating and managing receipt rules, you can refer to the official documentation.

Monitoring your incoming email

Configuring Amazon CloudWatch metrics

Once you have enabled email receiving in Amazon SES and created receipt rules for your emails, you can monitor and view the metrics using Amazon CloudWatch. Follow these steps to configure Amazon CloudWatch metrics for Amazon SES email receiving:

  1. Open the Amazon CloudWatch console.
  2. Navigate to the Metrics section and select All metrics.
  3. In the list of available metrics, locate and select SES to view SES-related metrics.
  4. Expand the Receipt Rule Set Metrics and Receipt Rule Metrics sections to access the specific metrics for your receipt rule sets and rules.
  5. Under Receipt Rule Set Metrics, you will find the following metrics:
    • “Received”: Indicates whether SES successfully received a message that has at least one rule applying. The metric value is always 1.
    • “PublishSuccess”: Indicates whether SES successfully executed all rules within a rule set.
    • “PublishFailure”: Indicates if SES encountered an error while executing rules within a rule set. The error may allow for retrying the execution.
    • “PublishExpired”: Indicates that SES will no longer retry executing the rules within a rule set after four hours.

These metrics can be filtered by the dimension RuleSetName to obtain data specific to individual rule sets.

  1. Under Receipt Rule Metrics, you will find the following metrics:
    • “Received”: Indicates whether SES successfully received a message and will try to process the applied rule. The metric value is always 1.
    • “PublishSuccess”: Indicates whether SES successfully executed a rule that applies to the received message.
    • “PublishFailure”: Indicates if SES encountered an error while executing the actions in a rule. The error may allow for retrying the execution.
    • “PublishExpired”: Indicates that SES will no longer retry executing the actions of a rule after four hours.

These metrics can be filtered by the dimension RuleName to obtain data specific to individual rules.

  1. Note that the metrics will only appear in the CloudWatch console if you have enabled email receiving, created receipt rules, and received mail that matches any of your rules.
  2. Keep in mind that changes made to fix your receipt rule set will only apply to emails received by Amazon SES after the update. Emails are always evaluated against the receipt rule set in place at the time of receipt.

Amazon SES also provides an Automatic Dashboard for SES in the CloudWatch console, which offers a preconfigured set of SES metrics and alarms to monitor your email sending and receiving activity. This dashboard provides a consolidated view of key metrics, making it easier to track the performance and health of your Amazon SES environment.

By configuring Amazon CloudWatch metrics, you can gain valuable insights into the performance and execution of your receipt rule sets and rules within Amazon SES. For more detailed information on viewing metrics for Amazon SES email receiving using Amazon CloudWatch, refer to the official documentation.

Using receipt rules effectively

Chaining Receipt Rules

Chaining receipt rules enable you to create sophisticated email processing workflows by linking multiple rules together, allowing each rule to apply specific actions based on the outcome of the previous rule. This advanced technique can help you achieve greater flexibility and precision in handling your incoming emails with Amazon SES. In this section, we’ll explain how to create chained receipt rules and provide examples of common use cases.

  1. Sign in to the AWS Management Console and open the Amazon SES console.
  2. Under the Email receiving pane, in the Receipt rule sets tab, select the name of your active rule set from the All rule sets section
  3. Review the existing rules in your rule set and ensure that they are ordered correctly. Chaining relies on the order of the rules, as each rule’s conditions and actions are evaluated sequentially. Under the Reorder tab, the rule orders can be modified by selecting the corresponding arrow associated with each.
  4. To chain additional rules, follow the steps previously outlined in the Adding Receipt Rules section and adjust the rule orders as necessary.

Chaining receipt rules can help you build complex email processing workflows with Amazon SES. Some common use cases include:

  • Executing multiple filtering criteria in an order that you specify. For example, adding a specific header value and then sending to additional AWS services such as Amazon S3, Amazon SNS, or AWS Lambda.
  • Creating multi-stage processing pipelines, where the output of one action (e.g., saving an email to Amazon S3) is used as the input for the next action (e.g., processing the email with AWS Lambda).
  • Implementing fallback actions, where the first rule in the chain attempts a specific action (e.g., saving an email to a primary S3 bucket), and if it fails, the next rule in the chain applies a different action (e.g., saving the email to a secondary S3 bucket).

The following figure shows how receipt rules, rule sets, and actions relate to each other.

SES Chaining multiple rules in a rule set

For more information on creating and managing receipt rules, you can refer to the official documentation.

Handling the 200 Receipt Rules per Rule Set limit

For each AWS account, Amazon SES imposes a limit of 200 receipt rules per receipt rule set. While this limit is sufficient for most use cases, there might be situations where you need to process a higher volume of incoming emails with more complex rule sets. These are some strategies to work around the 200 receipt rule limit using Amazon SES and other AWS services:

  • Utilize rule chaining: As mentioned earlier, chaining receipt rules allows you to link multiple rules together, effectively extending the number of actions you can perform for a single email. By chaining rules, you can create more complex processing workflows without exceeding the 200 rule limit.
  • Combine rules with actions: Instead of creating separate rules for each scenario, consider combining multiple actions within a single rule. This approach can help you reduce the total number of rules while still catering to various email processing requirements.
  • Use AWS Lambda for custom processing: Leverage AWS Lambda to perform custom processing on incoming emails. By incorporating Lambda functions in your receipt rules, you can handle more complex processing tasks without increasing the number of rules. This approach also allows you to offload some processing logic from Amazon SES to Lambda, providing additional flexibility.
  • Consolidate similar actions: If you have several rules performing similar actions, it is advisable to consolidate them into a single rule with multiple actions. This consolidation can help you reduce the total number of rules while maintaining the desired functionality.
  • Evaluate rule usage: Regularly review and evaluate your existing receipt rules to identify any rules that are no longer in use or can be optimized. Removing or consolidating unnecessary rules can help you stay within the 200 rule limit while still addressing your email processing requirements.

By implementing these strategies, you can effectively work around the 200 receipt rule limit in Amazon SES and build more complex email processing workflows to cater to your specific needs. Remember to monitor and optimize your rule sets regularly to make the most of the available resources and maintain efficient email processing.

For more information on the inbound quotas and limits in Amazon SES, you can refer to the official AWS documentation at Quotas related to email receiving.

Best Practices for multi-tenant scenarios

When dealing with multi-tenant scenarios in your application, it’s crucial to manage incoming emails efficiently to ensure smooth operation and a seamless experience for your users. In this section, we’ll provide best practices to handle incoming emails in multi-tenant environments using Amazon SES.

In a multi-tenant scenario, where multiple customers or tenants share a single AWS account, it’s important to consider the limit of 200 receipt rules per receipt rule set imposed by Amazon SES. To ensure compliance with this limit and maintain optimal email processing, the following practices are recommended:

  • Segregate tenants using email subdomains: Create unique subdomains for each tenant and route their incoming emails accordingly. This approach makes it easier to manage email processing rules and helps isolate tenants from potential issues.
  • Create separate rule sets for each tenant: By creating dedicated rule sets for each tenant, you can maintain better control over email processing rules and actions specific to their needs. This can simplify management and make it easier to update rules for individual tenants without affecting others.
  • Use tags to identify tenant-specific emails: Apply tags to incoming emails using the AddHeader action in your receipt rules. These tags can include tenant-specific identifiers, which will help you route and process emails correctly. You can later use these tags in other AWS services (e.g., AWS Lambda) to process tenant-specific emails.
  • Leverage conditional receipt rules: Utilize conditional receipt rules to apply tenant-specific processing based on email headers, recipients, or other criteria. This way, you can ensure that the right actions are taken for each tenant’s incoming emails.
  • Monitor tenant-specific metrics: Configure Amazon CloudWatch metrics and alarms for each tenant to track their email processing performance separately. This enables you to keep a close eye on individual tenants and take appropriate actions when needed.
  • Implement rate limiting: To prevent tenants from overwhelming your email processing pipeline, consider implementing rate limiting based on the number of incoming emails per tenant. This can help ensure fair resource allocation and prevent potential abuse.
  • Ensure security and privacy: Always encrypt tenant data at rest and in transit, and follow best practices for data protection and privacy. Consider using AWS Key Management Service (KMS) to manage encryption keys for each tenant.
  • Test and validate rule sets: Before deploying rule sets for tenants, thoroughly test and validate them to ensure they function as intended. This can help prevent unexpected behavior and maintain a high level of service quality.

By following these best practices for handling incoming emails in multi-tenant scenarios with Amazon SES, you can ensure a robust and efficient email processing pipeline that caters to each tenant’s unique requirements. As you continue to work with Amazon SES in multi-tenant environments, stay up to date with AWS documentation and best practices to further optimize your email processing workflows.

Conclusion

In this blog post, we’ve explored how to set up Amazon Simple Email Service (SES) for incoming email processing using receipt rules, rule sets, and various actions. We’ve covered domain verification, DKIM setup, creating and managing rule sets, adding receipt rules, and configuring Amazon CloudWatch metrics and alarms. We’ve also delved into advanced topics such as chaining receipt rules for more complex email processing workflows.

By following this guide, you can effectively leverage Amazon SES to process and manage your incoming emails, optimizing your email workflows, and maintaining high email deliverability standards. With Amazon SES, you can customize your email processing pipeline to meet your specific needs and seamlessly integrate with other AWS services such as Amazon S3, AWS Lambda, Amazon SNS, and Amazon CloudWatch.

In future blog posts, we will explore monitoring and alerting in more detail, providing you with additional insights on how to effectively monitor your email processing pipelines and set up alerts for critical events. Stay tuned for more information on this important aspect of managing your email infrastructure.

As you continue to work with Amazon SES and its email receiving capabilities, remember to review AWS best practices and documentation to stay up to date with new features and improvements. Don’t hesitate to experiment with different rule sets, actions, and conditions to find the perfect email processing solution for your use case.