Tag Archives: Partner solutions

AI-driven software delivery with Kiro, AWS DevOps Agent and Bluebox by Dynatrace

Post Syndicated from Philipp Ushiromiya original https://aws.amazon.com/blogs/devops/ai-driven-software-delivery-with-kiro-aws-devops-agent-and-bluebox-by-dynatrace/

This post was co-written with Michael Stephan, Senior Principal Product Manager, and Christian Kreuzberger, Principal Software Engineer, at Dynatrace.

AI-driven software delivery changes how code gets written, but not what production demands of it. A generated change still has to fit the traffic your service receives, the dependencies it calls, and the capacity limits it runs within. Without that context, you validate the change after it ships, which adds rework and deployment risk.

Kiro turns intent into specifications, code, and pull requests. AWS DevOps Agent investigates incidents and proposes mitigations. Bluebox by Dynatrace supplies the runtime topology, dependency, and traffic data that both draw on, so each change and each investigation is grounded in how the system behaves rather than how it’s expected to behave. In this post, we will follow a travel-booking example from feature design through post-deployment remediation. You’ll see how telemetry from Bluebox shapes a change in Kiro, how AWS DevOps Agent investigates an incident, and where human review and existing CI/CD controls remain in the process.

What are Kiro and AWS DevOps Agent?

Kiro is an agentic development environment that applies AI across the software development lifecycle. Its spec-driven workflow organizes a feature request into requirements, design, and implementation tasks before generating any code.

AWS DevOps Agent is a frontier agent for software delivery and operations across AWS, multicloud, and on-premises environments. It investigates incidents, identifies likely root causes, and recommends mitigations. Its release management capability (Preview) reviews code for release readiness and runs release tests before deployment.

Bluebox by Dynatrace: Helps agents ship the code you trust to production

To close the loop between code generation and production context, Kiro and AWS DevOps Agent rely on real-time production intelligence. This is where Bluebox by Dynatrace fits in. Bluebox provides the observability foundation that detects problems, measures their impact, and surfaces the runtime application topology, service dependencies, and actual traffic patterns that make AI-generated code and autonomous investigations truly production-aware.

Without production telemetry, AI-generated code operates in a vacuum – it cannot know that an endpoint handles 40:1 read-to-write ratios, that a service dependency has specific latency characteristics, or how API traffic fluctuates throughout the day. Bluebox grounds actions taken by Kiro and AWS DevOps Agent in how the system actually behaves, not in assumptions about how it should behave.

How the closed loop works

The combination of Kiro, AWS DevOps Agent, and Bluebox creates a continuous cycle from development through production and back:

  • Production-aware code generation: Before code is written, Kiro retrieves runtime context from Bluebox – service topology, traffic patterns, and resource utilization. Kiro’s spec-driven workflow translates this context into requirements and generates code that aligns with real production conditions from the first commit.
  • Confident code review: Kiro generates pull requests with production evidence attached. The release management capability in AWS DevOps Agent reviews the change for dependency impacts, drifts from internal standards, and production readiness – running autonomous tests in isolated environments.
  • Continuous monitoring: After deployment, Dynatrace continuously monitors application behavior. When an anomaly occurs, Bluebox detects it and surfaces full production context.
  • Autonomous investigation: Bluebox triggers AWS DevOps Agent with the relevant observability and topology data. AWS DevOps Agent performs a deep investigation, correlating telemetry, logs, infrastructure changes, and deployment history to pinpoint the root cause.
  • Automated remediation: AWS DevOps Agent generates the mitigation plan from the observability and runtime data that Bluebox provides. Bluebox adds that plan to the investigation report and files it as a GitHub issue. Kiro then proposes a production-aware fix as a pull request for your review, completing the loop.

Figure 1: Bluebox supports the closed loop from feature build to operations.

Next, we walk through a concrete example of this workflow in action.

Walkthrough

We follow a travel-booking application through two connected scenarios: shipping a new feature with production context, then responding to a production incident after it deploys.

Building a production-aware feature

Consider a team enhancing a travel booking application to improve customer experience. You begin by describing a new feature in Kiro, such as updating how products are displayed or adjusting backend logic to support new capabilities. In this case, we are using Kiro IDE.

Figure 2. A feature request in Kiro, with the project’s steering documents loaded for context.

Kiro’s spec-driven workflow expands this request into structured requirements before writing code. You connect Kiro to the Bluebox CLI to retrieve the full production context from Dynatrace: service dependencies, runtime topology, and observed traffic. The following figure shows how Kiro queries current load data for the flight-search path, including the ratio of Amazon DynamoDB reads to writes. Kiro composes and runs the CLI command on your behalf, so you don’t have to type it or set environment variables by hand. The command and its output stay visible in the session, so you can approve it before it runs and check what was retrieved before acting on it. In this case, the command queries the Bluebox API for the requested metrics. The output returns read and write counts per second for the DynamoDB table behind flight search, along with the services calling it.

Figure 3. Kiro runs the Bluebox CLI, then reads the codebase with production context before proposing changes.

The telemetry shows the flight-search endpoint is read-heavy. Users repeatedly query the same routes, at roughly 40 reads for every write against the DynamoDB table. Repeated identical reads are what a cache absorbs, so Kiro proposes an Amazon ElastiCache layer in front of the table, sized to the active working set derived from the observed request distribution. Without the read-to-write ratio, the same request could have produced a larger provisioned table or an added read replica, neither of which addresses repeated identical queries.

Kiro generates the code that implements the change and opens a pull request in GitHub for review. Nothing reaches production until a reviewer approves and merges it. The pull request carries the code changes and the Bluebox telemetry that justified them, so reviewers assess the decision against the same telemetry Kiro retrieved.

Figure 4. Kiro pushes a feature branch and opens a pull request in GitHub.

After review and approval through standard processes, a reviewer merges the pull request, and the existing CI/CD pipeline deploys the change.

Figure 5. The pull request is reviewed and merged through the standard GitHub workflow.

Responding to a production incident

With the feature live, Dynatrace continues monitoring the application. A marketing promotion then drives traffic above the observed baseline, and failed requests start to appear. The loop now runs from operations back to development.

Figure 6. Dynatrace detects a spike in failed requests, surfacing the production incident.

Bluebox collects the relevant observability and topology data, runs an initial root-cause analysis, then opens an autonomous investigation in AWS DevOps Agent. The AWS DevOps Agent multi-agent reasoning architecture decomposes the investigation across specialized capabilities that each examine one class of evidence: telemetry, logs, infrastructure configuration, and recent deployment activity.

Figure 7. Bluebox delegates an autonomous investigation to AWS DevOps Agent.

AWS DevOps Agent locates the cause in the DynamoDB table rather than the new cache. The table’s billing mode had been changed to PROVISIONED, with 5 read capacity units (RCU) and 5 write capacity units (WCU) and no auto scaling. The ElastiCache layer absorbs repeated reads, but cache misses and all writes still reach DynamoDB, and at promotion traffic that residual load exceeds 5 RCU and 5 WCU. AWS DevOps Agent produces a mitigation plan with specific remediation steps. This plan and the full investigation context from Bluebox, is documented as a GitHub issue.

Figure 8. GitHub issue is created with results from Bluebox and AWS DevOps Agent.

Kiro proposes a production-aware fix as a new pull request – including the root-cause analysis, supporting telemetry, and recommended configuration changes.

Figure 9. The Kiro coding session works on the GitHub issue and creates a remediation Pull Request.

The fix is reviewed, merged, and deployed like any other change. Dynatrace then confirms that error rates and response times return to baseline, which closes the loop.

Conclusion

In this post, we showed how Kiro, AWS DevOps Agent, and Bluebox by Dynatrace connect production telemetry with feature development and incident remediation. The travel-booking example keeps human review and existing CI/CD controls in the process while passing operational context from production back to development.

To get started pick one application and define a measurable outcome, such as investigation time, change-failure rate, or pull-request review time. Then:

  1. Download Kiro and start building with spec-driven development
  2. Enable AWS DevOps Agent for autonomous incident investigation and remediation
  3. Get started with Bluebox by Dynatrace to complete the loop with production intelligence

Simone Pomata

Simone is a Principal Solutions Architect at AWS. He has worked enthusiastically in the tech industry for more than 10 years. At AWS, he helps customers succeed in building new technologies every day.

Philipp Ushiromiya

Philipp Ushiromiya is a Solutions Architect at AWS. He helps customers drive organizational modernization through cloud-native solutions and DevOps practices. His passion for GenAI enables teams to accelerate development with cutting-edge technology.

Michael Stephan

Michael Stephan is a Senior Principal Product Manager at Dynatrace with over 15 years of experience in the IT industry. He specializes in helping Dynatrace customers effectively monitor and optimize their cloud environments.

Christian Kreuzberger

Christian Kreuzberger is a Principal Software Engineer at Dynatrace, with over 20 years of experience in the IT industry. At Dynatrace, he builds software that helps cloud-native and AI-native organizations automate their operations.

Add security context to operational investigations with AWS DevOps Agent and Wiz

Post Syndicated from Yuriy Prykhodko original https://aws.amazon.com/blogs/devops/add-security-context-to-operational-investigations-with-aws-devops-agent-and-wiz/

This post was co-authored by Ayelet Harcz (Product Manager), Hen Perez (CTO Architect), and Shani Gafni (Product Manager) at Wiz.

When an on-call engineer receives an alert at 2 AM, a CPU spike, a latency anomaly, or an unexpected API error, the first question is whether this is an operational issue or a security incident. A CPU spike could be a scaling problem or a cryptominer. A latency anomaly could be a bad deployment or data exfiltration. Without security context in the investigation loop, engineers lack the information to distinguish between the two, delaying resolution and increasing risk.

AWS DevOps Agent is a frontier agent that autonomously investigates incidents and identifies operational improvements across AWS, multicloud, and on-premises environments. It reduces mean time to resolution (MTTR) by performing the triage and investigation work that would otherwise take an on-call engineer hours of manual effort. With the Wiz integration, AWS DevOps Agent queries Wiz’s security graph during investigations through the Model Context Protocol (MCP), surfacing vulnerability data, security findings, and exposure analysis alongside operational telemetry so engineers can quickly determine whether an alert is a performance issue or a security incident.

In this post, we walk through how the integration works, demonstrate a real-world incident investigation where AWS DevOps Agent uses Wiz MCP to surface a critical vulnerability behind an API latency spike, and show how to configure the integration in your environment. If you already use Wiz to secure your AWS environment, this integration puts your existing security data to work during incident investigations.

AWS DevOps Agent

AWS DevOps Agent investigates incidents and identifies operational improvements as an experienced DevOps engineer would: by learning your resources and their relationships, working with your observability tools, runbooks, code repositories, and CI/CD pipelines, and correlating telemetry, code, and deployment data across all of them. For a deeper look at how it works, see How AWS DevOps Agent uses multi-agent reasoning to find root causes.

AWS DevOps Agent is extensible through MCP, which allows the agent to call external tools during its investigation without requiring custom development. This is the mechanism that makes the Wiz integration possible. When the agent identifies a resource under investigation, it queries Wiz MCP for security findings associated with that resource and incorporates the results into its analysis and recommendations.

Wiz MCP

Wiz is designed to secure cloud and AI applications through a unified, graph-powered platform. The Wiz Security Graph connects infrastructure, identities, data, AI components, and runtime activity into a single contextual view. This approach identifies toxic combinations across layers – where exposures, permissions, data access, AI vulnerabilities, and runtime behaviors intersect in ways attackers can realistically exploit.

The Wiz MCP Server acts as a standardized gateway that allows AWS DevOps Agent to query this security graph during investigations. Wiz knows whether your Amazon Elastic Compute Cloud (Amazon EC2) instance has an exploitable Common Vulnerabilities and Exposures (CVE), whether it is publicly exposed, and whether endpoint protection is in place. AWS DevOps Agent, looking at the same instance, knows that CPU spiked, and a deployment happened 20 minutes ago. Separately, each tool tells a partial story. Together, they give the engineer the complete picture needed to act.

Better together: how combined context changes triage

The value of this integration is easiest to understand through three scenarios. Each starts with the same operational signal: a CPU spike on an EC2 instance.

Figure 1 – AWS DevOps Agent sees operational telemetry, Wiz sees security posture. The combination changes the triage decision.

Figure 1 – AWS DevOps Agent sees operational telemetry, Wiz sees security posture. The combination changes the triage decision.

Scenario A: No security findings. A CPU spike fires on an instance. AWS DevOps Agent queries Wiz and confirms the instance is fully monitored, has no known vulnerabilities, and shows zero active threat detections. This is an operational issue. The engineer scales, investigates the deployment, tests, and moves on.

Scenario B: Security issue detected. The same CPU spike fires, the same Amazon CloudWatch alarm triggers, and the same engineer wakes up. But when AWS DevOps Agent queries Wiz, it finds a validated remote code execution vulnerability on that instance, confirmed exploitable, with the resource exposed to the internet. The operational symptoms are identical to Scenario A. The correct response is the opposite: isolate immediately, engage your security team, treat this as a potential compromise.

Scenario C: Wiz coverage gap. The resource isn’t in Wiz at all. AWS DevOps Agent includes this as a finding in the investigation report, noting that no security context was available for the resource. Your team can then address the coverage gap by onboarding the resource into Wiz.

Without the Wiz integration, all three scenarios look the same in your dashboard. With it, AWS DevOps Agent routes each to the correct response path before a human needs to context-switch between tools.

How the integration works: the MCP bridge

The integration uses MCP, the same protocol AWS DevOps Agent uses for many of its external tool connections. When the agent identifies affected resources during an investigation, it calls Wiz’s remote MCP server as part of its evidence collection – no separate step, no manual trigger. The security query happens alongside the operational investigation, not after it. During the MCP call, AWS DevOps Agent sends resource identifiers to Wiz’s MCP endpoint and receives security findings in response. No operational telemetry or broader investigation context is shared with Wiz.

Figure 2 – The investigation flow: operational alert triggers AWS DevOps Agent, which queries Wiz via MCP before reaching a triage decision.

During the MCP call, AWS DevOps Agent queries Wiz tools to build a complete risk picture of the affected resource, here are a few examples:

Wiz MCP Tool What it tells the agent
list_cloud_resources Whether Wiz monitors this resource at all (coverage check)
list_findings All finding types in one call: vulnerabilities, misconfigurations, secrets, data, and host config
list_vulnerability_findings Deep CVE detail – severity, fix version, and exploitability (CISA KEV / known exploit)
list_issues Prioritized risk issues, including toxic combinations (internet-facing + no Endpoint Detection and Response (EDR) + exploitable CVE)
list_threats / list_malware_findings Active threats and malware: cryptomining, data exfiltration, backdoors
list_detections Recent threat detection signals and anomalous activity
get_green_agent_analysis AI-generated remediation steps for the issues found

The agent runs these queries together through a single security-auditing skill that loads automatically when it connects to Wiz’s MCP server with the DevOps toolset, so the full security picture comes back in seconds. If the Wiz MCP server is unreachable, times out mid-query, or returns an authentication error, the agent continues its investigation with the operational data it has and flags the missing security context in the investigation findings (Scenario C). You can review exactly which MCP tools were called and what data was returned in the AWS DevOps Agent investigation log for full auditability.

Based on what comes back, the agent classifies the situation: no security findings (operational issue, proceed normally), compromised or at-risk (active threats, exploitable vulnerabilities, or toxic combinations – apply relevant security runbooks to isolate the resource or escalate to security, with Wiz Green Agent remediation steps attached), or unmonitored by Wiz (flag and close the coverage gap). The classification feeds directly into the investigation findings your team receives.

The following demonstration shows AWS DevOps Agent investigating a reported CPU spike. The agent queries Wiz MCP and identifies a critical, internet-exposed Remote Code Execution (RCE) under active exploitation – turning an ambiguous alert into a confirmed security incident.

Video 1 – AWS DevOps Agent investigates a CPU spike and uses Wiz MCP security context to identify a critical RCE exploited through a public endpoint

Getting started

Prerequisites

To use AWS DevOps Agent with Wiz MCP, you need:

  1. An active AWS DevOps Agent configuration with at least one Agent Space
  2. A Wiz tenant with a remote MCP server endpoint (Streamable HTTP transport)
  3. Authentication credentials for the Wiz MCP server. AWS DevOps Agent supports multiple MCP auth methods; for Wiz, use a Wiz service account (Client ID and Secret) or OAuth. Choose the method that matches your Wiz MCP server configuration. For setup details, see Connect remote Wiz MCP server in the Wiz documentation (requires Wiz login)

Enabling the integration

Step 1: Register the Wiz MCP server at account level

  1. Sign in to the AWS Management Console and navigate to the AWS DevOps Agent console.
  2. Go to the Capability Providers page from the side navigation.
  3. Find MCP Server in the Available providers section and choose Register.
  4. Enter the Wiz MCP server details:
    • Name: e.g., “Wiz Security”
    • Endpoint URL: https://mcp.app.wiz.io/?toolset=devops
    • Description: e.g., “Wiz security context for incident triage”
  5. Choose Next.
  6. Select the authentication method that matches your Wiz MCP server configuration.
  7. Review your configuration and choose Submit. AWS DevOps Agent validates the connection to the Wiz MCP server. Upon successful validation, the server is registered at the account level.

Step 2: Allowlist Wiz tools in your Agent Space

  1. In the AWS DevOps Agent console, select your Agent Space.
  2. Go to the Capabilities tab.
  3. In the MCP Servers section, choose Add.
  4. Select the registered Wiz MCP server.
  5. Select all the Wiz MCP tools.
  6. Choose Add.

Step 3: Choose how the Wiz security audit runs

Pick one of three options:

  1. Use the Wiz skill tool (recommended). With the Wiz MCP tools allowlisted, AWS DevOps Agent automatically runs the latest devops_resource_auditing_skill workflow from Wiz during investigations. You always get the most up-to-date version, maintained by Wiz.
  2. Import the ready-made skill. Import the wiz-security-context skill from the AWS DevOps Agent skills repo directly into your Agent Space. It is a lightweight skill that calls the Wiz workflow for you, so you get a one-step setup that stays current with Wiz.
  3. Create your own custom skill. Use AWS DevOps Agent’s Create skill with Chat to build a custom skill based on the devops_resource_auditing_skill workflow and tailor it to your environment. This lets you review and tailor the workflow to your environment.

For detailed MCP configuration guidance, refer to the AWS DevOps Agent documentation on connecting remote MCP servers.

The power of co-build: extending context through MCP

This integration started from a recurring customer question: how do I know if what I’m seeing is an operational problem or an active attack? We worked with Wiz to close this gap. AWS DevOps Agent provides operational investigation and reasoning; Wiz provides cloud security intelligence. MCP provided the integration path without either side needing to reimplement what the other already does well.

Because AWS DevOps Agent supports connecting remote MCP servers as a first-class extension mechanism, co-building new integrations with AWS Partners follows a repeatable pattern. Each integration adds a new dimension of context to the agent’s reasoning, and you benefit without writing custom code or middleware on your side. For example, connecting a change management MCP server would let the agent correlate deployment approvals with incident timing, adding change context alongside security context.

For you, this means the richer the toolset you run in your environment, the more context the agent brings to each investigation. Your existing investments get amplified rather than duplicated, and you benefit each time you connect a new partner MCP server to your Agent Space.

Conclusion

Operational incidents and security incidents often start with the same symptoms. The difference between the right response to each is context that lives in a different tool than the one that fired the alert. The AWS DevOps Agent and Wiz integration brings that context into the investigation loop automatically through MCP.

To get started, visit the AWS DevOps Agent console and follow the getting started guide. To learn more about Wiz’s MCP server, see Introducing the MCP Server for Wiz.

Wiz is an AWS Partner and AWS Marketplace Seller providing cloud security across the full development lifecycle. If you’re not already using Wiz, you can get started through the AWS Marketplace.

About the Authors

Yuriy Prykhodko

Yuriy Prykhodko is a Principal Technical Account Manager at AWS, based in Luxembourg. He partners with customers to architect highly reliable, cost-effective systems and drive operational excellence across their cloud workloads, with a focus on applying AI to streamline cloud operations. Yuriy is also an active contributor to the Cloud Operations Technical Field Community at AWS, where he leads several initiatives at the intersection of AI and cloud operations. Outside of work, he enjoys playing basketball and exploring new destinations around the world.

Ziv Shenhav

Ziv is a Principal Customer Solutions Manager at AWS. With nearly a decade at AWS, he helps ISV customers across EMEA accelerate modernization and transition into the agentic AI era. Outside of work, Ziv enjoys nature photography.

Yossi Lagstein

Yossi Lagstein is a Senior Solutions Architect at Amazon Web Services. Yossi has over 30 years of experience as specialist and manager in developing infrastructure components for a variety of projects and products. Yossi supports AWS customers to evolve, design and build well architected solutions. Outside of works, Yossi enjoys running , swimming and hiking.

Ayelet Harcz

Ayelet is a Product Manager at Wiz focused on the frontier of agentic AI engineering. She leads product initiatives and scaling coverage around Mika-Wiz’s AI assistant-and its expanding MCP ecosystem to deliver intelligent cybersecurity capabilities. She holds a degree in Computer Science and Cognitive Science, and outside of work, she enjoys practicing yoga

Hen Perez

Hen is a CTO Architect at Wiz, specializing in cloud security and agentic AI. He architected and co-built the patented Wiz MCP Server, enabling organizations to build AI-powered security agents on top of Wiz, and works across the Wiz Integration Network (WIN) and its MCP ecosystem. With over 19 years of experience spanning embedded systems, observability, and cybersecurity, he focuses on unlocking secure agentic workflows. In his free time, he enjoys playing the piano, experimenting with AI and synthesizers, and hacking life with his daughter.

Shani Gafni

Shani is Product Manager at Wiz, specializing in agentic AI engineering. Her work centers on Wiz’s core AI assistant, Mika along with Wiz Green agent, MCPs and related tools, delivering innovative cybersecurity solutions. In her free time, she enjoys books, music, nature, and photography.

Security Hub adds AI workload protection and multicloud support for Microsoft Azure

Post Syndicated from Michael Fuller original https://aws.amazon.com/blogs/security/security-hub-adds-ai-workload-protection-and-multicloud-support-for-microsoft-azure/

Security Hub is our foundation for full-stack enterprise security across clouds. It centralizes your security operations and turns raw signals into prioritized insights, so your team spends its time managing real risk instead of stitching tools together. Today that foundation grows in two directions our customers asked for most. We are adding purpose-built protection for AI workloads, and security monitoring for Microsoft Azure. Both are steps toward a bigger idea, that your best security tools should get smarter by working together.

These expansions came directly from customers, and they reflect where security is heading, not where it has been. The old promise of security tooling was a place to collect everything in one view. Collecting findings was never the hard part. The hard part is understanding them, connecting them, and acting before an attacker does, and doing it at the speed attacks now move. The programs that win from here will be the ones that see across their whole estate and respond fast, not the ones with the most dashboards. That is what we are building toward, and these launches are steps on that path.

Multicloud security management for Microsoft Azure

Customers across industries have made Security Hub a core part of how they run security on AWS. Most of them have run in more than one cloud for years, and they have been clear with us that they want Security Hub to also cover the rest of their estate. Today we do that for Microsoft Azure, with more clouds following quickly.

Security Hub now discovers Azure Virtual Machines, container images, Function Apps, and identities, then evaluates them for misconfigurations, internet exposure, and software vulnerabilities, with posture checks against the CIS Microsoft Azure Foundations Benchmark™. Azure findings are prioritized next to your AWS findings using the same finding format, automation, and response workflows, so your team works from one understanding of risk across your entire estate. Azure resources are priced at the same rates as equivalent AWS resources with no additional fees, and there’s an independent 30-day free trial. To learn more, see the What’s New post.

This is not actually our first move beyond AWS. Earlier this year we introduced Security Hub Extended, bringing best-in-class partner solutions across nine security categories into the same experience you already use. Those partner solutions protect endpoints, identities, email, browsers, and data wherever they run, across any cloud, on-premises, and everywhere your enterprise operates. Extended was already our first multicloud and multi-workload step. Today we broaden what our own native capabilities cover, and the two lines of work now advance together.

Protecting AI workloads

Every customer I talk to is building with AI. Generative AI on Amazon Bedrock, model training on SageMaker, agents orchestrating workflows through AgentCore. These workloads are reaching production faster than most security programs can keep up, and teams often don’t yet have the tools to monitor model invocations, track agent behavior, or even know what AI assets exist across the organization. One security leader told me his team only caught a compromised service account, one that had been invoking a foundation model thousands of times, because finance questioned the bill. They found a security incident through an accounting review. The visibility gap is real, and it is already expensive.

This summer we start closing it with three launches. Two are GuardDuty capabilities for threat detection and investigation, and a third is a new Security Hub AI inventory.

GuardDuty AI Protection (generally available)

Amazon GuardDuty AI Protection delivers threat detection purpose-built for Bedrock and SageMaker. It detects anomalous model invocations, cost harvesting attacks where adversaries abuse stolen credentials to run inference at your expense, and prompt injection attempts through integration with Bedrock Guardrails.

Cost harvesting is accelerating. When credentials are compromised, attackers increasingly use them to invoke foundation models. Inference is expensive, demand is high, and stolen access converts straight to value without deploying any infrastructure. GuardDuty analyzes CloudTrail data events, learns what normal invocation looks like at scale, and flags the deviations that signal compromise or abuse. This is detection that only works at AWS scale, because you have to see the signal across millions of workloads to know what normal is. GuardDuty AI Protection is now available to all GuardDuty customers with a 30-day free trial.

GuardDuty AI-powered investigations (preview)

AI-powered investigations take on the manual investigation work that drives alert fatigue and slows response. The capability automatically analyzes GuardDuty findings and the accounts around them to separate true threats from benign activity.

It examines finding context, related activity from the last 90 days, affected resources, and threat indicators, using knowledge graphs and threat intelligence to complete in minutes what used to take hours. Each investigation returns a disposition assessment with confidence scoring, MITRE ATT&CK® classification, supporting evidence, and clear recommendations to suppress, contain, or remediate. Your team focuses on genuine threats, whether across a single account or an entire AWS Organization, and mean time to resolution drops. GuardDuty AI-powered investigations is available in preview in 10 AWS Regions.

Security Hub AI inventory (generally available)

You can’t secure what you don’t know exists. Security Hub now provides an AI inventory, a continuously updated, organization-wide view of your AI assets and their security posture. As teams deploy models, agents, and pipelines, security often can’t see what’s running, and without connecting those assets to active threats and misconfigurations, it’s difficult to know what to secure first.

Security Hub AI inventory discovers and catalogs AI workloads across your AWS environment two ways. For managed services, it inventories AWS Config resources across Bedrock, SageMaker, and AgentCore. For self-hosted and external workloads, it finds models running on EC2, ECS, and EKS through runtime analysis, and identifies the external model endpoints your workloads make calls to. It maps each asset to the infrastructure beneath it, including compute, networking, IAM roles, and data stores, and correlates it with security signals such as GuardDuty findings. So when GuardDuty AI Protection flags an anomalous invocation, AI inventory immediately shows you which infrastructure is involved, what’s connected to it, and where it belongs in your priority order.

AI assets multiply fast. A developer spins up a Bedrock agent for a proof of concept. A data science team stands up a SageMaker endpoint for internal testing. Another team wires in an external model API through a Lambda function. Multiply that across hundreds or thousands of accounts and you can quickly lose track. AI inventory gives you that view across every account in your organization, available in your Security Hub Essentials plan at no additional cost.

A different approach to full-stack security

These launches share something worth pausing on. You didn’t procure AI protection as a separate product, and you won’t stand up separate operations for Azure. You add them to the Security Hub you already run, and they show up in your prioritized view of risk. That same idea is what Security Hub Extended extends to the rest of the security estate.

Security Hub Extended now has 21 curated partners across nine categories: 7AIBritiveCrowdStrike, Idira (CyberArk), CyeraIsland, LayerX, Native Security, NomaOktaOligoOptiProofpointSailPoint, SentinelOneSplunkSublime, Upwind, Varonis, Zenity, and Zscaler. These are best-in-class solutions across endpoint, identity, email, network, data, browser, cloud, AI, and security operations. None of them are here by default. Each one earned its place by committing to a shared view of where enterprise security is going, and by investing alongside us to build it. Curation is the point. A recommendation only means something if it can be turned down.

The commercial benefits of Extended are real today. Pay-as-you-go pricing, a single AWS bill, EDP eligibility, and no long-term commitments. But the work we’re most excited about goes further, and it’s not about procurement at all. Findings from every participating solution are emitted in the Open Cybersecurity Schema Framework (OCSF) and aggregated in Security Hub, and we’re building toward a single correlation across all of them, so a signal from an endpoint solution, an identity solution, and a cloud solution combine into one exposure and one attack path instead of three disconnected alerts. We’re working to reduce the deployment and onboarding effort between subscribing and seeing value. And we’re building the exchange that lets partner findings enrich each other, so the best-in-class tools you already trust become more than the sum of their parts. That is the differentiated future we’re investing in, and we’re building it in the open, guided by what customers ask for next. To learn more about Extended, see the What’s New post.

Accelerating forward

Step back and the shape of it is clear. Security Hub reaches across cloud providers, starting with Azure and expanding from there. It reaches across workload types with purpose-built AI protection and inventory. And it reaches across security categories through Extended and its curated partners. What began as a way to bring order to AWS security findings has become how more enterprises run full-stack security.

Detection and visibility are the foundation. What we build on top of them is a security experience that connects signals across every source you trust and helps you respond faster. It’s still Day 1, and Security Hub will keep extending as your environment, and the threats you face, continue to change.

If you have feedback about this post, submit comments in the Comments section below.


Michael Fuller

Michael has been with AWS for 16 years and led product for AWS Security Services for 11 years. Michael has 29 years in the industry and held several roles in product management, business development, and software development for IBM, Cisco, and Amazon. Michael has a Bachelor’s of Science in Computer Engineering from the University of Arizona and an MBA from the University of Washington.

Deploy modern data platforms in minutes with MDAA

Post Syndicated from Sudeshna Dash original https://aws.amazon.com/blogs/big-data/deploy-modern-data-platforms-in-minutes-with-mdaa/

Modern Data Architecture Accelerator (MDAA) is an open source framework that replaces infrastructure code with concise YAML configuration, so your team can deploy a governed, production-ready data architecture, reducing deployment time from months to weeks (depending on complexity and team experience).

Organizations building modern data architecture on AWS face a critical challenge: deploying production-ready, governed infrastructure traditionally requires 6–12 months of custom development, thousands of lines of infrastructure code, and continuous remediation cycles to maintain security and compliance. Governance is often added incrementally, treated as an afterthought that creates compliance gaps and engineering rework.

MDAA addresses this by replacing infrastructure code with concise YAML configuration, achieving up to 97.6 percent code reduction (from approximately 1,800 lines of AWS CloudFormation to 45 lines of MDAA YAML) while embedding governance from the start. The complete Governed Lakehouse Starter Kit deploys 491 AWS resources across 12 stacks from approximately 450 lines of YAML configuration, representing a 66x verbosity ratio where each line automatically expands into production-ready infrastructure.

In this post, we explore how MDAA transforms data architecture development from months of manual coding to production-ready deployment through configuration-driven infrastructure and embedded governance, examine a real customer transformation, and provide a clear implementation pathway for your own data modernization journey.

Customer use case and challenge

A university system office needed to modernize its analytics architecture across 17 campuses while managing sensitive educational data. Their third-party dependency created bottlenecks that slowed feature implementation from weeks to months, and their IT team lacked the cloud skillsets to build modern infrastructure independently.

With MDAA, they achieved:

  • 95 percent reduction in time-to-value for dashboard and feature implementation (from weeks to hours).
  • 17 campuses integrated into a unified, secure architecture.
  • 7.2TB of data and over 8,000 dashboards migrated successfully.
  • Significant cost savings by removing third-party dependencies and reducing license costs.
  • Enhanced security posture for external stakeholders accessing sensitive educational data.

The team used MDAA to implement a modernization strategy with continuous integration and continuous delivery (CI/CD) for automated deployment. The architecture now supports rapid response to stakeholder requests while maintaining strict data governance through AWS Lake Formation.

Their transformation demonstrates what becomes possible when governance is embedded from launch rather than added incrementally, moving from months-long manual development to weeks of production-ready deployment through configuration-driven infrastructure.

Solution: MDAA and its value propositions

MDAA’s capabilities stem from its modular, composable architecture. The accelerator provides over 40 pre-built modules that encapsulate AWS best practices for security, governance, and operational excellence. Organizations describe the outcomes they want in MDAA-specific YAML configuration files (not CloudFormation or Terraform YAML) and the accelerator automatically translates these configurations into AWS Cloud Development Kit (AWS CDK) constructs, which then deploy via CloudFormation with embedded governance.

Configuration over code. The MDAA framework takes a fundamentally different approach: describe the outcomes you want in YAML, and the accelerator deploys production-ready infrastructure with embedded governance. Consider deploying a governed data lake where fraud detection teams need write access to transaction data, while marketing analytics teams require read-only access to customer behavior data. Traditional approaches require over 1,800 lines of CloudFormation across Amazon Simple Storage Service (Amazon S3) buckets, AWS Key Management Service (AWS KMS) keys, AWS Identity and Access Management (IAM) policies, and Lake Formation permissions. With MDAA, the same governed data lake is expressed in 45 lines of configuration, a 97.6 percent reduction, while helping you apply encryption, least-privilege access, and cross-account governance as built-in defaults.

The configuration deploys multi-zone S3 storage with KMS encryption, Lake Formation permissions with tag-based access control (TBAC) enabled, Amazon SageMaker Unified Studio for data product discovery, and encrypted AWS Glue Data Catalog with automated crawlers. All permissions flow through Lake Formation rather than individual IAM policies.

Embedded governance from day one. Governance is declared in YAML and deployed alongside infrastructure from the first run. Fine-grained access controls, encrypted data catalogs, data quality validation, audit trails, and sensitive data classification are all part of the same configuration. MDAA’s Governed Lakehouse starter kit defines an entire governed data architecture in roughly 450 lines of YAML, which produces approximately 29,700 lines of CloudFormation across 12 stacks (a 98.5 percent reduction in infrastructure code).

Modular, composable architecture. Each module is purpose-built to handle a specific capability within the data architecture. Modules communicate through AWS Systems Manager Parameter Store, passing resource identifiers (Amazon Resource Names (ARNs), IDs, and names) between stacks. This approach removes hardcoded dependencies. A KMS key created in one module can be referenced by another through parameter resolution, with all dependencies resolved automatically at deployment time.

The diagram illustrates the deployed architecture and team-level access flow that MDAA generates from the 45-line configuration.

Progressive architecture patterns. MDAA provides four reference architecture patterns that align to progressive stages of data infrastructure maturity:

  • Basic Data Lake deploys a governed data lake with built-in security controls, data quality checks, centralized metadata management using AWS Lake Formation and AWS Glue.
  • Data Science Platform extends the data lake with Amazon SageMaker notebooks, feature stores, and machine learning (ML) pipelines so data science teams can experiment and train models on governed data.
  • SageMaker Unified Studio adds a single interface for analytics and ML collaboration, connecting data engineers, analysts, and data scientists in one workspace.
  • Generative AI Platform layers Amazon Bedrock and Retrieval Augmented Generation (RAG) capabilities on top of your existing data foundation, so teams can build generative AI applications grounded in enterprise data.

Each pattern builds the one before it. You can start with the Basic Data Lake and adopt additional patterns as your team’s needs grow. MDAA’s modular design means you add capabilities without rearchitecting what you already deployed.

The infrastructure is versioned through GitHub, repeatable across environments, and auditable through comprehensive AWS CloudTrail logging. Data engineers focus on data pipelines and business logic while MDAA manages infrastructure complexity and governance integration. This represents the fundamental shift: from writing infrastructure code to describing the outcomes you want through configuration, with governance embedded from the start.

Use case of MDAA: Governed data architecture

DataOps teams spend significant time on governance tasks, including permissions management, compliance validation, and access control, rather than building pipelines and analytics. These aren’t data problems, they’re governance problems that consume engineering capacity meant for higher-value work. MDAA addresses this at the architectural level. Governance is declared in YAML and deployed alongside infrastructure from the first run.

The following sections walk through how each governance module works in practice.

Publish, discover, subscribe, and consume data products between business units: SageMaker Unified Studio

Amazon SageMaker Unified Studio provides a governed data catalog where data producers publish data products, and consumers discover and subscribe to them. Your deployment with MDAA includes a pre-configured domain, blueprints (managed and custom), projects, and environment profiles, all defined in a single configuration file:

# sagemaker.yaml --- 16 lines that deploy 114 CloudFormation resources
domains:
  domain1:
    dataAdminRole:
      id: ssm:/{{org}}/govern1/generated-role/data-admin/id
    description: SMUS Domain 1
    userAssignment: MANUAL

    tooling:
      vpcId: '{{context:vpc_id}}'
      subnetIds:
        - '{{context:private_subnet_id1}}'
        - '{{context:private_subnet_id2}}'

    groups:
      team1:
        ssoId: '{{context:team1-group-sso-id}}'
      team2:
        ssoId: '{{context:team2-group-sso-id}}'

Behind this configuration, MDAA deploys an Amazon SageMaker Unified Studio domain with dedicated KMS keys, execution and provisioning roles, and single sign-on group profiles for team access. Data producers tag and publish assets with metadata, ownership, and classification. Consumers browse a searchable catalog, see only authorized assets, and request access through a governed workflow. Cross-account and cross-business-unit data sharing flows through a subscription model, ensuring every access grant is tracked, auditable, and revocable.

Use case of MDAA: Restricting access to cardholder data using Lake Formation

AWS Lake Formation provides fine-grained access control at database and table levels, removing manual IAM policy management. MDAA deploys AWS Lake Formation with pre-configured settings that disable IAMAllowedPrincipals, the critical governance setting that ensures all permissions flow through centralized governance:

# lakeformation-settings.yaml --- 6 lines that deploy 25 CloudFormation resources
lakeFormationAdminRoles:
  - id: generated-role-id:data-admin
createCdkLFAdmin: true
createDataZoneAdminRole: true
iamAllowedPrincipalsDefault: false

That last flag is the single most important governance setting in the platform. Without it, an IAM principal with glue:GetTable can read tables in the catalog, bypassing the entire access control model. Most manual setups miss this or defer it.

With the data lake configuration, you declare roles and access policies in YAML where admins get full control, engineers get read access to curated data, extract, transform, and load (ETL) roles get scoped write access, and MDAA compiles them into the correct S3 bucket policies and Lake Formation registrations.

Use case of MDAA: Ensuring data integrity with AWS Glue Data Quality

AWS Glue Data Quality runs automated validation rulesets continuously as part of the pipeline, not as periodic batch checks. MDAA’s data quality module supports over 15 built-in rule types, from completeness and uniqueness checks to statistical thresholds and data freshness validation:

# data-quality.yaml
projectName: example-project

rulesets:
  customer-data-quality:
    description: Validate customer data completeness and uniqueness
    targetTable:
      databaseName: project:databaseName/customer-data
      tableName: customers
    ruleset:
      - ruleType: IsComplete
        column: customer_id
      - ruleType: Uniqueness
        column: email
        comparisonOperator: ">"
        threshold: 0.95
      - ruleType: RowCount
        comparisonOperator: ">"
        value: 100

Quality metrics flow into Amazon CloudWatch for real-time alerting. If anomalies are detected, automated workflows quarantine affected records and alert data engineering teams before issues reach downstream consumers.

Protecting metadata at rest: AWS Glue Data Catalog encryption

Table schemas, column names, and partition structures can reveal sensitive information about an organization’s data architecture, even without access to the underlying data. AWS Glue Catalog Encryption secures metadata at rest using AWS KMS-managed keys. MDAA configures catalog encryption by default, so schema definitions and connection passwords are encrypted from initial deployment without requiring manual key management setup. Access to catalog metadata follows the same Lake Formation governance controls applied to the data itself, so teams see only the schemas that they’re authorized to query.

Auditing every data access event: CloudTrail integration

Every data access event must be logged and attributable to a specific identity. Without a complete audit trail, demonstrating compliance during a regulatory review becomes a manual, error-prone process. AWS CloudTrail captures API-level activity across the data infrastructure, recording who accesses what data, when, and from which service. MDAA configures CloudTrail integration by default, so audit logging is active from initial deployment rather than added retroactively. Log data flows into a centralized, tamper-resistant store, giving compliance teams a single location to query access history across all business units and accounts.

Identifying sensitive data automatically: Macie integration

In large environments, sensitive information spreads across dozens of S3 buckets through pipelines, transforms, and ad hoc data drops, and self-reporting data owners consistently produce gaps. Amazon Macie uses machine learning to automatically discover and classify sensitive data in S3, surfacing findings at the object level without manual tagging. MDAA configures Macie across your S3 buckets during deployment, routing findings to Amazon EventBridge where automated workflows can alert owners or trigger remediation.

Together, these controls form a layered defense: Lake Formation governs access to cataloged data, Glue Data Quality validates integrity on arrival, and Macie identifies sensitive data that lands outside governed pipelines to reduce compliance risk.

Multi-account data mesh

MDAA provides extensive support for multi-account data mesh setups, with decentralized data ownership across business units and centralized governance. The data mesh starter kit supports cross-account data product publishing and consumption, allowing organizations to scale data sharing while maintaining consistent security and compliance controls.

Technical implementation

Ready to deploy your modern data architecture? Here are the resources to get started:

MDAA Implementation Guide provides detailed instructions for deploying all starter packages, including architecture patterns, configuration examples, security best practices, and troubleshooting guidance.

MDAA Hands-on Workshop offers step-by-step guided implementation with AWS experts. The workshop covers configuration management best practices, implementation patterns, hands-on labs with real-world scenarios, and cleanup instructions.

GitHub Repository and Documentation provide source code, module reference, and comprehensive documentation.

Organizations approach MDAA from different starting points. Some modernize existing data architectures, migrating from on-premises infrastructure or legacy cloud architectures. Others build new architectures for artificial intelligence and machine learning (AI/ML) initiatives or generative AI applications. Financial services organizations require PCI-DSS compliance from day one. Healthcare organizations need controls that can help support HIPAA. Each journey benefits from MDAA’s configuration-driven approach and embedded governance.

Conclusion

MDAA transforms data architecture development from months of manual coding to production-ready deployment. Configuration-driven infrastructure reduces development time by 40–60 percent while embedding governance from the start. The university system’s 95 percent reduction in time-to-value demonstrates the outcome: organizations deploy secure, compliant, governed data architectures in weeks rather than months.

Financial services organizations can deploy architectures to help them align with PCI-DSS compliance requirements using Lake Formation access controls, Glue Data Quality validation, SageMaker Unified Studio data discovery, comprehensive CloudTrail audit trails, and automated Macie data classification, all inherited from configuration rather than built manually.

Data architecture journeys need not follow six-month timelines with governance added incrementally. MDAA provides an alternative: describe the outcomes you want through YAML configuration, inherit pre-validated security controls, and deploy production-ready infrastructure with comprehensive governance from initial deployment.

Security and compliance is a shared responsibility between AWS and the customer. For more information, see the AWS Shared Responsibility Model.

Need help or have questions? Contact AWS ProServe for personalized guidance on selecting the right package and deployment strategy for your organization.


About the author

Sudeshna Dash

Sudeshna Dash

Sudeshna is a Data Scientist at AWS Professional Services based in Berlin, Germany. She specializes in data architecture, generative AI, and agentic AI systems on AWS. Sudeshna is a contributor to the Modern Data Architecture Accelerator (MDAA) open-source project and helps customers design and deploy governed, production-ready data and AI/ML architectures on AWS.

John Reynolds

John Reynolds is a Principal Engineer with AWS Professional Services based in Seattle, Washington. He leads the architecture and development of Modern Data Architecture Accelerator (MDAA), focusing on turning proven delivery patterns into reusable, production-ready foundations that customers can adopt and extend at scale.

Reducing SMS OTP fraud with Vonage network-powered solutions and Amazon Cognito

Post Syndicated from Tito Milla original https://aws.amazon.com/blogs/architecture/reducing-sms-otp-fraud-with-vonage-network-powered-solutions-and-amazon-cognito/

User authentication remains one of the most targeted touchpoints in application security. With the industrialization of fraud threats by generative AI, cybercrime costs are expected to reach $23 trillion in 2027, an increase of 175 percent from 2022. 20 percent of fraud is attributed to synthetic identity and authentication exploits, with account takeover (ATO) surging 141 percent since 2021.

But the damage goes beyond security. SMS One-time passcodes (OTPs) achieve only approximately 80 percent conversion on authentication flows, meaning 1 in 5 legitimate users is lost at the point of verification. Enterprises absorb hundreds of thousands of password recovery helpdesk tickets annually, representing significant support costs tied to OTP-based verification. Every abandoned authentication attempt today represents an opportunity to maximize your conversion rates across checkout, account recovery, and onboarding flows. The industry has long assumed that stronger security requires more user friction. That isn’t a law of physics. It’s a limitation of the tools available. Mobile operator network data removes that constraint and provides stronger identity assurance and a smoother experience, not one at the expense of the other.

In this post, we show how Vonage network-powered solutions work with Amazon Cognito to enhance many mobile-first use cases with network-level identity verification. Vonage network-powered solutions are a composable stack of real-time mobile operator intelligence, silent authentication, and integrated fraud protection, which uses the CUSTOM_AUTH flow to complete identity verification in under 5 seconds, with zero user interaction.

About Vonage

Vonage, part of Ericsson, is an AWS Partner with multiple AWS Marketplace listings. The company provides enterprise and CIAM deployments with cloud-based access to mobile operator network APIs, including real-time mobile identity and authentication across key regions. These complement Vonage’s global communications, voice, and video APIs backed by Ericsson’s global telecommunications infrastructure.

What network-powered means and why it matters

Before diving into architecture, it’s worth being precise about what separates Vonage’s network-powered solutions from the identity and fraud tools enterprises already have in their stack.

Most identity verification signals today are derived from aggregated, cached, or behavioral data. Traditional phone number lookup services query static databases that may be days or weeks out of date. Device fingerprinting analyzes browser characteristics that might be spoofed. Behavioral biometrics builds models from historical sessions. This is useful, but a lagging indicator by definition.

Enterprise customers who implement Vonage’s network-powered solutions operate from a fundamentally different layer: real-time data sourced directly from mobile network operators (MNOs). When you query whether a SIM was recently swapped, you’re querying the network that performed the swap. When Silent Authentication verifies a user, the proof of possession is the cellular data session itself. This session can’t be phished, intercepted, or socially engineered.

In fraud scenarios where SIM swaps are weaponized for account takeover (ATO), “recently” means minutes or hours, not days. Static databases refreshed weekly are not detecting these events. They’re logging them after the fact. Real-time operator queries close that window entirely.

The three pillars: Identity Insights, Verify, and Fraud Defender

Vonage network-powered solutions combine three API service components into a composable security stack that integrates with Amazon Cognito through the CUSTOM_AUTH flow:

1. Identity Insights: Pre-verification intelligence

Identity Insights runs before verification channels are initiated, surfacing real-time operator signals that are directly actionable in authentication policy decisions. The following list shows a representative set of JSON elements that might be returned by a request. Customers have the option to select which data is most valuable given a specific authentication use case and industry combination.

  • format and network_type: Filters invalid numbers, VoIP, landline, and premium-rate numbers used in synthetic account creation and bot-driven fraud.
  • sim_swap: Detects SIM swaps within a configurable look-back window, a leading indicator of ATO events in progress.
  • subscriber_match: Compares subscriber identity (name, address) against operator Know Your Customer (KYC) records.
  • device_swap: A recent change in the mobile device associated with a phone number signals that a bad actor might have taken control of the SIM card. (coming soon)
  • recycled_number: Numbers previously deactivated and reassigned to a new subscriber can trigger false identity matches in onboarding flows, creating risk in account creation. (coming soon)

These pre-checks trigger your defined risk policy: step-up challenge, hard block, or silent logging. Critically, fraudulent attempts are identified and blocked before a single OTP is sent, before verification costs are incurred, and before fraud processing overhead is generated.

2. Verify with Silent Authentication: Alleviating the friction tax

Every additional step a user must finish during authentication carries a measurable cost: abandoned sign-ups, failed conversions, and support tickets from users who don’t receive or mistyped a code. We call this cumulative loss the friction tax. For SMS OTP flows with approximately 80 percent completion rates, the friction tax means roughly 20 percent of legitimate users drop off before they ever reach your application.

After a number passes the risk pre-checks, the Verify API delivers the authentication challenge. The primary authentication method is Silent Authentication.

When a user initiates sign-in from a mobile device, Vonage routes an HTTP request through the user’s cellular data connection. The mobile operator confirms that the SIM registered to the phone number matches the session making the request. The exchange happens in the background, in seconds. The user doesn’t see, type, copy, or enter any code.

If Silent Authentication can’t finish or is unavailable, Verify automatically falls back to traditional SMS, RCS, Voice, WhatsApp, or email, remaining transparent to the user.

Key benefit: Silent Authentication alleviates the three primary exploit vectors against SMS OTP: SIM swap (bad actor receives the code), SS7 interception (message diverted in transit), and social engineering (user tricked into sharing the code). All without additional input from the end user.

3. Fraud Defender: Protecting the verification channel

Fraud Defender addresses a threat familiar to enterprise finance teams: artificially inflated traffic (AIT) and SMS pumping. Automated systems trigger high volumes of OTPs sent to premium-rate numbers that bad actors control. At enterprise verification volumes, these events can run undetected for extended periods.

Fraud Defender provides real-time traffic monitoring and intelligent blocking at the point of outbound delivery, intercepting these malicious events before costs accumulate. The financial impact is immediate and measurable. Fraud Defender typically absorbs its own cost in toll fraud prevention within the first billing cycle. For most enterprises, it quickly becomes a net revenue-positive investment. Vonage customers have collectively saved over $3M in SMS-related fraud costs since deployment. The savings continue to compound as the blocking algorithm evolves to counter new exploit patterns. For Verify customers, the value is even more compelling: Fraud Defender activates automatically with the Vonage Verify API at no additional cost. This makes it one of the highest-ROI fraud protections available.

Prerequisites

To implement this solution, you need:

  • An AWS account with permissions to create and manage Amazon Cognito, AWS Lambda, AWS Secrets Manager, Amazon CloudWatch, and AWS WAF resources.
  • An Amazon Cognito user pool (existing or new).
  • A Vonage API account with access to Identity Insights and Verify APIs.
  • AWS Command Line Interface (AWS CLI) or AWS Serverless Application Model (AWS SAM) CLI installed and configured.
  • For client integration: the Vonage Silent Authentication SDK for your mobile platform (iOS/Android).

Solution architecture with Amazon Cognito

Enterprise customers that integrate the Vonage solution use the Amazon Cognito CUSTOM_AUTH flow, which uses three AWS Lambda functions that orchestrate the solution stack without changing your existing user pool configuration or downstream service integrations.

Architecture diagram showing the Risk-Adaptive Customer Sign-In flow with layers including user devices, edge protection with Amazon CloudFront and AWS WAF, Amazon API Gateway, identity layer with Amazon Cognito, verification layer with Vonage Identity Insights, Verify API, and Fraud Defender, and the carrier network with Mobile Network Operators.

Architecture components

The solution connects five layers, each handling a distinct step in the authentication flow:

  • Client app (mobile/web) – Initiates the CUSTOM_AUTH flow with the Vonage Silent Authentication SDK, follows check_url redirects over the cellular network, and submits the verification code back to Amazon Cognito.
  • Amazon Cognito user pool – Orchestrates the CUSTOM_AUTH challenge flow and issues JWT tokens upon successful verification.
  • AWS Lambda triggers – Define Auth Challenge (orchestrator), Create Auth Challenge (calls Vonage APIs), and Verify Auth Challenge (validates response).
  • Vonage Network APIs – Identity Insights pre-check, Verify with Silent Auth and OTP (built-in failover), and Fraud Defender (automatic).
  • Mobile network operators – SIM-level identity verification through CAMARA/Open Gateway APIs.

Authentication flow

The following steps represent an authentication workflow sequence between Amazon Cognito and Vonage network-powered solutions:

  1. The client calls InitiateAuth with CUSTOM_AUTH, passing the user’s phone number.
  2. The Define Auth Challenge Lambda function instructs Amazon Cognito to issue a CUSTOM_CHALLENGE.
  3. The Create Auth Challenge Lambda function calls Identity Insights for pre-verification risk assessment. If the number passes pre-checks, Lambda calls Vonage Verify to initiate Silent Authentication and returns the check_url to the client.
  4. Upon receiving the check_url, the client opens an HTTPS connection to it, triggering HTTP redirects to the mobile carrier’s network for direct mobile-device-to-mobile-network-operator verification. Upon completion, the client receives a verification code from the operator.
  5. The client calls RespondToAuthChallenge with the code.
  6. The Verify Auth Challenge Lambda function submits the code to Vonage’s check endpoint. On success, it returns answerCorrect = true and Amazon Cognito issues the appropriate session token.

Sequence diagram showing the User Login flow with SIM-swap pre-check using Vonage Identity Insights and Silent Authentication via Vonage Verify, orchestrated through the Amazon Cognito CUSTOM_AUTH flow with Lambda triggers.

Coexistence and phased rollout

A critical design principle: zero disruption to existing infrastructure. The Vonage Network API plugs into the Amazon Cognito CUSTOM_AUTH flow without changes to your existing user pool, app client configurations, or downstream service integrations. Deployment requires a single sam deploy command.

This design approach allows for a phased rollout. Start with the highest-risk journeys (password recovery, high-value transactions) where security ROI is clearest, then expand to daily login and onboarding as you measure impact. Traditional SMS, RCS, and Voice OTP remain options for lower-risk flows during the transition.

Risk-aware workflows by journey type

The strategic value of combining Vonage’s network-powered solutions with the Amazon Cognito policy-driven CUSTOM_AUTH flow is context-aware authentication calibrated to actual risk. CRITICAL journeys are recommended for the first phase of implementation as they aim to meaningfully mitigate synthetic identity and account takeover. The following table describes risk-aware workflows by journey type.

Journey Risk Vonage Workflow
New account signup CRITICAL Identity Insights filters invalid/non-mobile numbers + Subscriber Match validates KYC → Silent Auth for zero-tap onboarding
Daily login MEDIUM SIM swap recency + device consistency check → Silent Auth passively, step-up only on elevated signals
Password recovery, profile change (contacts), 2FA settings change HIGH Mandatory SIM swap hard-check (tight lookback window) + Subscriber Match → Silent Auth required, no passive bypass
High-value transaction CRITICAL Full signal stack (line type, SIM swap, subscriber match) → Silent Auth + secondary challenge if risk elevated

Low-risk actions (for example, viewing account details, browsing content, or checking order history) generate no friction and no unnecessary verification cost. High-risk actions trigger the full assurance stack. The calibration is policy-driven and configurable per journey.

Implementation considerations

Configuring Amazon Cognito starts with setting up the user pool to allow the CUSTOM_AUTH authentication flow and accept phone numbers as the primary sign-in attribute. After the user pool is in place, associate the three required Lambda functions with their corresponding Amazon Cognito trigger hooks and store your Vonage API credentials in AWS Secrets Manager.

Layer in security from the start, following the AWS Well-Architected Security Pillar. Scope each Lambda function’s AWS Identity and Access Management (IAM) role to only what it needs: Amazon Cognito trigger invocations and AWS Secrets Manager access. Enforce TLS 1.2+ on all communication for encryption in transit. For observability, turn on Amazon CloudWatch logging on each Lambda function and turn on AWS CloudTrail to capture Amazon Cognito API audit trails. Finally, deploy AWS WAF with rate-limiting rules in front of the authentication endpoint to protect against brute-force attempts.

To configure the solution, follow these steps:

  1. Set up the Amazon Cognito user pool to allow the CUSTOM_AUTH authentication flow.
  2. Configure the user pool to accept phone numbers as the primary sign-in attribute.
  3. Associate the three required Lambda functions with their corresponding Amazon Cognito trigger hooks.
  4. Store your Vonage API credentials in AWS Secrets Manager.

Important: This solution creates AWS resources that incur charges. These include Amazon Cognito (per monthly active user), AWS Lambda (per invocation), AWS Secrets Manager (per secret per month), Amazon CloudWatch Logs, AWS CloudTrail, and AWS WAF (per rule and request). See the pricing page for each service and delete resources when no longer needed.

Privacy and compliance

The architecture is designed so that PII doesn’t leave the mobile operator. Subscriber Match performs a comparison within the operator’s environment and returns only a match score. The underlying subscriber data isn’t transmitted. Silent Authentication operates without PII exchange. The cellular session is the credential.

  • GDPR: Only match scores are returned. No subscriber PII is stored or transmitted, supporting GDPR data minimization.
  • PSD2 / Open Banking: Silent Authentication qualifies as a possession-factor for Strong Customer Authentication (SCA).
  • HIPAA: Subscriber Match supports identity assurance for healthcare applications.
  • DORA: Multi-channel fallback achieves > 99.9 percent verification availability.
  • CCPA: Same data-minimization architecture as GDPR.

Production results: Lydia Solutions

Lydia Solutions, one of Europe’s fastest-growing mobile financial services applications, deployed Vonage Verify with Silent Authentication in October 2024. The results demonstrate the real-world impact at scale, including up to 50 percent reduction in latency when compared to Lydia Solutions’s previous authentication services.

“Vonage Verify with Silent Authentication has been a real innovation for us. The solution has elevated our ability to deliver a simpler, seamless and more secure user experience while protecting against increasingly sophisticated threats and fraud patterns.”

— William Brulin, Senior VP, Lydia Solutions

Lydia’s results sit at the high end of outcomes observed. Across deployments in ecommerce, digital banking, and consumer services, conversion improvements of 2–8.5 percent compared to SMS-only are the norm, with authentication journey latency reductions of 50–75 percent.

Conclusion

This is where mobile operator data shifts the approach. Rather than applying identical verification friction to every session, enterprises can use real-time network signals to make adaptive authentication decisions. Verify silently when conditions are right, step up when risk indicators appear, and block when fraud is detected.

Enterprise implementation of the offering makes those risk signals and authentication methods accessible through a composable API layer. The combination of Identity Insights for pre-verification intelligence, Verify for network-layer authentication, and Fraud Defender for channel protection delivers risk-proportionate authentication that’s in production at scale today.

The solution deploys with minimal changes to your existing Amazon Cognito user pool. Start with high-risk journeys, measure impact, and expand. Vonage Verify API is available across over 700 MNOs in over 200 countries and territories, and the integration requires only three Lambda functions.

Next steps

Vonage is an AWS Partner. To learn more, visit the Vonage partner page.

The content and opinions in this post are those of the third-party author and AWS is not responsible for the content or accuracy of this post.


About the authors

Accelerate Incident Resolution with PagerDuty and AWS DevOps Agent

Post Syndicated from Shan Kandaswamy original https://aws.amazon.com/blogs/devops/accelerate-incident-resolution-with-pagerduty-and-aws-devops-agent/

When something breaks in production, you find out fast. Understanding why it broke, before the damage spreads, is the hard part. That is where Site Reliability Engineering (SRE) teams lose the most time.

Think about the last time you got paged at 2 a.m. The alert said something broke, not why. You open four or five dashboards, cross-reference deployment logs with AWS CloudTrail events, and scroll through metrics. Twenty or thirty minutes burn before the picture comes together. That manual correlation is where resolution time balloons.

What if the investigation started before you opened your first dashboard?

That’s the idea behind connecting the new native PagerDuty Capability Provider in AWS DevOps Agent. The two systems now talk directly over a built-in OAuth 2.0 connection. When a PagerDuty incident triggers, the DevOps Agent starts investigating while responders are still getting oriented. Connecting them takes a few fields in a console.

What AWS DevOps Agent does

AWS DevOps Agent is a frontier agent built to help engineering teams investigate and resolve production incidents faster. The DevOps Agent works as a first responder, conducting federated investigations across your observability stack, tracing incidents from code changes all the way through to cloud infrastructure impact, and producing detailed mitigation plans. Beyond reactive investigations, it also proactively recommends improvements to your observability, infrastructure, and deployment pipelines to help prevent recurring issues. Through the AWS DevOps Agent web app, you can observe investigations as they unfold, access findings, and steer the analysis in real time.

The central concept is the Agent Space. Think of it as the boundary that defines what your agent can access. Your AWS account serves as the primary source, and from there you layer on secondary capabilities from telemetry providers like Datadog, Dynatrace, New Relic, or Splunk; pipeline tools like GitHub and GitLab; communications from PagerDuty and Slack; and custom Model Context Protocol (MCP) servers for anything else. Every investigation the agent runs, it learns. It maps relationships between your resources such as load balancers to services, services to databases, and deployments to config changes. One team we’ve worked with had the agent map hundreds of infrastructure relationships, and that number keeps growing with each investigation it completes.

PagerDuty, of course, needs no introduction to anyone who’s been responsible for resolving critical, customer-impacting incidents. Engineering teams rely on it to detect, triage, resolve, and learn from incidents. The native PagerDuty Capability Provider in AWS DevOps Agent connects the two directly. PagerDuty incident events drive AWS DevOps Agent investigations automatically. Findings flow back to the originating PagerDuty incident record, including root cause analysis and recommended mitigation steps. They are also available in the AWS DevOps Agent console and web app, giving your whole team visibility into what the agent discovered.

There’s a second piece to this integration worth understanding. By adding the PagerDuty MCP Server as a capability and configuring an AWS DevOps Agent skill for working with PagerDuty, you enable AWS DevOps Agent to query PagerDuty’s institutional memory during investigations. This includes past incidents, diagnostics, resolution patterns, and operational context across both AWS and non-AWS environments. This PagerDuty MCP Server-based connection is separate from the Capability Provider event flow and requires its own setup (covered in Step 6 below). The result is investigations informed by both current signals and prior incident history.

Why this matters

These are practical, tangible changes for your team:

Faster time to root cause. When a PagerDuty incident triggers, AWS DevOps Agent kicks off an investigation automatically. No one has to sign in to another tool, step through a wizard, or remember to initiate anything. The investigation is already running by the time you acknowledge your alert.

Real contextual analysis. The agent correlates PagerDuty incident data with Amazon CloudWatch metrics, AWS CloudTrail logs, application topology, and deployment history, plus telemetry from whichever third-party observability providers you’ve connected, like Datadog, Splunk, New Relic, or Dynatrace. It connects dots that would otherwise take humans significant time to even start connecting.

Investigations start when an incident triggers. AWS DevOps Agent automatically conducts the deep-dive investigation behind the scenes. It reports back its root cause analysis and proposed mitigation steps into the originating PagerDuty incident, with a link to the AWS DevOps Agent web app for more details.

Less time playing detective, more time fixing things. That manual data correlation across four or five tools? The agent handles it. Your people can focus on actually resolving the issue instead of building the investigation timeline by hand.

Nothing extra to host. The native PagerDuty Capability Provider means you’re not standing up additional infrastructure. No servers to manage, no endpoints to maintain on your side.

How the integration works

The architecture is straightforward. Here’s the flow:

High-level architecture diagram showing PagerDuty connected to AWS DevOps Agent through a native OAuth 2.0 Capability Provider, with the agent investigating across AWS CloudWatch, AWS CloudTrail, and connected telemetry and pipeline tools

Figure 1: High-level architecture, native PagerDuty Capability Provider in AWS DevOps Agent.

AWS DevOps Agent and PagerDuty authenticate to each other using OAuth 2.0 Scoped OAuth. You register PagerDuty once at the AWS account level as a Capability Provider, and then add it to whichever Agent Spaces need it. Registration is shared across Agent Spaces in the account, so you don’t have to repeat the setup per team.

Once a PagerDuty incident triggers, AWS DevOps Agent picks up the event over the native connection and begins investigating:

  1. Receives the PagerDuty incident event (service, severity, and initial context) via the native Capability Provider connection
  2. If the PagerDuty MCP capability and AWS DevOps Agent skill are configured, queries PagerDuty for related historical incidents, past diagnostics, and resolution patterns to enrich the investigation
  3. Examines AWS resource topology and the relationships between your infrastructure components through its knowledge graph
  4. Reviews AWS CloudTrail logs for recent changes or anything that looks off
  5. Queries Amazon CloudWatch and connected telemetry providers (Datadog, Dynatrace, New Relic, Splunk) for relevant metrics and traces
  6. Cross-references deployment events from configured pipeline tools (GitHub, GitLab) against the incident timeline
  7. Synthesizes potential root causes from all the evidence it’s gathered

The agent builds up a comprehensive picture by introspecting AWS observability data, pulling from connected capability providers, and leveraging the topology mapping that creates a knowledge graph of your application infrastructure. Every investigation it runs expands its understanding of how your resources connect. It discovers relationships you might not have explicitly documented, building a richer map with each incident it works.

Beyond raw data, the agent produces detailed mitigation plans with specific actions to resolve the issue, validate the fix, and revert if needed. The agent posts its findings, root cause summary, and recommended next steps directly to the originating PagerDuty incident record, giving your on-call team actionable information without them having to go digging.

A quick note on security, because it matters. The native connection uses OAuth 2.0 Scoped OAuth with a minimum set of PagerDuty scopes (incidents.read incidents.write services.read webhook_subscriptions.read webhook_subscriptions.write). AWS DevOps Agent only supports the newer scoped OAuth flow; legacy PagerDuty OAuth with a redirect URI is not supported. For inbound events from PagerDuty, only V3 webhooks are supported. Earlier webhook versions won’t work. Traffic flows over HTTPS.

Getting it set up

Setup comes in four phases: register PagerDuty as a Capability Provider at the account level, attach it to your Agent Space, configure the PagerDuty MCP server and AWS DevOps Agent skill for working with PagerDuty to enrich investigations, and verify things work end to end.

What you’ll need

  • An active AWS account with permissions to use AWS DevOps Agent
  • AWS DevOps Agent enabled in a supported AWS Region. You’ll create an Agent Space, which needs two AWS Identity and Access Management (IAM) roles (one for Agent Space operations, one for web app functionality). Both can be auto-created during setup
  • A PagerDuty account with permission to register OAuth apps, plus an Administrator role for Events Integration
  • A PagerDuty Advance license and a PagerDuty User API token (for the MCP integration in Step 6)
  • Your PagerDuty account subdomain (so if your PagerDuty URL is https://your-company.pagerduty.com, the subdomain is your-company)
  • An OAuth client ID and client secret from a PagerDuty app registered with OAuth 2.0 Scoped OAuth

Step 1: Create your Agent Space

Stand up an Agent Space in the AWS DevOps Agent console. This defines the boundary for what the agent can reach into and investigate.

  1. Head to the AWS DevOps Agent console home page.
AWS DevOps Agent console home page with a Begin setup call to action to create your first Agent Space

AWS DevOps Agent console home page.

  1. Create a new Agent Space with a name and a short description, usually scoped to a service or application team’s responsibilities
Create Agent Space form with fields for Agent Space name, optional description, and agent response language

Creating a new Agent Space with a name and description.

  1. Create the Agent Space IAM roles (AWS DevOps Agent requires two IAM roles: one for Agent Space operations and another for its associated web app functionality). You can auto-create them during setup
Agent Space setup screen showing the two IAM roles required, one for Agent Space operations and one for web app functionality

Configuring the two IAM roles required for the Agent Space.

Detailed view of IAM role auto-creation options during Agent Space setup

IAM roles can be auto-created during setup.

  1. Your primary source (the AWS account you’re creating the Agent Space in) is added automatically. If you need the agent to investigate resources in other accounts, add those as secondary sources
Agent Space sources screen showing the AWS account added automatically as the primary source

The AWS account is added automatically as the primary source.

Step 2: Add supporting capabilities

Out of the box, the agent connects to Amazon CloudWatch for metrics, logs, and alarms, and can investigate AWS CloudTrail API activity and AWS X-Ray traces through its read-only permissions. That said, most teams don’t live entirely inside AWS tooling, and that’s where third-party capability providers pull their weight. You can wire in external tools to give the agent a fuller picture of your world:

  • Telemetry: Datadog, Dynatrace, New Relic, or Splunk, so the agent can pull metrics and traces beyond Amazon CloudWatch during investigations
  • Pipelines: GitHub or GitLab, so it can correlate deployments and code changes with incidents
  • Communications: Slack, for team coordination and investigation updates (PagerDuty is configured separately as a Capability Provider in Step 4)
  • MCP Servers: Custom integrations via OAuth or API keys for anything else in your stack

You don’t need everything connected on day one. Start with what makes sense and add more as you go. Each new capability helps the agent discover more infrastructure relationships and investigate more effectively.

Step 3: Set up application topology

Help the agent understand what your application landscape looks like:

  1. Configure IAM roles to define the AWS topology scope for your Agent Space. The agent uses these permissions to determine which resources it can see and investigate
  2. Give the agent time to discover and map the relationships between your resources (it does this automatically as it runs investigations)
  3. Check the interactive topology visualization in the console and make sure your critical components are showing up correctly
  4. If you want the agent to focus on certain tags or resource subsets, add those instructions to your skills

Step 4: Register PagerDuty as a Capability Provider

You register PagerDuty once at the AWS account level. From there, it’s shared across every Agent Space in the account.

First, create the OAuth app in PagerDuty:

  1. In a separate browser tab, sign in to PagerDuty and go to Integrations > App Registration
PagerDuty Integrations menu showing the App Registration option

In PagerDuty, navigate to Integrations then App Registration.

PagerDuty App Registration page for creating a new app

PagerDuty App Registration page.

  1. Create a new app using OAuth 2.0 Scoped OAuth. AWS DevOps Agent does not support legacy PagerDuty OAuth with redirect URI
PagerDuty new app form with OAuth 2.0 Scoped OAuth selected as the authentication type

Create the app using OAuth 2.0 Scoped OAuth.

  1. Under Permissions, grant the minimum scopes: incidents.read incidents.write services.read webhook_subscriptions.read webhook_subscriptions.write
PagerDuty OAuth permissions screen showing the minimum required scopes for incidents, services, and webhook subscriptions

Granting the minimum required OAuth scopes.

  1. Turn on Events Integration so AWS DevOps Agent and PagerDuty can talk in both directions
PagerDuty app configuration with Events Integration enabled

Turn on Events Integration for two-way communication.

  1. Copy your Client ID and Client Secret. You’ll paste them into the AWS console in a minute
PagerDuty app credentials screen displaying the Client ID and Client Secret

Copy the Client ID and Client Secret from PagerDuty.

Then, register PagerDuty in the AWS DevOps Agent console:

  1. In the AWS DevOps Agent console, open the Capability Providers page from the side navigation
  2. In the Available providers section, find PagerDuty under Communication and choose Register
AWS DevOps Agent Capability Providers page with PagerDuty listed under Communication and a Register button

Find PagerDuty under Communication and choose Register.

  1. On the Configure access in PagerDuty page, pick your PagerDuty region (US or EU) and enter your PagerDuty subdomain (if your PagerDuty URL is https://your-company.pagerduty.com, the subdomain is your-company)
  2. Paste in the OAuth Client name, Client ID, and Client secret from PagerDuty. Confirm the minimum scopes (incidents.read incidents.write services.read webhook_subscriptions.read webhook_subscriptions.write)
Configure access in PagerDuty form with fields for region, subdomain, OAuth client name, client ID, and client secret

Enter your PagerDuty region, subdomain, and OAuth credentials.

  1. Review the configuration and choose Add
Review screen for the PagerDuty Capability Provider configuration before adding

Review the configuration and choose Add.

Once registration goes through, PagerDuty shows up under the Currently registered section of the Capability Providers page.

Capability Providers page showing PagerDuty under the Currently registered section

PagerDuty appears under Currently registered after registration.

Step 5: Add PagerDuty to your Agent Space

PagerDuty is registered at the account level. Now connect it to the Agent Space that needs it:

  1. In the AWS DevOps Agent console, pick your Agent Space
  2. Open the Capabilities tab
  3. In the Communications section, choose Add
Agent Space Capabilities tab with the Add button in the Communications section

On the Capabilities tab, choose Add in the Communications section.

  1. Select PagerDuty from the list of available providers
Provider selection list with PagerDuty available to add to the Agent Space

Select PagerDuty from the list of available providers.

  1. Choose Associate service

To update OAuth credentials or remove PagerDuty from an Agent Space, see the AWS DevOps Agent documentation.

Step 6: Add PagerDuty MCP Server and configure the agent skill

The Capability Provider from the previous steps handles the event flow. When a PagerDuty incident triggers, AWS DevOps Agent investigates and posts findings back to the originating PagerDuty incident. To let the agent also pull context from PagerDuty during those investigations, you add two things: the PagerDuty MCP server as a custom MCP capability, and an AWS DevOps Agent skill for working with PagerDuty that tells the agent when and how to use it.

Prerequisites:

  • PagerDuty Advance license
  • A PagerDuty User API token (generate one at User Settings > API Access in PagerDuty)

Add the PagerDuty MCP server:

  1. In your Agent Space, go to Capabilities tab > MCP Servers
Agent Space Capabilities tab showing the MCP Servers section

Open the MCP Servers section on the Capabilities tab.

  1. Add a new custom MCP server with the following configuration:
    • Server URL: https://mcp.pagerduty.com/mcp
    • For EU region PagerDuty accounts, use https://mcp.eu.pagerduty.com/mcp instead
    • Authentication: PagerDuty User API token in the format Token token=<your-pagerduty-api-key>
Add custom MCP server form in the AWS DevOps Agent console

Add a new custom MCP server.

MCP server configuration showing the server URL field and PagerDuty User API token authentication field

Configure the server URL and PagerDuty User API token.

Add the AWS DevOps Agent skill for working with PagerDuty:

The MCP server gives the agent access to PagerDuty tools. The skill tells the agent when and how to use them during investigations.

  1. In your Agent Space, choose Operator access to open the web app in a separate browser window
Agent Space console with the Operator access option to open the web app

Choose Operator access to open the web app.

  1. In the Agent Space Operator web app, navigate to Knowledge and the Skills tab, then choose Add skill
AWS DevOps Agent Operator web app Skills page with the Add skill button

On the Skills page, choose Add skill.

  1. You can select Create skill to create a skill through a wizard, interactively chat with the agent to create a skill, or upload a skill zip file if you already have one
Create skill options showing wizard, interactive chat, and zip upload methods

Choose how to create the skill.

  1. Choose Create skill and fill out the skill instructions from the table below to create a skill
Skill creation form with fields for name, description, status, agent type, and instructions

Fill out the skill instructions.

  1. You should see the pagerduty-aws-devops-agent skill added to the AWS DevOps Agent
Skills page showing the pagerduty-aws-devops-agent skill successfully added and active alongside the core skills

The pagerduty-aws-devops-agent skill added to AWS DevOps Agent.

Skill form instructions:

Field Value
Name pagerduty-aws-devops-agent
Description Use this skill to interact with the PagerDuty Advance SRE Agent for incident response, troubleshooting, runbook generation, and log search. Invoke when the agent is investigating incidents, performing triage, root cause analysis, or resolving operational issues. This skill calls the sre_agent_tool from the pagerduty-advance-mcp MCP server to access PagerDuty’s historical incident data, diagnostics, and resolution patterns.
Status Active
Agent Type Generic
Instructions See the skill instructions code block below.

Skill instructions (paste into the Instructions field):

# PagerDuty Advance SRE Agent

Use the PagerDuty MCP Server to call the `sre_agent_tool` for incident response and technical troubleshooting.

## Prerequisites

This skill requires the `pagerduty-advance-mcp` MCP server to be configured in the Agent Space under Capabilities > MCP Servers.

1. Extract the PagerDuty incident ID from the investigation context
2. Call the `sre_agent_tool` from the `pagerduty-advance-mcp` MCP server with:
   - `message`: a natural language question about the incident
   - `incident_id`: the PagerDuty incident ID
3. If follow-up queries are needed, continue calling `sre_agent_tool` with the same `incident_id` and a new `message`. Pass the `session_id` from the previous response to maintain conversation

## Tool Details

- **Tool name:** `sre_agent_tool`
- **MCP Server:** `pagerduty-advance-mcp`
- **Parameters:**
  - `message` (string, required) — natural language question about the incident
  - `incident_id` (string, required) — the PagerDuty incident ID
  - `session_id` (string, optional) — reuse from previous response for conversation continuity

## What the SRE Agent Can Help With

- Active incident analysis, triage, and resolution
- Root cause analysis and technical explanations
- Incident summaries and catch-ups
- Status updates for stakeholders
- Diagnostic checks and remediation recommendations
- Log interpretation and troubleshooting guidance
- Alert trigger analysis and explanations
- Change event analysis and impact assessment
- Playbook and runbook generation
- Past incident correlation and pattern recognition
- Service dependencies and related system analysis
- Real-time incident monitoring and alerting questions

Step 7: Test and validate

Before you call it finished, confirm things work end to end:

  1. Create a test incident in PagerDuty
  2. Confirm AWS DevOps Agent picks up the event and starts an investigation
  3. Watch the investigation move along in the AWS DevOps Agent console or web app
  4. Review the root cause summary, the mitigation plan, and the investigation findings
  5. Verify that root cause analysis and mitigation steps appear on the originating PagerDuty incident record. If you’ve also connected Slack, check that updates land in your configured channel

Start with a limited scope for your initial Agent Space. Focus on a single application or service first. Get comfortable with the integration, tune your configuration, and then expand from there.

Troubleshooting

A handful of things we’ve seen trip people up:

Registration fails with invalid credentials. Double-check that the Client ID and Client secret were copied from the right PagerDuty OAuth 2.0 Scoped OAuth app. Legacy PagerDuty OAuth apps (the ones configured with a redirect URI) aren’t supported. When credentials do need to change, deregister from the Capability Providers page and re-register with the new values, rather than trying to edit in place.

Webhook events don’t trigger an investigation. AWS DevOps Agent only supports PagerDuty V3 webhooks. If your PagerDuty subscription is still on an older webhook version, upgrade to V3. Full details live in Webhooks Overview in the PagerDuty developer documentation.

PagerDuty shows as registered, but isn’t active in an Agent Space. Registering at the account level and adding the provider to an Agent Space are two separate actions. On the Agent Space’s Capabilities tab, check that PagerDuty appears under Communications. If it doesn’t, add it there.

Region or subdomain mismatch. If your PagerDuty account is on the EU service region, make sure you picked EU during registration. The subdomain has to match the first label of your PagerDuty URL exactly (for example, your-company from https://your-company.pagerduty.com).

Conclusion

Most of what happens in the first several minutes of incident response is undifferentiated heavy lifting like opening dashboards, tailing logs, correlating deployments with AWS CloudTrail events. With the native PagerDuty Capability Provider in AWS DevOps Agent, investigations automatically begin by the time you’ve acknowledged your alert, giving your engineers a head start on root cause analysis before responders have finished triaging.

To get started, check out the AWS DevOps Agent documentation or reach out to your PagerDuty or AWS account team.

Resources

About the authors

Shan Kandaswamy

Shan Kandaswamy

Shan is a Senior Partner Solutions Architect specializing in generative AI at AWS, dedicated to solving complex user challenges. He advocates for innovative AI solutions, distributed architecture, and serverless technologies, helping users harness the power of generative AI in their cloud journey. You can reach him on LinkedIn.

Laith Al-Saadoon

Laith Al-Saadoon

Laith Al-Saadoon is a Principal AI Engineer at AWS. He created and launched AWS MCP Servers (30M+ PyPI downloads) and contributes to Strands Agents SDK — AWS’s open-source framework for building AI agents — along with other agentic AI open-source projects like Mem0 and Agno. He drives AWS’s autonomous software development and agentic AI strategy and builds production agentic systems that make agents work for the world’s largest companies. In his personal time, Laith enjoys the outdoors — fishing, photography, drone flights, and hiking with his wife.

Scott Schreckengaust

Scott Schreckengaust

Scott Schreckengaust brings a biomedical engineering degree and decades of deep domain expertise in healthcare and life sciences to emerging technologies and AI. He’s spent his career building—from automating lab workflows and integrating enterprise systems to architecting full-stack software deployments in regulated environments. Now working as an AI engineer, Scott continues what he’s always done best: partner with customers to uncover their scientific and operational challenges, then engineer solutions that scale. His journey from the bench to the cloud reflects a consistent belief: the best technology is invisible—it just works.

 

AWS Security Hub Extended: Why enterprise security products should sell themselves

Post Syndicated from Michael Fuller original https://aws.amazon.com/blogs/security/aws-security-hub-extended-why-enterprise-security-products-should-sell-themselves/

Our largest security services customers started the same way every customer does – with a click. They enabled Amazon GuardDuty, Amazon Inspector, AWS WAF, and AWS Security Hub, experienced the benefits in real time, and evaluated with transparent pay-as-you-go pricing. No RFP. No six-month evaluation. No multi-year commitment up front. Our field teams played a critical role in that growth, not by selling the first click, but by building the trusted relationships that turned early adoption into deep, long-term commitment. We believe customers should have this same frictionless adoption experience and flexibility for all best-in-class security products and that’s why we developed Security Hub Extended.

In our first post, we introduced Security Hub Extended, a significant expansion of Security Hub that brings together curated partner solutions in a single, unified experience. In our second post, we walked through how it works technically, including the onboarding flow, the pricing model, the unified operations layer built on the Open Cybersecurity Schema Framework (OCSF). In this post, I want to step back and talk about why we built it the way we did and why I believe the way enterprises discover, evaluate, and adopt security solutions is ready for a fundamental shift.

The shift

If you’ve ever tried to evaluate a new enterprise security product, you know the drill. Request a demo. Wait. Take the demo. Request a PoC. Wait for professional services (or your team to stop building) to set it up. Negotiate pricing, which isn’t published, so you’re starting blind. Loop in procurement. Sign a multi-year commitment. Then, months later, find out whether the product actually solves your problem in your unique environment.

Meanwhile, an ambitious security engineer on your team has already spun up an open-source tool, connected real data, and knows in two hours whether it’s going to work for your use cases. They didn’t need a slide deck. They needed a solution they could put their hands on.

A Fortune 500 CISO recently told me: “I spent 9 months procuring a security solution and it still doesn’t work the way the demo showed.” That frustration isn’t unique. It’s the norm.

This isn’t a criticism of the sales motion. Sales-led has evolved for good reason. Enterprise procurement is complex, products need customization, customers need support. I respect the craft and have poured a significant portion of my career into trying to perfect it. Even the most product-driven companies still need great sales, marketing, field enablement, and support.

It doesn’t change the fact that threats are evolving constantly, and defenders need the flexibility to discover and deploy new solutions as fast as the landscape shifts. Having the best solutions discoverable and deployable in that moment of need isn’t just a convenience, it’s a competitive advantage that customers are demanding. A new threat emerges, security teams have access to industry-leading solutions, and in a few clicks they’ve found their answer and are already seeing value. That’s the model every security company should be building toward.

What we’ve learned at AWS

At AWS, we’ve spent two decades learning what it takes to let customers adopt complex enterprise technology on their own terms, at massive scale. We haven’t always gotten it right, but we learn fast and adjust. The result is one of the largest cloud businesses in the world. I bring up that scale for one reason. It’s proof that complex, enterprise-grade technology can be adopted without requiring a traditional procurement gauntlet. Compute, storage, databases, AI/ML, networking, and yes, security — adopted all through a console, on each customer’s own timeline, and scaled when they were ready.

The proof is in the adoption

Amazon GuardDuty, Amazon Inspector, AWS Shield, AWS Security Hub are all available through the AWS Management Console. All pay-as-you-go. All activated with a click. Tens of thousands of customers rely on these security services today. When you make it easy to get started and deliver outcomes that earn confidence, expansion follows naturally.

These are sophisticated, enterprise-grade security solutions. And customers, from two-person startups to the world’s largest financial institutions, adopt them the same way. They try it, see the value, expand, and lean on the AWS team to go deeper.

We didn’t get here by accident, and we definitely didn’t get here without making mistakes. Building products that can be adopted and scaled on their own, without a sales engineer explaining away UX problems, without a solutions architect doing the first deployment, requires a different kind of product mindset. Time-to-value becomes your most important metric. Onboarding friction becomes your biggest enemy. Transparent pricing becomes non-negotiable. It’s hard. We’ve gotten a lot wrong along the way. And we’re still iterating.

But the results are clear. When customers adopt based on experience rather than commitment, they don’t just stay, they expand. They bring their teams. They become advocates. I’ve spent 15 years at AWS, the last 10 building security services like GuardDuty and Security Hub. When we launch a new security service or major feature, we consistently see rapid organic adoption at a pace that would be impossible through traditional sales cycles alone. These products are built to deliver value the moment customers turn them on and we make that as easy as we possibly can. That’s the scale a product-led motion unlocks.

Security Hub Extended

So, we asked ourselves: why can’t we build a similar approach that can expand to include industry leading partner solutions? Why can’t the CrowdStrikes, the Splunks, the Zscalers, and the fast-growing innovators solving tomorrow’s problems like Cyera, Noma, and 7AI also reach customers with the same frictionless motion that AWS services enjoy? Why can’t a security team that discovers a new threat on Monday have a proven solution deployed and delivering value by Tuesday? Our partners have built incredible products. What they haven’t always had is an avenue to put those products directly in the hands of the customers who need them most, at the moment they need them, at scale, in a way that feels as natural as turning on an AWS service. Not by replacing how our partners build or sell, but by giving them infrastructure that lets their products speak for themselves.

That’s what Security Hub Extended is. Security teams already using Security Hub can discover curated partner solutions right alongside their AWS security services. One click to evaluate, one click to deploy, pay-as-you-go pricing on your existing AWS bill with Enterprise Discount Program (EDP) discounts automatically applied. No separate procurement cycle. No long-term commitments required. Start fast, validate at scale, and commit for deeper discounts when you’re ready, versus making a three-year bet based on a few months of testing.

For customers, industry-leading enterprise security solutions become as easy to adopt as GuardDuty or WAF. For our partners, Security Hub Extended is a growth channel where the product leads and the customer experience mirrors what we’ve spent 20 years building at AWS. For the industry, it’s an invitation to reimagine what the relationship between a security product and a security practitioner can look like when you remove the friction standing between them.

But Security Hub Extended isn’t just a simpler way to buy security products. It’s a unified solution. When a customer enables a solution through Extended, we’re working toward an experience where AWS handles the rest. Sensors that deploy automatically across Amazon EC2, Amazon EKS, and AWS Fargate workloads using the same mechanism that powers GuardDuty Runtime Monitoring. IAM roles that provision across a customer’s Organization in one click. Resource inventory is automated from day one – S3 buckets, databases, AI workloads – without manual work.

Once enabled, solutions in Security Hub Extended emit findings in OCSF, automatically aggregated in Security Hub alongside findings from GuardDuty, Amazon Inspector, and every other AWS security service. Security Hub applies risk scoring and correlated risk analytics across all of them. AWS-native and third-party findings together, weighted and prioritized as a single view of your security posture. For example, an endpoint detection from CrowdStrike, correlated with a credential theft in GuardDuty, and a data access event from Cyera, produces an attack path that none of those solutions can produce alone. The correlation uses AWS context (IAM topology, VPC exposure, resource criticality) to improve the context of each attack path for security analysts. Deploying a solution through Security Hub Extended doesn’t add another pane of glass. It deepens the intelligence of the one you already have.

We’re also building toward automated response. Customers will be able to opt in to pre-built playbooks that take action through AWS-native services when a threat is detected, such as isolating compromised resources, revoking credentials, or containing active threats. The goal is detect-to-respond in seconds, not the hours it takes to context-switch across five consoles and two ticketing systems.

Where we are and where we’re headed

We’re still in the first inning — or Day 1, as we like to say at Amazon. We launched in February 2026 with 14 partners, now 21, spanning endpoint, identity, email, network, data, browser, cloud, AI, and security operations, and we’re continuously working backwards from customers as we operationalize for scale. We are building this because our customers asked for it. We’re learning alongside our partners and customers every week, identifying what works, what needs improvement, where the friction still lives, and iterating quickly.

We’re building and delivering at the speed of our customers. That means shipping fast, iterating faster, and not waiting for perfection. We’re not where we want to be just yet, and we need your feedback to get us there. What’s encouraging is that our partners aren’t waiting to be asked. They’re investing in this alongside us. Not because we’re demanding it, but because they see the same thing we do, that companies that make it effortless for customers to get started are the ones that will win at scale.

The early signals are encouraging. Customer response has exceeded our expectations, and the feedback we hear most often is that the procurement simplification and flexibility of pay-as-you-go with public pricing alone, even before the unified operations and data normalization benefits, is a meaningful differentiator.

If you’re a security leader: Security Hub Extended is live now. Log into Security Hub, look for the Security Hub Extended Plan (or visit the Security Hub Extended Pricing Page), and explore what’s available for your use cases. Start with what solves your most urgent problem. Pay-as-you-go, no commitment. Your team will tell you if it’s working in days, not months.

The vision is bigger than what’s live today, and we’re iterating fast. Share your feedback on AWS re:Post for Security Hub, reach out through contact AWS Support, or connect with me directly.


Michael Fuller

Michael Fuller

Michael has been with AWS for 16 years and led product for AWS Security Services for 11 years. Michael has 29 years in the industry and held several roles in product management, business development, and software development for IBM, Cisco, and Amazon. Michael has a Bachelor’s of Science in Computer Engineering from the University of Arizona and an MBA from the University of Washington.

 

Enable real-time mainframe analytics with Precisely Connect and Amazon S3

Post Syndicated from Supreet Padhi, Rochelle Grubbs original https://aws.amazon.com/blogs/big-data/enable-real-time-mainframe-analytics-with-precisely-connect-and-amazon-s3/

This is a guest post by Supreet Padhi, Technology Architect, Strategic Technologies, and Rochelle Grubbs, Senior Director, Solution Architect at Precisely in partnership with AWS.

Business leaders face a critical challenge to enable real-time analytics. Their most valuable data sits in mainframe systems that reliably process billions of transactions daily, but extracting value for modern analytics and AI remains complex and costly. Traditional mainframe-to-cloud integration approaches require multi-step replication with intermediary systems, creating operational overhead, latency, and data integrity risks. This complexity delays insights, increases infrastructure costs, limits agility, and blocks organizations from using AI and machine learning on their mainframe data.

Precisely, a global leader in data integrity with over 12,000 customers including 95 of the Fortune 100, has announced an expansion of its collaboration with AWS through new enhancements to Precisely Connect. Precisely is an AWS Data and Analytics ISV Competency and AWS Migration and Modernization ISV Competency partner. Precisely has service specializations in Amazon Redshift and Amazon Relational Database Service (Amazon RDS).

In Stream mainframe data to AWS in near-real time with Precisely and Amazon MSK, we showed you how to set up mainframe CDC and the AWS Mainframe Modernization – Data Replication for IBM z/OS Amazon Machine Image (AMI) available in AWS Marketplace. In this post, we discuss how you can use Precisely Connect to enable real-time, direct replication of mainframe data to Amazon Simple Storage Service (Amazon S3), and how your organization can extend this foundation using Amazon S3 Tables for advanced analytics.

Real-time mainframe data access

Organizations that can connect their mainframe environments with modern cloud platforms can gain advantages through improved agility, reduced operational costs, and enhanced analytics capabilities.For example, moving appropriate analytics and reporting workloads to the cloud can significantly reduce mainframe operational costs while maintaining performance and reliability. Real-time data access makes insights available within seconds rather than waiting for batch processing cycles, enabling faster responses to market changes and customer needs. Eliminating bulk data extracts and intermediary systems also reduces infrastructure and maintenance expenses. This frees IT resources to focus on higher-value initiatives.

However, implementing mainframe-to-cloud integrations presents unique technical challenges that require specialized solutions. These include converting mainframe character encoding (EBCDIC) to standard ASCII format and handling mainframe-specific data types such as packed decimal (COMP) fields. You also need to manage the complexity of VSAM (Virtual Storage Access Method) files that can store multiple record types in a single file, and maintain real-time synchronization without impacting mainframe performance.

Change Data Capture (CDC) technology addresses these challenges through incremental data movement that eliminates disruptive bulk extracts by streaming only changed data to cloud targets, minimizing system impact and ensuring data currency. Real-time synchronization keeps cloud applications in sync with mainframe systems, enabling immediate insights and responsive operations.

Precisely Connect: Real-time data replication to Amazon S3

With Precisely Connect, you can replicate data directly from mainframes to Amazon S3 in real time, eliminating the need for intermediaries and simplifying modernization.Data flows directly from mainframe sources, including Db2 z/OS, IMS, and VSAM, to Amazon S3, eliminating intermediary steps and reducing both latency and operational complexity. You can move mainframe data directly to Amazon S3 data lakes and analytics platforms without managing complex, multi-step replication processes.

The simplicity of this approach reduces maintenance overhead and integration complexity by removing the need for staging servers, middleware, or batch processing systems. After data lands in Amazon S3, it becomes immediately available for downstream AWS workloads. You can use Amazon Athena for SQL queries, AWS Glue for ETL and data cataloging, Amazon EMR for big data processing, Amazon SageMaker AI for machine learning, and Amazon Quick Sight for business intelligence dashboards.

Solution overview

Here we present a solution architecture for streaming mainframe data changes from Db2z through AWS Mainframe Modernization – Data Replication for IBM z/OS AMI directly to Amazon S3 and then using Amazon S3 Tables for advanced analytics capabilities.

By introducing direct S3 replication and streamlining deployment through the pre-configured AWS Marketplace AMI, you can deploy in minutes rather than weeks. This creates new possibilities for data distribution, transformation, and consumption. This architecture offers several key benefits:

  1. Simplified deployment – Accelerate implementation using the preconfigured AWS Marketplace AMI
  2. Direct replication – Eliminate intermediary systems by streaming data directly to Amazon S3, reducing latency and operational overhead
  3. Real-time synchronization – Capture changes as they occur on the mainframe, ensuring downstream applications operate on current data
  4. Flexible analytics options – Use S3 Tables for Iceberg-compatible tabular data storage
  5. Comprehensive AWS integration – Gain immediate access to Amazon EMR, Amazon Athena, AWS Glue, Amazon SageMaker AI, and Amazon Quick Sight
  6. Natural language data access – Through the MCP Server for Amazon S3 Tables, AI assistants can interact with structured data using conversational interfaces without needing to write SQL queries.

Prerequisites

To complete the solution, you need the following prerequisites:

Precisely components

  1. AWS Mainframe Modernization – Data Replication for IBM z/OS – Deploy this Precisely Connect AMI from AWS Marketplace. This pre-configured image contains the Apply Engine and Controller Daemon components required for replicating mainframe data changes to Amazon S3.
  2. Precisely Connect CDC Capture/Publisher – Deploy the Precisely Connect CDC Capture/Publisher on your mainframe environment. This component captures changes from Db2z logs and streams them to the Apply Engine over TCP/IP.

For detailed setup and configuration steps for Precisely components, refer to our previous post Stream mainframe data to AWS in near-real time with Precisely and Amazon MSK.

Connectivity requirements

  1. Have network connectivity established between your mainframe environment and AWS using your organization’s approved connectivity method (such as AWS Direct Connect or VPN).
  2. Verify that firewall rules allow TCP/IP communication between the mainframe Capture/Publisher and the Apply Engine.

AWS analytics components (optional extension)

After mainframe data lands in Amazon S3, your organization can extend its analytics capabilities using AWS services. One approach is to use Amazon EMR streaming jobs to process and write data to Amazon S3 Tables. After the data is stored in S3 Tables, the data can be queried directly using Amazon Athena for ad-hoc SQL analysis. This extension is optional and represents one of several ways to consume and analyze mainframe data after it reaches Amazon S3.

The following diagram illustrates the solution architecture.

image-BDB-5540-1-architecture

  1. Capture/Publisher – Connect CDC Capture/Publisher captures Db2 changes from Db2 logs using IFI 306 Read and communicates captured data changes to a target engine through TCP/IP.
  2. Controller Daemon – The Controller Daemon authenticates all connection requests, managing secure communication between the source and target environments.
  3. Apply Engine – The Apply Engine receives the changes from the Publisher agent and applies the changed data to the target Amazon S3.
  4. Amazon S3 – Serves as the scalable data lake foundation where replicated mainframe data lands.
  5. Amazon EMR streaming job – As data arrives, an instance of the Amazon EMR streaming job writes the data to target tables in Amazon S3 Tables.
  6. Amazon Athena – Queries data stored in Amazon S3 Tables using standard SQL.

This architecture provides a clean separation between the data capture process and the data consumption process, allowing each to scale independently. When CDC data arrives in Amazon S3, you can use Amazon S3 Tables to store Db2 z/OS, VSAM, and IMS data in an open table format (Apache Iceberg) that is ready for analytics, providing a flexible path to mainframe modernization.

Quantifiable business value

Organizations implementing this solution typically see significant reductions in mainframe operational costs by offloading analytics and reporting workloads to the cloud. The elimination of intermediary infrastructure reduces both capital and operational expenses. The reduced maintenance burden frees IT resources to focus on strategic initiatives rather than managing complex replication systems. Speed and agility improvements are equally significant. Near real-time data availability, measured in seconds to minutes rather than hours to days, enables organizations to respond rapidly to market changes and operational events. The rapid deployment of new analytics use cases without requiring mainframe changes accelerates innovation. Organizations gain access to the full breadth of AWS services that can be used immediately after data lands in Amazon S3.

From an analytics and AI perspective, the solution creates a unified data platform that brings together mainframe, cloud-native, and third-party data sources. This unified view enables advanced machine learning on historical and current data, delivering predictive insights that drive proactive decision-making across the organization.

Customer story

A leading global payments provider put this into practice. The payments provider was struggling to generate timely analytics and insights from Point of Sale (POS) transaction data. As one of the world’s largest payment providers, they process hundreds of thousands of transactions per second. Users expect to swipe their card and have their transaction approved in seconds. New architecture was needed to keep up with customer demands and volume. By streaming mission-critical mainframe data directly to AWS in real time using Precisely Connect and landing it in Amazon S3 Tables, the company used storage built on the Apache Iceberg open standard. This approach enables high-performance analytics directly on mainframe data alongside cloud-native sources.

Conclusion

In this post, we demonstrated how Precisely Connect enables real-time, direct data replication from mainframes to Amazon S3, eliminating intermediaries and simplifying mainframe modernization.

Your organization can further extend this foundation with Amazon S3 Tables, purpose-built storage for Apache Iceberg tables in S3, enabling analytical applications to query the most current mainframe data using tools such as Amazon Athena, Amazon EMR, and Amazon Redshift.

Get started by deploying AWS Mainframe Modernization – Data Replication for IBM z/OS from AWS Marketplace and use Amazon S3 as a target for your mainframe use cases. Learn more about Precisely’s mainframe data integration capabilities at precisely.com. Contact AWS and Precisely experts to discuss your specific modernization challenges and design a proof-of-concept that demonstrates business value quickly.


About the authors

image-BDB-5540-2

Supreet Padhi

Supreet is a Technology Architect at Precisely. He has been with Precisely for more than 14 years, with specialty in streaming data use cases and technology, with emphasis on data warehouse architecture. He is responsible for research and development in areas such as Change Data Capture (CDC), streaming ETL, metadata management, and VectorDBs.

image-BDB-5540-3

Rochelle Grubbs

Rochelle is a Senior Director and Solution Architect for Precisely’s Data Integration solutions and has been with Precisely for over 11 years. She has spent the last several years focusing on databases, analytics, data trends, data integration, and GenAI. Rochelle is an expert on Precisely’s OEM AWS Mainframe Migration offering and is driven to help customers successfully migrate their applications and workloads to the cloud.

image-BDB-5540-4

Tamara Astakhova

Tamara is a Sr. Partner Solutions Architect in Data and Analytics at AWS with over two decades of expertise in architecting and developing large-scale data analytics systems. In her current role, she collaborates with strategic partners to design and implement sophisticated AWS-optimized architectures. Her deep technical knowledge and experience make her an invaluable resource in helping organizations transform their data infrastructure and analytics capabilities.

AWS Transform custom: Enterprise Code Modernization with the Learn-Scale-Improve Flywheel

Post Syndicated from Venugopalan Vasudevan original https://aws.amazon.com/blogs/devops/aws-transform-custom-enterprise-code-modernization-with-the-learn-scale-improve-flywheel/

Enterprise modernization has reached an inflection point. You can transform one repository easily. Existing tools, including AWS Transform custom, work well for individual repositories, and the process is understood. But what about 50 repositories? 100? 200? When you need to modernize at enterprise scale, transforming code is only part of the challenge. Coordinating people, capturing knowledge, and maintaining quality across your entire portfolio are also important.

In this post, we explore how AWS Transform custom’s bulk automation capabilities address the enterprise coordination problem through intelligent learning and scaled execution. You will see how one customer reduced end-to-end modernization timelines from 7-12 weeks to 2.5 weeks, delivering a 3-5x reduction in delivery time and 10-20x reduction in total effort hours. Most importantly, you will learn how to start your own transformation journey immediately.

The Coordination Problem at Enterprise Scale

Ask any enterprise architect about their last major modernization initiative, and you will hear familiar stories. As an example, an enterprise software company needed to migrate a large legacy codebase to a modern platform. Their projection: 12 weeks of intensive work coordinating across multiple teams.

The code transformation itself took days. The remaining weeks were consumed by the end-to-end activities surrounding it: orchestrating teams across time zones, ensuring consistent patterns across codebases with different histories, and managing dependencies so upstream changes did not break downstream systems. Teams tracked status through meetings and spreadsheets and captured tribal knowledge that existed only in senior developer heads.

This is the enterprise coordination problem. When you scale from one repository to hundreds, coordination overhead explodes. Each additional repository adds not just its own complexity, but new integration points, edge cases, and unanticipated coordination requirements.

The Hidden 70% Gap

In enterprise engagements, we have observed that code transformation represents approximately 30% of the modernization effort. The remaining 70% include things like test generation, validation, comprehensive documentation, business analysis, and organizational coordination across hundreds of moving pieces.

This gap explains why productivity gains from transformation tools rarely materialize. The tools handle code changes, but organizations still struggle with coordination, validation, and knowledge capture. The transformation is completed quickly, but the project takes months.

Here is what we see: traditional approaches fail at enterprise scale because they treat each repository as an independent challenge. Teams repeat work across codebases, make inconsistent decisions, and lose learnings when developers move to different projects. Organizational knowledge remains trapped in individual heads rather than becoming reusable assets.

A New Approach to Enterprise Modernization

AWS Transform custom takes a different approach to enterprise modernization. Rather than repeating the same operation hundreds of times, the service learns from every execution and applies that knowledge to improve future transformations.

The Learn-Scale-Improve Flywheel

The workflow follows a deliberate progression designed to maximize learning while minimizing risk. It begins with a focused learn pilot, scales through bulk automation, and improves through deliberate review, creating a flywheel where each cycle produces better results than the last (Figure 1).

Iterative transformation workflow with three stages: LEARN (interactive pilot, refine TD), SCALE (bulk execution, overnight processing), and IMPROVE (review and approve knowledge items). Arrows show the cycle: org knowledge captured flows from Learn to Scale, edge cases observed flow from Scale to Improve, and TD improves flows from Improve back to Learn.

Figure 1: Learn Scale and Improve Flywheel for AWS Transform custom transformation

Learn — You start with two to three representative repositories and execute transformations in interactive mode. You work directly with the AI agent, providing feedback on decisions and validating quality at each step. When the agent encounters ambiguity, it asks questions. You provide guidance, and the system captures that context. At the end of the pilot, you review the feedback and modify the transformation definition. The result is a transformation definition enhanced with your organizational knowledge, ready to scale.

Scale — You shift to non-interactive mode for bulk execution. The system processes dozens or hundreds of repositories overnight without manual intervention, applying patterns learned during the pilot. It validates transformations using your build and test commands and tracks progress across your portfolio in real time. What previously required weeks of team coordination happens overnight. During execution, the system captures observations: new edge cases, unexpected patterns, and optimization opportunities the pilot did not encounter.

Improve — After each round of bulk execution, you review the knowledge items the system captured during processing. These observations surface patterns and edge cases specific to repositories the pilot did not cover. You approve the valuable learnings, and your transformation definition improves for the next iteration. This review step ensures quality control. The system does not self-modify. Transformation owners decide which learnings get incorporated.

The Scale-Improve cycle repeats. Each round of bulk execution generates insights that make the next round more effective. Transformation success rates increase, manual intervention decreases, and edge case handling improves with every iteration.

This flywheel transforms how enterprises capture and share institutional knowledge. Transformation definitions are not automation scripts. They are organizational assets that encode how your company approaches specific modernization scenarios. When an architect defines a transformation strategy, that strategy becomes a reusable definition stored in your registry. When your team identifies best practices, those practices become embedded within the transformation definition and automatically apply across all repositories. Previously, when a senior developer left your team, that knowledge disappears with them. With AWS Transform custom, their expertise is captured in transformation definitions and knowledge items available to the entire organization. Individual expertise becomes an organizational capability.

An Enterprise Customer Modernization Case Study

These productivity gains are production outcomes, not theoretical projections. An enterprise software company needed to migrate a large volume of production-grade Control-M workflows to Apache Airflow, a modernization requiring both technical precision and consistency across a complex, interdependent codebase. Their estimate was 12 weeks of intensive coordination across multiple teams, with risk of inconsistency and integration failures.

Using AWS Transform custom, the company executed an iterative learn-scale-improve workflow. During the pilot phase, they ran interactive transformations on representative repositories, reviewed results, and refined transformation definitions. With each iteration, transformation definitions improved in edge case handling and accuracy. They then shifted to non-interactive bulk execution across their portfolio and completed the full migration in 2.5 weeks.

The validation achieved a 100% success rate across all workflows in scope. Edge case handling improved by 60% compared to the customer’s existing approach, and the transformed code demonstrated a 19% runtime performance improvement while meeting industry expert code quality standards. This proves that organizations can achieve both migration speed and production readiness, with 3-5x faster delivery timelines and 10-20x reduction in total effort hours compared to traditional approaches.

Get Started: Transform Your Repository Portfolio

AWS Transform custom bulk automation capabilities are available as a solution in this Github repo. Follow the learn-scale-improve workflow to begin your transformation journey.

Prerequisites

Before beginning, ensure you have:

  • An AWS account with AWS Transform custom access enabled
  • AWS CLI configured with appropriate credentials
  • Git installed on your local machine or CI/CD environment
  • IAM permissions for AWS Transform custom operations

Your Implementation Path

AWS Transform custom supports Java upgrades (e.g., 8 to 17, 17 to 21), Python migrations (e.g., 3.7 to 3.11), Node.js updates (e.g., 14 to 20), AWS SDK migrations (e.g., boto2 to boto3, SDK v1 to v2), and other transformations. Beyond these AWS-managed transformations, you can create custom transformation definitions for organization-specific standards, proprietary framework migrations, and architectural patterns unique to your environment.

AWS Transform custom integrates naturally into your existing development processes. The CLI connects with CI/CD pipelines like Jenkins, GitLab CI, or GitHub Actions. Transformations create code in local Git branches that flow through your standard code review and merge processes. The web interface provides centralized visibility for tracking progress across teams. Validation commands execute automatically during transformation, ensuring code builds successfully and tests pass before changes are considered complete. At the end of the transformation, if validation criteria fail, the transformation is marked as failed.

To accelerate your path to scaled execution, AWS provides an open-source sample repository that gives you a production-ready starting point for running transformations across multiple repositories and transformation definitions simultaneously. The aws-transform-custom-samples scaled execution repository includes scripts that orchestrate bulk execution, manage repository queuing, and handle status tracking across your portfolio. Rather than building orchestration from scratch, you clone the sample, configure it with your repository list and transformation definitions, and begin executing scaled transformations immediately.

Conclusion

Enterprise modernization at scale requires more than code transformation tools. The real challenges are coordination across teams, learning from execution, and capturing knowledge as organizational assets. AWS Transform custom learn-scale-improve workflow addresses these challenges through continual learning that improves quality with every execution, organizational knowledge capture that transforms tribal expertise into reusable assets, and bulk automation that scales consistently across hundreds of repositories. When the next critical security vulnerability requires framework updates across your repositories, or a new runtime version unlocks performance improvements, you respond in days rather than months — using transformation definitions you have already proven.

Real customers have reduced delivery timelines by 3-5x and total effort hours by 10-20x, compressing modernization from months to weeks. These are not aspirational goals. They are production results from organizations using AWS Transform custom today.

Begin Your Transformation Today

Follow the learn-scale-improve workflow on two to three representative repositories, refine your transformation definitions, then scale across your portfolio.
To dive deeper into AWS Transform custom bulk automation capabilities, explore these resources:

      • AWS Transform custom Documentation — Technical documentation covering all capabilities, API references, and integration guides: AWS Transform custom
      • Scaled Execution Sample Repository — Open-source scripts for running transformations across multiple repositories and transformation definitions: aws-transform-custom-samples
      • Transformation Registry — Discover AWS-managed transformations and create custom definitions: aws-transform-custom-samples

Contact your AWS account team or visit the AWS Transform custom documentation to begin your journey.

 


 

About the authors

meghan-author

Meghan Kothari

Meghan Kothari is a Senior Technical Product Manager with the Customer Experience and Business Trends team, where he partners with AWS leadership on strategic deep dives to discover evolving trends in agentic AI-driven application development and modernization. His background as a solutions architect and full-stack developer gives him a unique hands-on perspective to help shape the developer experience. 

Venu-author

Venugopalan Vasudevan

Venugopalan Vasudevan (Venu) is a Principal Specialist Solutions Architect at AWS, where he leads Agentic AI initiatives focused on AWS Transform. He helps customers adopt and scale AI-powered developer and modernization solutions to accelerate innovation and business outcomes.

grilli-author

Rodney Grilli

Rodney Grilli is a Principal Technologist at AWS, specializing in product and code modernization using agentic AI services. He builds solutions that help customers modernize their product portfolios and accelerate their transformations into AI-Native Enterprises.

Modernizing KYC with AWS serverless solutions and agentic AI for financial services

Post Syndicated from Neeraj Kaushik original https://aws.amazon.com/blogs/architecture/modernizing-kyc-with-aws-serverless-solutions-and-agentic-ai-for-financial-services/

Regulators worldwide require financial institutions to implement Know Your Customer (KYC) processes that help prevent money laundering, terrorist financing, fraud, and identity theft. KYC has evolved from a compliance checkbox to a core security function for financial institutions. Financial institutions must modernize their KYC architectures because of several factors: rising transaction volumes, increasing regulatory complexity, and customer demands for instant onboarding. Legacy systems create multiple problems. They slow down compliance processes and expose institutions to both operational risks and regulatory penalties. However, traditional KYC orchestration systems, often built on monolithic architectures, struggle to meet these demands because of latency, availability, and scalability challenges. Their reliance on batch processing and manual handoffs leads to higher operational costs and impediments to real-time compliance validation, reinforcing the need for architectural modernization.

This post extends IBM’s approach to real-time KYC validation using generative AI, as previously discussed in the post IBM Digital KYC on AWS uses Generative AI to transform Client Onboarding and KYC Operations. It transforms compliance operations through autonomous decision-making and intelligent automation using agentic AI, event-driven architecture, and AWS serverless services. The solution addresses the fundamental limitations of traditional rule-based systems. It provides autonomous decision-making, dynamic adaptation, and intelligent automation that transforms compliance operations.

Financial institutions can break down KYC workflows into separate business functions. Amazon Managed Streaming for Apache Kafka (Amazon MSK) handles real-time event streaming, which speeds up processing. Amazon Bedrock automates document analysis and risk assessment with AI. AWS Lambda provides serverless computing that scales on demand and supports instant customer onboarding.

The critical role of KYC

KYC protects financial systems by verifying customer identities and detecting fraud in four ways. It supports regulatory compliance with anti-money laundering (AML) and counter-terrorist financing (CTF) regulations. It helps prevent fraud by detecting identity theft and forged documents. It manages risk by assessing customer profiles and monitoring transactions. And it builds customer trust through transparency. As financial institutions broaden their footprint across products, industries, and regions, KYC compliance becomes increasingly complex. Each financial service offering presents unique requirements, from traditional banking to digital wallets, investment systems, and cryptocurrency services. Expansion into retail, SME, and corporate segments brings diverse identity structures and risk profiles. Operating across multiple jurisdictions requires navigation of various regulatory frameworks. These frameworks include the Bank Secrecy Act (BSA) and USA PATRIOT Act in the US, Anti-Money Laundering Directives (AMLD) in the EU, and guidelines from international regulators like the Monetary Authority of Singapore (MAS) and Financial Action Task Force (FATF).

Traditional KYC

Traditional KYC processes verify customer identities, assess risk, and monitor for money laundering. They rely on manual document collection, identity checks across multiple databases, and periodic reviews. While these established processes have served the financial industry for decades, they were designed for a different era with lower transaction volumes, simpler product offerings, and less sophisticated threat landscapes. Today’s digital-first financial environment demands a fundamental reimagining of KYC at scale.

Current challenges

Legacy systems create several bottlenecks. They process requests in batches rather than real-time, making instant onboarding impossible. Manual validation across jurisdictions leads to inconsistent compliance. Without event-driven capabilities, these systems can’t integrate with modern AI and machine learning (ML) services or adapt to new fraud patterns without manual reconfiguration.

Cloud-native KYC solution architecture using agentic AI

This architecture illustrates a comprehensive cloud-native real-time KYC validation system designed to process live customer onboarding requests and validate identity information using AI-powered automation. The architecture uses an event-driven pipeline to process high-volume KYC validations securely in under 5 minutes. The system processes real-time KYC requests containing sensitive financial data including PII while maintaining strict security and regulatory compliance requirements across multiple geographies.

High-level Agentic Architecture for real-time KYC

High-level Agentic Architecture for real-time KYC

This architecture diagram illustrates an AI-driven Know Your Customer (KYC) Orchestration Framework built using Amazon Bedrock AgentCore and Amazon Managed Streaming for Apache Kafka (Amazon MSK). The design showcases how multiple specialized AI agents collaborate to automate and optimize KYC workflows, from document ingestion to compliance validation and fraud detection, while maintaining real-time integration with on-premises financial systems.

At the heart of the architecture is the AgentCore Runtime Environment, which provides native orchestration capabilities, session management, and memory persistence. Within this runtime, the KYC Orchestration Supervisor Agent acts as the intelligent coordinator, delegating tasks to five domain-specific sub-agents: Identity Verification, Document Analysis, Fraud Detection, Compliance & Risk, and Customer Experience. Unlike traditional multi-agent systems, AgentCore provides built-in session state management, shared memory across sub-agents, and automatic context preservation throughout asynchronous processing workflows.

The architecture uses asynchronous invocation patterns where MSK consumers trigger AgentCore processing without blocking, enabling sub-5-minute processing times while handling thousands of concurrent KYC requests. Lambda functions serve as the integration layer, consuming events from MSK, invoking AgentCore asynchronously, and publishing results back to Kafka topics for downstream system consumption.

Each sub-agent uses foundation models hosted on Amazon Bedrock for tasks such as optical character recognition (OCR), language processing, behavioral analysis, and regulatory interpretation. These agents operate within the AgentCore Runtime, sharing context through AgentCore Memory (a built-in feature of Bedrock AgentCore that automatically manages session state and context) and accessing external systems through tools defined using OpenAPI schemas and Lambda targets.

The agents use KYC Knowledge Bases, powered by Amazon OpenSearch Serverless and Amazon Simple Storage Service (Amazon S3), to access contextual information from internal policies, compliance rules, vendor documentation, and regulations. This approach provides consistent, explainable, and policy-aligned decision-making. These knowledge bases integrate with AgentCore’s retrieval mechanisms, providing sub-agents with grounded information during processing.

Finally, the solution connects with existing on-premises systems, such as customer management, transaction monitoring, case management, risk/AML systems, and core banking systems. These connections use tools defined with OpenAPI schemas as targets and Lambda-based integrations using AgentCore Gateway. AgentCore Gateway uses these OpenAPI specifications to understand API contracts, handle authentication, validate requests and responses, and manage retries. AgentCore Identity manages authentication and authorization for agents and their tool access, so that only authorized sub-agents can invoke specific tools and access the Knowledge Base. With this approach, financial institutions can achieve an intelligent, scalable, and compliance-aligned KYC process that minimizes manual intervention, improves onboarding speed, and reduces fraud and regulatory risks.

Solution Components

Event-Driven Communication Infrastructure with Amazon MSK

Amazon MSK serves as the communication backbone, enabling asynchronous, real-time message exchange between agentic AI components and enterprise systems. The streaming infrastructure organizes into distinct topic categories supporting bi-directional flows.

Inbound topics capture customer interactions through KYC requests (new applications), document uploads (identity documents), ID verification results (third-party vendor responses), and transaction events (fraud/risk signals). Event listeners pre-process these streams. These listeners filter onboarding requests, prepare documents for OCR, normalize vendor data formats, and correlate transaction signals with customer profiles.

Outbound topics publish KYC decisions with confidence scores and audit trails to core banking systems, route complex cases to human reviewers through case management events, and trigger fraud alerts to security teams. With this decoupled architecture, you can achieve sub-5-minute processing while maintaining full event auditability and allowing independent scaling of individual agents based on workload patterns.

Agentic AI Orchestration Layer

KYC Orchestration Supervisor Agent

The Supervisor Agent implements intelligent routing logic using Amazon Bedrock AgentCore to dynamically determine optimal sub-agent collaboration patterns. Unlike rule-based systems following rigid workflows, the supervisor analyzes case characteristics (document types, customer geography, risk indicators, and historical patterns) to construct context-aware execution plans that invoke sub-agents in parallel or sequentially based on dependencies. The supervisor monitors sub-agent confidence scores to guide decision-making: high confidence (>95%) results in automatic approvals, medium confidence (75-95%) triggers additional verification, and low confidence (<75%) escalates to human review with comprehensive context.

Five Specialized Sub-Agents operate as autonomous decision-makers, each using foundation models for domain-specific tasks:

  • Identity Verification Sub-Agent validates customer identities against watchlists and sanctions databases. It calls third-party verification APIs and uses natural language processing to handle name variations.
  • Document Analysis Sub-Agent extracts data from identity documents using OCR. The agent handles poor image quality and multiple languages and detects forgery by analyzing watermarks and security features.
  • Fraud Detection Sub-Agent identifies suspicious patterns through behavioral analysis. The agent detects multiple applications from the same IP address or inconsistent information across form fields. It correlates current applications with historical fraud cases using semantic similarity search and maintains dynamic risk scores with explainable fraud assessments.
  • Compliance & Risk Sub-Agent supports regulatory adherence by interpreting jurisdiction-specific KYC requirements across different geographies. It translates regulatory frameworks into concrete validation actions and generates compliance attestations with audit trails for regulatory examinations.
  • Customer Experience Sub-Agent optimizes the onboarding journey by analyzing application progress in real time, identifying friction points, and recommending strategies to reduce abandonment while identifying upselling opportunities based on customer profiles.

Intelligent Knowledge Management Architecture

The KYC Knowledge Base implements a retrieval augmented generation (RAG) pattern that grounds agent decisions in factual, current information rather than relying solely on foundation model training. Amazon S3 stores source documents, including regulations from financial authorities, institution-specific compliance rules, internal policies, and vendor documentation, enabled to track changes over time. Documents undergo automated preprocessing for text extraction, metadata enrichment, and quality validation before the system indexes them. Amazon OpenSearch Serverless provides semantic search using vector embeddings generated by Amazon Bedrock. When agents query using natural language questions, the system embeds queries into the same vector space and identifies semantically relevant document chunks through cosine similarity search, improving retrieval accuracy over keyword matching.

Context-aware retrieval enriches queries with case-specific information, including customer jurisdiction, document types, and risk levels – facilitating highly relevant regulatory guidance. This continuous knowledge access keeps agent decisions grounded in institutional knowledge rather than hallucinating responses.

Real-Time Decision Store (Amazon DynamoDB) complements the Knowledge Base with sub-millisecond access to frequently accessed structured data, including current KYC decision status, risk scores, customer interaction history, and dynamic configuration parameters controlling agent behavior.

Secure integration with on-premises financial systems

The architecture integrates with on-premises financial systems through Action Groups bridging the cloud-native agentic layer and existing enterprise infrastructure.

Customer Management Systems receive real-time KYC decisions, updating verification status and account activation flags. Transaction Monitoring Systems consume fraud alerts and risk scores, enabling immediate action on suspicious patterns. Case Management Systems receive escalated cases with comprehensive agent analysis context, accelerating human review. Risk and AML Systems integrate bidirectionally to maintain consistent risk assessments. Core Banking Systems receive approved validations, triggering account activation.

Secure connectivity through AWS Direct Connect or AWS Site-to-Site VPN provides encrypted data transmission over dedicated network paths. API calls include comprehensive audit logging through AWS CloudTrail and Amazon CloudWatch, satisfying regulatory requirements.

Security Considerations

The solution should incorporate multi-layered security controls, continuous monitoring, and automated compliance auditing to meet the rigorous expectations of financial regulators and internal risk teams. Financial institutions should conduct a comprehensive threat modelling to identify risks including introduced by agentic AI systems. For further information please refer Security Guidance.

Conclusion

This KYC architecture uses AWS serverless services and Amazon Bedrock to process validations faster and at scale. The parallel agent execution model is designed to reduce KYC validation time from the typical 3-5 days to near-real time for standard cases. This approach enables exponentially faster processing through simultaneous operation of Document Analysis, Identity Verification, and Fraud Detection agents rather than sequential workflows.

With this architecture, financial institutions can handle high-volume validations through elastic scaling, optimize costs through serverless pay-per-use pricing, and improve accuracy through multi-agent collaboration. Automated document processing and intelligent routing are expected to reduce manual review workload, allowing each compliance specialist to handle up to 4x their current caseload while focusing on complex cases requiring human expertise. Explainable AI decisions with comprehensive audit trails support regulatory compliance and enable rapid audit responses.

Event-driven architecture and agentic AI help financial institutions compete in digital landscapes while meeting regulatory requirements.

Note: The architecture presented here is for reference purposes only. IBM and AWS will work closely with you to execute a Proof of Concept and implementation plan in accordance with industry standards and compliance requirements.

Further Reading

IBM Consulting is an AWS Premier Tier Services Partner that helps customers who use AWS to harness the power of innovation and drive their business transformation. They are recognized as a Global Systems Integrator (GSI) for over 30 competencies, including Financial Services Consulting. For additional information, please contact an IBM Representative.


About the authors

A technical walkthrough of multicloud full-stack security using AWS Security Hub Extended

Post Syndicated from Matt Meck original https://aws.amazon.com/blogs/security/a-technical-walkthrough-of-multicloud-full-stack-security-using-aws-security-hub-extended/

Building on our recent announcement of AWS Security Hub Extended —our full-stack enterprise security offering — we want to show you how we’re simplifying security procurement and operations for your multicloud environments. Whether you’re a security architect evaluating solutions or a CISO looking to streamline vendor management, this post walks through the streamlined experience that transforms how you acquire, deploy, and manage end-to-end enterprise security solutions across endpoint, identity, email, network, data, browser, cloud, AI, and security operations. Security Hub Extended brings together AWS security services with carefully curated security partners. Delivering better outcomes together through unified procurement, billing, and operations that significantly reduce vendor management overhead so you can focus on what matters most: protecting your organization.

The challenge we’re addressing

Security teams today spend too much time on vendor management, evaluating services, negotiating contracts, and managing multiple billing cycles instead of focusing on what matters most: managing risk. But the procurement challenge runs even deeper. Until now, customers really only had one option: sign multi-year agreements based solely on proof-of-concept testing and estimated annual usage. This forces organizations to commit budget before they can validate whether a solution will work for them at scale.

AWS Security Hub Extended transforms this procurement model. Security Hub Extended offers customers the option to get started with pay-as-you-go pricing and no commitments, so they can move fast and validate solutions in their actual environment. After they’ve confirmed a solution works at scale, they can then align their vendor strategy and sign longer-term commitments for even more favorable pricing.

Security Hub Extended provides a curated set of carefully chosen partner solutions with competitive pricing, unified billing through your AWS account, and seamless integration. Our initial launch partners, selected by customers for their proven value, include 7AI, Britive, CrowdStrike, Cyera, Island, Noma, Okta, Oligo, Opti, Proofpoint, SailPoint, Splunk, Upwind, and Zscaler.

Getting started with Security Hub Extended

AWS Security Hub consolidates threat analytics from Amazon GuardDuty, vulnerability management from Amazon Inspector, and sensitive data discovery from Amazon Macie, correlating these signals with Security Hub Exposure findings to determine overall risk, reachability, and assumability. Security Hub Extended builds on this foundation by adding curated partner solutions, extending these unified security operations across your entire organization including multicloud, on-premises, and endpoint environments. If you’re already using Security Hub, you can navigate directly to the Extended plan section.

Getting started with Security Hub is straightforward. From the AWS Management Console, search for Security Hub to start the onboarding walkthrough. If you’re not already a Security Hub customer, you can quickly complete onboarding by designating an AWS organization delegated administrator (DA) account. You can then centrally enable and manage Security Hub across your entire organization’s accounts and AWS Regions from a single location (see Introduction to AWS Security Hub). After you’ve onboarded, navigate to the Extended plan section to add curated partner solutions.

Figure 1- Security Hub centralized configuration

Figure 1: Security Hub centralized configuration

From this single interface, you can enable detection and response capabilities across your entire organization, provide granular configurations at the organizational unit or member account level, select specific Regions, and turn individual features on or off as needed.

Understanding risk through attack paths

The Security Hub risk correlation engine identifies potential exposures by correlating threats, vulnerabilities, and misconfigurations to reveal how they connect and could lead to compromise of critical resources.

Figure 2 - Security Hub exposure attack path visualization

Figure 2: Security Hub exposure attack path visualization

The attack path visualization in the preceding figure reveals critical insights including upstream root causes and blast radius, showing the potential impact if a threat actor exploits a vulnerability. You can use this visualization to focus on fixing the root cause rather than addressing symptoms. For example, updating one security group configuration can eliminate the entire attack path, cutting off all downstream exposure.

Accessing Security Hub Extended

You can find Security Hub Extended, shown in the following figure, in the left navigation pane under Management in your Security Hub delegated administrator (DA) account; Security Hub Extended will only be visible from the delegated administrator account. The Extended plan brings curated third-party security solutions directly into the Security Hub experience. Because Extended is built into Security Hub, there’s no separate console to manage. You discover, subscribe to, and operate curated partner solutions from the same place you manage enterprise security, delivering unified operations across your entire security estate.

Figure 3- Security Hub Extended partners

Figure 3: Security Hub Extended partners



Transparent, competitive pricing consolidated with Security Hub

Unlike traditional third-party engagements that require lengthy negotiations, private pricing deals, and multi-year commitments, Security Hub Extended offers complete pricing transparency. Every partner solution displays clear, competitive monthly pay-as-you-go rates billed directly with Security Hub requiring no commitments. For example, Cloud Security from Upwind costs $3.75 per resource per month, and Identity Security from Okta costs $20 per user per month.

All Security Hub Extended offerings are also eligible for AWS Enterprise Discount Program (EDP) discounts that will be applied automatically. If you have an existing AWS enterprise discount agreement, those discounts automatically apply to Security Hub Extended offerings, further reducing your effective costs. All partner solutions you deploy through Security Hub Extended appear on your consolidated AWS bill, no separate invoices or payment processes.

Streamlined onboarding

Adopting curated partner solutions through Security Hub Extended is straightforward. Choose View Product to initiate an automated workflow. Depending on the solution, you’ll either be directed to the partner onboarding console or provide information for the partner to guide you through their onboarding process tailored to your environment.

Billing begins only after you’re fully activated on the partner solution and starts automatically, no additional action is required to benefit from the unified billing. If you’re already using one of the curated partner solutions, transitioning to Security Hub Extended for consolidated billing and flexible pricing won’t disrupt your current services. Now, instead of receiving separate invoices for each partner in addition to Amazon Inspector, GuardDuty, and Security Hub CSPM you get one unified bill through Security Hub. This consolidates visibility to support better understanding of spend and to manage cost.

Unified operations

Security Hub Extended unifies security operations by consolidating findings from AWS and curated partner solutions. All findings use the Open Cybersecurity Schema Framework (OCSF) for consistency, without the need for complex data normalization, transformation, and extract, transform, and load (ETL) processes.

When you deploy solutions such as CrowdStrike, Noma, and Upwind alongside Splunk and 7AI through Security Hub Extended, security findings automatically flow into Security Hub and then seamlessly route to Splunk and 7AI. All in OCSF format so your security team can focus on responding to threats, not managing pipelines, so you can quickly identify and respond to security risks that span boundaries—from endpoint compromises to cloud infrastructure—without spending valuable time on manual integration work.

The full-stack security vision

Security Hub Extended represents a shift in how you discover, procure, and build comprehensive security programs. Instead of managing dozens of vendor relationships, negotiating separate contracts, agreeing to multi-year annual commitments, and integrating disparate tools, you now have one procurement process through AWS, one bill with transparent competitive pay-as-you-go pricing, one console for unified security operations, one support channel for AWS Enterprise Support customers, and one schema (OCSF) for all security findings. The result: reduced security risk, improved team productivity, and a more unified approach to security operations across your enterprise.

Get started

Try Security Hub Extended today and experience how simplified procurement and unified operations can transform your security program. Security Hub Extended is generally available globally in all AWS commercial Regions where Security Hub is available. We’ve also published a walk through video to further explain how Security Hub Extended works.

It’s still Day 1, but we’re iterating fast, so share your feedback with us on AWS re:Post for Security Hub or through your AWS Support contacts and watch for future blog posts on our progress.


Matt Meck

Matt Meck

Matt is a Worldwide Security Specialist at Amazon Web Services, based in New York, with 10 years of experience in the tech industry. For the past 4 years at AWS, he’s focused on Detection and Response, helping solve complex security challenges in the rapidly evolving security space. He works closely with product teams, customers, partners, and field teams to deliver effective security solutions.

 

Michael Fuller

Michael Fuller

Michael has been with AWS for 16 years and led product for AWS Security Services for 11 years. Michael has 29 years in the industry and held several roles in product management, business development, and software development for IBM, Cisco, and Amazon. Michael has a Bachelor’s of Science in Computer Engineering from the University of Arizona and an MBA from the University of Washington.

 

Enhancing Identity Intelligence with Babel Street Match and Amazon OpenSearch

Post Syndicated from Kunal Sharma original https://aws.amazon.com/blogs/big-data/enhancing-identity-intelligence-with-babel-street-match-and-amazon-opensearch/

This post is co-authored with Gil Irizarry, Mae Wells-Kress and Craig Harmon from Babel Street. 

Can your system tell “John Smith” apart from “John Smith”?

Organizations requiring identity intelligence increasingly face challenges due to complexity of matching names and entities across vast, multilingual, and constantly evolving datasets. Whether helping border security, combating financial crimes, or maintaining regulatory compliance, the accuracy of identity and entity resolution directly determines whether threats are detected, investigations succeed, and regulatory requirements are met. Yet, linguistic diversity, transliterations, inconsistent data formats, and legacy system limitations continue to create friction, leading to false positives, missed matches, and costly manual reviews. As customers ingest and analyze petabytes of unstructured and structured data in Amazon OpenSearch Service, the need for intelligent, scalable, and multilingual matching becomes increasingly important. This is where the integration of Babel Street (an AWS Partner) with OpenSearch Service provides a solution that helps organizations enhance precision, reduce noise, and accelerate insights from their high-volume data environments.

This post explores how combining Babel Street Match with OpenSearch Service provides a solution that helps your organization to handle large-scale, multilingual data.

The growing complexity of identity and entity resolution

As organizations ingest and analyze massive volumes of multilingual and inconsistently formatted data, accurately matching names and entities becomes increasingly difficult. Variations in spelling, transliterations, semantic differences, cultural naming conventions, and incomplete or noisy records can contribute to mismatches. These challenges are compounded by legacy systems, fragmented data pipelines, operational inefficiencies, and evolving regulatory requirements—especially in sectors where precision is a requirement.

Evaluating and enhancing identity in high-volume enterprise environments

Amazon OpenSearch Service is a fully managed, scalable search and analytics service that enables organizations to ingest, search, visualize, and analyze massive volumes of data in near real time. Built to handle structured and unstructured information from diverse sources, it powers use cases ranging from security analytics and log monitoring to enterprise search and advanced analytical applications.

Babel Street delivers risk intelligence trusted by organizations across government, defense, and the private sector. The offering combines access to vast volumes of multilingual data with advanced analytics to uncover hidden identities, secure vendor networks, and identify emerging risks with precision, speed, and scale. From national security to regulatory compliance and enterprise resilience, Babel Street provides the strategic advantage needed to stay ahead of risk, safeguard operations, and protect missions.

Babel Street Match, an offering from Babel Street incorporates advanced identity risk intelligence capabilities, which enhance the precision and reliability of screening processes. This advanced solution uses sophisticated matching techniques to verify identities and identify variations in personal data—including aliases, alternate spellings, and differences in biographical details, helping organizations separate legitimate individuals from potential threats. The ability to screen names, addresses, dates, and other identifiers across different scripts and languages helps reduce false positives and negatives, helps accurately detect critical risks with transparent scoring to meet compliance and audit requirements. Further, Babel Street Match streamlines screening workflows, reduces the burden of manual reviews, and elevates the accuracy of threat detection.

The following diagram shows the details of OpenSearch Service and Babel Street Match Plugin integration.

Architecture diagram showing Babel Street Match Plugin integration with AWS services, including AWS Marketplace, Amazon S3, and Amazon OpenSearch Service across two AWS accounts for secure entity matching.

Babel Street Match integrates directly with the OpenSearch Service domain through a lightweight plugin that runs inside your own AWS account where you have full control of your data. The Match plugin sends encrypted match requests to Babel Street’s fully managed Match engine, where the core matching engine performs the entity-resolution logic. The results return to you in real time, enhancing your existing OpenSearch Service workflows with advanced name- and entity-matching capabilities. Meanwhile, Babel Street’s control plane handles licensing, monitoring, and AWS Marketplace integration behind the scenes, provides continuous validation, automated updates, and a seamless operational experience.

Example use cases

The solution combines enterprise-scale search and analytics with AI-powered, multilingual identity intelligence. This section showcases example use cases where integration has enhanced organizations’ capabilities.

  • Border Screening: Help agencies identify high-risk travelers, cargo, and networks to strengthen point-of-entry security with faster, automated risk assessment.
  • Financial Services Compliance: Help Financial institutions and the FinTechs that serve them by offering AI-driven solutions for name screening, adverse media monitoring, and know your customer (KYC)/know your vendor (KYV) due diligence.
  • Identity and Organization Screening: Help businesses needing identity and organization screening by providing AI, analytics, and advanced matching technologies to assist in addressing complex screening challenges.
  • Customer and Vendor Onboarding: Help governments and financial institutions by providing research, analytics, and advanced matching technologies needed to quickly and confidently onboard customers and vendors at scale.

Customer Success Stories

Here’s how leading organizations are leveraging Babel Street Match and Amazon OpenSearch Service to solve real-world identity challenges:

  • A European online brokerage faced AML (anti-money laundering) compliance challenges with its outdated name-matching system, which produced excessive false positives and couldn’t process longer multilingual names. After implementing Babel Street Match on OpenSearch Service, the firm achieved up to 70% better accuracy across 25 languages—significantly reducing manual work and speeding customer payments.
    Babel Street Match Improves FI’s Name-Matching Accuracy by Up to 70% on OpenSearch
  • A major border agency struggled with an outdated screening system that flagged 15% of travelers as potential watchlist matches—overwhelming agents and creating long queues. After implementing Babel Street Match, false positives dropped dramatically (from 80,000 to just 100 in one test), hardware needs fell by 70%, and travelers with common names can now pass through faster. As one stakeholder put it: “Name matching is not our biggest problem anymore.”
    Enabling Stronger, Safer Borders with AI-powered Screening by Babel Street Match

Getting Started with Babel Street Match for Amazon OpenSearch Service

Amazon OpenSearch Service supports third-party plugins like Babel Street Match for OpenSearch. This plugin is supported on OpenSearch version 2.15 or higher and licenses can be obtained through AWS Marketplace.

Installing Babel Street Match for Amazon OpenSearch Service

Prerequisites: Obtain the license file from Babel Street and upload it to an S3 bucket in the same AWS Region as your OpenSearch domain.

Installation Steps:

  1. Create packages – In the OpenSearch Service console, create a package for your license file and select the Babel Street Match plugin from the available options
  2. Associate packages – Link both the license and plugin packages to your OpenSearch domain
  3. Verify – Monitor the domain update and confirm the plugin is active

For details, refer to AWS documentation “Installing third-party plugins in Amazon OpenSearch Service” and Babel Street installation guide which provides detailed guidance on pre-requisites, installation and using the plugin.

Conclusion

Together, Babel Street Match and OpenSearch Service help organizations cut through false positives and catch true matches faster. The result? Greater precision, efficiency, and speed—whether protecting entities, maintaining compliance, or securing supply chains. That’s business-critical identity intelligence in action.

Explore how Babel Street Match on Amazon OpenSearch Service can elevate your organization’s identity intelligence capabilities and transform the screening operations through an interactive or customized demo on Babel Street’s website.

Portions of this content describing Babel Street products and services are provided by Babel Street. AWS is not responsible for the accuracy of third-party product information.


About the Authors

Kunal Sharma

Kunal Sharma is a Sr. Solutions Architect at AWS. He works with AWS Worldwide Public Sector (WWPS) partners to build and scale cloud-native solutions. As an SA, he thrives on turning complex customer challenges into elegant, well-architected solutions — one whiteboard session at a time.

Gil Irizarry

Gil is the Chief Innovation Officer at Babel Street. He specializes in applying natural language processing and AI to identity resolution use cases. Gil’s work combines computational linguistics, machine learning and AI to produce state-of-the-art entity extraction and resolution applications. Gil’s focus on innovation led to his winning of Babel Street’s internal hackathon two years in a row.

Mae Wells-Kress

Mae Wells-Kress is the Vice President of Strategic Marketing at Babel Street. She has extensive experience across strategic and creative marketing roles, she implements process-driven lead generation efforts and develops strategic campaigns, events, and messaging that connect with audiences and helps organizations advance their missions in high stakes environments.

Craig Harmon

Craig is the Director of Partner Management at Babel Street. He leads the company’s strategic alliance with Amazon Web Services (AWS). A former Senior Partner Account Manager at AWS, Craig brings a hyperscaler‑native perspective to building and scaling partnerships that drive revenue growth and deepen technical collaboration. He is passionate about operational excellence and the design of high‑performance partner models that translate cloud innovation into measurable outcomes for customers and partners.

Streamlining access to powerful disaster recovery capabilities of AWS

Post Syndicated from Jennifer Moran original https://aws.amazon.com/blogs/architecture/streamlining-access-to-powerful-disaster-recovery-capabilities-of-aws/

Learn how you can use AWS services like AWS Backup and AWS Elastic Disaster Recovery (AWS DRS), along with AWS Resilience Competency Partner solutions like Arpio to implement powerful and comprehensive Disaster Recovery solutions.

Resilience is the ability of your application to keep running even when “bad stuff” happens. A critical part of your resilience strategy is Disaster Recovery (DR). DR is what protects you against less frequent, but bigger faults like natural disasters, technical faults, and bad actors. To maintain critical business continuity, disaster recovery requires recovering your workload to a new site, such as a different AWS Region or AWS account.

AWS provides powerful tools for all aspects of resilience. However, achieving a comprehensive Disaster Recovery solution for your cloud workloads using native AWS services requires planning and engineering effort. This is because, as the Shared Responsibility Model for Resiliency states, resilience is a shared responsibility between AWS and the customer. This blog post will help you understand your responsibilities and show you how to reduce the work required to access the powerful DR capabilities of AWS.

In this blog post, we take a building blocks approach. Starting with the tools like AWS Backup to protect your data, we then add protection for Amazon Elastic Compute Cloud (Amazon EC2) compute using AWS Elastic Disaster Recovery (AWS DRS). Finally, we show how to use the full capabilities of AWS to restore your entire workload—data, infrastructure, networking, and configuration, using Arpio disaster recovery automation.

Your recovery site

For DR, your recovery site is usually going to be a different AWS Region (cross-Region) or a different AWS account (cross-account) than where your workload runs.

Cross-Region backup and recovery are essential for disaster recovery. This helps to keep your workloads protected if an event causes your source Region to be unable to run your workload. AWS Regions are strong fault isolation boundaries, so the event in your source is highly unlikely to affect your recovery Region.

Cross-account backup is a critical security measure to enable recovery from malware and ransomware. By storing copies of your data in a separate clean room recovery account with distinct credentials, you create an isolated environment that can’t be accessed, even if the source account is compromised.

Protecting your data

We start with your data—your data is the foundation of your workload.

Each AWS data storage resource offers the ability to back up or replicate your data. For example, Amazon Elastic Block Store (Amazon EBS) snapshots, Amazon Relational Database Service (Amazon RDS) for Db2) Backups, and Amazon Simple Storage Service (Amazon S3) replication, offer data protection for Amazon EBS volumes, Amazon RDS Databases instances, and Amazon S3 Buckets respectively. Figure 1, for example, illustrates the several methods and destinations of backup and replication for Amazon RDS.

AWS Backup and replication for Amazon RDS

Figure 1. AWS Backup and replication for Amazon RDS

AWS Backup takes this further, tying together many of these disparate backup technologies, giving a single plane of glass to configure data backup plans across resources. AWS Backup also added backup capabilities for AWS resources that previously didn’t have them such as Amazon Elastic File System (Amazon EFS) and Amazon FSx. It also provides the ability to back up your data to a different AWS Region or AWS account. It even enabled cross-Region backup for services like Amazon DynamoDB, which previously didn’t have that capability.

AWS Backup is a powerful tool for protecting your data. With your data protected, you will then need additional automation to get to a fully recovered workload. If you want to build this yourself, AWS offers the tools to do this. In this prior blog post on Backup and Restore, we go more into detail about adding automation using Amazon EventBridge and AWS Lambda functions for automated recovery. For more information, see figures 6 and 7.

With its ability to create vaults for secure storage, define policies for governance, and set schedules for automation, AWS Backup centralizes and streamlines the backup process. Instead of managing backups service by service, you can enforce consistent protection across resources, reduce manual effort, and streamline recovery when it matters most.

Protecting your Amazon EC2 compute

As important as data is, only restoring your data isn’t a complete solution to recovering from disaster. You must also restore your compute resources.

For static Amazon EC2 instances, you can create snapshots of your instances as Amazon Machine Images (AMIs), or use AWS Backup to manage this for you. By static instances, we mean those you create directly and maintain, as opposed to those created by Amazon EC2 Auto Scaling. Such a strategy can deliver a Recovery Points Objective (RPO) and Recovery Time Objective (RTO) of minutes to hours. The size (and growth in size) of your EC2 instances determines the time to back them up. Their size and launch time determines the time to restore them.

If you need real-time RPO (near-zero data loss) and RTO (recovery) in minutes or less, then AWS DRS is the solution here. AWS DRS provides a nearly continuous block-level replication, recovery orchestration, and automated server conversion capabilities. With these, you to achieve a crash-consistent recovery point objective of seconds, and a recovery time objective typically ranging between 5–20 minutes. You can also use AWS DRS to configure your recovery Amazon Virtual Private Cloud (Amazon VPC). So, with the right settings, you can get your EC2 networking to look like your primary environment.

Protecting everything in your entire workload

Restoring data and static EC2 instances is only part of the disaster recovery solution that you need. Modern workloads often rely on a broader range of compute services, including EC2 Auto Scaling, AWS Lambda, Amazon ECS, and Amazon Elastic Kubernetes Service (Amazon EKS). For ECS and EKS, you can run on EC2 instances or go serverless with AWS Fargate. You will need a solution that can restore either of these.

The challenge with these services is making sure that they are recreated with the right configuration and metadata. For example, EC2 instance types and volume sizes, EC2 user data, or AWS Lambda function code, and not everything here is stateless. Both ECS and EKS can rely on persistent Amazon EBS volumes or Amazon Elastic File System (Amazon EFS). In those cases, recovery requires restoring the data and reattaching volumes restored from backup to the correct ECS tasks or EKS pods.

You can build automation to do all of this, or you can rely on an AWS Resilience Software Competency Partner solution to take care of this for you. Arpio is a software as a service (SaaS) product focused on discovering and backing up everything it takes to run your workload on AWS, and recovering it cross-Region and cross-account as a fully functional workload.

Figure 2 illustrates how AWS tools (left) establish a powerful foundation for robust workload recovery. Beyond these foundational building blocks, full recovery requires additional resources (right), including AWS compute options as discussed. Furthermore, complex networking (potentially spanning VPCs and accounts), infrastructure, and IAM principals are critical. Arpio uses and extends AWS Backup, AWS DRS, and other AWS service capabilities to back up and restore a functional AWS workload, including all its necessary components. This unburdens you from the undifferentiated heavy lifting of building your own automation. You still have responsibilities in the shared responsibility model, but Arpio takes on most of the work of getting you backed up and recovered.

AWS tools on the left provide a powerful foundation for full recovery on the right

Figure 2. AWS tools on the left provide a powerful foundation for full recovery on the right

Even with data, compute, networking, infrastructure, and IAM principals restored, there is another requirement to achieve full recovery: translation of your configuration. For example, an application that accesses the Amazon RDS database requires configuration information about the DB endpoint and credentials. When restoring your RDS instance into your recovery environment, it will have a new endpoint. Arpio addresses this using a two-fold strategy. First Arpio will find all references to the early database endpoint name and translate them to the new database endpoint name. Next, Arpio will also create an Amazon Route 53 private hosted zone in the recovered VPC, mapping the early endpoint to the new one using a CNAME record. This way, applications still using the early name still connect to the newly recovered database. Arpio also securely backs up the credentials in your recovery account, for every database backup taken, ready to be recovered for the point in time that you recover your database from. Figure 3 shows how your recovered application can seamlessly access your restored database.

Arpio automation ensures your applications can access your restored database in the recovery environment

Figure 3. Arpio automation ensures your applications can access your restored database in the recovery environment

Figure 4 shows a sample AWS workload protected by Arpio. In the standby state, you can see how Arpio is coordinating multiple AWS services. When a disaster or ransomware event occurs, you can launch a recovery. This will create a fully recovered workload as seen in the recovery stage on the right.

Sample AWS workload protected by Arpio

Figure 4. Sample AWS workload protected by Arpio

Arpio does all of this in your accounts, on your behalf. To enable this, Arpio applies AWS Well-Architected Tool (AWS WA Tool) best practices for security, using only IAM roles with least-privilege permissions. For example, the IAM role used to access your source AWS account is incapable of changing or mutating your source workload and is explicitly denied from reading or exfiltrating any data.

With its ability to back up over 140 AWS resources and restore them as fully functioning AWS workloads in a cross-Region cross-account recovery environment, Arpio builds on top of the powerful AWS tooling to streamline your complete workload recovery.

Conclusion

Disaster recovery (DR) is essential for a robust resilience strategy. By using the powerful tools offered by AWS and complementing them with AWS Resilience Competency Partner solutions like Arpio, organizations can significantly streamline access to comprehensive and powerful disaster recovery capabilities for their AWS workloads.


About the authors

Amazon threat intelligence teams identify Interlock ransomware campaign targeting enterprise firewalls

Post Syndicated from CJ Moses original https://aws.amazon.com/blogs/security/amazon-threat-intelligence-teams-identify-interlock-ransomware-campaign-targeting-enterprise-firewalls/

Amazon threat intelligence has identified an active Interlock ransomware campaign exploiting CVE-2026-20131, a critical vulnerability in Cisco Secure Firewall Management Center (FMC) Software that could allow an unauthenticated, remote attacker to execute arbitrary Java code as root on an affected device, which was disclosed by Cisco on March 4, 2026.

After Cisco’s disclosure, Amazon threat intelligence began research into this vulnerability using Amazon MadPot’s global sensor network—a system of honeypot servers that attract and monitor cybercriminal activity. While looking for any current or past exploits of this vulnerability, our research found that Interlock was exploiting this vulnerability 36 days before its public disclosure, beginning January 26, 2026. This wasn’t just another vulnerability exploit, Interlock had a zero-day in their hands, giving them a week’s head start to compromise organizations before defenders even knew to look. Upon making this discovery, we shared our findings with Cisco to help support their investigation and protect customers.

A misconfigured infrastructure server—essentially, a poorly secured staging area used by the attackers—exposed Interlock’s complete operational toolkit. This rare mistake provided Amazon’s security teams with visibility into the ransomware group’s multi-stage attack chain, custom remote access trojans (backdoor programs that give attackers control of compromised systems), reconnaissance scripts (automated tools for mapping victim networks), and evasion techniques.

AWS infrastructure and customer workloads on AWS were not observed to be involved in this campaign. This advisory shares comprehensive technical analysis and indicators of compromise to help organizations identify potential compromise and defend against Interlock’s operations. Organizations running Cisco Secure Firewall Management Center should immediately apply Cisco’s security patches and review the indicators provided below.

Discovery and investigation timeline

Amazon threat intelligence identified threat activity potentially related to CVE-2026-20131 beginning January 26, 2026, predating the public disclosure. Observed activity involved HTTP requests to a specific path in the affected software. Request bodies contained Java code execution attempts and two embedded URLs: one used to deliver configuration data supporting the exploit, and another designed to confirm successful exploitation by causing a vulnerable target to perform an HTTP PUT request and upload a generated file. Multiple variations of these URLs were observed across different exploit attempts.

To advance the investigation and obtain additional threat intelligence, we performed the expected HTTP PUT request with the anticipated file content—essentially, we pretended to be a successfully compromised system. This successfully prompted Interlock to proceed to the next stage, issuing commands to fetch and execute a malicious ELF binary (a Linux executable file) from a remote server.

When analysts retrieved the binary, they discovered the same host (attacker-controlled server) is used for distributing Interlock’s entire operational toolkit. The exposed infrastructure organized artifacts into separate paths corresponding to individual targets, with the same paths used for both downloading tools to compromised hosts and uploading operational artifacts back to the staging server.

Attribution to Interlock ransomware

The ELF binary and associated artifacts are attributable to the Interlock ransomware family based on convergent technical and operational indicators. The embedded ransom note and TOR negotiation portal are consistent with Interlock’s established branding and infrastructure. The ransom note’s invocation of multiple data protection regulations reflects Interlock’s documented practice of citing regulatory exposure to pressure victims, essentially threatening organizations not just with data encryption, but with regulatory fines and compliance violations. The campaign-specific organization identifier embedded in the note aligns with Interlock’s per-victim tracking model.

Interlock has historically targeted specific sectors where operational disruption creates maximum pressure for payment. Education represents the largest share of their activity, followed by engineering, architecture, and construction firms, manufacturing and industrial organizations, healthcare providers, and government and public sector entities.

Temporal analysis performed on timestamps from observed threat activities, artifacts stored on the misconfigured infrastructure server, and metadata embedded within recovered threat artifacts indicates the actor most likely operates in UTC+3 with 75–80% confidence. Systematic analysis across all UTC offsets showed UTC+3 produced the best fit: first activity around 08:30, peak activity between 12:00 and 18:00, and a probable sleep window of 00:30–08:30.

Interlock ransomware negotiation portal where victims enter their organization ID and email address to receive an auth token to begin a negotiation chat session.

Figure 1: Interlock ransomware negotiation portal where victims enter their organization ID and email address to receive an auth token to begin a negotiation chat session.

Technical analysis: Interlock’s operational toolkit

Post-compromise reconnaissance script

Once Interlock gains initial access, they use a variety of priority tools to complete their attack. Amazon threat intelligence teams recovered a PowerShell script designed for systematic Windows environment enumeration (automated information gathering about the victim’s network). The script collects operating system and hardware details, running services, installed software, storage configuration, Hyper-V virtual machine inventory, user file listings across Desktop, Documents, and Downloads directories, browser artifacts from Chrome, Edge, Firefox, Internet Explorer, and 360 browser (including history, bookmarks, stored credentials, and extensions), active network connections correlated with responsible processes, ARP tables, iSCSI session data, and RDP authentication events from Windows event logs.

The script stages results to a centralized network share (\JK-DC2\Temp) using each system’s fully qualified hostname to create dedicated directories—essentially creating a folder for each compromised computer. Following collection, it compresses data into ZIP archives named after each hostname and removes original raw data. This structured per-host output format indicates the script operates across multiple machines within a network—a hallmark of ransomware intrusion chains that prepare for organization-wide encryption.

Custom remote access trojans

Remote access trojans (RATs) are malicious programs that give attackers persistent control over compromised systems, functioning like unauthorized remote desktop software.

JavaScript implant: Amazon threat intelligence recovered an obfuscated JavaScript remote access trojan that suppresses debugging output by overriding browser console methods (hiding its activity from basic detection tools). On execution, it profiles the infected host using PowerShell and Windows Management Instrumentation (WMI), collecting system identity, domain membership, username, OS version, and privilege context before transmitting this data during an encrypted initialization handshake.

Command-and-control communication occurs over persistent WebSocket connections with RC4-encrypted messages using per-message 16-byte random keys embedded in packet headers—essentially, each message uses a different encryption key, making interception more difficult. The implant cycles through multiple operator-controlled hostnames and IP addresses in randomized order with exponential backoff between reconnection attempts.

The implant provides interactive shell access, arbitrary command execution, bidirectional file transfer, and SOCKS5 proxy capability for tunneling TCP traffic (routing malicious traffic through other systems to hide its origin). Self-update and self-delete capabilities allow operators to replace or remove the implant without reinfection, supporting operational cleanup to hinder forensic investigation.

Java implant: A functionally equivalent client implemented in Java provides identical command-and-control capabilities. Built on GlassFish ecosystem libraries, it uses Grizzly for non-blocking I/O transport and Tyrus for WebSocket protocol communication. In simpler terms, Interlock built the same backdoor in two different programming languages, ensuring they maintain access even if defenders detect one version.

Infrastructure laundering script

Sophisticated threat actors don’t attack from their own infrastructure, they build disposable relay networks to hide their tracks. Amazon threat intelligence teams identified a Bash script that configures Linux servers as HTTP reverse proxies (intermediary servers that forward traffic to hide the attacker’s true location). The script performs system updates, installs fail2ban with SSH brute-force protection, and compiles HAProxy 3.1.2 from source. The HAProxy instance listens on port 80 and forwards all inbound HTTP traffic to a hardcoded target IP, with systemd ensuring persistence across reboots.

A notable component is a log erasure routine running as a cron job every five minutes. The routine truncates all *.log files under /var/log and suppresses shell history by unsetting the HISTFILE variable. This aggressive evidence destruction, wiping logs every five minutes, combined with the purpose-built HTTP forwarding proxy, indicates the script establishes disposable traffic-laundering relay nodes. These nodes obscure exploit traffic origin, relay command-and-control communications, or proxy data exfiltration, making it nearly impossible to trace attacks back to their source.

Memory-resident webshell

Amazon threat intelligence teams observed a Java class file delivered as an alternative to the ELF binary drop. When loaded by the Java Virtual Machine (JVM), its static initializer registers a ServletRequestListener with the server’s StandardContext, essentially installing a persistent memory-resident backdoor that intercepts HTTP requests without writing files to disk. This “fileless” approach evades traditional antivirus scanning that looks for malicious files.

The listener inspects incoming requests for specially crafted parameters containing encrypted command payloads. Payloads are decrypted using AES-128 with a key derived from the MD5 hash of the hardcoded seed “geckoformboundary99fec155ea301140cbe26faf55ed2f40″ (using the first 16 characters: 09b1a8422e8faed0). Decrypted payloads are treated as compiled Java bytecode, dynamically loaded into the JVM, and executed—a technique designed to evade file-based detection by running malicious code entirely in memory.

Connectivity verification tool

Amazon threat intelligence teams recovered Java class files implementing a basic TCP server listening on port 45588 (encoded as Unicode character 넔 to obscure the port number from static analysis). The server accepts connections, logs connecting IP addresses, sends a greeting message, and immediately closes connections. This operational profile is consistent with a lightweight network beacon—essentially a “phone home” tool used to verify successful code execution or confirm network port reachability following initial exploitation.

Legitimate tool abuse

Interlock deployed ConnectWise ScreenConnect, a legitimate commercial remote desktop tool, alongside custom implants. When ransomware operators deploy legitimate remote access tools alongside their custom malware, they’re buying insurance—if defenders find and remove one backdoor, they still have another way in. This indicates multiple redundant remote access mechanisms—a pattern consistent with ransomware operators seeking to maintain access even if individual footholds are removed. The tool’s legitimate network footprint helps blend with authorized remote administration traffic, making detection more challenging.

Amazon threat intelligence teams also recovered Volatility, an open-source memory forensics framework typically used by incident responders (the same tool defenders use to investigate attacks). While no artifacts indicated automated use, its presence alongside custom implants and reconnaissance scripts is consistent with advanced threat operations. Both ransomware groups and nation-state actors have been observed deploying Volatility during intrusions. The tool’s focus on parsing memory dumps provides access to sensitive data such as credentials stored in RAM, which can enable lateral movement (spreading through the network) and deeper environment compromise in support of ransom operations or espionage objectives.

Interlock also used Certify, an open source offensive security tool designed to exploit misconfigurations in Active Directory Certificate Services (AD CS). For ransomware operators, Certify provides a pathway to identify vulnerable certificate templates and enrollment permissions that allow requesting authentication-capable certificates. These certificates can be used to impersonate users, escalate privileges, or maintain persistent access. These capabilities directly support both initial compromise and long-term persistence objectives in ransomware operations.

Indicators of compromise (IoCs)

The following indicators support defensive measures by organizations that may be affected. Due to Interlock’s use of content variation techniques, most file hashes are not included as reliable indicators. The threat actor modified most artifacts like scripts and binaries downloaded to different targets. This resulted in different file hashes for functionally identical tools. The customization allowed each attack to evade signature-based detection that looks for exact file matches.

206.251.239[.]164

Exploit source IP

Active Jan 2026

199.217.98[.]153

Exploit source IP

Active Mar 2026

89.46.237[.]33

Exploit source IP

Active Mar 2026

Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:136.0) Gecko/20100101 Firefox/136.0

Exploit HTTP User-Agent

Observed Jan 2026 and Mar 2026

b885946e72ad51dca6c70abc2f773506

Exploit TLS JA3

Observed Jan 2026 and Mar 2026

f80d3d09f61892c5846c854dd84ac403

Exploit TLS JA3

Observed Mar 2026

t13i1811h1_85036bcba153_b26ce05bbdd6

Exploit TLS JA4

Observed Jan 2026 and Mar 2026

t13i4311h1_c7886603b240_b26ce05bbdd6

Exploit TLS JA4

Observed Mar 2026

144.172.94[.]59

C2 Fallback IP

Active Mar 2026

199.217.99[.]121

C2 Fallback IP

Active Mar 2026

188.245.41[.]78

C2 Fallback IP

Active Mar 2026

144.172.110[.]106

Backend C2 IP

Active Mar 2026

95.217.22[.]175

Backend C2 IP

Active Mar 2026

37.27.244[.]222

Staging host IP

Active Mar 2026

hxxp://ebhmkoohccl45qesdbvrjqtyro2hmhkmh6vkyfyjjzfllm3ix72aqaid[.]onion/chat.php

Ransom negotiation portal

Active Mar 2026

cherryberry[.]click

Exploit Support Domain

Active Jan 2026

ms-server-default[.]com

Exploit Support Domain

Active Mar 2026

initialize-configs[.]com

Exploit Support Domain

Active Mar 2026

ms-global.first-update-server[.]com

Exploit Support Domain

Active Mar 2026

ms-sql-auth[.]com

Exploit Support Domain

Active Mar 2026

kolonialeru[.]com

Exploit Support Domain

Active Mar 2026

sclair.it[.]com

Exploit Support Domain

Active Mar 2026

browser-updater[.]com

C2 domain

Active Mar 2026

browser-updater[.]live

C2 domain

Active Mar 2026

os-update-server[.]com

C2 domain

Active Mar 2026

os-update-server[.]org

C2 domain

Active Mar 2026

os-update-server[.]live

C2 domain

Active Mar 2026

os-update-server[.]top

C2 domain

Active Mar 2026

d1caa376cb45b6a1eb3a45c5633c5ef75f7466b8601ed72c8022a8b3f6c1f3be

Offensive security tool (Certify)

Observed Mar 2026

6c8efbcef3af80a574cb2aa2224c145bb2e37c2f3d3f091571708288ceb22d5f

Screen locker

Observed Mar 2026

Defensive recommendations

Organizations should take the following actions to protect against Interlock ransomware operations.

Immediate actions:

  • Apply Cisco’s security patches for Cisco Secure Firewall Management Center
  • Review logs for the indicators of compromise listed above
  • Conduct security assessments to identify potential compromise
  • Review ScreenConnect deployments for unauthorized installations

Detection opportunities:

  • Monitor for PowerShell scripts staging data to network shares with hostname-based directory structures
  • Detect Java ServletRequestListener registrations in web application contexts (unusual modifications to Java web applications)
  • Identify HAProxy installations with aggressive log deletion cron jobs (proxy servers that erase their own logs every five minutes)
  • Watch for TCP connections to unusual high-numbered ports (e.g., 45588)

Long-term measures:

  • Implement defense-in-depth strategies with multiple layers of security controls
  • Maintain continuous threat monitoring and hunting capabilities
  • Ensure comprehensive logging with secure, centralized log storage (stored separately from systems that could be compromised)
  • Regularly test incident response procedures for ransomware scenarios
  • Educate security teams on Interlock’s tactics, techniques, and procedures

The real story here isn’t just about one vulnerability or one ransomware group—it’s about the fundamental challenge zero-day exploits pose to every security model. When attackers exploit vulnerabilities before patches exist, even the most diligent patching programs can’t protect you in that critical window. This is precisely why defense in depth is essential—layered security controls provide protection when any single control fails or hasn’t yet been deployed. Rapid patching remains foundational in vulnerability management, but defense in depth helps organizations not to be defenseless during the window between exploit and patch.

Amazon Threat Intelligence teams continue to monitor Interlock ransomware operations and will provide updates as additional information becomes available. The intelligence gathered from this campaign is being integrated into AWS security services to protect customers proactively.


If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, contact AWS Support.

CJ Moses

CJ Moses

CJ Moses is the CISO of Amazon Integrated Security. In his role, CJ leads security engineering and operations across Amazon. His mission is to enable Amazon businesses by making the benefits of security the path of least resistance. CJ joined Amazon in December 2007, holding various roles including Consumer CISO, and most recently AWS CISO, before becoming CISO of Amazon Integrated Security September of 2023.

Prior to joining Amazon, CJ led the technical analysis of computer and network intrusion efforts at the Federal Bureau of Investigation’s Cyber Division. CJ also served as a Special Agent with the Air Force Office of Special Investigations (AFOSI). CJ led several computer intrusion investigations seen as foundational to the security industry today.

CJ holds degrees in Computer Science and Criminal Justice, and is an active SRO GT America GT2 race car driver.

AWS Security Hub Extended offers full-stack enterprise security with curated partner solutions

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/aws-security-hub-extended-offers-full-stack-enterprise-security-with-curated-partner-solutions/

At re:Invent 2025, we introduced a completely re-imagined AWS Security Hub that unifies AWS security services, including Amazon GuardDuty and Amazon Inspector into a single experience. This unified experience automatically and continuously analyzes security findings in combination to help you prioritize and respond to your critical security risks.

Today, we’re announcing AWS Security Hub Extended, a plan of Security Hub that simplifies how you procure, deploy, and integrate a full-stack enterprise security solution across endpoint, identity, email, network, data, browser, cloud, AI, and security operations. With the Extended plan, you can expand your security portfolio beyond AWS to help protect your enterprise estate through a curated selection of AWS Partner solutions, including 7AI, Britive, CrowdStrike, Cyera, Island, Noma, Okta, Oligo, Opti, Proofpoint, SailPoint, Splunk, a Cisco company, Upwind, and Zscaler.

With AWS as the seller of record, you benefit from pre-negotiated pay-as-you-go pricing, a single bill, and no long-term commitments. You can also get unified security operations experience within Security Hub and unified Level 1 support for AWS Enterprise Support customers. You told us that managing multiple procurement cycles and vendor negotiations was creating unnecessary complexity, costing you time and resources. In response, we’ve curated these partner offerings for you to establish more comprehensive protection across your entire technology stack through a single, simplified experience.

Security findings from all participating solutions are emitted in the Open Cybersecurity Schema Framework (OCSF) schema and automatically aggregated in AWS Security Hub. With the Extended plan, you can combine AWS and partner security solutions to quickly identify and respond to risks that span boundaries.

The Security Hub Extended plan in action
You can access the partner solutions directly within the Security Hub console by selecting Extended plan under the Management menu. From there, you can review and deploy any combination of curated and partner offerings.

You can review details of each partner offering directly in the Security Hub console and subscribe. When you subscribe, you’ll be directed to an automated on-boarding experience from each partner. Once onboarded, consumption-based metering is automatic and you are billed monthly as part of your Security Hub bill.

Security findings from all solutions are automatically consolidated in AWS Security Hub. This gives you immediate and direct access to all security findings in normalized OCSF schema.

To learn more about how to enhance your security posture with these integrations for AWS Security Hub, visit the AWS Security Hub User Guide.

Now available
The AWS Security Hub Extended plan is now generally available in all AWS commercial Regions where Security Hub is available. You can use flexible pay-as-you-go or flat-rate pricing—no upfront investments or long-term commitments required. For more information about pricing, visit the AWS Security Hub pricing page.

Give it a try today in the Security Hub console and send feedback to AWS re:Post for Security Hub or through your usual AWS Support contacts.

Channy

Digital Transformation at Santander: How Platform Engineering is Revolutionizing Cloud Infrastructure

Post Syndicated from Julio Bando original https://aws.amazon.com/blogs/architecture/digital-transformation-at-santander-how-platform-engineering-is-revolutionizing-cloud-infrastructure/

This post is cowritten by Julio Bando from Santander.

Santander faced a significant technical challenge in managing an infrastructure that processes billions of daily transactions across more than 200 critical systems. The expansion into diverse financial services, including investment banking, wealth management, insurance, and payment solutions, had created unprecedented technological complexity, requiring a robust, agile, and scalable infrastructure solution. This raised two main issues. Santander needed to ensure that provisioned services followed established architecture definitions, and they needed to reduce infrastructure provisioning time, which took up to 90 days. This situation demanded intensive operational effort. The solution emerged through an innovative platform engineering initiative called Catalyst, which transformed the bank’s cloud infrastructure and development management. This post analyzes the main cases, benefits, and results obtained with this initiative.

The Catalyst solution

Santander is a global financial services company present in more than 10 countries, with over 160 million customers worldwide. They conceived Catalyst in conjunction with the Platform Strategy Program (PSP), an Amazon Web Services (AWS) program specialized in infrastructure platform design. Implemented through a partnership between AWS Professional Services and Santander, the platform was designed to abstract infrastructure provisioning complexity, standardize architectural compliance, and create a framework that enables new technologies in the bank.

The platform’s in-house frontend was developed as an intuitive developer portal, offering a unified interface for all provisioning and resource management needs. At the platform’s core is the control plane cluster, based on Amazon Elastic Kubernetes Service (Amazon EKS). This cluster is the brain of the operation, orchestrating all components and workflows. Within the cluster, Crossplane plays a fundamental role, acting as a universal resource provisioner that Santander uses to manage resources across multiple cloud providers consistently and declaratively.

The control plane cluster has three components:

  • Data plane claims – Managed by ArgoCD, a continuous delivery tool, the component is responsible for continuous synchronization and deployment of application stacks (integrated sets of cloud resources) and configurations, exploring the GitOps concept.
  • Policies catalog – A central repository of policies ensuring compliance and security across all operations using Open Policy Agent (OPA).
  • Stacks catalog – A library of composite resource definitions and Compositions enabling quick and standardized creation of complex environments.

Santander used this innovative architecture to significantly reduce provisioning time from 90 days to only a few hours and in some cases only minutes. Catalyst brought significant benefits in terms of standardization, security, and governance. The provisioning cycle decreased from 30 days to 2 days, and proof of concept preparation time jumped from 90 days to only 1 hour. The consolidation of over 100 pipelines into a single control plane will further simplify infrastructure management. The following diagram shows the Santander catalyst architecture.

This diagram shows the AWS architecture of Santander's Catalyst platform that provides AI capabilities to teams across the company.

Key platform capabilities

Catalyst’s implementation enabled the creation of strategic workloads demonstrating the platform’s versatility and robustness:

  • Generative AI agents stack – The first success case was implementing a complete stack for AI agents integrating:
  • Modern data platform – One of the most complex workloads implemented through Catalyst was the new data platform, including:
    • Built-in integration with Databricks
    • Data lakes
    • Automated extract, transform, and load (ETL) workflows
    • Integration with centralized data catalog
    • Segregated environments for experimentation. With this implementation, the bank significantly reduces approximately 3,000 monthly tickets related to data experimentation environment provisioning.
  • Cloud process orchestration – creation of a modern process orchestration environment with significant results:
    • Migration of legacy workflows to AWS Step Functions
    • Implementation of retry patterns and error handling
    • Centralized process monitoring

Overall result

This stack reduced AI agent implementation time from 105 days to only 24 hours, eliminating dozens of provisioning tickets per environment. The success of these workloads demonstrates Catalyst’s technical capability and the solution’s versatility in meeting different business needs. Each implementation brought valuable learnings that were incorporated into the platform, creating a virtuous cycle of continuous improvement. The variety of implemented workloads also shows how Catalyst has the potential to be a universal platform, capable of supporting everything from traditional use cases to the most innovative ones involving AI and legacy system modernization. Catalyst’s success wasn’t limited to operational efficiency. The platform also catalyzed a cultural change within Santander, promoting an automation and self-service mindset among development teams. This resulted in faster overall development velocity, more agile teams, and enhanced capability to respond quickly to market changes.

Conclusion

Catalyst represents more than merely a technological tool—it’s a digital transformation enabler that’s redefining cloud development standards at the bank. With the platform, Santander addressed the challenges of a scaled environment and established a solid foundation for continuous innovation and future growth.

With these practical cases, Santander proves that investment in platform engineering solves technical problems and enables new business possibilities, keeping the bank at the forefront of digital transformation in the financial sector.


About the authors

AWS Partner Central now available in AWS Management Console

Post Syndicated from Sébastien Stormacq original https://aws.amazon.com/blogs/aws/aws-partner-central-now-available-in-aws-management-console/

Today, we’re announcing that AWS Partner Central is now available directly in the AWS Management Console, creating a unified experience that transforms how you engage with AWS as both customers and AWS Partners.

As someone who has worked with countless AWS customers over the years, I’ve observed how organizations evolve in their AWS journey. Many of our most successful Partners began as AWS customers—first using our services to build their own infrastructure and solutions, then expanding to create offerings for others. Seeing this natural progression from customer to Partner, we recognized an opportunity to streamline these traditionally separate experiences into one unified journey.

As AWS evolved, so did the needs of our Partner community. Organizations today operate in multiple capacities: using AWS services for their own infrastructure while simultaneously building and delivering solutions for their customers. Modern businesses need streamlined workflows that support their growth from AWS customer to Partner to AWS Marketplace Seller, with enterprise-grade security features that match how they actually work with AWS today.

A new unified console experience
The integration of AWS Partner Central into the Console represents a fundamental shift in partnership accessibility. For existing AWS customers, such as you, becoming an AWS Partner is now as clear as accessing any other AWS service. The familiar console interface provides direct access to partnership opportunities, program benefits, and AWS Marketplace capabilities without needing separate logins or navigation between different systems.

Getting started as an AWS Partner now takes only a few clicks within your existing console environment. You can discover partnership opportunities, understand program requirements, and begin your Partner journey without leaving the AWS interface you already know and trust.

The console integration creates an intuitive pathway for existing customers to transition into AWS Marketplace Sellers. You can now access AWS Marketplace Seller capabilities alongside your existing AWS services, managing both your infrastructure and AWS Marketplace business from a single interface. Private offer requests and negotiations can be managed directly within AWS Partner Central, and you can manage your AWS Marketplace listings alongside your other AWS activities through streamlined workflows.

Becoming an AWS Partner
The unified console experience provides access to comprehensive partnership benefits designed to accelerate your business growth.

Join the AWS Partner Network (APN) and complete your Partner and AWS Marketplace Seller requirements seamlessly within the same interface. Enroll in Partner Paths that align with your customer solutions to build, market, list, and sell in AWS Marketplace while growing alongside AWS. When you are established, use the Partner programs to differentiate your solution, list in AWS Marketplace to improve your go-to-market discoverability, and build AWS expertise through certifications to drive profitability by capturing new revenue streams. Scale your business by selling or reselling software and professional services in AWS Marketplace, helping you accelerate deals, boost revenue, and expand your customer reach to new geographies, industries, and segments.

Throughout your journey, you can continue using Amazon Q in the console, which provides personalized guidance through AWS Partner Assistant.

Let’s see the new Partner Central console
The new AWS Partner Central is accessible like any other AWS service from the console. Among many new capabilities, it provides four key sections that support Partner operations and business growth within the AWS Partner Network:

1. It helps you sell your solutions

AWS Partner Central - Solutions

You can create and publish solutions that address specific customer needs through AWS Marketplace. Solutions are made up of products such as software as a service (SaaS), Amazon Machine Images (AMI), containers, professional services, AI agents and tools, and more. The solutions management capability guides you through building offerings that include both products you own and those you are authorized to resell. You can craft compelling value propositions and descriptions that clearly communicate your solution benefits to potential buyers browsing AWS Marketplace.

I choose Create solution to start listing a new solution in the AWS Marketplace, as shown in the following figure.

AWS Partner Central - Create solution

2. It helps you update and manage your Partner profile

AWS Partner Central - Manage profile

Your Partner profile showcases your organization’s expertise and capabilities to the AWS community. You control how your business appears to potential customers and Partners by highlighting the industry segments you serve and describing your primary products or services. Profile visibility settings provide you with the option to choose whether your information is public or private.

3. It helps you track opportunities

AWS Partner Central - Track Opportunities

You can manage your pipeline of AWS customers, supporting joint collaborations with AWS on customer engagements. You monitor these prospects using clear status indicators: approved, rejected, draft, and pending approval. The opportunity dashboard shows stages, estimated AWS Monthly Recurring Revenue, and other key metrics that help you understand your pipeline. You can create more opportunities directly within the console and export data for your own reporting and analysis.

4. It provides you with the ability to discover and connect with other Partners

After becoming an AWS Partner, you get access to the AWS Partners network, where you can search for other Partners. You can connect with them to collaborate on sales opportunities and expand your customer outreach.

AWS Partner Central - Discover and Search for partners

You search through available Partners using filters for industry, location, Partner program type, and specialization. The centralized dashboard shows your active connections, pending requests, and connection history, so that you can manage business relationships and identify collaboration opportunities that can expand your reach. Like all other AWS services, these Partner connection capabilities are now available as APIs, which provide automation and integration into your existing workflows.

AWS Partner Central - Manage contact requests

These capabilities work together within the new AWS Partner Central console, accessible directly from the console, helping you transition from AWS customer to successful Partner with enterprise-grade security and streamlined workflows.

The technical foundation: Migrating the identity system
This unified console experience is made possible by our migration to a modern identity system built on AWS Identity and Access Management (IAM). We’ve transitioned from legacy identity infrastructure to IAM Identity Center, providing enterprise-grade security capabilities including single sign-on capabilities and multi-factor authentication. With security as job zero, this migration provides new and existing Partners with the possibility to connect their own identity providers to AWS Partner Central. It provides seamless integration with existing enterprise authentication systems while removing the complexity of managing separate credentials across different services.

One more thing
APIs are the core of what we do at AWS, and AWS Partner Central is no different. You can automate and streamline your co-sell workflows by connecting your business tools to AWS Partner Central. The APIs offered by AWS Partner Central help you accelerate APN benefits—from Account Management (Account API) and Solution Management (Solution API) to co-selling with Opportunity and Leads APIs, and Benefits APIs for faster benefit activation.

You can use these APIs to engage with AWS and grow your Partner business from your own CRM tools.

Get started today
This integration between the console and AWS Partner Central reflects our commitment to reducing complexity and improving the Partner experience. We’re bringing AWS Partner Central into the console to create a more intuitive path for organizations to grow with AWS from initial customer adoption through to full partnership engagement and AWS Marketplace success.

Your journey from AWS customer to successful AWS Partner and AWS Marketplace Seller starts with a few clicks in your console. I encourage you to explore the new unified experience today and discover how AWS Partner Central in the console can accelerate your organization’s growth and success within the AWS community.

Ready to get started? Visit AWS Partner Central in your console to learn more about the AWS Partner Network and discover the partnership path that’s right for your organization.

— seb

Modernization of real-time payment orchestration on AWS

Post Syndicated from Neeraj Kaushik original https://aws.amazon.com/blogs/architecture/modernization-of-real-time-payment-orchestration-on-aws/

The global real-time payments market is experiencing significant growth. According to Fortune Business Insights, the market was valued at USD 24.91 billion in 2024 and is projected to grow to USD 284.49 billion by 2032, with a CAGR of 35.4%. Similarly, Grand View Research reports that the global mobile payment market, valued at USD 88.50 billion in 2024, is expected to grow at a CAGR of 38.0% from 2025 to 2030. (Disclaimer: Third-party market research and statistics are provided for informational purposed only. AWS and IBM make no representations about the accuracy of this information.)

This rapid expansion underscores the urgency for financial institutions to modernize their payment processing infrastructure. Financial institutions often need to process high volume of transactions with near-zero latency to meet stringent service level agreements (SLAs) to support surging mobile payments volume.

However, traditional payment orchestration systems, often built on monolithic architectures, struggle to meet these demands due to latency, availability, and scalability challenges. Additionally, their reliance on on-premises infrastructure leads to higher costs and an impediment to innovation, reinforcing the need for modernization.

As sustainability becomes a priority, organizations are turning to cloud-based solutions to optimize infrastructure, reduce carbon footprints, and enhance energy efficiency. This shift provides scalability and performance, and aligns with global sustainability goals, securing the future of real-time payments.

In this post, we discuss the real-time payment orchestration framework. It uses an event-driven architecture and AWS serverless services to enhance the resiliency, efficiency, and scalability of real-time payments. By decomposing payment processing into distinct business capabilities, financial institutions can improve modularity and flexibility. Implementing tenant-based segregation helps with data isolation and security. Additionally, adopting asynchronous communication through Amazon Managed Streaming for Apache Kafka (Amazon MSK) enhances scalability and resilience.

Traditional real-time payment orchestration

Payment orchestration serves as a middleware solution, streamlining transaction processing across multiple payment methods, gateways, and financial institutions. It orchestrates key business functions such as payment authorization, payment processing, settlement and clearing, compliance and risk management, and account management for both inbound and outbound payment flows.

The following diagram depicts the high-level business capabilities supported by payment orchestrators across various payment flows, including real-time payments, digital disbursements, tax payments, wires, and more.

Payment processing system flowchart showing main components from acceptance to billing

Detailed flowchart depicting a payment processing system with multiple components. The diagram shows primary payment types at the top (including Realtime Payments, Digital Disbursement, Credit Transfer, and Peer to Peer Payments) flowing down through core processing stages including Payment Acceptance, Execution, Clearing, Reporting, Tracking, Reversals, and Billing.

Many financial institutions adopt a tenant-based approach organized by geography due to varying clearing processes, localized regulations, and transaction requirements across AWS Regions. However, without proper separation of services, teams often continue to add region-specific logic to existing services, gradually increasing their monolithic complexity and using the same infrastructure for all payment flows.

Traditional payment systems process transactions linearly, with each step waiting for the previous one to complete. However, analysis of payment workflows reveals numerous opportunities for parallel execution:

  • Sanctions screening and fraud detection – Compliance and fraud checks can run simultaneously with initial routing decisions, rather than sequentially blocking all subsequent processing
  • Payment routing and authorization requests – When basic validations are complete, routing and authorization can proceed in parallel rather than one after another
  • Payment execution and ledger updates – The actual payment execution doesn’t need to wait for ledger records to be updated—these can occur concurrently
  • Settlement, reconciliation, and tracking – These post-transaction processes can be initiated independently as soon as the primary transaction is complete

This parallel approach can dramatically improve throughput and reduce latency compared to traditional queue-based systems where operations form a sequential chain that extends processing time and creates bottlenecks.

Most legacy payment orchestration systems rely heavily on on-premises virtual machines (VMs), leading to several challenges:

  • Multi-Region support for disaster recovery and multi-tenancy resulting in significant capital expenditure and operational overhead
  • High latency and SLA issues caused by sequential message processing and delays between globally separated data centers
  • Limited reusability of payment flows as monolithic architectures require region-specific changes for local clearing mechanisms and regulations, increasing complexity and costs
  • Scalability challenges and high memory consumption due to inefficient resource utilization and execution of irrelevant logic across regions
  • Complex cross-border payment routing caused by variations in clearing rules, transaction limits, and local regulations, increasing latency and routing errors
  • Integration challenges with diverse data formats because legacy systems rely on proprietary standards (for example, ISO 20022, SWIFT MT), complicating data conversion and compliance
  • High deployment complexity for new payment flows due to monolithic architectures requiring extensive region-specific modifications, slowing time to market
  • Environmental impact and high carbon footprint from on-premises infrastructure consuming excessive energy, whereas cloud-based approaches improve efficiency

Solution overview

To overcome these challenges, the proposed architecture embraces the following design principles to build a future-ready, real-time payment orchestration solution:

  • Performance at scale – Handling over 1,000 transactions per second (TPS) with consistent low latency under varying load conditions.
  • High availability – Achieving 99.999% uptime to meet the strict requirements of financial transactions.
  • Geographic resilience – Supporting global operations with region-specific compliance while maintaining consistent performance.
  • Cost optimization – Reducing total cost of ownership through efficient resource utilization and serverless technologies.
  • Security and compliance – Supporting data protection and regulatory adherence across different jurisdictions.
  • Operational simplicity – Streamlining deployment, monitoring, and maintenance across the payment ecosystem.
  • Microservices – Decomposing payment processing into distinct business capabilities, so financial institutions can improve modularity and flexibility. This microservices-based approach allows for independent scaling and development of critical components.

The following diagram depicts the high-level solution architecture for real-time payments. The existing channels using synchronous or asynchronous APIs can be modified to use edge-optimized endpoints to reduce latency.

Event-driven payment orchestration system with pub/sub channels connecting multiple payment processing modules

Architecture diagram detailing an AWS-based payment orchestration platform utilizing event-driven principles. Features reusable components across two regions, with dedicated modules for payment initiation, execution, reconciliation, billing, and risk management. Implements pub/sub messaging patterns for inter-component communication and connects to enterprise systems including accounting, compliance, and analytics.

An event-driven architecture is used for payment orchestration, which handles communication through a pub/sub pattern. This architecture maintains persistent connections, improving performance of the end-to-end real-time payment processing.

The event-driven architecture for real-time payment processing allows multiple payment operations to occur simultaneously using different adaptors, as opposed to the traditional systems where payment processes are sequential and flow through a single pipeline. Payment events are distributed to specialized payment processor microservices based on their function (initiation, execution, tracking, settlements), enabling each to process independently without waiting for others to complete.

Because we’re transitioning from sequential processing to distributed, maintaining transaction traceability is crucial. The payment tracking adapters shown in the preceding diagram connect to enterprise analytics systems, creating a specialized layer for monitoring transactions. The pub/sub model allows for attaching correlation IDs to events, enabling systems to track related events across different topics and processing stages.

A standardized event schema serves as the foundation for this architecture, providing consistency across regional deployments while allowing for customization at the adapter level. This schema defines uniform event structures containing tenant-specific metadata and supports versioning to accommodate evolving requirements. By isolating region-specific variations to the adapter layer, the solution maintains core functionality while interfacing with diverse enterprise systems through configuration-driven customization rather than code changes.

For most payment processes, especially those with independent processing steps that can run in parallel, this architecture delivers net performance gains despite the topic switching overhead, particularly for complex transactions where multiple independent validations or processing steps are required.

Deployment on the AWS Cloud

The solution uses edge-optimized Amazon API Gateway for channels. An edge-optimized API endpoint routes requests to the nearest Amazon CloudFront Point of Presence (POP), which can help in cases where your clients are geographically distributed to enable efficient routing within each geographical region, enhancing global responsiveness by minimizing network round trips and making sure requests take the shortest possible path before transitioning from the public internet to the client network.

The following diagram illustrates the high-level solution architecture for real-time payments.

Multi-region AWS payment architecture with managed Kafka topics connecting Lambda microservices and DynamoDB storage

Comprehensive AWS payment orchestration solution implementing modern cloud-native architecture principles. Core processing logic implemented as Lambda functions covering initiation, execution, reconciliation, billing, tracking, risk management, and settlement workflows. Leverages Amazon MSK for reliable event streaming between components, with dedicated Kafka topics for each processing stage. Data persistence handled by Amazon DynamoDB, supporting cross-region operations. Architecture demonstrates AWS best practices for financial services, including regional redundancy, serverless computing, managed services, and event-driven design patterns. System integrates with external banking infrastructure and enterprise systems while maintaining separation of concerns through microservices architecture. Features built-in support for compliance monitoring, risk management, and payment tracking through specialized Lambda functions.

The solution uses Amazon MSK to implement an event-driven architecture that efficiently handles both inbound and outbound channels traffic through API requests and asynchronous message-based events. Amazon MSK communicates using a high-performance binary protocol between producers, consumers, and brokers, providing low latency and high throughput. Real-time payments are logically partitioned across multiple tenants within geographical regions—North America, EMEA, LATAM, and Asia-Pacific.

Each real-time payment tenant follows an active/active disaster recovery strategy by deploying MSK clusters across multiple AWS Regions, designed to achieve high availability and resilience. Amazon MSK offer both serverless and provisioned cluster options. The team can decide to select one or the other depending on the non-functional requirements and team expertise. Amazon MSK automatically manages partition leadership with leaders in primary Regions and followers in secondary Regions. During failover, leaders are re-elected in healthy Regions, designed to help maintain processing capabilities during regional incidents. Sticky partitioning uses consistent hashing for deterministic routing, and cooperative rebalancing enables efficient failover. Multi-AZ deployment provides zone redundancy and isolated clusters per Region for data sovereignty compliance through programmatic AWS Identity and Access Management (IAM) and virtual private cloud (VPC) boundaries.

To support seamless cross-Region replication and maintain message continuity, Amazon MSK Replicator—a fully managed feature of Amazon MSK—is used to replicate topics and synchronize consumer group offsets across clusters. MSK Replicator simplifies the process of building multi-Region Kafka applications by not needing custom code, open-source tool configuration, or infrastructure management. It automatically provisions and scales the necessary resources, so teams can focus on business logic while only paying for the data being replicated. In the event of a regional outage or failover, traffic can be automatically redirected to a healthy Region without data loss or service disruption, providing near-zero Recovery Time Objectives (RTOs) and uninterrupted operations for downstream services such as payment processors and audit trail consumers.

In addition to regional redundancy, the architecture uses an event-driven architecture to enable parallel and decoupled processing of payment transactions. Events such as transaction initiation, validation, and settlement are emitted asynchronously and consumed by various microservices independently, which drastically reduces end-to-end latency.

To process these events at scale, the architecture can use AWS Lambda, Amazon Elastic Container Service (Amazon ECS), or Amazon Elastic Kubernetes Service (Amazon EKS) depending upon non-functional requirements. Automatic scaling responds to Amazon CloudWatch metrics, and exponential backoff retry logic with dead-letter queues (DLQs) handles throttling scenarios. Circuit breakers prevent cascade failures during high error rates.

One of the key benefits of the solution is the reusability of payment flows across different regions. Although each region has its own unique compliance requirements and settlement rules, the core functionalities of real-time payments (payment authorization, payment processing, settlement and clearing) are largely similar. This reusability enables rapid deployment of payment solutions across new regions without rearchitecting the entire system. For example, the real-time payment system in the US and UK might share similar business logic for real-time gross settlement but differ in the clearing and compliance requirements. The solution treats these as bounded contexts within the microservices architecture, providing flexibility while making sure each region can handle its own specific rules and regulations.

Sustainability

AWS relentlessly innovates its infrastructure design, build, and operations to make progress towards net-zero carbon by 2040 and being water positive by 2030. Amazon MSK with AWS Graviton based instances use up to 60% less energy than comparable M5 instances, helping you achieve your sustainability goals. Lambda is inherently sustainable by design. Its serverless model makes sure compute resources are only used when needed, drastically reducing idle infrastructure and wasted energy. Instead of keeping always-on servers for infrequent tasks, Lambda provisions compute power just-in-time, achieving near-zero idle capacity.

Security and compliance in financial services

Given the sensitive nature of payment transactions and financial data, you should apply the security controls required to meet financial regulations such as AWS PCI DSS and AWS Federal Information Processing Standard (FIPS) 140-3 according to your organization’s needs.

The solution should incorporate multi-layered security controls, continuous monitoring, and automated compliance auditing to meet the rigorous expectations of banking regulators and internal risk teams. For more information, refer to Security Guidance.

Conclusion

The modernization of payment orchestration systems using an event-driven architecture and AWS serverless technologies marks a significant advancement in meeting the demands of today’s rapidly evolving financial services landscape. This solution addresses the key challenges faced by traditional payment systems while delivering substantial benefits in performance, scalability, cost optimization, global resilience, sustainability, and compliance. By using cutting-edge cloud technologies and robust security controls, financial institutions can now build a future-ready foundation that adapts to evolving business needs while maintaining the highest standards of performance, security, and reliability. As the real-time payments market continues its explosive growth, this modern architecture provides a solution that meets today’s demands and is also well-positioned to support tomorrow’s payment innovations. Organizations looking to modernize their payment infrastructure can use this blueprint to accelerate their digital transformation journey, supporting sustainable, secure, and efficient payment processing at scale in an increasingly competitive global marketplace.

The architecture presented here is for reference purposes only. IBM will work closely with you to deploy the solution in accordance with industry standards and compliance requirements.For additional resources, refer to:

IBM Consulting is an AWS Premier Tier Services Partner that helps customers who use AWS to harness the power of innovation and drive their business transformation. They are recognized as a Global Systems Integrator (GSI) for over 22 competencies, including Financial Services Consulting. For additional information, please contact an IBM Representative.

Break down data silos and seamlessly query Iceberg tables in Amazon SageMaker from Snowflake

Post Syndicated from Nidhi Gupta original https://aws.amazon.com/blogs/big-data/break-down-data-silos-and-seamlessly-query-iceberg-tables-in-amazon-sagemaker-from-snowflake/

Organizations often struggle to unify their data ecosystems across multiple platforms and services. The connectivity between Amazon SageMaker and Snowflake’s AI Data Cloud offers a powerful solution to this challenge, so businesses can take advantage of the strengths of both environments while maintaining a cohesive data strategy.

In this post, we demonstrate how you can break down data silos and enhance your analytical capabilities by querying Apache Iceberg tables in the lakehouse architecture of SageMaker directly from Snowflake. With this capability, you can access and analyze data stored in Amazon Simple Storage Service (Amazon S3) through AWS Glue Data Catalog using an AWS Glue Iceberg REST endpoint, all secured by AWS Lake Formation, without the need for complex extract, transform, and load (ETL) processes or data duplication. You can also automate table discovery and refresh using Snowflake catalog-linked databases for Iceberg. In the following sections, we show how to set up this integration so Snowflake users can seamlessly query and analyze data stored in AWS, thereby improving data accessibility, reducing redundancy, and enabling more comprehensive analytics across your entire data ecosystem.

Business use cases and key benefits

The capability to query Iceberg tables in SageMaker from Snowflake delivers significant value across multiple industries:

  • Financial services – Enhance fraud detection through unified analysis of transaction data and customer behavior patterns
  • Healthcare – Improve patient outcomes through integrated access to clinical, claims, and research data
  • Retail – Increase customer retention rates by connecting sales, inventory, and customer behavior data for personalized experiences
  • Manufacturing – Boost production efficiency through unified sensor and operational data analytics
  • Telecommunications – Reduce customer churn with comprehensive analysis of network performance and customer usage data

Key benefits of this capability include:

  • Accelerated decision-making – Reduce time to insight through integrated data access across platforms
  • Cost optimization – Accelerate time to insight by querying data directly in storage without the need for ingestion
  • Improved data fidelity – Reduce data inconsistencies by establishing a single source of truth
  • Enhanced collaboration – Increase cross-functional productivity through simplified data sharing between data scientists and analysts

By using the lakehouse architecture of SageMaker with Snowflake’s serverless and zero-tuning computational power, you can break down data silos, enabling comprehensive analytics and democratizing data access. This integration supports a modern data architecture that prioritizes flexibility, security, and analytical performance, ultimately driving faster, more informed decision-making across the enterprise.

Solution overview

The following diagram shows the architecture for catalog integration between Snowflake and Iceberg tables in the lakehouse.

Catalog integration to query Iceberg tables in S3 bucket using Iceberg REST Catalog (IRC) with credential vending

The workflow consists of the following components:

  • Data storage and management:
    • Amazon S3 serves as the primary storage layer, hosting the Iceberg table data
    • The Data Catalog maintains the metadata for these tables
    • Lake Formation provides credential vending
  • Authentication flow:
    • Snowflake initiates queries using a catalog integration configuration
    • Lake Formation vends temporary credentials through AWS Security Token Service (AWS STS)
    • These credentials are automatically refreshed based on the configured refresh interval
  • Query flow:
    • Snowflake users submit queries against the mounted Iceberg tables
    • The AWS Glue Iceberg REST endpoint processes these requests
    • Query execution uses Snowflake’s compute resources while reading directly from Amazon S3
    • Results are returned to Snowflake users while maintaining all security controls

There are four patterns to query Iceberg tables in SageMaker from Snowflake:

  • Iceberg tables in an S3 bucket using an AWS Glue Iceberg REST endpoint and Snowflake Iceberg REST catalog integration, with credential vending from Lake Formation
  • Iceberg tables in an S3 bucket using an AWS Glue Iceberg REST endpoint and Snowflake Iceberg REST catalog integration, using Snowflake external volumes to Amazon S3 data storage
  • Iceberg tables in an S3 bucket using AWS Glue API catalog integration, also using Snowflake external volumes to Amazon S3
  • Amazon S3 Tables using Iceberg REST catalog integration with credential vending from Lake Formation

In this post, we implement the first of these four access patterns using catalog integration for the AWS Glue Iceberg REST endpoint with Signature Version 4 (SigV4) authentication in Snowflake.

Prerequisites

You must have the following prerequisites:

The solution takes approximately 30–45 minutes to set up. Cost varies based on data volume and query frequency. Use the AWS Pricing Calculator for specific estimates.

Create an IAM role for Snowflake

To create an IAM role for Snowflake, you first create a policy for the role:

  1. On the IAM console, choose Policies in the navigation pane.
  2. Choose Create policy.
  3. Choose the JSON editor and enter the following policy (provide your AWS Region and account ID), then choose Next.
{
     "Version": "2012-10-17",
     "Statement": [
         {
             "Sid": "AllowGlueCatalogTableAccess",
             "Effect": "Allow",
             "Action": [
                 "glue:GetCatalog",
                 "glue:GetCatalogs",
                 "glue:GetPartitions",
                 "glue:GetPartition",
                 "glue:GetDatabase",
                 "glue:GetDatabases",
                 "glue:GetTable",
                 "glue:GetTables",
                 "glue:UpdateTable"
             ],
             "Resource": [
                 "arn:aws:glue:<region>:<account-id>:catalog",
                 "arn:aws:glue:<region>:<account-id>:database/iceberg_db",
                 "arn:aws:glue:<region>:<account-id>:table/iceberg_db/*",
             ]
         },
         {
             "Effect": "Allow",
             "Action": [
                 "lakeformation:GetDataAccess"
             ],
             "Resource": "*"
         }
     ]
 }
  1. Enter iceberg-table-access as the policy name.
  2. Choose Create policy.

Now you can create the role and attach the policy you created.

  1. Choose Roles in the navigation pane.
  2. Choose Create role.
  3. Choose AWS account.
  4. Under Options, select Require External Id and enter an external ID of your choice.
  5. Choose Next.
  6. Choose the policy you created (iceberg-table-access policy).
  7. Enter snowflake_access_role as the role name.
  8. Choose Create role.

Configure Lake Formation access controls

To configure your Lake Formation access controls, first set up the application integration:

  1. Sign in to the Lake Formation console as a data lake administrator.
  2. Choose Administration in the navigation pane.
  3. Select Application integration settings.
  4. Enable Allow external engines to access data in Amazon S3 locations with full table access.
  5. Choose Save.

Now you can grant permissions to the IAM role.

  1. Choose Data permissions in the navigation pane.
  2. Choose Grant.
  3. Configure the following settings:
    1. For Principals, select IAM users and roles and choose snowflake_access_role.
    2. For Resources, select Named Data Catalog resources.
    3. For Catalog, choose your AWS account ID.
    4. For Database, choose iceberg_db.
    5. For Table, choose customer.
    6. For Permissions, select SUPER.
  4. Choose Grant.

SUPER access is required for mounting the Iceberg table in Amazon S3 as a Snowflake table.

Register the S3 data lake location

Complete the following steps to register the S3 data lake location:

  1. As data lake administrator on the Lake Formation console, choose Data lake locations in the navigation pane.
  2. Choose Register location.
  3. Configure the following:
    1. For S3 path, enter the S3 path to the bucket where you will store your data.
    2. For IAM role, choose LakeFormationLocationRegistrationRole.
    3. For Permission mode, choose Lake Formation.
  4. Choose Register location.

Set up the Iceberg REST integration in Snowflake

Complete the following steps to set up the Iceberg REST integration in Snowflake:

  1. Log in to Snowflake as an admin user.
  2. Execute the following SQL command (provide your Region, account ID, and external ID that you provided during IAM role creation):
CREATE OR REPLACE CATALOG INTEGRATION glue_irc_catalog_int
CATALOG_SOURCE = ICEBERG_REST
TABLE_FORMAT = ICEBERG
CATALOG_NAMESPACE = 'iceberg_db'
REST_CONFIG = (
    CATALOG_URI = 'https://glue.<region>.amazonaws.com/iceberg'
    CATALOG_API_TYPE = AWS_GLUE
    CATALOG_NAME = '<account-id>'
    ACCESS_DELEGATION_MODE = VENDED_CREDENTIALS
)
REST_AUTHENTICATION = (
    TYPE = SIGV4
    SIGV4_IAM_ROLE = 'arn:aws:iam::<account-id>:role/snowflake_access_role'
    SIGV4_SIGNING_REGION = '<region>'
    SIGV4_EXTERNAL_ID = '<external-id>'
)
REFRESH_INTERVAL_SECONDS = 120
ENABLED = TRUE;
  1. Execute the following SQL command and retrieve the value for API_AWS_IAM_USER_ARN:

DESCRIBE CATALOG INTEGRATION glue_irc_catalog_int;

  1. On the IAM console, update the trust relationship for snowflake_access_role with the value for API_AWS_IAM_USER_ARN:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                   "<API_AWS_IAM_USER_ARN>"
                ]
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "sts:ExternalId": [
                        "<external-id>"
                    ]
                }
            }
        }
    ]
}
  1. Verify the catalog integration:

SELECT SYSTEM$VERIFY_CATALOG_INTEGRATION('glue_irc_catalog_int');

  1. Mount the S3 table as a Snowflake table:
CREATE OR REPLACE ICEBERG TABLE s3iceberg_customer
 CATALOG = 'glue_irc_catalog_int'
 CATALOG_NAMESPACE = 'iceberg_db'
 CATALOG_TABLE_NAME = 'customer'
 AUTO_REFRESH = TRUE;

Query the Iceberg table from Snowflake

To test the configuration, log in to Snowflake as an admin user and run the following sample query:SELECT * FROM s3iceberg_customer LIMIT 10;

Clean up

To clean up your resources, complete the following steps:

  1. Delete the database and table in AWS Glue.
  2. Drop the Iceberg table, catalog integration, and database in Snowflake:
DROP ICEBERG TABLE iceberg_customer;
DROP CATALOG INTEGRATION glue_irc_catalog_int;

Make sure all resources are properly cleaned up to avoid unexpected charges.

Conclusion

In this post, we demonstrated how to establish a secure and efficient connection between your Snowflake environment and SageMaker to query Iceberg tables in Amazon S3. This capability can help your organization maintain a single source of truth while also letting teams use their preferred analytics tools, ultimately breaking down data silos and enhancing collaborative analysis capabilities.

To further explore and implement this solution in your environment, consider the following resources:

These resources can help you to implement and optimize this integration pattern for your specific use case. As you begin this journey, remember to start small, validate your architecture with test data, and gradually scale your implementation based on your organization’s needs.


About the authors

Nidhi Gupta

Nidhi Gupta

Nidhi is a Senior Partner Solutions Architect at AWS, specializing in data and analytics. She helps customers and partners build and optimize Snowflake workloads on AWS. Nidhi has extensive experience leading production releases and deployments, with focus on Data, AI, ML, generative AI, and Advanced Analytics.

Andries Engelbrecht

Andries Engelbrecht

Andries is a Principal Partner Solutions Engineer at Snowflake working with AWS. He supports product and service integrations, as well the development of joint solutions with AWS. Andries has over 25 years of experience in the field of data and analytics.