Tag Archives: Amazon Aurora

Transforming fleet analytics for Novus Hi-Tech through agentic AI

Post Syndicated from Sameer Shrivastava original https://aws.amazon.com/blogs/architecture/transforming-fleet-analytics-for-novus-hi-tech-through-agentic-ai/

Fleet analytics sounds straightforward until you have thousands of vehicles streaming dashcam footage, driver alerts, GPS pings, and speed warnings across 10 countries, all at the same time. At that scale, no team can manually triage every drowsiness alert or investigate every over-speeding event before it becomes a real incident. With over 120 enterprise customers relying on their fleet, Novus Hi-Tech needed a system that would act on the data, not only show it. With AI-powered automation, they achieved a 46 percent reduction in road-behavior violations and a 55 percent drop in drowsiness alerts per 1,000 km.

In this post, we walk you through how Novus Hi-Tech, in collaboration with the AWS Prototyping and AI Customer Engineering (PACE) team, designed and implemented an agentic AI solution called FleetGPT. We architected the multi-agent system using Strands Agents SDK and AWS managed AI services, turning IoT data into real-time safety interventions through natural language querying, automated investigation, and AI-powered voice calls to drivers.

For years, extracting value from fleet telemetry required significant manual effort. When a drowsiness alert triggered, operators had to validate video footage, assess severity across multiple signals, contact the driver, and log the outcome. Existing fixed dashboards showed the same views regardless of what teams actually needed to know.

At scale, this model breaks down because one can’t staff enough operators to catch every warning in real time, and manual analysis becomes the bottleneck. The cost of a missed warning could be a preventable accident.

Novus Hi-Tech needed a system that understood the difference between what a fleet manager needs to know, what a customer success analyst is looking for, and what a Command Center operator must act on immediately.

The bottleneck

The friction between question and answer wasn’t uniform. It created different bottlenecks depending on what teams were trying to accomplish.

Every warning that reached the Command Center demanded the same judgment calls. Operators cycled through this decision tree hundreds of times per shift.

On the other hand, fleet managers didn’t need individual event response, but pattern recognition: which drivers are trending toward accidents, which routes carry disproportionate risk, which fleets are drifting on safety compliance.

Both roles shared the same friction: the time between asking a question and getting an answer. Operators spent too long investigating individual events. Fleet managers couldn’t analyze trends fast enough. Neither could move from question to insight without switching tools or waiting on data teams.

Solution overview

FleetGPT replaces fixed dashboards with a natural language interface for dynamic querying, investigation, and insight generation. It surfaces video evidence, driver history, and policy in single interactions. Beyond analytics, FleetGPT automates real-time safety interventions. When a critical event occurs, such as repeated drowsiness, over-speeding, or an SOS trigger, the system initiates a two-way AI-powered voice call in the driver’s native language, with user approval. It tracks acknowledgment and escalates when necessary.

FleetGPT is built with two core capabilities.

  1. Analyst: A conversational intelligence layer where operators and fleet managers query the fleet data, retrieve video evidence, and search policy documents through natural language.
  2. Intervention engine: An agentic layer that monitors incoming events, evaluates severity, and executes driver interventions with human approval.

Analyst: Natural language intelligence

Novus Hi-Tech’s data landscape included separate systems for telemetry, video, and policy documents. FleetGPT unifies these through a single data layer accessed by an agentic conversational interface.

Data storage and processing layer

Data store Role Key design choice
Amazon Aurora PostgreSQL Structured data Stores structured telemetry data
Amazon DynamoDB High velocity event stream from devices Low latency read-write operations with composite keys optimized for agentic query patterns
Amazon Bedrock Knowledge Bases Implement RAG with agentic system Vector embeddings stored in Aurora PostgreSQL with pgvector for semantic retrieval

Amazon Aurora PostgreSQL manages the structured analytical backbone. It stores per-vehicle daily aggregates covering distance, fuel, safety scores, event counts, and GPS statistics, plus live snapshots of current vehicle status. It’s the primary target for the Text-to-SQL pipeline. A YAML schema file with column aliases, descriptions, and worked examples is injected into every SQL generation prompt. This grounds the large language model (LLM) on the actual schema and reduces hallucination on column names and join paths.

Amazon DynamoDB handles the high-velocity warning event stream. The warning event table uses a composite sort key of warning_code + datetime, enabling efficient range queries by warning type and time window without full table scans. A Global Secondary Index on fleet_id supports fleet-scoped queries without touching data from other tenants. A separate device registry table maps vehicle IDs to country metadata which is the source of truth for language routing in audio alerts.

Amazon Bedrock Knowledge Bases, the fully managed RAG capability, indexes unstructured documents including SOPs, compliance policies, product documentation which are stored in Amazon Simple Storage Service (Amazon S3). The embeddings are generated using Amazon Titan Text Embeddings V2 for storage in Amazon Aurora PostgreSQL with pgvector. This gives operators and fleet managers a single interface for both data questions and policy questions, with no context switching.

Orchestration: Strands Agents

FleetGPT multi-agent architecture over Amazon Aurora, DynamoDB, and Amazon Bedrock Knowledge Bases

Figure 1: Solution architecture

The system uses a multi-agent architecture built with Strands Agents, where each agent has a distinct role and model assignment based on the complexity of its task.

At the top sits the Supervisor agent, running on Claude Sonnet 4.6 (for model availability by AWS Region, refer to Region availability by models), which handles intent decomposition and multi-step reasoning. It decides what kind of question the user is asking and routes it to the right sub-agent. The Supervisor uses a SummarizingConversationManager from Strands Agents, which preserves the last six messages verbatim while summarizing older turns. This keeps context window usage bounded without losing conversational continuity.

For structured data queries, the Supervisor delegates to a Text-to-SQL sub-agent (also Claude Sonnet 4.6). This agent follows a plan-and-execute approach: it first classifies the query type (vehicle data, driver behavior, or operational metrics), then extracts key entities and maps them to database tables and columns. From there, it generates a query plan outlining the relevant tables and operations, and finally produces and executes the SQL. If an error occurs, the agent retries with the error message, original plan, and failed query included in the prompt for self-correction.

Text-to-SQL sub-agent flow: classify the query, extract entities, map to tables, generate a plan, then produce and run SQL

Figure 2: Text-to-SQL sub-agent flow

When a query needs computation beyond SQL (correlation analysis, statistical profiling, trend detection), the Supervisor delegates to a Data Processing sub-agent running on Claude Haiku 3.0 (for model availability by AWS Region, refer to Region availability by models). Rather than pushing complex processing onto the database, this agent generates executable Python, runs it through a Python REPL tool, and returns the shaped results to the orchestrator. Claude Haiku 3.0, a smaller model than Claude Sonnet 4.6, helps keep the inference cost down for this tier of the pipeline.

Each agent has a focused tool surface and scoped system prompt, reducing hallucination and making failures straightforward to isolate.

The agent is hosted on AWS Fargate for Amazon Elastic Container Service (Amazon ECS), which provides a serverless container environment. An Application Load Balancer (ALB) serves as the entry point, routing incoming requests to the ECS-hosted application.

Video evidence retrieval

Video retrieval demonstrates the practical impact of this architecture. Earlier retrieving dashcam footage required operators to know vehicle IDs, navigate a separate system, and apply filters manually.

Operators now retrieve video using natural language, specifying fleet, vehicle, warning type, date range, speed, or city. The agent resolves these parameters into a DynamoDB query using the composite sort key, applies the remaining criteria as in-memory filters, and returns a paginated result set with vehicle ID, timestamp, GPS coordinates, and a streamable video link in a single conversation.

Multi-tenancy and conversation persistence

Multi-tenancy is enforced at the data layer. Every query is scoped to the authenticated user’s authorized fleet IDs, injected into the agent’s context at session initialization. This helps reduce the risk of cross-tenant data access, whether through natural language, prompt injection, or direct tool calls.

Conversation history is stored in DynamoDB with a composite key of user_id + conversation_id + timestamp, to support per-user time-ordered retrieval. At session start, the system also injects live operational context including today’s warning counts, the latest critical events and the currently open video. This gives the agent situational awareness along with the conversation history.

Intervention engine: Automated event response

The intervention agent acts on critical events. When warnings such as drowsiness, dangerous speeding, or SOS arrive, the system evaluates severity and queues interventions for user approval.

Real-time event push through Amazon DynamoDB Streams

ADAS (Advanced Driver Assistance Systems) edge devices send warning events to DynamoDB. Amazon DynamoDB Streams capture these events, which a background worker forwards to the notification API for delivery to the Command Center.

The worker uses exponential backoff on idle periods: if no new events are found, the poll interval doubles up to a maximum of five minutes, then resets to the base interval on the next event. A fallback polling job provides redundancy, scanning a configurable recent time window on a fixed schedule in case the stream worker misses an event.

Two-way AI voice calling

When a safety event needs direct driver contact, the system automatically initiates a fresh conversation session that is pre-configured with the driver’s language. The LLM system prompt, STT/TTS voice model, and language hint are all set at session start. You can refer to Sample Indic Language Voicebot for reference implementation.

The voice calling pipeline operates in real time:

  1. Speech-to-Text: The driver’s voice is captured through the in-cabin microphone and transcribed by a high-accuracy multilingual STT (Speech-to-Text) model, with the language hint set at the beginning of the session for accuracy across regional fleets.
  2. LLM Processing: Transcriptions are sent to an LLM on Amazon Bedrock that generates contextually appropriate responses in the driver’s native language.
  3. Text-to-Speech and delivery: Responses are synthesized using a multilingual TTS (Text-to-Speech) model and streamed to the vehicle’s IVI (In-Vehicle Infotainment) head unit over WebSocket.
  4. Interrupt handling: VAD (Voice Activity Detection) monitors for driver speech during system responses, immediately canceling active streams to allow natural conversation flow.

Key design decisions

Building FleetGPT required deliberate choices about frameworks, data stores, and agent architecture. The following sections explain the reasoning behind the decisions that shaped the system.

Why Strands Agents?

Strands Agents was chosen for fine-grained control over tool invocation, conversation lifecycle management, and per-session agent state within a Python-native environment. Capabilities such as summarizing conversation memory, multi-model routing, and caching intermediate tool outputs are easier to implement and tune directly at the SDK level than through higher-level abstraction layers.

DynamoDB

Amazon DynamoDB provides the low-latency reads and stream processing needed for real-time fleet monitoring. The composite sort key design deliberately matches agent query patterns to allow efficient filtering without table scans. A Global Secondary Index on fleet_id helps provide tenant isolation. Amazon DynamoDB Streams trigger the intervention pipeline so the system reacts to critical safety events within seconds.

Amazon Bedrock

Amazon Bedrock provides the foundation model infrastructure for FleetGPT, offering access to multiple models. Amazon Bedrock Knowledge Bases handles the complete RAG pipeline for policy and documentation retrieval, alleviating the need for custom RAG infrastructure while maintaining the security and access controls already established for structured data.

Sub-agent delegation for specialization without coupling

The three-tier data analysis chain exists because each step requires fundamentally different reasoning. Consolidating these responsibilities into a single agent would degrade performance across multiple dimensions.

A monolithic agent must balance competing concerns in its system prompt, reducing effectiveness for each task. It must select from a larger tool inventory, increasing decision complexity and error rates. And when failures occur, diagnosing the root cause becomes significantly harder.

Separating these concerns means each agent can be prompted, tested, and improved independently, with clear failure boundaries at each tier.

Atomic system prompts for maintainability and debugging

System prompts are structured as composable, tagged instruction blocks rather than a monolithic string. Each instruction is a discrete, auditable unit which is tagged by category, named, and independently editable. When two instruction blocks produce conflicting behavior, the tag and block name tell you exactly where to look, without having to reason about the prompt as a whole. This made iterating on specific behaviors fast and safe.

Results

FleetGPT has delivered measurable operational impact across Novus Hi-Tech’s customer base:

  1. 46 percent annual reductions in road-behavior violations.
  2. Insights from data led to a 3–6 percent increase in fuel efficiency.
  3. 55 percent reduction in drowsiness alerts per 1,000 km.

These improvements stem from four key operational transformations:

  1. Warning validation time dropped from minutes to seconds: Investigating a single warning previously meant navigating multiple dashboard screens, manually filtering video feeds, cross-referencing driver history in separate tools, and looking up policy documents. Now it’s a single conversational query that surfaces video evidence, driver history, and fleet context together.
  2. Real-time alert delivery within seconds: The DynamoDB Streams based pipeline replaced a polling-based system with multi-minute latency. Command Center operators now receive critical warnings within seconds of an event being recorded by an edge device in the field.
  3. Self-service analytics for fleet managers: Fleet managers generate driver performance reports, fleet health summaries, and trend analyses without engaging the data team. Questions that once took hours or days are now answered in conversational turns.
  4. Improved driver response rates in multilingual fleets: Delivering audio alerts in each driver’s native language has measurably improved acknowledgment rates across regional fleet segments, where a single-language approach previously left a significant portion of drivers underserved.

Conclusion and next steps

FleetGPT demonstrates how agentic AI can transform operational workflows when applied to domain-specific problems. By combining natural language interfaces with automated intervention systems, Novus Hi-Tech moved from reactive dashboard monitoring to proactive, context-aware fleet management.

Looking ahead, Novus Hi-Tech is expanding FleetGPT’s capabilities in three directions:

Amazon Bedrock AgentCore: Host the developed agent on Amazon Bedrock AgentCore, a platform to build, connect, and optimize agents at scale, with any framework or model.

Predictive risk scoring: The dataset now carries enough behavioral history to surface vehicles trending toward incidents before they occur. The next capability will score drivers against their own baselines and flag deteriorating patterns proactively.

Embedded chart and visualization generation: Fleet managers think visually. Rather than returning tabular data, the agent will embed generated charts directly in responses for trend queries which will close the gap between insight and presentation without a separate business intelligence tool in the loop.

If you manage real-time operational data across distributed systems, FleetGPT offers a blueprint: start with the questions your teams ask most often, build agents that can answer them reliably, and then extend those agents to act on the answers.

For more information and relevant reads, see the following resources:

 


About the authors

AWS Weekly Roundup: AgentCore Runtime V2, Amazon DynamoDB bulk put items, and more (November 9, 2026)

Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-agentcore-runtime-v2-amazon-dynamodb-bulk-put-items-and-more-november-9-2026/

Most of the cost of adopting something new isn’t the new thing, it’s the move. Moving a Hive table to Apache Iceberg has typically meant weeks of engineering work. Last week, Amazon S3 Tables turned that into three steps: start, sync as often as you need, complete. The source stays writable except for a few minutes at cutover. The same week, Amazon DynamoDB started loading data from S3 into tables that already exist.

The pattern I like is that neither one asks you to stop what’s running. That’s also the way to read the agent news this week. The new Amazon Bedrock AgentCore Runtime is a single field on the runtime you already have, and Web Answer is a tool you attach to a gateway you already run. Now, let’s get into this week’s AWS news…

Headlines
The new AgentCore Runtime: elastic, optimized, and consistently fast starts – If you run agents on AgentCore and haven’t tried the new runtime since it became generally available, it’s worth a look. It changes two things. First, memory: sessions start small, page in more on demand, and give memory back when it goes cold instead of holding the peak, so you pay for what your agent uses. Second, cold starts: AgentCore prepares your environment once, snapshots it, and restores the snapshot for every new instance. In the team’s tests, that kept P75 cold starts at about 2 seconds for images from 200 MB up to 2 GB, while the original runtime rose from roughly 5.4 seconds to nearly 30 seconds as images grew. The per-unit rate is higher, but you use far fewer GB-hours, so the team expects most agents to see a lower bill, though that isn’t guaranteed.

To opt in, you set platformVersion to V2 when you create or update a runtime. Create and update take several minutes while the snapshot is prepared, and your container must report healthy on /ping within 120 seconds, so report health only after initialization finishes. The docs cover how to optimize your agent for V2. V2 is available in 10 Regions, and committed baseline pricing is coming soon.

Amazon S3 Tables now offer managed migration of Apache Hive tables to Apache Iceberg – You can now migrate Parquet tables registered in the AWS Glue Data Catalog into S3 Tables without building a pipeline or provisioning compute. S3 Tables validates the source and produces a compatibility report covering every object in scope. Each sync copies only what changed, so production keeps writing until cutover. Afterward you can query the table with Amazon Athena, Amazon SageMaker Unified Studio, or any engine that speaks the Iceberg REST API. You pay only for the data copied, with no charge for the conversion, and objects in S3 Intelligent-Tiering keep their tier. At launch it works within the same account and Region, in 10 Regions.

Last week’s launches
Here are some launches and updates from this past week that caught my attention:

Amazon DynamoDB now supports bulk put items – You can now load line-delimited DynamoDB JSON or Amazon Ion files from Amazon S3 into existing tables as a managed, asynchronous job. Each record can carry its own condition expression, so one file can mix plain, insert-only, and version-guarded writes. Writes use dedicated system capacity and cost less than the equivalent per-request writes. An Accelerated tier handles deadline-bound jobs. The Database blog post gives illustrative Accelerated times of less than 1 hour for 100 million 1 KB items. One thing to know before you start: canceling doesn’t roll back items already written.

Announcing Web Answer on Amazon Bedrock AgentCore – You can now give an agent a single tool call that returns a synthesized, cited answer to a natural-language question, instead of building your own search, retrieval, and citation pipeline. Web Answer plans its own searches, runs several in parallel, and decides when it has enough evidence. You attach it as a target on AgentCore Gateway and call it over MCP, and you can limit it with domain include and deny lists and publish-date filters. It’s available in US East (N. Virginia). The getting started post explains when to use it instead of Web Search, which returns ranked results for your own model to synthesize.

AWS Lambda now supports queues for Apache Kafka in self-managed Kafka event sources – You can now process records from a single Kafka partition across many parallel AWS Lambda invocations. Classic consumer groups cap parallelism at the partition count. Queue mode uses Kafka share groups and needs Kafka 4.2 or later with share groups enabled. It works with self-managed clusters and managed offerings such as Confluent Cloud. You give up ordering and get at-least-once delivery, so your functions need to be idempotent. The Compute blog walkthrough covers poller scaling and when to keep stream mode.

AWS announces AWS Interconnect – last mile preview with Equinix – You can now set up a private, managed connection from Equinix’s network of 282 data centers to AWS without ordering cross-connects or configuring routers. Capacity is quad-redundant across two locations and four links, with BGP and VLANs configured for you, a 99.99% availability SLA, and MACsec by default. You pick a Region, bandwidth, and AWS Direct Connect gateway, then paste the activation key into Equinix Fabric. The preview covers the San Jose metro and, as of last Thursday, Northern Virginia, with 1 Gbps connections at no charge.

AWS Security Hub now provides an SBOM inventory for your compute resources – You can now see every software package installed across your organization’s Amazon EC2 instances, Amazon ECR images, and Lambda functions in AWS Security Hub, with each package’s version and how many resources run it. That’s the list you want the next time a widely used library has a vulnerability. The inventory builds on the SBOMs that Amazon Inspector already generates, so it fills in automatically wherever Inspector is enabled. It’s included in Security Hub Essentials at no additional cost.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news
Here are some additional posts and resources that you might find interesting:

Serverless ICYMI Q3 2026 – Julian Wood’s quarterly recap is a quick serverless catch-up before re:Invent: Lambda SnapStart for container images, a 90-minute timeout on Lambda Managed Instances, Lambda MicroVMs in 10 Regions, 1 MiB payloads in Amazon SNS, and the new AgentCore Runtime.

AWS contributes a large-transaction binary log optimization to MySQL – With binlog on, a huge transaction holds a global commit lock while its events are copied, blocking every other commit. The optimization, which Amazon Aurora MySQL has run in production since 2019, promotes the spill file into the binlog instead of copying it. In AWS’s tests, commit latency for a 50 GiB transaction dropped from about 7 minutes to about 2 to 2.5 seconds. It has now shipped to community MySQL.

How OrderServ achieved up to 64% cost savings with multi-tenant ECS – Cognizant’s food-ordering platform went from one Amazon ECS cluster per brand on AWS Fargate to a shared cluster on EC2 Auto Scaling groups with warm pools. The 64% applies to a 3-year Compute Savings Plan (47% with a 1-year plan). Scaling time also fell from 5–7 minutes to 30–60 seconds.

Upcoming AWS events
Check your calendar and sign up for upcoming AWS events:

AWS re:Invent – re:Invent returns to Las Vegas from November 30 to December 4. The event catalog is live and reserved seating is open, so register and claim your spots before they fill up.

AWS Community Days – Community-led conferences planned and delivered by community leaders, with a few more before the year ends: Belém, Brazil (November 14), JAWS FESTA 2026 in Akita, Japan (November 17), and Port Harcourt, Nigeria (December 12).

Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development. Browse here for upcoming AWS-led in-person and virtual events and developer-focused events.

That’s all for this week. Check back next Monday for another Weekly Roundup!

– Daniel Abib

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!

AWS Weekly Roundup: GPT-6.1 Terra on Bedrock, Step Functions as agent tools, and more (November 2, 2026)

Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-gpt-6-1-terra-on-bedrock-step-functions-as-agent-tools-and-more-november-2-2026/

The agentic launches that stayed with me last week had nothing to do with how capable a model is. They were about reach. AWS Step Functions, which already orchestrates over 220 AWS services, can now be handed to an agent as a tool, so workflows teams already trust became callable without a rewrite. In the same week, the AI blog published how one engagement platform exposed roughly 400 internal API operations the same way. Neither story involves rebuilding anything, only describing what exists and letting the agent find it.

The other thread was cost. OpenAI’s newest Terra model reached Amazon Bedrock priced well below the model beside it, and two post-quantum launches landed for a deadline nobody can put on a calendar yet. Now, let’s get into this week’s AWS news…

Headlines
OpenAI GPT-6.1 Terra is now generally available on Amazon Bedrock, and what interests me is where it sits rather than what it tops. Terra is the everyday-workload line of the GPT-6 family. In OpenAI’s Agents’ Last Exam evaluation it scored 15 percentage points higher than GPT-5.6 Terra, performed on par with GPT-6 Astra, and completed tasks more than three times faster than both. Those are OpenAI’s figures, not an independent test. The price is what I would check first: input and output tokens are at least 50% lower than GPT-6.1 Sol and GPT-5.6 Terra. It takes up to 1 million input tokens with text and image input, and you can reach it from the Bedrock APIs or ChatGPT Work and Codex.

AWS Step Functions workflows are now available as AI agent tools through Amazon Bedrock AgentCore Gateway, which connects agents to tools over the Model Context Protocol (MCP). The division of labor is the idea: the agent decides when to run a workflow and what input to pass; Step Functions runs each step exactly as you defined it. A support agent can trigger a refund, and the eligibility check, the manager approval, and the credit still happen in that order every time, because the ordering was never the model’s to improvise. Your existing Standard and Express workflows qualify as connector targets, and agents can also look up runs. It’s available in all Regions where AgentCore Gateway is, at standard rates.

Last week’s launches
Here are some launches and updates from this past week that caught my attention:

AWS Key Management Service now supports ML-KEM for post-quantum key encapsulation – You can now establish a shared secret with a post-quantum algorithm using AWS KMS keys, with private keys in FIPS 140-3 Level 3 validated HSMs. All three NIST FIPS 203 parameter sets are supported through Encapsulate, which produces a random 32-byte shared secret and its ciphertext, and Decapsulate, which recovers it. The threat is harvest now, decrypt later: an adversary records today’s key exchange and recovers the secret once quantum computers are powerful enough. It’s available everywhere KMS runs except the Middle East (UAE and Bahrain) and China Regions. Amazon Aurora PostgreSQL added post-quantum TLS key exchange the same week, on versions 18 and higher.

AWS IAM Identity Center now supports cross-Region trusted identity propagation – You can now carry a user’s identity with their request to AWS services in another Region. This removes a genuine architectural constraint: trusted identity propagation used to require every service accessing that identity to sit in the same Region, which left two unappealing options, broad IAM roles that cannot distinguish individual users, or consolidating all your services into a single Region. Now the identity travels with the request automatically, so you grant access by user and group membership and audit who accessed what. It’s available at no additional cost everywhere IAM Identity Center runs.

Amazon ECS now supports GPU sharing with NVIDIA Multi-Process Service (MPS) – You can now run multiple Amazon ECS tasks concurrently on a single NVIDIA GPU, the practical answer to work that only ever uses a fraction of one, like small-model inference and multi-model serving. You declare the GPU memory each container needs and can optionally cap its share of GPU compute, and ECS places tasks on GPUs with available capacity as your services scale, with no code changes. It works on ECS Managed Instances and ECS on Amazon EC2, and on Managed Instances it watches the MPS control daemon and replaces the instance if it turns unhealthy. It’s available in all commercial and AWS GovCloud (US) Regions.

AWS Lambda supports Seekable OCI (SOCI) lazy loading for Lambda Managed Instances – You can now have AWS Lambda fetch only the files needed at each stage of startup instead of pulling a whole container image first, which AWS puts at up to 70% off container startup time. You create a SOCI index, push it to Amazon ECR, point the image URI at the indexed tag, and Lambda detects it automatically. The technique also reached Amazon SageMaker AI training and processing jobs, with benchmarks: lazy loading held at 3 to 11 seconds regardless of image size, against a median 3 minutes 42 seconds for a 13.2 GB image, a range they report as 8x to 46x.

Amazon EKS Auto Mode now supports Kubernetes Gateway API – You can now declare how traffic reaches your applications with Gateway and HTTPRoute resources on Amazon EKS without installing a separate controller, and EKS Auto Mode provisions and manages the underlying AWS load balancing infrastructure. You get HTTP and gRPC routing, cross-namespace routing, weighted traffic splitting, header-based routing, and automatic TLS certificate discovery through AWS Certificate Manager. Existing Ingress workloads keep working alongside them, so this is additive rather than a migration. Gateway API support is included with EKS Auto Mode; standard load balancing charges apply.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news
Here are some additional posts and resources that you might find interesting:

How Nubank automated disaster recovery with the AWS DevOps Agent – Nubank serves more than 140 million customers across Brazil, Mexico, and Colombia, and classifies over 300 business flows as critical, each tested on a regulator’s schedule. Nubank reports that handing evidence collection and report writing to the AWS DevOps Agent cut a test cycle from about 4.3 hours to roughly 40 minutes, and quadrupled flow coverage in 2026 to more than 160. One execution terminated more than 900 EKS instances under synthetic traffic with AWS Fault Injection Service, producing the evidence its regulator needed.

How MoEngage turned over 400 APIs into agent tools with AgentCore Gateway – Roughly 400 internal REST API operations, built by 25 teams, sitting in private VPCs across seven Regions, with tenant identity that could never become a parameter the model gets to set. MoEngage estimates the managed path saved about six months of platform engineering. The detail worth stealing: a full tool listing would have crossed 20,000 tokens per connection, so agents search for tools by describing a task instead of receiving the catalog.

Reducing petabyte-scale S3 storage costs with evidence-based lifecycle rules – The hard part isn’t deciding to act, it’s knowing which data has gone cold and where it can safely live. This uses S3 Storage Lens groups and Storage Class Analysis to prove access patterns before moving anything. In the engagement described, 5.8 PB of logs in Amazon S3 Standard at approximately $241,000 per month showed an access ratio below 0.1% past 90 days, and a 90-day transition to Glacier Instant Retrieval cut the bill by roughly 70%, a figure the author ties to that workload.

Upcoming AWS events
Check your calendar and sign up for upcoming AWS events:

AWS re:Invent – re:Invent returns to Las Vegas from November 30 to December 4, with more than 2,200 sessions. The event catalog is live and reserved seating is open, so register and claim your spots before they fill up.

AWS Community Days – Community-led conferences planned and delivered by community leaders. The 2026 season is winding down as re:Invent approaches, so check the hub for what’s still on.

Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development. Browse here for upcoming AWS-led in-person and virtual events and developer-focused events.

That’s all for this week. Check back next Monday for another Weekly Roundup!

– Daniel Abib

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!

AWS Weekly Roundup: AWS Lambda Web Functions, Amazon S3 on the Vercel Marketplace, and more (October 19, 2026)

Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-aws-lambda-web-functions-amazon-s3-on-the-vercel-marketplace-and-more-october-19-2026/

Nothing last week added a new kind of compute. What it did instead was shorten the distance between the code you already have and that code being live. AWS Lambda learned to run a plain HTTP server with no adapter in front of it, and Amazon S3 became something you can provision from inside the tool you were already using, without opening the AWS Management Console. Different services, same direction.

The counterweight arrived on Thursday. As it gets easier to put things into production, it matters more that you can say what those things are allowed to do, and this week’s most interesting read is about exactly that. Now, let’s get into this week’s AWS news…

Headlines

AWS Lambda Web Functions is a new form factor that runs a standard HTTP web application on Lambda without rewriting it into an event handler. You take a port-listening server built with Express, Hono, Fastify, or Next.js, run aws lambda-web deploy, and get back a live HTTPS URL in under a minute. No adapters, no infrastructure to set up first, no framework lock-in. Lambda now has three form factors on the same Firecracker isolation, which compose inside one application: Event Functions (the model you already know) for async work, Web Functions for the user-facing layer, and MicroVMs for stateful sandboxes running untrusted code.

Two details make this more than repackaging. Each execution environment handles up to 128 concurrent requests, so a burst of traffic shares warm environments instead of mapping one request to one environment, and billing meters the CPU time your code actually uses rather than wall-clock time. For server-side rendering, API backends, and inference endpoints streaming tokens, where most of the elapsed time is spent waiting, that changes the arithmetic. Adding --regions to the same deploy command replicates your code and configures latency-based routing behind a single global URL, and you can split traffic across revisions by weight for a canary rollout. It’s available in five Regions, running Node.js 24 on Arm64 with VPC connectivity. The AWS Compute Blog has a longer walkthrough, and Amazon CloudFront can now use a Web Function endpoint as an origin with origin access control.

The week’s other headline isn’t a launch. Trust in Depth: giving agents more autonomy without losing control makes an argument I keep running into in practice: putting a person in the loop for every agent action works at small scale and then quietly stops working, because people asked to check everything stop catching what matters. The alternative is four questions answered at four moments, which AWS calls scope, know, bound, and observe. The part worth sitting with is that an agent’s limits belong outside its own reasoning, in Amazon Bedrock AgentCore policies enforced at the gateway, because an agent cannot turn off a control it does not run. AWS has run automated reasoning in production for more than a decade, including AWS Identity and Access Management at more than 2.1 billion requests per second, and the case here is for pointing it at the rules you write for agents.

Last week’s launches

Here are some launches and updates from this past week that caught my attention:

  • Amazon S3 is now available on the Vercel Marketplace and v0 by Vercel – You can now provision S3 general purpose buckets, S3 Tables, and S3 Vectors from the Vercel dashboard, or by describing what you want to build in v0. It joins existing integrations for Amazon Aurora, Amazon DynamoDB, and Amazon OpenSearch Serverless, so the whole stack sits in one AWS account with one bill. The authentication detail is the good part: a short-lived OIDC token is exchanged at runtime with AWS Security Token Service for credentials scoped to that resource, so no static keys are stored anywhere.
  • Amazon EC2 T8i instances are now available in South America (São Paulo) – The newest burstable generation reached the São Paulo Region, which matters for the unglamorous half of a workload: login gateways, small databases, CI/CD runners, low-traffic sites. Amazon EC2 T8i delivers up to 30 percent better price performance than T3, with up to 70 percent higher compute performance and 2.4 times the Amazon EBS bandwidth. It keeps the same CPU credit system as T3, so moving over is a size change rather than a redesign, and t8i.micro and t8i.small are available under the AWS Free Tier.
  • Reflection Beam, an open-weight model for agentic coding, is now on AWS – You can now deploy Beam, Reflection AI’s open-weight model for agentic coding and long-horizon tasks, from Amazon SageMaker JumpStart. It’s a sparse mixture-of-experts model with 501 billion total and 23 billion active parameters, released on Hugging Face under Apache 2.0, and Reflection reports an effective context length of 1 million tokens. Start with a JumpStart real-time endpoint, or use SageMaker HyperPod Inference with SGLang or vLLM when you need Kubernetes-native control. It was also ported and optimized for AWS Trainium3, and launches in N. Virginia and Ohio.
  • AWS Lambda Durable Execution SDK for Go is now generally available – You can now write multi-step Go workflows that checkpoint their own progress and pause for up to a year while waiting on an external event, without building progress tracking yourself. That covers deployment rollouts with approval gates and pipelines that call a model and then stop for human review. It behaves like ordinary Go tooling: go get to add it, deploy a single static binary on the provided.al2023 runtime, and test a full workflow locally with the durabletest package, no network access or credentials required.
  • Amazon CloudWatch announces log routing and chained log parsers – You can now define routes in an Amazon CloudWatch pipeline that send each event to a different log group based on conditions evaluated against processed fields, so severity, source, or event type can each land somewhere with its own retention and access controls. Chained parsers handle the other half: parsers run in sequence, with later ones consuming fields that earlier ones produced, so you can parse a JSON envelope first and then run a Grok parser over a field it extracted.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news

Here are some additional posts and resources that you might find interesting:

Upcoming AWS events

Check your calendar and sign up for upcoming AWS events:

  • AWS re:Invent – re:Invent returns to Las Vegas from November 30 to December 4, with more than 2,200 sessions. The event catalog is live and reserved seating is open, so register and claim your spots before they fill up.
  • AWS Community Days – Community-led conferences planned and delivered by community leaders. Here in Brazil, the 2026 season closes with AWS Community Day Brasil Norte in Belém on October 31, the final stop in a year that covered every region of the country for the first time.

Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development. Browse here for upcoming AWS-led in-person and virtual events and developer-focused events.

That’s all for this week. Check back next Monday for another Weekly Roundup!

— Daniel Abib

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!

AWS Weekly Roundup: Prime Day 2026 by the numbers, Claude Haiku 5.5, Amazon Nova 2.5 Sonic, and more (October 13, 2026)

Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-prime-day-2026-by-the-numbers-claude-haiku-5-5-amazon-nova-2-5-sonic-and-more-october-13-2026/

Some numbers are hard to picture until you sit with them for a moment. Last week we published the Prime Day 2026 metrics, and the one that stuck with me is 2.3 quadrillion: the daily requests Amazon ElastiCache peaked at during the event. Prime Day ran back in June, so this post always lands as a retrospective, and that gap is part of what makes it worth reading. By the time the numbers reach you, the only evidence that any of it happened is that nothing went wrong.

The rest of the week pulled in a different direction: smaller, cheaper, and more constrained. Claude Haiku 5.5 arrived on Amazon Bedrock at roughly 75 percent less than Haiku 4.5, Amazon Nova 2.5 Sonic made voice agents quicker on their feet, and Strands Box showed up to put real boundaries around what an agent is allowed to do. Now, let’s get into this week’s AWS news…

Headlines

All the numbers: Amazon Prime Day 2026 powered by AWS is the latest entry in a tradition that goes back to 2016, and the scale is still difficult to hold in your head. Over the four days of Prime Day, Amazon DynamoDB processed more than 59 trillion requests while peaking at 192 million requests per second, and kept single-digit millisecond responses throughout. AWS Lambda handled over 2.3 trillion invocations per day. Amazon EBS peaked above 24.8 trillion I/O operations, moving more than an exabyte of data daily. Amazon ECS launched an average of 158.3 million tasks per day on AWS Fargate, a 47.7 percent jump over last year, and AWS Graviton powered up to 49 percent of the Amazon EC2 compute that Amazon.com used.

The number I keep coming back to, though, isn’t a throughput record. The team ran more than 44,000 AWS Fault Injection Service experiments ahead of the event, more than six times what they ran in 2025. Those are deliberate failures injected on purpose, months in advance, so that the trillions of requests above turn into an ordinary shopping afternoon. If you’re preparing for your own peak moment, AWS Countdown Premium is built for exactly that kind of planning.

Last week’s launches

Here are some launches and updates from this past week that caught my attention:

  • Claude Haiku 5.5 is now available on AWS – You can now use the fastest model in the Claude 5.5 family on Amazon Bedrock and Claude Platform on AWS, at around 75 percent less than Claude Haiku 4.5 for most tasks. It’s the first Haiku with effort controls, so you can tune cost against intelligence per task instead of fixing one setting for a whole workload. It pairs naturally with Claude Opus 5.5: Opus plans and makes the judgment calls, and because Haiku is fast and cheap, you can run many Haiku subagents in parallel for routing, classification, and small code changes across many files.
  • Amazon Nova 2.5 Sonic for voice agents – You can now build real-time voice agents on a speech-to-speech Amazon Nova model with better reasoning, instruction following, and tool-calling accuracy, plus lower latency. That combination matters for multi-step work: a support agent can look up an order, check return eligibility, and start the return inside one conversation. It keeps expressive voices in seven languages, controllable turn-taking, and a 256K context window, and pairs with Strands Bidi Agents, now generally available, to get a production voice agent going in a few lines of code.
  • Amazon GuardDuty RDS Protection now detects data exfiltration and destruction – Amazon GuardDuty now looks past login anomalies to the queries themselves on Amazon Aurora PostgreSQL and Amazon RDS for PostgreSQL. RDS Protection for data activity builds behavioral baselines and uses ML-based anomaly detection plus threat intelligence to flag unusual data access, with no agents, extra infrastructure, or database configuration changes. This closes a real gap: once an attacker has valid credentials from phishing or a misconfiguration, their queries can look indistinguishable from your application’s.
  • AWS Lambda supports OAuth for self-managed Apache Kafka event sources – You can now authenticate Lambda Kafka consumers through your enterprise identity provider, such as Amazon Cognito or Okta. Event source mappings previously supported only SASL/PLAIN, SASL/SCRAM, and mTLS, which left teams in regulated industries unable to use Kafka ESM at all. Now you get the automatic scaling, batching, error handling, and event filtering of Kafka ESM with the same identity governance you apply everywhere else. It works with self-hosted clusters and managed offerings including Confluent Cloud, Aiven, and Redpanda.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news

Here are some additional posts and resources that you might find interesting:

  • Introducing Strands Box: AI agent sandboxes powered by Dogwood – As more of us let agents run shell commands and call APIs without approving every step, isolation alone stops being enough. Strands Box is an open source sandbox (Apache 2.0, developer preview) that combines OS-level containment with fine-grained policies written in Dogwood, so permissions can depend on what the agent has already done. You can say “let the agent post to the incident channel, but no more than three times every 10 minutes” and have that enforced from outside the agent’s process. It also keeps real credentials out of the agent’s environment by attaching them at an egress gateway.

Upcoming AWS events

Check your calendar and sign up for upcoming AWS events:

  • AWS re:Invent – re:Invent returns to Las Vegas from November 30 to December 4, and reserved seating is now open. The session catalog is live, so register and claim your spots in chalk talks, workshops, and builders’ sessions before they fill up.

Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development. Browse here for upcoming AWS-led in-person and virtual events and developer-focused events.

That’s all for this week. Check back next Monday for another Weekly Roundup!

– Daniel Abib

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!

All the numbers: Amazon Prime Day 2026 powered by AWS

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/all-the-numbers-amazon-prime-day-2026-powered-by-aws/

Amazon Prime Day 2026 was exclusively for Prime members and ran June 23–26, 2026 with millions of deals across more than 35 categories.

As part of our annual tradition of sharing how AWS powered Prime Day (2016, 2017, 2019, 2020, 2021, 2022, 2023, 2024, and 2025), I want to share the services and chart-topping metrics from AWS that made your amazing shopping experience possible.

Prime Day 2026 – all the numbers

As always, Prime Day was powered by AWS. Here are some of the most interesting and/or mind-blowing metrics:

Amazon Elastic Compute Cloud (Amazon EC2) and AWS Graviton – During Amazon Prime Day 2026, AWS Graviton powered up to 49% of the Amazon EC2 compute used by Amazon.com.

Amazon Elastic Block Store (Amazon EBS) – During Prime Day 2026, Amazon EBS, our high-performance block storage service, peaked at over 24.8 trillion I/O operations, moving over an exabyte of data daily.

AWS Lambda – AWS Lambda handled over 2.3 trillion invocations per day during Prime Day 2026.

Amazon Elastic Container Service (ECS) and Fargate – During Prime Day 2026, Amazon ECS launched an average of 158.3 million tasks per day on AWS Fargate, representing a 47.7 percent increase from the previous year’s Prime Day average.

Amazon CloudFront – Amazon CloudFront delivered over 2.1 trillion HTTP requests during the global week of Prime Day 2026, a 5 percent increase in total requests compared to Prime Day 2025.

Amazon DynamoDB – Amazon DynamoDB, a serverless, fully managed, distributed NoSQL database, powers multiple high-traffic Amazon properties and systems including Alexa, the Amazon.com sites, and all Amazon fulfillment centers. Over the course of Prime Day 2026, between June 23 and June 26, 2026, DynamoDB processed over 59 trillion requests. DynamoDB maintained high availability while delivering single-digit millisecond responses and peaking at 192 million requests per second.

Amazon Aurora – On Prime Day, Amazon Aurora, a relational database management system (RDBMS) built for high performance and availability at global scale for PostgreSQL, MySQL, and DSQL, processed hundreds of billions of transactions, stored 5,491 terabytes of data, and transferred 1,194 terabytes of data.

Amazon ElastiCache – During Prime Day, Amazon ElastiCache peaked at serving over 2.3 quadrillion daily requests and 2.1 trillion requests in a minute.

Amazon Kinesis Data Streams – Amazon Kinesis Data Streams, a fully managed serverless data streaming service, processed a peak of 988 million records per second during Prime Day 2026.

Amazon Simple Notification Service (SNS) – Amazon SNS, a fully managed pub/sub messaging service for application-to-application and application-to-person communication, delivered 5 trillion messages in a single day during Prime Day 2026.

Amazon Simple Queue Service (SQS) – Amazon SQS, a fully managed message queuing service for microservices, distributed systems, and serverless applications, received a peak of 213 million messages per second during Prime Day 2026

AWS CloudTrail – AWS CloudTrail processes billions of API activity events per day for governance, compliance, and operational auditing. During Prime Day 2026, that volume surged to 3.6 trillion events in just four days, a 44% increase over Prime Day 2025.

AWS CloudWatch – Amazon CloudWatch processed over 2.15 quadrillion metric observations per day during Prime Day 2026.

Amazon GuardDuty – During Prime Day 2026, Amazon GuardDuty monitored an average of 14.08 trillion log events per hour, a 59% increase from last year’s Prime Day.

AWS Fault Injection Service (FIS) – We ran over 44,000 AWS FIS experiments – more than six times what we conducted in 2025 – to help ensure Amazon.com remains highly available on Prime Day.

Prepare to scale

If you’re preparing for similar business-critical events, product launches, and migrations, I recommend that you take advantage of AWS Countdown Premium. From retail peak seasons to major sporting events, elections, and healthcare enrollment periods, we help you deliver flawless experiences when it matters most. Our experts help you manage your infrastructure to handle massive traffic spikes while maintaining security and performance. We work alongside your team to scale infrastructure, optimize costs during demand surges, strengthen security measures, and monitor real-time demands. To learn more, visit AWS Countdown Premium for business critical events.

I look forward to seeing what other records will be broken next year!

— Channy

Amazon Aurora PostgreSQL now supports direct querying of Apache Iceberg and Parquet data in your data lake

Post Syndicated from Esra Kayabali original https://aws.amazon.com/blogs/aws/amazon-aurora-postgresql-now-supports-direct-querying-of-apache-iceberg-and-parquet-data-in-your-data-lake/

Today, we’re announcing a new capability for Amazon Aurora PostgreSQL that you can use to directly query operational data together with data stored in your data lake in Apache Iceberg and Apache Parquet formats, using your existing PostgreSQL applications and tools. By eliminating the need to extract, transform, and load (ETL) structured data from data lakes into your operational database, you can reduce operational complexity and simplify application development. You can also use Aurora PostgreSQL to query data from data lakes managed in Iceberg REST Catalog (IRC)-compatible catalogs, giving you access to data across a breadth of analytics systems without moving or duplicating it. Whether you’re powering real-time dashboards, enriching transactions with historical context, or building AI agents that reason over both live and archived data, you can now do it all through a single, familiar interface.

Previously, if your application needed to combine recent transactional data in Aurora with historical records stored in Amazon S3, a common approach was to build reverse ETL pipelines that duplicated data, increased infrastructure costs, and required ongoing engineering effort to keep everything synchronized. This challenge only grows as you increasingly embed AI agents into your applications, where it is impractical to predict and pre-replicate every dataset an agent might need.

DuckLabs, the team that maintains the DuckDB project, recently joined Amazon, and this capability is an example of how the efficiency of DuckDB is being integrated into our services. DuckDB is now embedded directly within Aurora PostgreSQL, so you can query live operational data (including uncommitted writes) alongside your data lake in a single query. Query processing stays within Aurora, with no additional network hops and no ETL pipelines that duplicate data. You can query Apache Iceberg tables managed through the AWS Glue Data Catalog, as well as Parquet and Iceberg data stored in Amazon S3 and S3 Tables. You do all of this using familiar PostgreSQL syntax and your existing applications and tools.

We’re excited to bring the speed and simplicity of DuckDB directly into Aurora PostgreSQL, so you and your agents can query and combine operational and Iceberg data using the familiar PostgreSQL applications, tools, and endpoints already in use. By building this capability around DuckDB, future improvements to the open source engine can continue to bring performance and functionality gains to Aurora and other AWS services.

What is new

This capability is supported on two Aurora PostgreSQL major versions: 17 (starting with 17.11) and 18 (starting with 18.6). To use it, you create an Aurora PostgreSQL cluster, attach an IAM role with the AuroraAnalytics feature, and enable the aurora_analytics extension. The IAM role is what gives Aurora access to your data in Amazon S3 and the AWS Glue Data Catalog. You then create foreign tables that point to your Iceberg or Parquet data in the data lake, and query them using familiar PostgreSQL syntax. You can complete this setup through the Amazon RDS console, or with any PostgreSQL client such as psql. The process is well documented in the Aurora PostgreSQL documentation.

You can query data across external IRC-compatible catalogs through AWS Glue Data Catalog federation. You register the external catalog once with Glue, and then create foreign tables for the tables you want to query, the same way you would for any Glue-native table. A single query can then join data stored in Aurora with Iceberg tables registered across multiple catalogs, so applications get a unified view without moving data or replacing your existing catalog investments.

Aurora also applies optimizations such as predicate pushdown and column pruning so that only the relevant data is read. This keeps queries efficient even as the underlying data grows. Frequently accessed data is also cached in your Aurora instance, so subsequent queries against the same data return faster. You can inspect this behavior per query using aurora_analytics_stat_statements(), which reports metrics such as rows scanned, bytes read from Amazon S3, and cache hits.

To see how direct querying works, I connected to my Aurora PostgreSQL database using psql and created the extension:

CREATE EXTENSION aurora_analytics;

For my walkthrough, I set up a simple financial scenario. I have a recent_transactions table in Aurora with the last 7 days of customer transactions, and a Parquet file in Amazon S3 containing 5 years of historical transaction data. To make Aurora aware of the historical data, I created a foreign table pointing at the Parquet file in S3:

CREATE FOREIGN TABLE transaction_history ()
SERVER aurora_analytics_server
OPTIONS (
    location 's3://<my-bucket>/finance/transaction_history.parquet',
    format 'parquet'
);

Notice the empty parentheses in the CREATE FOREIGN TABLE statement. Aurora automatically reads the schema from the Parquet file metadata, so you do not need to define columns manually. For workloads with many tables, you can skip creating them one at a time: a single IMPORT FOREIGN SCHEMA statement bulk-creates foreign tables for every Iceberg or Parquet table in an AWS Glue Data Catalog database, inferring schemas automatically.

With both tables in place, I ran a single query that combines the recent operational data in Aurora with the historical data in S3:

SELECT merchant, category, amount, transaction_date, 'recent' AS source
FROM recent_transactions
WHERE customer_id = 'C-1001'
UNION ALL
SELECT merchant, category, amount, transaction_date, 'historical' AS source
FROM transaction_history
WHERE customer_id = 'C-1001'
  AND transaction_date >= CURRENT_DATE - INTERVAL '5 years'
ORDER BY transaction_date DESC
LIMIT 15;

The result shows both recent and historical transactions in a single result set. The 7 most recent rows come from Aurora, and the rest come directly from the Parquet file in S3. DuckDB handles the analytical scan of the Parquet data under the hood, while Aurora handles the operational data. That single query would have previously required a pipeline to move the historical data into the database first.

If a query pattern needs single-digit-millisecond latency, you can materialize data from the data lake into a native Aurora PostgreSQL table using familiar commands such as CREATE TABLE AS SELECT, INSERT INTO ... SELECT, or MERGE INTO. The materialized table lives in Aurora and is queried like any other PostgreSQL table, giving you a low-latency path for hot data without operating a separate ingestion pipeline. The read queries can run on any Aurora PostgreSQL instance in your cluster, whether the writer or a read replica, so you can offload analytical scans from your operational workload. The materialization commands write data into Aurora, so they run on the writer instance.

Get started today

Direct querying of Apache Iceberg and Parquet data from Amazon Aurora PostgreSQL is available today in all commercial AWS Regions and AWS GovCloud (US) Regions, at no additional charge. You pay only for the incremental Aurora compute the queries consume and Amazon S3 request costs for reading data lake files.

To learn more, visit the Amazon Aurora features page, read the Aurora PostgreSQL documentation, or try it in the Amazon RDS console. We welcome your feedback through AWS re:Post or through your usual AWS Support contacts.

— Esra

Aurora PostgreSQL zero-ETL integration with Amazon SageMaker

Post Syndicated from Apurwa Pawar original https://aws.amazon.com/blogs/big-data/aurora-postgresql-zero-etl-integration-with-amazon-sagemaker/

When you need quick insights from your Amazon Aurora PostgreSQL operational data, traditional analytics approaches force you to build complex extract, transform, and load (ETL) pipelines. These pipelines introduce latency, operational overhead, and data silos, which slow down decision making and increase cost. AWS introduced the support for Amazon Aurora PostgreSQL zero-ETL integration with Amazon SageMaker, providing near real-time data availability for analytics workloads.

The zero-ETL integration automatically replicates the data from your Amazon Aurora PostgreSQL database into a target AWS Glue managed catalog, where it’s available as Apache Iceberg tables. You can then analyze this data through Amazon SageMaker alongside data from other sources using your preferred analytics and machine learning (ML) tools. The data is compatible with Apache Iceberg open standards, so you can use SQL, Apache Spark, business intelligence, and artificial intelligence and machine learning (AI/ML) tools.

In this post, you explore the benefits of this integration, the architectural concepts, and the underlying change data capture (CDC) mechanics. You also go through the setup process and learn how to query your Aurora PostgreSQL data in Amazon SageMaker AI.

Zero-ETL in the lakehouse architecture

The lakehouse architecture of Amazon SageMaker AI brings together data across Amazon Simple Storage Service (Amazon S3) data lakes and Amazon Redshift data warehouses. Because it’s built on open standards, you can build analytics and AI/ML applications on a single copy of data, without moving it between systems.

Amazon SageMaker AI uses AWS Glue Data Catalog and AWS Lake Formation to provide integrated access controls across S3 data lakes and Amazon Redshift data warehouses from a single governance plane.

Understanding change data capture mechanics

At its core, Aurora PostgreSQL zero-ETL integration is powered by CDC. CDC continuously monitors the database transaction log and streams every insert, update, and delete to a downstream target in near real time.

Aurora PostgreSQL uses enhanced logical replication as its CDC engine. Standard PostgreSQL logical replication publishes row-level changes from the write-ahead log (WAL). The enhanced logical replication in Aurora offers added capabilities that make it well-suited for zero-ETL integrations, including automatic DDL propagation and continuous streaming of transactional changes.

Solution overview

With Amazon Aurora PostgreSQL zero-ETL integration with Amazon SageMaker AI, you can:

  • Remove ETL complexity – Automatically replicate data without building custom ETL pipelines.
  • Near real-time analytics – Access operational data in Amazon SageMaker AI within seconds of changes in Aurora PostgreSQL.
  • Unify data analysis – Combine Aurora PostgreSQL data with data from other sources in a single lakehouse architecture.
  • Reduce costs – Minimize operational overhead and infrastructure costs associated with maintaining ETL pipelines.
  • Accelerate insights – Query data using familiar SQL tools and integrate with ML workflows in Amazon SageMaker AI.

The following diagram illustrates the architecture of this solution:

Aurora PostgreSQL zero-ETL integration replicating data into an AWS Glue managed catalog queried through Amazon SageMaker

Figure 1: Architecture of the Aurora PostgreSQL zero-ETL integration with Amazon SageMaker

The workflow includes the following steps:

  1. Your application writes data to an Amazon Aurora PostgreSQL database cluster.
  2. The zero-ETL integration automatically captures changes from the Aurora PostgreSQL database.
  3. Data is replicated to the target AWS Glue managed catalog in near real time.
  4. You can query and analyze the data using Amazon Athena, Amazon Redshift, or other analytics tools integrated with Amazon SageMaker AI.
  5. Data scientists can build and train ML models using Amazon SageMaker AI with direct access to the Apache Iceberg tables in the target AWS Glue managed catalog.

Prerequisites

Before setting up the zero-ETL integration, verify that you have the following:

Configure the source PostgreSQL database for zero-ETL integration

When you have all the prerequisites in place, you can configure the source PostgreSQL database for zero-ETL integration.

Create a custom Aurora PostgreSQL cluster parameter group

Your Aurora PostgreSQL database needs to have parameters configured for real-time replication. In this section, you will create the DB cluster parameter group and configure parameters. For more information, see Getting started with Aurora zero-ETL integrations.

Use the following AWS CLI command to create an Aurora PostgreSQL cluster parameter group:

aws rds create-db-cluster-parameter-group \
    --db-cluster-parameter-group-name aurora-pgsql-zetl-cluster-pg \
    --db-parameter-group-family aurora-postgresql16 \
    --description "Aurora PostgreSQL with enhanced logical replication" \
    --region us-east-1 --output json

Now set the parameters by modifying the parameter group:

aws rds modify-db-cluster-parameter-group --db-cluster-parameter-group-name <aurora-pgsql-zetl-cluster-pg> \
    --parameters \
    ParameterName=rds.logical_replication,ParameterValue=1,ApplyMethod=pending-reboot \
    ParameterName=aurora.enhanced_logical_replication,ParameterValue=1,ApplyMethod=pending-reboot \
    ParameterName=aurora.logical_replication_backup,ParameterValue=0,ApplyMethod=pending-reboot \
    ParameterName=aurora.logical_replication_globaldb,ParameterValue=0,ApplyMethod=pending-reboot \
    --region us-east-1 --output json

The parameter group is now fully configured and ready to be applied to your Aurora PostgreSQL cluster.

Select or create a source Aurora PostgreSQL cluster

If you already have an Aurora PostgreSQL cluster, you can use it, or you can create a new Aurora PostgreSQL cluster.

Note: Your source DB cluster must be running a supported version of Aurora PostgreSQL. For a list of supported versions, see Regions and database engines supported for Aurora zero-ETL integrations.

While creating an Aurora PostgreSQL cluster, use the parameter group (aurora-pgsql-zetl-cluster-pg) you created earlier:

Note: Throughout this post, make sure to replace the with your own information.

aws rds create-db-cluster \
    --db-cluster-identifier <aurora-pgsql-zetl> \
    --engine aurora-postgresql \
    --engine-version 16 \
    --master-username <admin> \
    --master-user-password <password> \
    --database-name <my_db> \
    --db-cluster-parameter-group-name <aurora-pgsql-zetl-cluster-pg> \
    --storage-encrypted \
    --kms-key-id alias/aws/rds \
    --backup-retention-period 7 \
    --db-subnet-group-name <dbsubnet> \
    --vpc-security-group-ids <sg-c14219ba> \
    --region <us-east-1> \
    --output json
aws rds create-db-instance \
    --db-instance-identifier <aurora-pgsql-zetl-instance-1> \
    --db-instance-class db.r5.large \
    --engine aurora-postgresql \
    --db-cluster-identifier <aurora-pgsql-zetl> \
    --region us-east-1 \
    --output json

If you’re creating a new Aurora PostgreSQL cluster, wait for your DB instance(s) to be in an “Available” status. You can verify DB instance status by using the describe-db-instances API call:

aws rds describe-db-instances --filters 'Name=db-cluster-id,Values=<aurora-pgsql-zetl>' --output json | grep -o '"DBInstanceStatus": "[^"]*"'
"DBInstanceStatus": "available"

Reboot the cluster to apply parameter changes

A cluster reboot is needed before zero-ETL integration can function correctly:

aws rds reboot-db-instance \
    --db-instance-identifier <aurora-pgsql-zetl-instance-1> \
    --region <us-east-1>

Wait until the cluster and the primary instance are back in Available status. For more information, see reboot-db-instance.

Create a target AWS Glue managed catalog

With your source PostgreSQL database configured for enhanced logical replication, the next step is setting up your target Amazon SageMaker AI. Zero-ETL integration uses AWS Glue Data Catalog backed by Amazon Redshift managed storage as its target. To have this functionality, you need to create a managed catalog, configure IAM permissions for Amazon SageMaker AI to access and query the managed catalog, and set up authorization for incoming integration requests from your source database.

Create an AWS Glue managed catalog

You must create a new catalog (if it doesn’t exist already) managed by AWS Glue to store table metadata and serve as the landing zone for your replicated datasets. Zero-ETL integration streams the data into Amazon Redshift managed storage, and AWS Glue keeps track of table definitions so that tools such as SageMaker AI, Athena, and Amazon Redshift Spectrum can query the data.

Create an IAM role for AWS Glue and Amazon Redshift to access the AWS Glue managed catalog

Now, use the following command to create an IAM role so that AWS Glue and Amazon Redshift can interact with the catalog. This role serves two key functions: It allows AWS Glue and Amazon Redshift to perform catalog operations, and it authorizes incoming integration requests from your source database.

aws iam create-role \
    --role-name <GlueDataCatalogDataTransferRole> \
    --assume-role-policy-document '{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "Service": [
                        "glue.amazonaws.com",
                        "redshift.amazonaws.com"
                    ]
                },
                "Action": "sts:AssumeRole"
            }
        ]
    }'

Next, attach a policy to this IAM role that provides the minimum required permissions for AWS Glue and Amazon Redshift. This policy should also include the necessary permissions for encryption key actions to help maintain secure data handling throughout the integration process:

aws iam put-role-policy \
    --role-name <GlueDataCatalogDataTransferRole> \
    --policy-name <GlueDataTransferPolicy> \
    --policy-document '{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "DataTransferRolePolicy",
                "Effect": "Allow",
                "Action": [
                    "kms:GenerateDataKey",
                    "kms:Decrypt",
                    "glue:GetDatabase",
                    "glue:GetCatalog"
                ],
                "Resource": ["*"]
            }
        ]
    }'

Set up AWS Lake Formation access

Before using the managed catalog for zero-ETL integration, you must configure data lake administrators in AWS Lake Formation who have administrative or read-only permissions on the managed resources. Additionally, you need to grant ReadOnlyAdmin permissions to the Amazon Redshift service-linked role, AWSServiceRoleForRedshift, in your account. If this role doesn’t exist in your account or you need to verify its permissions, see Using service-linked roles for Amazon Redshift.

aws lakeformation put-data-lake-settings \
    --region <us-east-1> \
    --cli-input-json '{
        "DataLakeSettings": {
            "DataLakeAdmins": [
                {
                    "DataLakePrincipalIdentifier": "<arn:aws:iam::111122223333:role/Admin>"
                }
            ],
            "ReadOnlyAdmins": [
                {
                    "DataLakePrincipalIdentifier": "<arn:aws:iam::111122223333:role/aws-service-role/redshift.amazonaws.com/AWSServiceRoleForRedshift>"
                }
            ],
            "CreateDatabaseDefaultPermissions": [],
            "CreateTableDefaultPermissions": [],
            "Parameters": {
                "CROSS_ACCOUNT_VERSION": "4",
                "SET_CONTEXT": "TRUE"
            },
            "AllowExternalDataFiltering": false,
            "ExternalDataFilteringAllowList": []
        }
    }'

Create the AWS Glue managed catalog backed by Amazon Redshift managed storage

Because you have configured IAM permissions and Lake Formation settings, you can now create the AWS Glue managed catalog.

aws glue create-catalog \
    --region <us-east-1> \
    --cli-input-json '{
        "Name": "<zetl-catalog>",
        "CatalogInput": {
            "Description": "A Glue Data Catalog backed by Redshift Managed Storage",
            "CreateDatabaseDefaultPermissions": [],
            "CreateTableDefaultPermissions": [],
            "CatalogProperties": {
                "DataLakeAccessProperties": {
                    "DataLakeAccess": true,
                    "DataTransferRole": "<arn:aws:iam::111122223333:role/GlueDataCatalogDataTransferRole>",
                    "CatalogType": "aws:redshift"
                }
            }
        }
    }'

Register the catalog as a zero-ETL integration target

To prepare your target AWS Glue managed catalog for zero-ETL integration, use the create-integration-resource-property command with these required parameters:

  • The –resource-arn parameter specifies the Amazon Resource Name (ARN) of your AWS Glue managed catalog that will serve as the integration target.
  • The –target-processing-properties parameter requires the ARN of an IAM role that has describe permissions on the target AWS Glue managed catalog.

You can use the GlueDataCatalogDataTransferRole created in the earlier step because it already includes the minimal describe permissions needed for this integration. Alternatively, you can create a new IAM role specifically for this purpose and attach the necessary minimal permissions to meet your company’s security requirements.

aws glue create-integration-resource-property \
    --region <us-east-1> \
    <arn:aws:glue:us-east-1:111122223333:catalog/zetl-catalog> \
    '{"RoleArn": "<arn:aws:iam::111122223333:role/GlueDataCatalogDataTransferRole>"}'

Example output:

{
    "ResourceArn": "arn:aws:glue:us-east-1:111122223333:catalog/zetl-catalog",
    "TargetProcessingProperties": {
        "RoleArn": "arn:aws:iam::111122223333:role/GlueDataCatalogDataTransferRole"
    }
}

Configure authorization for inbound integration requests

The last step in creating a target managed catalog is to define a resource-based access policy that authorizes zero-ETL integration to push data into your catalog. This policy grants AWS Glue the necessary permissions to create and authorize incoming integration requests from your source database. Apply this resource policy by using the AWS Glue put-resource-policy API call to complete the catalog configuration for your zero-ETL integration:

aws glue put-resource-policy \
    --region <us-east-1> \
    --policy-in-json '{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Principal": {
                    "AWS": [
                        "111122223333"
                    ]
                },
                "Effect": "Allow",
                "Action": [
                    "glue:CreateInboundIntegration"
                ],
                "Resource": [
                    "arn:aws:glue:us-east-1:111122223333:catalog/zetl-catalog"
                ],
                "Condition": {
                    "StringEquals": {
                        "aws:SourceArn": "arn:aws:rds:us-east-1:111122223333:cluster:aurora-pgsql-zetl"
                    }
                }
            },
            {
                "Principal": {
                    "Service": [
                        "glue.amazonaws.com"
                    ]
                },
                "Effect": "Allow",
                "Action": [
                    "glue:AuthorizeInboundIntegration"
                ],
                "Resource": [
                    "arn:aws:glue:us-east-1:111122223333:catalog/zetl-catalog"
                ],
                "Condition": {
                    "StringEquals": {
                        "aws:SourceArn": "arn:aws:rds:us-east-1:111122223333:cluster:aurora-pgsql-zetl"
                    }
                }
            }
        ]
    }'

Your AWS Glue managed catalog is now ready to receive data from the zero-ETL integration.

Load data in the source Aurora PostgreSQL database

Now that your Aurora PostgreSQL database is configured and ready, you must populate it with sample data that serves as the historical baseline for your zero-ETL integration. This first dataset provides the foundation for testing and demonstrating the integration capabilities. After you set up the zero-ETL integration, subsequent database changes stream automatically in near real time to your target AWS Glue managed catalog.

Connect to the source Aurora PostgreSQL cluster

Use the following commands to create a connection to your source Aurora PostgreSQL cluster:

psql --host aurora-pgsql-zetl-xxxxx.us-east-1.rds.amazonaws.com --username admin --port 5432 --dbname my_db --password

Create a database and table

Create a table named products to store product information:

CREATE TABLE products (product_id SERIAL PRIMARY KEY,product_name VARCHAR(100) NOT NULL, description TEXT,category VARCHAR(50),price NUMERIC(10,2) NOT NULL,stock_quantity INTEGER DEFAULT 0,is_active BOOLEAN DEFAULT TRUE,created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP );

Insert historical data

Use the following code to insert a row:

INSERT INTO products (product_name, description, category, price, stock_quantity) VALUES ('Laptop', 'High-performance laptop with 16GB RAM and 512GB SSD', 'Electronics', 1299.99, 50);

This table serves as a representative dataset to demonstrate the data capture and streaming capabilities of the zero-ETL integration. After your zero-ETL integration is active, all database changes, including inserts, updates, and deletes, are automatically captured and streamed to your AWS Glue managed catalog. This creates a data pipeline from your Aurora PostgreSQL database to your Amazon SageMaker for real-time analytics on your operational data.

Create a zero-ETL integration

Because your Aurora PostgreSQL database is now populated with historical data, you can set up the zero-ETL integration that continuously streams database changes to your AWS Glue managed catalog backed by Amazon Redshift managed storage.

Create the integration

Create the integration between your source PostgreSQL database and target AWS Glue catalog by using the aws rds create-integration AWS CLI command. You can customize the integration by specifying added configurations, such as data filters, to control which data gets replicated to your target environment:

aws rds create-integration \
    --source-arn <arn:aws:rds:us-east-1:111122223333:cluster:aurora-pgsql-zetl> \
    --target-arn <arn:aws:glue:us-east-1:111122223333:catalog/zetl-catalog> \
    --integration-name <zetl-test-integration> \
    --data-filter "include: *.*" \
    --region us-east-1

When you run the command, the zero-ETL integration begins provisioning and enters a ‘creating’ state. The AWS CLI response provides key details about the integration configuration.

Example CLI output:

{
    "SourceArn": "<arn:aws:rds:us-east-1:111122223333:cluster:aurora-pgsql-zetl>",
    "TargetArn": "<arn:aws:glue:us-east-1:111122223333:catalog/zetl-catalog>",
    "IntegrationName": "<zetl-test-integration>",
    "IntegrationArn": "<arn:aws:rds:us-east-1:111122223333:integration:4c4d81b9-xxxx>",
    "KMSKeyId": "<arn:aws:kms:us-east-1:111122223333:key/b9130ae1-exxxx>",
    "Status": "creating",
    "Tags": [],
    "CreateTime": "2025-07-04T05:42:56.841000+00:00",
    "DataFilter": "include: *.*"
}

When the integration status changes to “active”, your zero-ETL integration pipeline is fully operational.

Monitor the integration

Before generating new live data, verify that the integration has reached an “active” state by running the describe-integrations AWS CLI command. This monitoring step is important to confirm that changes from your source Aurora cluster are successfully streaming to the AWS Glue managed catalog without errors:

aws rds describe-integrations
{
    "Integrations": [
        {
            "SourceArn": "<arn:aws:rds:us-east-1:111122223333:cluster:aurora-pgsql-zetl>",
            "TargetArn": "<arn:aws:glue:us-east-1:111122223333:catalog/zetl-catalog>",
            "IntegrationName": "<zetl-test-integration>",
            "IntegrationArn": "<arn:aws:rds:us-east-1:111122223333:integration:4c4d81b9-xxxx>",
            "KMSKeyId": "<arn:aws:kms:us-east-1:111122223333:key/b9130ae1-xxxx>",
            "Status": "active",
            "Tags": [],
            "CreateTime": "2025-07-04T05:42:56.841000+00:00",
            "DataFilter": "include: *.*"
        }
    ]
}

Verify the zero-ETL integration

Now that your historical data is loaded and the zero-ETL integration is “active”, you must confirm that the data has been successfully replicated.

Grant Lake Formation permissions

Before you can query the AWS Glue managed catalog by using the Amazon Redshift Data API, you must make sure the IAM user or role has the right permissions to create and manage tables within the catalog. Use the Lake Formation grant-permissions API to provide these necessary permissions so that Amazon Redshift can access your AWS Glue managed catalog for the zero-ETL integration. For more information, see Creating an Amazon Redshift managed catalog in the AWS Glue Data Catalog.

aws lakeformation grant-permissions \
    --region <us-east-1> \
    --cli-input-json '{
        "Principal": {
            "DataLakePrincipalIdentifier": "<arn:aws:iam::111122223333:role/Admin>"
        },
        "Resource": {
            "Table": {
                "DatabaseName": "my_db",
                "CatalogId": "<111122223333:zetl-catalog/zetl_0dff6d97-xxxx>",
            },
            "Permissions": [
                "CREATE_CATALOG",
                "DESCRIBE",
                "CREATE_DATABASE",
                "DROP",
                "ALTER"
            ],
            "PermissionsWithGrantOption": [
                "CREATE_CATALOG",
                "DESCRIBE",
                "CREATE_DATABASE",
                "DROP",
                "ALTER"
            ]
        }'

These permissions allow for query execution and metadata inspection on the managed catalog.

Query historical data by using the Amazon Redshift Data API

With the necessary permissions in place, you can now verify your historical data by querying the AWS Glue managed catalog through the Amazon Redshift execute-statement Data API. Begin this verification process by running a SELECT statement against the catalog:

aws redshift-data execute-statement --sql 'SELECT * FROM "zetl_0dff6d97-xxxx@zetl-catalog"."my_db"."products" LIMIT 10;' --database "<arn:aws:glue:us-east-1:111122223333:catalog/pg-zetl-catalog>"

The following command returns a unique query ID that you can use to monitor the execution status and retrieve results from your query:

{
    "CreatedAt": "2025-07-15T00:31:48.778000+00:00",
    "Database": "arn:aws:glue:us-east-1:111122223333:catalog/zetl-catalog",
    "DbUser": "IAMR:Admin",
    "Id": "ce1ff0el-xxxx",
}

Monitor your query’s progress by using the describe-statement API with the query ID. Continue checking until the status shows that your query has completed successfully:

//use the Id to make the describe-statement API call to verify execution status is Started
aws redshift-data describe-statement --id <ce1ff0el-xxxx>
{
    "CreatedAt": "2025-07-15T00:31:48.778000+00:00",
    "Database": "arn:aws:glue:us-east-1:111122223333:catalog/zetl-catalog",
    "DbUser": "IAMR:Admin",
    "Duration": 6238051060,
    "HasResultSet": true,
    "Id": "2ca8fedf-xxxx",
    "QueryString": "SELECT * FROM \"zetl_0dff6d97-xxxx_zeroetl@pg-zetl-catalog\".\"zetl_default\".\"products\" LIMIT 10;",
    "RedshiftPid": 1073791309,
    "RedshiftQueryId": 1018598,
    "ResultFormat": "json",
    "ResultRows": 1,
    "ResultSize": 149,
    "Status": "FINISHED",
    "UpdatedAt": "2025-07-15T00:31:55.491000+00:00"
}

To complete the verification process and view your historical data now available in Amazon SageMaker AI, retrieve the query results by using the get-statement-result API call:

aws redshift-data get-statement-result --id <ce1ff0el-xxxx>
{
    "Records": [
        [
            {
                "longValue": 1
            },
            {
                "stringValue": "Laptop"
            },
            {
                "stringValue": "High-performance laptop with 16GB RAM and 512GB SSD"
            },
            {
                "stringValue": "Electronics"
            },
            {
                "stringValue": "1299.99"
            },
            {
                "longValue": 50
            },
            {
                "booleanValue": true
            },
            {
                "stringValue": "2026-02-27 13:32:02.697117"
            },
            {
                "stringValue": "2026-02-27 13:32:02.697117"
            }
        ]
    ],
    "ColumnMetadata": [
        ....
        //Skipping metadata
    ],
    "TotalNumRows": 1
}

With your zero-ETL integration now active, you can demonstrate real-time data streaming by adding new data to your source Aurora PostgreSQL instance. Run the following INSERT query to add a new row, which shows how changes are automatically replicated in near real time:

INSERT INTO products (product_name, description, category, price, stock_quantity)
VALUES ('Wireless Mouse', 'Ergonomic wireless mouse with USB receiver and long battery life', 'Electronics', 29.99, 150);

You can verify that the recent changes from your source database have been replicated to the target environment within seconds. Use the same Amazon Redshift Data API workflow you used earlier to confirm the real-time replication:

aws redshift-data execute-statement --sql 'SELECT * FROM "zetl_0dff6d97-xxxx@zetl-catalog"."my_db"."products" LIMIT 10;' --database "<arn:aws:glue:us-east-1:111122223333:catalog/pg-zetl-catalog>"
{
    "CreatedAt": "2025-07-15T00:31:48.778000+00:00",
    "Database": "arn:aws:glue:us-east-1:111122223333:catalog/zetl-catalog",
    "DbUser": "IAMR:Admin",
    "Id": "2ca8fedf-a604-4c87-a183-3a553d62354c",
}

Use the describe-statement API call to monitor the query execution and confirm that the status shows ‘FINISHED’ before proceeding to retrieve the results:

aws redshift-data describe-statement --id <ce1ff0ef-xxxx>
{
    "CreatedAt": "2025-07-15T00:31:48.778000+00:00",
    "Database": "arn:aws:glue:us-east-1:111122223333:catalog/zetl-catalog",
    "DbUser": "IAMR:Admin",
    "Duration": 6238051060,
    "HasResultSet": true,
    "Id": "2ca8fedf-xxxx",
    "QueryString": "SELECT * FROM \"zetl_0dff6d97-xxxx_zeroetl@pg-zetl-catalog\".\"zetl_default\".\"products\" LIMIT 10;",
    "RedshiftPid": 1073791309,
    "RedshiftQueryId": 1018598,
    "ResultFormat": "json",
    "ResultRows": 2,
    "ResultSize": 317,
    "Status": "FINISHED",
    "UpdatedAt": "2025-07-15T00:31:55.491000+00:00"
}

Finally, retrieve the query results by using the get-statement-result API call:

aws redshift-data get-statement-result --id <ce1ff0ef-xxxx>
{
    "Records": [
        [
            {
                "longValue": 1
            },
            {
                "stringValue": "Laptop"
            },
            {
                "stringValue": "High-performance laptop with 16GB RAM and 512GB SSD"
            },
            {
                "stringValue": "Electronics"
            },
            {
                "stringValue": "1299.99"
            },
            {
                "longValue": 50
            },
            {
                "booleanValue": true
            },
            {
                "stringValue": "2026-02-27 13:32:02.697117"
            },
            {
                "stringValue": "2026-02-27 13:32:02.697117"
            }
        ],
        [
            {
                "longValue": 2
            },
            {
                "stringValue": "Wireless Mouse"
            },
            {
                "stringValue": "Ergonomic wireless mouse with USB receiver and long battery life"
            },
            {
                "stringValue": "Electronics"
            },
            {
                "stringValue": "29.99"
            },
            {
                "longValue": 150
            },
            {
                "booleanValue": true
            },
            {
                "stringValue": "2026-02-27 15:41:00.206273"
            },
            {
                "stringValue": "2026-02-27 15:41:00.206273"
            }
        ]
    ],
    "ColumnMetadata": [
        ....
        //Skipping metadata
    ],
    "TotalNumRows": 2
}

This verification process confirms that your zero-ETL integration from Aurora PostgreSQL to Amazon SageMaker AI is working and continuously replicating both historical and real-time data. Although zero-ETL integration significantly simplifies data replication, it’s important to understand certain limitations on supported data types, schema change handling, and data filtering capabilities. For more details about these considerations and best practices, see Aurora zero-ETL integrations and Amazon RDS zero-ETL integrations.

Clean up

This section guides you through the cleanup process to remove the resources and components you created during this walkthrough. When you delete a zero-ETL integration, Amazon Aurora removes it from the source Aurora DB cluster. Your transactional data isn’t removed from Amazon Aurora or the analytics destination, but Aurora doesn’t send new data to Amazon SageMaker AI.

Delete the zero-ETL integration: Begin the cleanup process by removing the integration between your source Amazon Relational Database Service (Amazon RDS) database and the AWS Glue managed catalog. Run the following command to delete the integration:

aws rds delete-integration --integration-identifier <arn:aws:rds:us-east-1:111122223333:integration:4c4d81b9-af2a-4b09-b922-007636ba7f66>

Delete the AWS Glue managed catalog: After you successfully delete the integration, delete the AWS Glue managed catalog that served as your zero-ETL target destination. Use the following command to remove the catalog:

aws glue delete-catalog --catalog-id <111122223333:zetl-catalog>

This permanently removes all associated table metadata and Amazon Redshift managed storage references.

Delete the Aurora DB cluster: If you created the source Aurora DB cluster for this demonstration and you no longer need it, you can complete the cleanup by deleting the entire DB cluster. By skipping the final snapshot option, you avoid retaining any test data and confirm complete resource removal:

aws rds delete-db-instance --db-instance-identifier <aurora-pgsql-zetl-instance-1> --skip-final-snapshot --region <us-east-1>

aws rds delete-db-cluster --db-cluster-identifier <aurora-pgsql-zetl> --skip-final-snapshot --region <us-east-1>

aws rds delete-db-cluster-parameter-group --db-cluster-parameter-group-name <aurora-zetl-cluster-pg> --region <us-east-1>

Conclusion

In this post, you learned how to configure zero-ETL integration between Aurora PostgreSQL and your Amazon SageMaker AI using AWS CLI. This integration automatically replicates your PostgreSQL data to a lakehouse in near real time, removing the need for custom ETL pipelines.

As you move forward, consider expanding this zero-ETL approach to more supported data sources, such as Amazon RDS for MySQL and Amazon DynamoDB. This creates a centralized data access strategy across your company. You can also explore advanced analytics scenarios by combining zero-ETL integrations with Amazon Redshift capabilities. These include large-scale SQL analytics, Amazon Redshift ML for in-database ML, and federated queries that span multiple data lakes and warehouses. These integrations provide the foundation for building a near real-time data platform that scales with your business needs.

To get started, see the AWS zero-ETL documentation for setup guidance, supported configurations, troubleshooting integrations, and architectural best practices.

Related posts and references:


About the authors

Apurwa Pawar

Apurwa Pawar

Apurwa is a Solutions Architect at AWS and a Data Analytics and AI enthusiast. She helps customers build their modern data strategy and cloud-native innovative solutions on AWS. She works with enterprise organizations across industries including healthcare, life sciences, financial services, and hospitality, partnering with engineering and business leadership to turn data into insight and action.

Sarika Subramaniam

Sarika Subramaniam

Sarika is a Solutions Architect at AWS, specializing in analytics and data platforms. She helps customers design scalable, secure, and cloud-based modern data architectures on AWS. She works with enterprise customers across industries, partnering with engineering teams to build innovative data solutions and drive business outcomes.

Consistency is the new latency: AI at the data layer

Post Syndicated from Suman Chatterjee original https://aws.amazon.com/blogs/architecture/consistency-is-the-new-latency-ai-at-the-data-layer/

As AI applications scale from reactive bots to autonomous agents, their reliability is bound to the speed and accuracy of the data layer beneath them.

The integrity crisis nobody is talking about

There’s a quiet assumption baked into most AI architectures today regarding data layer consistency, and it’s costing companies more than they realize. The assumption is that the data your AI agent reads is the current state of reality.

In a world of distributed systems, cross-region replication, and autonomous agents making millisecond decisions, this assumption breaks down.

I’ve spent extensive time working with enterprise teams building agentic AI, and a recurring failure pattern emerges.

The breakdown isn’t in the model or the prompts. It’s in how we manage replication consistency when an agent performs the reading.

The context window is the new database row

In a modern agentic Retrieval-Augmented Generation (RAG) architecture, the database is the active memory of your AI. When an agent performs a task, it retrieves data to build its context window, forming the foundation of the large language model’s (LLM) reasoning.

If that data is even slightly out of date, the agent’s entire reasoning chain is invalidated. We must shift from simply managing data availability to strictly verifying contextual integrity.

The silent poison of asynchronous lag

In traditional web applications, asynchronous replication scales global reads with minimal write impact. If a user sees a post 500ms late, nobody notices.

For an autonomous AI agent, a 500ms delay is silent poison. If an agent writes a decision to a primary node and immediately reads from a lagging replica, it treats stale data as ground truth. It then executes a logically coherent, multi-step plan based on factually incorrect inputs.

In the age of AI, a fast answer that is wrong is more expensive than a slightly slower answer that is right.

The anatomy of a stale-read failure: When memory betrays logic

Consider an autonomous Inventory Reconciliation Agent managing a flash sale:

  1. The write: The agent updates available_stock to 500 units on the primary database in us-east-1.
  2. The lag: Network congestion causes a 2-second replication lag to the ap-south-1 (Mumbai) replica.
  3. The read: A secondary agent instance in Mumbai queries the replica and retrieves the old value: 0 units.
  4. The failure: The agent triggers a “Sold Out” notification and halts the sale, despite having 500 units in the warehouse.

The agent didn’t make a reasoning error. It performed logical operations on poisoned context.

Diagram of the stale read cascade, showing how replication lag feeds outdated data into an AI agent’s context

Figure 1: The stale read cascade, showing how replication lag poisons an AI agent’s context

The hallucination debt problem

When an agent writes an incorrect conclusion back to the database, that error becomes long-term memory. Future retrievals pull this poisoned history, creating a self-reinforcing cycle of “Hallucination Debt.”

LLMs amplify this because they lack a temporal compass. They cooperatively treat retrieved database results as current facts without hesitation. The burden of verifying contextual integrity falls entirely on the architecture.

The replication trinity: Choosing your truth

Not all AI tasks have the same consistency requirements. You must match your replication model to the specific “truth requirement” of the task.

Here are three architectural patterns I’ve found most effective.

Pattern A: Precision through global consistency

When an agent manages high-stakes data (user permissions, security policies, financial records, core system instructions), the cost of a stale read is unacceptable. You need strong consistency.

For many workloads, Amazon Aurora Global Database provides the necessary foundation. While its cross-region storage replication is asynchronous by default, you can close the consistency gap by turning on Global Write Forwarding with a GLOBAL consistency level.

To verify Read-Your-Own-Writes integrity, you configure the SESSION consistency level, which makes an agent wait for its own forwarded writes to replicate back before reading.

For the strongest consistency, the GLOBAL level makes a read query wait for replication to catch up to the exact point in time when the read started.

For the next generation of globally distributed AI, Amazon Aurora DSQL addresses this need. Aurora DSQL offers native synchronous strong consistency across multiple regions, so multi-agent systems can scale globally without compromising accuracy.

Every agent, regardless of location, operates on the exact same ground truth.

Best for: Identity metadata, financial ledgers, immutable system prompts.

Why it matters: Eliminates “mid-thought” state changes that cause contradictory behavior between agent instances.

Pattern B: Global availability at scale

For global AI agents that need ultra-low latency at massive scale, Amazon DynamoDB Global Tables offer a multi-leader architecture where data replicates across regions. For replication details, refer to the DynamoDB documentation.

The key technique here is Conditional Writes. By using a ConditionExpression that checks a version timestamp or whether an attribute exists, an agent updates a record only if the data hasn’t changed since it was last retrieved.

If the condition fails, DynamoDB returns a ConditionalCheckFailedException. This is a critical signal: it tells the agent to re-read the current state and reconsider its decision, rather than blindly overwriting another agent’s work.

This pattern prevents the “Lost Update” anomaly (where two agents running in parallel overwrite each other’s reasoning) without requiring synchronous global coordination.

Best for: Conversational history, user session state, personalized agent memory.

Why it matters: Handles concurrent updates from distributed agents while maintaining a shared memory that’s resilient to race conditions.

Pattern C: High-velocity intake

Some AI agents perform real-time anomaly detection or trend analysis on massive streams of telemetry data. In these cases, you need unthrottled ingestion above all else.

A leaderless architecture like Amazon Keyspaces (for Apache Cassandra) is designed for this workload.

Keyspaces provides highly available, predictable performance by automatically replicating data across three Availability Zones.

Every write is durably committed using LOCAL_QUORUM.

To make sure your AI agent doesn’t miss a critical spike in telemetry, you enforce strong consistency by setting its read operations to LOCAL_QUORUM rather than the eventually consistent LOCAL_ONE.

This quorum overlap means the agent retrieves the latest data without slowing down the high-speed ingestion pipeline.

It transforms a noisy, high-frequency data stream into a reliable foundation for real-time AI decision-making.

Best for: Internet of Things (IoT) telemetry, real-time log analysis, high-frequency sensor data.

Why it matters: Throughput is the priority, but you still need a safety valve to confirm the agent doesn’t miss critical spike data.

Conclusion: Becoming a context architect

Our role as architects has evolved.

We can no longer treat database replication as a background infrastructure concern, something to configure once and forget. In the era of autonomous agents, the stability of the data layer is the direct prerequisite for the trustworthiness of the AI. The two are inseparable.

By matching your replication model to your agent’s reasoning requirements, you move beyond simply managing data. You become a Context Architect, someone who works to confirm that every decision your AI makes is grounded in a synchronized version of the truth.

Because in the end, an AI is only as good as the context it operates in. And context is only as good as the data it’s built on.

Get the database layer right, and everything else follows.

References:


About the author

Eclipse Dataspace Components on AWS: Cost optimization strategies

Post Syndicated from Jorge Hernández Suárez original https://aws.amazon.com/blogs/architecture/eclipse-dataspace-components-on-aws-cost-optimization-strategies/

When you deploy Eclipse Dataspace Components (EDC) connectors on AWS, one of the first challenges you face is predicting and controlling the cost of the required infrastructure. Without clear benchmarks, it is difficult to make informed decisions about workload sizing, environment configuration, and long-term investment.

Part 1 of this 3-part blog series covered the fundamentals of data space architectures and the EDC per the International Data Space Association’s (IDSA) standards. Part 2 explored production-ready architecture patterns for deploying EDC connectors on Amazon Web Services (AWS), discussing operational excellence, security, and reliability principles. This final post covers the remaining three AWS Well-Architected Framework pillars: Performance Efficiency, Cost Optimization, and Sustainability.

In this post, you will learn which AWS services drive cost in an EDC connector deployment, how to estimate monthly costs for business-critical and non-critical workloads, and how to apply optimization strategies that can reduce your spending by up to 58%.

Understanding cost drivers in data space deployments

Data spaces are secure and sovereign data environments that enable data sharing across independent organizations. With these architectures, you can collaborate with external organizations while maintaining full control over your data and compliance with data sovereignty principles. Your infrastructure costs can vary significantly. The main factors are your performance and reliability requirements, along with data volume and velocity across the network. It’s also important to distinguish between two types of infrastructure. A Dataspace Governance Authority (DSGA) centrally establishes components such as management, identity, and discovery functions. Participants host other components themselves, including the connector. This blog post focuses only on costs associated with EDC connector deployment on the participant, that is, the data provider and consumer sides.

Fictional usage assumptions

Before diving into the numbers, here are technical and operational assumptions you can use as a baseline for your own estimates.

Technical assumptions

Category Assumption Justification
Data Volume 5 GB per participant Includes 6 months of historical data, and backups
Network Traffic 20 GB/month per participant Data transfers between participants
API Calls 100,000/month per participant Catalog queries, contract negotiations, and data transfers
OAuth Token Requests 1,000/month per participant Machine-to-machine authentication for data plane operations

Table 1: Technical assumptions for EDC connector cost estimation

Operational assumptions

  • Single AWS Region: Spain (eu-south-2)
  • Operating hours: 24/7/365.
  • Growth rate: Not considered in baseline estimates.
  • Disaster recovery: Automated backups only (no cross-region replication)

Deployment architecture and scenarios

Figure 1 shows the reference architecture for deploying production-ready EDC connectors on AWS, covered in depth in Part 2 of this series.

Production-ready EDC connector deployment architecture diagram showing AWS services including Amazon ECS, Amazon Aurora, Network Load Balancer, and supporting services

Figure 1: Production-ready EDC connector deployment

This post considers two cost scenarios depending on the criticality of the workload:

  • Business-critical workloads: Designed for high availability, performance, and reliability of use cases supporting critical business functions.
  • Non-critical workloads: Designed for use cases that tolerate interruptions, testing environments, or production workloads where brief disruptions are acceptable.

Both scenarios follow the architecture patterns described in Part 2 of this post series, with the primary differences being in sizing of compute and database resources.

Cost estimation: Business-critical workloads

Note: These estimates use the assumptions above and illustrate the relative cost contribution of each service. Your actual costs may vary based on your specific usage patterns, data volumes and regional pricing. This post highlights which components represent the highest cost drivers and therefore come with the highest potential for optimization.

AWS Service Configuration Monthly Cost (USD)
Amazon Aurora PostgreSQL-Compatible Edition db.r6g.large (2 vCPU, 16 GB), 20 GB storage + 10 GB backup 276.00
Amazon Elastic Container Service (Amazon ECS) with AWS Fargate 2 vCPU, 4 GB RAM, always on 83.00
Network Load Balancer 20 GB processed data 20.00
AWS Secrets Manager 10 secrets 4.00
Amazon Cognito 1K machine-to-machine (M2M) token requests 2.25
Amazon Elastic Container Registry (Amazon ECR) 2 GB storage, 10 GB transfer 1.00
Amazon API Gateway 100K REST API calls 0.40
Amazon Simple Storage Service (Amazon S3) 5 GB Standard tier 0.10
Total 387.00

Table 2: Estimated monthly cost for business-critical EDC connector deployment

These estimates help identify where your budget goes and where optimization has the most impact. In the business-critical scenario, the main cost driver is Amazon Aurora PostgreSQL. The db.r6g.large configuration is selected for constant workloads that require reliability and speed with high memory and performance. Amazon ECS with AWS Fargate is the second largest cost contributor as it runs containers continuously to maintain environment availability. Network Load Balancer represents a third notable cost component, while the remaining services contribute only a small portion of the total cost.

Cost estimation: Non-critical workloads

If you are running development, testing, or experimentation environments, you can reduce costs by up to 58% through rightsizing and use of Amazon EC2 Spot capacity.

AWS Service Configuration Monthly Cost (USD)
Amazon Aurora PostgreSQL-Compatible db.t4g.medium (2 vCPU, 4 GB), 20 GB storage + 10 GB backup 110.00
Amazon ECS with AWS Fargate Spot 2 vCPU, 4 GB RAM, always on 26.00
Network Load Balancer 20 GB processed data 20.00
AWS Secrets Manager 10 secrets 4.00
Amazon Cognito 1K M2M token requests 2.25
Amazon ECR 2 GB storage, 10 GB transfer 1.00
Amazon API Gateway 100K REST API calls 0.40
Amazon S3 5 GB Standard tier 0.10
Total 164.00

Table 3: Estimated monthly cost for non-critical EDC connector deployment

These figures show that a non-critical configuration can cut costs significantly while maintaining the same data throughput and API capacity. Costs are reduced through the use of smaller and more flexible resources. Amazon Aurora PostgreSQL remains the main cost driver, but the smaller instance type (db.t4g.medium) reduces cost significantly. From a compute perspective, using Amazon ECS with AWS Fargate Spot capacity cuts costs by almost 70% compared to the business-critical setup. In total, this configuration reduces the monthly cost by approximately 58%, while maintaining identical assumptions for data throughput, API calls, and storage.

Key takeaways on cost optimization

This comparison shows that the primary cost contributors in both scenarios are database, compute and load balancing resources, which represent baseline infrastructure costs rather than usage-based charges. Services like Amazon S3, API Gateway, and data transfer charges contribute marginally to overall costs at these volumes. This cost structure indicates that the architecture scales efficiently with increased usage. As you onboard more use cases and increase data volume and velocity, you get more value from your existing infrastructure investment without proportional cost increases.

Well-Architected pillars: Performance efficiency, cost optimization, and sustainability

Part 2 of this series covered EDC best practices along the Operational Excellence, Security, and Reliability pillars of the AWS Well-Architected Framework. This section covers the remaining three pillars as they apply to EDC deployments.

Performance efficiency

Right-size compute resources: Match your Amazon ECS task definitions to actual workload requirements. Start with smaller configurations and scale up based on observed metrics rather than over-provisioning from the start. Amazon CloudWatch Container Insights provides the visibility needed to make informed sizing decisions.

Use the flexibility of Amazon Aurora: For workloads with variable demand patterns, consider Amazon Aurora Serverless v2 which automatically scales database capacity based on application needs. This eliminates the need to provision for peak capacity while maintaining performance during high-demand periods.

Optimize data transfer patterns: Design your data plane operations to minimize unnecessary data movement. Use Amazon S3 Transfer Acceleration for large transfers across geographic distances and consider data compression where appropriate to reduce both transfer times and costs.

Cost optimization

Reduce compute costs for fault-tolerant workloads: With AWS Fargate Spot, you can save up to 70% for workloads that can tolerate interruptions. Non-critical environments, batch processing, and development workloads are ideal candidates. Implement graceful shutdown handling to manage Spot interruptions effectively.

Lower storage costs over time: Configure Amazon S3 Lifecycle policies to automatically transition infrequently accessed data to lower-cost storage classes such as S3 Intelligent-Tiering or S3 Glacier Instant Retrieval. For EDC connector deployments, historical transfer logs and archived assets are good candidates for tiered storage.

Monitor for unexpected cost increases: Use AWS Cost Explorer and set up AWS Budgets with alerts to help detect unexpected cost increases. Tag EDC-related AWS resources consistently so you can accurately allocate costs and identify optimization opportunities.

Lock in lower rates for predictable workloads: For business-critical connectors with predictable, steady-state usage, Savings Plans for Amazon Aurora and AWS Fargate can provide significant discounts compared to On-Demand pricing.

Sustainability

Optimize resource utilization: Higher utilization of provisioned resources means less waste. Use automatic scaling policies to match capacity with demand and shut down non-production environments outside of business hours when possible.

Select efficient instance types: AWS Graviton-based instances (such as the r6g and t4g families used in our example) deliver better price-performance and energy efficiency compared to equivalent x86 instances. AWS Graviton processors offer improved performance per watt of energy use.

Minimize data movement: Each data transfer consumes energy. Design your data space integrations to avoid redundant transfers, cache frequently accessed catalog data of peers locally using the Federated Catalog, and batch operations where possible to reduce the total number of network round trips.

Summary

By rightsizing AWS infrastructure to match actual compute and database capacity needs, data space participants can achieve significant cost savings without compromising on data security and sovereignty aspects that make data spaces valuable. The comparison between business-critical and non-critical workload configurations demonstrates how AWS services like Amazon Aurora, AWS Fargate Spot, and Amazon S3 can be combined effectively to balance data sovereignty, performance, and cost efficiency.

As data spaces grow in adoption across industries and geographies, understanding these cost dynamics becomes increasingly important as you plan your network participation. The patterns and estimates in this post series offer a foundation for planning your cross-organizational data strategy and data spaces journey on AWS.

To get started, assess your workload criticality to determine whether a business-critical or non-critical configuration fits your needs. Then use the AWS Pricing Calculator to estimate costs for your specific data volumes, regions, and usage patterns. For an end-to-end reference implementation, explore the Dataspace Connector on AWS project which combines Infrastructure-as-Code with custom EDC extensions and AI tooling integration.

References

About the authors

Eclipse Dataspace Components on AWS: Architecture patterns in production

Post Syndicated from Jonas Bürkel original https://aws.amazon.com/blogs/architecture/eclipse-dataspace-components-on-aws-architecture-patterns-in-production/

Running Eclipse Dataspace Components (EDC) connectors in production on AWS requires deliberate architecture decisions around isolation, managed services, and security layering. In Part 1 of this series, we covered the fundamentals of data space architectures and EDC per the International Data Space Association’s (IDSA) standards. If you are new to EDC, we recommend starting there. We showed how connector functionality can be customized to support native integration with Amazon Web Services (AWS) Cloud services. Examples include Amazon Simple Storage Service (Amazon S3) for data storage and AWS Secrets Manager for credentials management. In this post, we dive deeper into the connector deployment architecture on AWS and present patterns and practices for production-grade deployments.

Fundamental architecture building blocks

The EDC connector consists of a control plane and a data plane that customers typically ship and deploy as containers. Depending on data integration requirements and support for specific protocols and capabilities, a custom EDC build process may need to be implemented as described in Part 1 of this series. For example, you may need OAuth 2.0 client credentials for the data plane to connect to backend systems. You store the resulting EDC container images in a container registry, such as Amazon Elastic Container Registry (Amazon ECR). Figure 1 shows an example architecture for EDC connector deployments on AWS following best practices for production use.

Architecture diagram showing production-ready EDC connector deployment on AWS with Amazon ECS, Aurora, S3, and API Gateway components

Figure 1: Production-ready EDC connector deployment on AWS

You can split the architecture into four sub-components:

  • Amazon Elastic Container Service (Amazon ECS) and AWS Fargate provide serverless container orchestration. This allows for scalable EDC deployment without managing any of the underlying infrastructure.
  • EDC requires persistence to store secrets and relational control plane data, and a means of vending OAuth 2.0 client credentials. AWS Secrets Manager, Amazon Aurora and Amazon Cognito can provide these capabilities as managed services.
  • Amazon S3 provides durable data storage for handling both inbound and outbound data that is shared and received through the data space.
  • Finally, Amazon API Gateway and Network Load Balancer provide secure, private network connectivity to EDC APIs in an isolated Amazon Virtual Private Cloud (Amazon VPC) using VPC links.

With this approach, all cloud resources belonging to a single EDC connector instance form an isolated architecture cell. You access this cell through the S3 bucket to move in data that is to be shared as part of an EDC asset, or to retrieve data received from a third party as part of an EDC data transfer. Secondly, the API Gateway can be configured to expose selected EDC API resources from its management API, data plane API and Dataspace Protocol (DSP) API. You can protect both means of interacting with the EDC architecture cell using AWS Identity and Access Management (AWS IAM) and the AWS Signature Version 4 (SigV4) protocol.

Larger enterprises participating in data spaces may decide to operate multiple EDCs depending on their requirements on failure isolation, data governance, and separation of shared and received data. A common pattern is to deploy separate connector instances per use case. Infrastructure-as-code such as AWS Cloud Development Kit (CDK) allows for automated, templatized deployment and management of EDC connectors over time while keeping operational effort at bay. Using the Dataspace Connector on AWS reference implementation, a full connector cell deploys from a single CDK command, ready to negotiate contracts and transfer data. Amazon API Gateway also comes with Model Context Protocol (MCP) proxy support. This allows EDC APIs to be consumed by authorized AI agents and MCP clients for autonomous data collection and sharing. Besides integration with agentic systems, customers often follow a workflow-based approach for connecting EDCs with their cloud-based data environments. They interact with both APIs and the peripheral S3 bucket to securely expose and retrieve external information.

Real-world validation of these architecture patterns can be seen in production deployments like the Prometheus-X Data Space Connector, for education sector use cases. This implementation uses the same core architecture we recommend: Amazon ECS with AWS Fargate for container orchestration, S3 for data storage, and event-driven processing with AWS Lambda and Amazon EventBridge. This demonstrates how these patterns work effectively in production environments across different industry sectors.

Key principles for production-readiness

We discuss some of the architecture principles that inform the best practices diagram highlighted in Figure 1 along three of the AWS Well-Architected Framework’s pillars.

Operational Excellence

Infrastructure as Code for Consistency: Define all infrastructure declaratively to support repeatable, version-controlled, and testable deployments. Automated validation, for example using CDK Nag, helps catch misconfigurations and security issues before deployment, shifting security left in the development lifecycle. The code itself serves as living documentation of the architecture.

Observability as a First-Class Concern: Treat monitoring and logging as core infrastructure components. Amazon CloudWatch Container Insights, Amazon CloudWatch Logs, and EDC’s structured health check endpoints provide visibility into system behavior, supporting proactive issue detection and faster troubleshooting. EDC APIs for health checks can be similarly exposed with restricted access through API Gateway and IAM.

Managed Services Over Self-Managed Infrastructure: Use AWS managed services (Aurora, Secrets Manager, Fargate, Cognito) instead of deploying and maintaining compatible self-managed solutions. This shifts undifferentiated heavy lifting to AWS and reduces operational burden. You gain high availability, built-in security best practices, compliance certifications, and automatic updates.

Security

Defense in Depth: Implement security through multiple independent layers rather than relying on a single control. Network isolation (VPC private subnets), security group segmentation (restricting traffic between components), IAM least privilege (scoped permissions per service), and encryption (at rest and in transit) each provide independent controls. These layers work together so that if one layer is bypassed, others continue to provide protection.

Principle of Least Privilege: Every component receives only the minimum permissions required for its specific function. Scope IAM roles to individual services (control plane, data plane) and restrict security groups to necessary ports and sources. The internal-only Network Load Balancer fronted by API Gateway prevents unintended public exposure of EDC APIs and data. This may also support security review and approval of EDC as open-source software, since APIs can be allowlisted and validated individually.

Encryption Everywhere: Encrypt data by default at every stage: at rest (Aurora, S3, Secrets Manager), in transit (TLS enforcement, HTTPS-only egress), and during processing (encrypted environment variables). This provides comprehensive data protection regardless of where information resides in the system.

Reliability

Fail Fast, Recover Automatically: Systems detect failures quickly and recover without manual intervention. ECS circuit breakers can automatically roll back failed deployments, automated health checks remove unhealthy targets, and point-in-time recovery supports rapid database restoration. This minimizes mean time to recovery (MTTR) and reduces the scope of failures, even within a single EDC architecture cell.

Design for Regional Resilience: Cross-zone load balancing distributes traffic across multiple Availability Zones (AZs), Aurora automatically replicates data across AZs, and Fargate tasks can be scheduled in any AZ. The highlighted architecture can tolerate Availability Zone failures in an AWS Region without service disruption. For more information about Availability Zones and Regions, see AWS Global Infrastructure.

Decoupled Components with Clear Boundaries: Deploy the control plane and data plane as independent services with distinct responsibilities, security contexts, and scaling characteristics. This separation enables independent updates, targeted scaling, and failure isolation between coordination logic and data transfer operations.

The remaining three Well-Architected Framework pillars of Performance Efficiency, Cost Optimization, and Sustainability are covered in the third post of this series where we discuss cost optimization strategies for running EDC connectors on AWS.

Conclusion

With the growing popularity of data spaces and EDC as a data space connector, it is important to distinguish between a setup suitable for testing and experimentation and one that is ready for production. Production environments require that business-critical processes depend on timely, successful transmission of confidential information between participants. The architecture defined in this post combines EDC deployment best practices from the community with AWS recommendations to achieve fault tolerance, scalability, and security while keeping operational complexity at a minimum.

In part 3, you will learn about cost optimization strategies to run your production-ready connector efficiently and maximize the value it returns by supporting business use cases for data sharing along your supply network. In the meantime, explore the Dataspace Connector on AWS project and see how the patterns and best practices covered in this post come together in an end-to-end reference implementation.

References

About the authors

Eclipse Dataspace Components on AWS: Data sharing fundamentals

Post Syndicated from Alejandro Esquivias Cañadas original https://aws.amazon.com/blogs/architecture/eclipse-dataspace-components-on-aws-data-sharing-fundamentals/

This three-part blog series guides you through implementing Eclipse Dataspace Components (EDC) on AWS, from foundational concept to production deployment. Part 1 establishes the theoretical foundation with IDSA standards, the Dataspace Protocol (DSP), and core EDC architecture. Part 2 provides production-ready AWS deployment patterns using services like Amazon Elastic Container Service (Amazon ECS), Amazon Aurora, and Amazon API Gateway. Part 3 completes the journey with cost optimization strategies and practical guidance for running efficient, scalable data space infrastructure on AWS.

The International Data Spaces Association (IDSA) is a non-profit organization focused on establishing and promoting standards for data spaces. A data space is defined as a “set of technical services that facilitate interoperable dataset sharing between distinct entities”. At a technical level, a data space has participants: data consumers and data providers. Centrally, there is a Dataspace Governance Authority (DSGA). The DSGA manages the data space, enforces rules and policies and issues membership credentials to participants.

IDSA rules and specifications are implemented in the Dataspace Protocol (DSP). The DSP has become an Eclipse Specification project and can be found in the Dataspace Protocol GitHub repository. Standardization is under development and is currently in the approval phase as ISO/IEC DIS 20151. The Eclipse Dataspace Components (EDC) provide the technical components to implement data spaces according to IDSA requirements.

These components include the federated catalog (FC), the connector, and the identity hub. The FC contains an aggregated repository of catalogs gathered from multiple participants in the data space. These are obtained by periodically crawling participants’ data assets and storing them in a local cache, to eliminate the need to query each participant individually on demand. The connector is the software that enables data to be shared between participants, and identity hub manages a participant’s credentials in the data space. Amazon Web Services (AWS) provides a comprehensive cloud infrastructure that supports the deployment and operation of data space components like the EDC connector. AWS offers scalable compute, storage, and networking services that help you build secure, compliant, and interoperable data spaces aligned with IDSA standards and the ISO/IEC DIS 20151 specifications.

To verify identities in a data space in a decentralized manner, the Decentralized Claims Protocol (DCP) is used. DCP represents an overlay on top of DSP to establish trust between network participants. When an issuer needs to prove their identity to a verifier, they first generate a Decentralized Identifier (DID), which contains information about their identity. The issuer stores their Verifiable Credential (VC) in their identity hub. A verifiable credential is similar to a certificate: while a certificate authority validates a public key, the credential issuer validates the DID with specific attributes. The verifier looks up the DID of the issuer and with the information obtained from the DID document verifies the VC.

High-level diagram showing how DCP establishes trust between an issuer and verifier using DIDs and Verifiable Credentials

Figure 1: High-level diagram of the Decentralized Claims Protocol (DCP)

Simplified overview showing how a verifier resolves a DID document to verify credentials

Figure 2: Simplified overview of DID document processing

The final element of the EDC are the different types of policies, including membership, access, contract and usage policies.

The role of the connector

The EDC connector is divided into two parts: the control plane and the data plane. The control plane handles contract negotiation and sends messages to the data plane to initiate a data transfer. The data plane is responsible for transferring data from a provider’s to a consumer’s EDC connector across distinct legal entities.

Diagram showing the EDC connector architecture with separate control plane and data plane

Figure 3: Overview of the connector control and data plane

Structure of the EDC connector open-source project

The Connector repository as part of the EDC project defines how the connector control and data planes are implemented. We provide an overview of how the repository is structured:

  • /spi – The Service Provider Interface (SPI) is the architectural foundation that defines how modules communicate within the EDC connector. It contains foundational interfaces and contracts that every component must implement, establishing standardized integration patterns. This layer also acts as a blueprint for developers to extend EDC connector functionality while maintaining clean separation of concerns and ensuring compatibility between core and custom components.
  • /core – The Core module represents the core SPI implementation. It houses the actual working code for the connector’s essential operations, including default implementations of key services and business logic for data sharing.
  • /extensions – The Extensions layer showcases the connector’s modular plugin architecture through real-world integrations, including connections to major cloud providers like AWS. These extensions serve both as reference implementations and ready-to-use components for common enterprise integration scenarios.

SPI defines the contracts, Core implements the basics, and Extensions add specialized functionality. These three layers work together to create a flexible, extensible data space connector system. The “vanilla” Eclipse EDC connector does not bundle the AWS extensions by default. To add AWS service-specific functionality, such as integration with Amazon Simple Storage Service (Amazon S3), AWS Secrets Manager, or Amazon DynamoDB, you need to include the respective AWS extensions into a custom EDC control and data plane build.

High-level EDC customization guide

Customizing the EDC connector for native AWS service integration creates a purpose-built solution that can use AWS managed services for storage, security, and scalability. The following steps help your connector natively integrate with services like Amazon S3 for data storage (used for example as Asset Administration Shell “Submodel Server”) and AWS Secrets Manager for credentials management, rather than relying on operations of self-managed components.

EDC uses a modular, plugin-based architecture built on Gradle. The vanilla connector ships only core functionality. To integrate with specific cloud services or persistence backends, you assemble a custom build that combines EDC’s core modules with the extensions you need. The customization revolves around three concepts: a version catalog, launcher modules, and a project settings file.

Step 1: Version catalog

The version catalog (gradle/libs.versions.toml) is the centralized registry of all dependencies and their versions. Here you declare which EDC modules, cloud provider extensions, and third-party libraries your connector needs to use. EDC publishes its AWS extensions under the org.eclipse.edc.aws Maven group. You reference these alongside the core EDC artifacts (org.eclipse.edc) at compatible version numbers. This single file ensures all modules in your project share consistent dependency versions.

Step 2: Launcher modules

The launcher modules are the actual deployable units: one for the control plane and one for the data plane. Each launcher is a small Gradle subproject whose build.gradle.kts lists the extensions to bundle at runtime. A typical control plane launcher might include the core control plane module, a metadata persistence extension (such as PostgreSQL or DynamoDB), a vault extension (such as HashiCorp Vault or AWS Secrets Manager), and any provisioning extensions for your target storage system. The data plane launcher follows the same pattern with data plane-specific modules. Both launchers typically use Gradle plugins to produce a single executable JAR. You only include what you actually use, keeping the connector lightweight and tailored to your unique requirements.

Step 3: Project settings

The project settings file (settings.gradle.kts) registers all modules with Gradle: your launcher subprojects and any custom extensions you may develop locally. If you write your own extension, for example, a DynamoDB-backed asset store, you place it in an extensions/ directory and register it here so your launchers can reference it as a project dependency.

The resulting project structure typically looks like this:

my-connector/
├── control-plane/
│   └── build.gradle.kts       # Control plane launcher
├── data-plane/
│   └── build.gradle.kts       # Data plane launcher
├── extensions/                # Optional: Custom extension code
├── gradle/libs.versions.toml  # All dependency versions
├── build.gradle.kts           # Root build config
└── settings.gradle.kts        # Subproject registrations

An open-source reference implementation for EDC customization on AWS is provided as part of the Dataspace Connector on AWS project. In this post, you learned about the fundamentals behind secure, cross-organizational data sharing using emerging data space architectures. We covered the two protocols included in ISO/IEC DIS 20151 data space standardization, DSP and DCP, and discussed the EDC connector, the main software required for data space participants, and how it can be customized.

In part 2, we explore production-ready deployment patterns for EDC connectors on AWS, examining architecture best practices that support your data space infrastructure’s security, reliability, and operational efficiency. To get started, explore the Dataspace Connector on AWS project and see how its Gradle build settings are configured to support Amazon DynamoDB for serverless, inexpensive metadata persistence.

References

About the authors

AWS Weekly Roundup: AWS Builder Center at 1 year, Network Scanning in Security Hub, Loom for AWS, and more (July 13, 2026)

Post Syndicated from Esra Kayabali original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-aws-builder-center-at-one-year-network-scanning-in-security-hub-loom-for-aws-and-more-july-13-2026/

AWS Builder Center turned one year old last week. Launched on July 9, 2025, the platform has grown from a community hub with Wishlist voting, community profiles, and a toolbox into a full ecosystem with sandbox environments, workshops, Spaces, and a Builders’ Library. To mark the anniversary, Rick Suttles published a full feature timeline covering everything shipped over the past year: AWS Capabilities by Region (1,500+ services across 37 Regions), Spaces for community-created groups, workshops with category and complexity filters, badges and streaks, article series, view counts, saved items, student status, availability notifications, sign-in with GitHub and Amazon, and sandbox environments.

Jeff Barr published a retrospective summarizing Builder Center’s first year. Since launch, 5,548 authors have published 6,448 articles with more than 10.4 million page views combined. Builders have earned 99,226 badges since the badge system launched in March 2026. Community members have submitted 565 wishes, 10 of which have shipped with another 20 on the near-term roadmap.

The top community article Building an AWS Study Buddy with MCP + Strands Agents SDK by Dineshraj Dhanapathy reached 50,000+ views. Chris Miller’s Migrating an EOL Linux Server to AWS in 8 Hours with Kiro followed at 45,000+, and Yash Aggarwal’s AIdeas: NeuroVoice – Multimodal AI for Early Screening of Neurological Diseases article reached 38,000+.

The week’s headline addition is Sandbox Environments by Rick Suttles. Sandboxes give you a free, pre-provisioned AWS account to complete a workshop exercise. Each environment is active for 8 hours, after which the account and all its resources are automatically de-provisioned. You can have one active sandbox at a time and request one per week. No personal AWS account, credit card, or manual cleanup required.

Last week’s launches
Here’s what else happened this week.

  • AWS Security Hub introduces Network Scanning – Security Hub introduced Network Scanning, a capability that identifies resources in your environment that are reachable from the public internet. Network Scanning probes your resources from the internet to detect actual reachability, complementing the existing network reachability findings in Security Hub that identify configurations that could make a resource reachable. It discovers public IP addresses, virtual machines, and load balancers across your AWS and Azure environments, identifies reachable ports, and determines what services are running behind them. Each reachable port generates a Security Hub finding with evidence of the port and service discovered. Security Hub Exposures then automatically correlates these findings with other findings and resource configurations to determine broader risk. Existing customers can enable Network Scanning in individual accounts and Regions, or across an organization through a configuration policy. For new customers, Network Scanning is on by default. It is included with Security Hub Essentials at no additional cost.
  • Security Hub also extends unified security management to Microsoft Azure – Security Hub now monitors Microsoft Azure resources, providing unified posture management, vulnerability management, and security response across both clouds. It automatically discovers Azure VMs, container images, Function Apps, and identities, and evaluates them for misconfigurations, internet exposure, and software vulnerabilities. AWS and Azure findings appear in the same prioritized view with the same formats and automation workflows.
  • Amazon SageMaker Studio integrates with Hugging Face for one-click model deployment and customization – You can now go from discovering a model on Hugging Face to working with it in SageMaker Studio in a single click. Select any supported model on Hugging Face and choose “Customize on SageMaker AI” or “Deploy on SageMaker AI” to land directly on the corresponding workflow page with the model pre-loaded. New customers receive a Studio environment created in seconds with pre-configured permissions for serverless model customization (including fine-tuning with custom reward functions for reinforcement learning), model evaluation, and deployment to SageMaker or Bedrock endpoints. Verified customers receive default GPU access to G5, G6, and G4dn instances without requesting quota increases, and quota utilization is visible directly inside the Studio environment.
  • Amazon EKS Auto Mode and Amazon ECS Managed Instances reduce GPU management fees by up to 60% – Beginning July 1, 2026, EKS Auto Mode and ECS Managed Instances reduce management fees for accelerated instance types: G-series fees are down 35%, and P-series and AWS Trainium fees are down 60%. The reductions apply automatically to existing clusters and require no action from customers. Both services include capabilities built for accelerated workloads. EKS Auto Mode provides automatic parallel image pulling on GPU instances with local NVMe storage and accelerator-aware node repair. ECS Managed Instances provides GPU metrics through Amazon CloudWatch Container Insights and automatic health monitoring for GPU hardware failures.
  • Amazon Aurora DSQL change data capture (CDC) is now generally available – Aurora DSQL CDC streams the results of insert, update, and delete operations as change events to Amazon Kinesis Data Streams. You can use it to synchronize data across microservices, trigger Lambda functions, or deliver changes to S3, Redshift, and OpenSearch Service through Amazon Data Firehose. CDC streaming is designed to have zero impact on database workload performance and requires no infrastructure to manage.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news
Here are some additional posts you may find useful:

  • Building secure AI agents at scale: Introducing Loom for AWS – Loom is an open-source enterprise platform for building agents with AWS Strands Agents and deploying them on Amazon Bedrock AgentCore Runtime. It provides a unified management UI and backend API with identity provider integration, scope-based authorization, multi-persona navigation, and full lifecycle management for agents, memory, MCP servers, and agent-to-agent integrations. Loom enforces automated resource tagging for cost attribution, implements RBAC and ABAC for multi-tenant security, uses paved-path blueprints for agent deployments, manages identity propagation through delegated actor chains, integrates with AWS Agent Registry for discovery and governance, and supports human-in-the-loop review before sensitive actions. The project is available in AWS Labs on GitHub.
  • Introducing Claude apps gateway for AWS – The Claude apps gateway is a self-hosted control plane that gives organizations centralized control over access, cost, and policy for Claude Code and Claude Desktop. It connects to any OIDC-compliant identity provider, enforces managed settings on every request, routes inference to Amazon Bedrock or Claude Platform on AWS, and supports per-user and per-group spend caps. The gateway runs as a stateless container in your private network, backed by a PostgreSQL database for short-lived sign-in state. No long-lived secrets are stored on developer machines. Deploy it through Amazon Bedrock to keep data within the AWS security boundary, or through Claude Platform on AWS for the native Claude platform experience.
  • Introducing OAuth support for AWS MCP Server – You can now connect agents to the AWS MCP Server using browser-based OAuth with the same credentials you use for the AWS Console or CLI. The new sign-in path supports IAM federation, AWS IAM Identity Center, and root or IAM users. AWS Sign-In issues short-lived access tokens and refresh tokens, with automatic token management so developers stay authenticated across restarts. For headless use cases, a non-interactive flow lets applications with existing AWS credentials obtain OAuth access tokens through the create-oauth2-token-with-iam API. New governance controls include OAuth-specific IAM condition keys, token introspection and revocation, dynamic client registration, and CloudTrail audit elements.

For a full list of AWS blog posts, be sure to keep an eye on the AWS Blogs page.

Upcoming AWS events
Check your calendar and sign up for upcoming AWS events:

Visit the AWS Builder Center to meet other builders, contribute solutions, and find resources that help you keep building.

Wishing everyone a restful and enjoyable summer. Whether you’re building, learning, or recharging, I hope you find time for all three. I’ll be heading to Scandinavia for a few weeks to trade the heat for some cooler weather and longer evenings. Come back next week for more news!

— Esra

AWS Weekly Roundup: Claude Opus 4.8 on AWS, Aurora MySQL with Kiro Powers, and more (June 1, 2026)

Post Syndicated from Micah Walter original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-claude-opus-4-8-on-aws-aurora-mysql-with-kiro-powers-and-more-june-1-2026/

In my last Week in Review post, I shared what I’d been hearing from customers in the AI-Driven Development Lifecycle (AI-DLC) workshops I’ve been delivering. Last week I was back at it, this time in Denver for a two-day AI-DLC workshop, where I helped facilitate 17 teams to deliver nearly 20 separate use cases in just two days. The pace of acceleration that AI-DLC unlocks—especially when paired with tools like Claude Code on Amazon Bedrock—is fundamentally changing how businesses operate. Traditional roles within software development teams are collapsing into smaller, AI-augmented squads, and the paradigm shift is beginning to take place right in front of us. To learn more about how to utilize various AI tools, visit the GitHub repository of AI-DLC workflow.

This shift is also reshaping how AWS account teams (solutions architects, customer solutions managers, and technical account managers) collaborate with customers. It’s becoming less about handing off advisory design documents and more about building alongside them in real time. It’s a genuinely exciting moment to be in the middle of the change, and this week’s headline launch — Anthropic’s most capable model yet, now on AWS — is going to push that pace even further.

Now, let’s get into this week’s AWS news…

Headlines
Claude Opus 4.8 on AWS — Anthropic’s most capable generally available model is now accessible through both Amazon Bedrock and the Claude Platform on AWS. Opus 4.8 is built for agentic coding, knowledge work, and extended autonomous task execution — it sustains longer autonomous sessions with deeper reasoning, recovers from errors, and synthesizes information across lengthy documents. For coding workloads, it reads codebases like an engineer, plans before it edits, and holds context across long sessions. On Amazon Bedrock, you get AWS-managed features like Guardrails, Knowledge Bases, and data residency; on the Claude Platform on AWS, you get Anthropic’s native APIs unified with AWS billing. To learn more, visit the deep-dive blog post.

Last week’s launches
Here are some launches and updates from this past week that caught my attention:

  • Introducing the next generation of AWS Resilience Hub — A reimagined Resilience Hub gives SREs and developers a unified framework to define resilience standards, evaluate applications against them, and demonstrate compliance across an entire portfolio. It introduces modular resilience policies (covering service-level objectives (SLOs), multi-AZ/Region DR, and data recovery), business-oriented application modeling, generative AI-powered assessments aligned with the Well-Architected and Resilience Analysis Frameworks, and automatic dependency discovery via DNS query log analysis. Integration with AWS Organizations enables organization-wide resilience management from a single delegated administrator account.
  • Introducing the next generation of Amazon OpenSearch Serverless for building agentic AI applications — Amazon OpenSearch Serverless is now a fully managed search and vector engine purpose-built for agentic AI applications. It scales from zero to thousands of requests per second—roughly 20x faster than the prior generation—delivers up to 60% cost savings versus peak-provisioned clusters, and adds GPU acceleration plus new SEARCH and VECTORSEARCH collection types. Native integrations with Vercel, Kiro, Claude Code, and Cursor through OpenSearch Agent Skills make it straightforward to plug into your agent stack.
  • New assessment capabilities in AWS Transform — AWS Transform expands with new tools to help you build migration business cases and evaluate TCO before moving workloads to AWS. You can ingest data from RVTools exports, CMDB data, the AWS Transform discovery tool, and third-party discovery tools, then run what-if scenarios across region, utilization, and service mapping for EC2, FSx, S3, SQL Server on EC2, and virtual desktops. The release also adds Agentic Readiness Analysis (ARA) and Modernization Analysis (MODA), which scan code repositories in 5 to 30 minutes per repo to surface severity-tagged findings with file-level evidence and AWS-mapped remediation guidance.
  • Amazon Aurora MySQL with Kiro Powers — Aurora MySQL now integrates with Kiro Powers, drawing from a curated repository of pre-packaged MCP servers, steering files, and hooks validated by Kiro partners. Developers can execute both data plane tasks (queries, schema management) and control plane tasks (cluster management) in natural language, with dynamic guidance for Aurora MySQL Serverless scaling, RDS-to-Aurora migration, and replication setup. The companion Database Blog post explains how the agent produces the API calls, SQL, and configuration for you to review and run — available via one-click install from the Kiro IDE or webpage.
  • Amazon WorkSpaces Applications now supports Windows Desktop OS — You can now bring your own Windows Desktop licenses to Amazon WorkSpaces Applications and stream full Windows desktops and applications from AWS-hosted dedicated hardware. BYOL eliminates OS fees (you pay only for compute and streaming infrastructure), supports eligible Microsoft 365 Apps for enterprise, and gives users a matching experience between local and remote environments — same workflows, shortcuts, and navigation in both.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news
Here are some additional posts and resources that you might find interesting:

For a full list of AWS blog posts, be sure to keep an eye on the AWS Blogs page.

Learn more about AWS, browse and join upcoming AWS-led in-person and virtual events, startup events, and developer-focused events as well as AWS Summits and AWS Community Days. Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development.

That’s all for this week. Check back next Monday for another Weekly Roundup!

-Micah

AWS Weekly Roundup: Anthropic & Meta partnership, AWS Lambda S3 Files, Amazon Bedrock AgentCore CLI, and more (April 27, 2026)

Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-anthropic-meta-partnership-aws-lambda-s3-files-amazon-bedrock-agentcore-cli-and-more-april-27-2026/

Late March took me to Seattle for the Specialist Tech Conference, one of the most energizing gatherings of AWS specialists from around the world. It was an incredible opportunity to connect with peers, exchange experiences, and go deep on the latest advancements in Generative AI and Amazon Bedrock — and a powerful reminder of something I truly believe in: when specialists come together to challenge each other, explore edge cases, and co-create solutions, the impact goes far beyond the meeting room. In a fast-moving space like AI, having a strong internal community isn’t a nice-to-have — it’s a competitive advantage.

Now, let’s get into this week’s AWS news…

Headlines

Anthropic partnership: Claude on AWS Trainium and Graviton, and Claude Cowork in Amazon Bedrock – This week, AWS and Anthropic deepened their product collaboration in meaningful ways for builders. Anthropic is now training its most advanced foundation models on AWS Trainium and Graviton infrastructure, co-engineering directly at the silicon level with Annapurna Labs to maximize computational efficiency from the hardware up through the full stack.

Claude Cowork is now available in Amazon Bedrock — Claude Cowork brings Anthropic’s collaborative AI capabilities directly to enterprise builders within the AWS ecosystem, enabling teams to work alongside Claude as a true collaborator, not just a tool. You can now deploy Claude Cowork within your existing Amazon Bedrock environment, keeping your data secure within AWS while leveraging the full power of Claude for team-based AI workflows.

Claude Platform on AWS (Coming soon) — A unified developer experience to build, deploy, and scale Claude-powered applications without leaving AWS. If you’re building with Generative AI on AWS, this is a significant step forward in what you’ll be able to do with Claude directly through Amazon Bedrock.

Meta signs agreement with AWS to power agentic AI on Amazon’s Graviton chips — Meta has signed an agreement to deploy AWS Graviton processors at scale, starting with tens of millions of Graviton cores to power CPU-intensive agentic AI workloads — including real-time reasoning, code generation, search, and multi-step task orchestration.

Last week’s launches

Here are some launches and updates from this past week that caught my attention:

  • AWS Lambda functions can now mount Amazon S3 buckets as file systems with S3 Files — You can now mount Amazon S3 buckets as file systems in AWS Lambda using S3 Files, enabling your functions to perform standard file operations without downloading data for processing. Built on Amazon EFS, S3 Files provides the simplicity of a file system with the scalability, durability, and cost-effectiveness of S3 — and multiple Lambda functions can connect to the same file system simultaneously, sharing data through a common workspace. This is particularly valuable for AI and machine learning workloads where agents need to persist memory and share state across pipeline steps.
  • Amazon EKS Hybrid Nodes gateway for hybrid Kubernetes networking — Amazon Elastic Kubernetes Service now offers the Amazon EKS Hybrid Nodes gateway, which automates networking between your EKS cluster VPC and Kubernetes Pods running on EKS Hybrid Nodes. You can now eliminate the need to make on-premises pod networks routable or coordinate network infrastructure changes, greatly simplifying hybrid Kubernetes environments. The gateway automatically enables pod-to-pod traffic across cloud and on-premises environments, control plane-to-webhook communication, and connectivity for AWS services like Application Load Balancers, and is available at no additional charge.
  • Amazon Aurora Serverless: Up to 30% better performance, smarter scaling, and still scales to zero — Amazon Aurora Serverless just got faster and smarter, with up to 30% better performance than the previous version and an enhanced scaling algorithm designed to handle workloads where multiple tasks compete for resources — like busy APIs and agentic AI applications with bursts of activity and long idle windows. You can now run even more demanding workloads serverlessly, paying only for what you use, and automatically scaling to zero when not in use. All improvements are available in platform version 4 at no additional cost.
  • Amazon Bedrock AgentCore adds new features to help developers build agents faster — Amazon Bedrock AgentCore introduces a managed harness (preview), the AgentCore CLI, and AgentCore skills for coding assistants, helping developers go from idea to working agent prototype faster. The managed harness lets you define an agent by specifying a model, system prompt, and tools and run it immediately with no orchestration code required — and when you’re ready for full control, you can export the harness orchestration as Strands-based code. The AgentCore CLI deploys your agents with the governance and auditability of infrastructure-as-code (AWS CDK today, Terraform coming soon), and is available in 14 AWS Regions at no additional charge.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news

Here are some additional posts and resources that you might find interesting:

  • Introducing granular cost attribution for Amazon Bedrock — This post walks through how Amazon Bedrock’s granular cost attribution works and covers practical example cost tracking scenarios. You can now tag and track Bedrock usage costs at a finer level of detail — useful for organizations running multiple teams or projects on Bedrock who need precise cost visibility and chargeback capabilities.
  • Automating Incident Investigation with AWS DevOps Agent and Salesforce MCP Server — This post (co-written with Salesforce) shows how AWS DevOps Agent, integrated with the Salesforce MCP Server, automates the full lifecycle of infrastructure incident investigation — from identifying issues and diagnosing root causes to notifying customers through Salesforce Service Cloud. It’s a compelling real-world example of how AI agents and MCP-based tool connectivity are reshaping DevOps workflows in production, dramatically reducing mean time to resolution.
  • Microcredentials from AWS are now free — Here’s why that matters — You can now access AWS microcredentials at no cost through AWS Skill Builder in all countries where the platform is offered. Unlike traditional multiple-choice certifications, microcredentials are hands-on assessments that place builders in simulated business scenarios where they configure, troubleshoot, and optimize directly in a live AWS environment — the same way they would on the job. A great opportunity to validate real cloud skills without a cost barrier.
  • Amazon SageMaker AI now supports optimized generative AI inference recommendations — You can now use Amazon SageMaker AI to automatically identify optimized deployment configurations for your generative AI models, including instance type, container, and inference parameters. This new capability takes the guesswork out of tuning inference infrastructure, helping you reduce costs and improve latency for your AI applications in production.

Upcoming AWS events

Check your calendar and sign up for upcoming AWS events:

  • What’s Next with AWS — Tune in on April 28 for What’s Next with AWS, a virtual event featuring the latest announcements and product updates directly from AWS teams. A great opportunity to get up to speed on what’s new before diving into the week’s launches.
  • AWS Summits — AWS Summits are free in-person events where you can explore the latest in cloud and AI innovation, learn best practices, and network with builders and experts. Coming up in May: Singapore (May 6), Tel Aviv (May 6), Warsaw (May 6), Stockholm (May 7), Sydney (May 13–14), Hamburg (May 20), Seoul (May 20), Amsterdam (May 27), Bangkok (May 28), Milan (May 28), and Mumbai (May 28). And in June, join us in Los Angeles (June 10). Check the full schedule and register at the link above.
  • AWS Community Days — Community-led conferences where content is planned, sourced, and delivered by community leaders, featuring technical discussions, workshops, and hands-on labs. Upcoming events include Athens, Greece (April 28), Vancouver, Canada (May 1), İstanbul, Türkiye (May 9), and Panama City, Panama (May 23). If you’re in Latin America, mark your calendar for the AWS Community Day Belo Horizonte (August 22) — registration is open at awscommunityday.com.br.

Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development. Browse here for upcoming AWS-led in-person and virtual events and developer-focused events.

That’s all for this week. Check back next Monday for another Weekly Roundup!

— Daniel Abib

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!

Real-time analytics: Oldcastle integrates Infor with Amazon Aurora and Amazon Quick Sight

Post Syndicated from Avdhesh Paliwal original https://aws.amazon.com/blogs/architecture/real-time-analytics-oldcastle-integrates-infor-with-amazon-aurora-and-amazon-quick-sight/

This post is cowritten with Avdhesh Paliwal from Oldcastle.

Oldcastle APG is one of the largest suppliers of construction materials in North America, including asphalt and concrete. The company also provides construction and paving services across more than 150 facilities. As the company migrated from on-premises systems to Infor Cloud ERP hosted on Amazon Web Services (AWS), they faced a critical challenge: maintaining the real-time operational reporting capabilities that hundreds of users across customer service, finance, logistics, and manufacturing depended on daily.

This post explores how Oldcastle used AWS services to transform their analytics and AI capabilities by integrating Infor ERP with Amazon Aurora and Amazon Quick Sight. We discuss how they overcame the limitations of traditional cloud ERP reporting to deploy real-time dashboards and build a scalable analytics system. This practical, enterprise-grade approach offers a blueprint that organizations can adapt when extending ERP capabilities with cloud-native analytics and AI.

Challenges with cloud ERP reporting

The primary challenge that we faced was finding a solution that could accomplish the following:

  • Maintain real-time data access – Our on-premises environment supported hundreds of complex real-time reports, but Infor ERP Cloud’s configuration-based reporting covered minimal reports of our operational needs.
  • Support complex reporting requirements – Users needed multi-dimensional analysis across customer service, finance, logistics, and manufacturing functions.
  • Provide seamless user experience – Business users demanded integrated reporting within the ERP network without switching between multiple systems.
  • Enable advanced analytics –We needed capabilities for demand forecasting, machine learning (ML) capabilities, and intelligent search across real-time data.
  • Scale efficiently – The solution needed to support over 100 concurrent users and process millions of transactions while maintaining performance.
  • Expose Data using API – The solution needed to expose data through APIs, allowing both external and internal applications to access and consume the data securely and efficiently.

Our existing batch reporting process created significant operational challenges across our organization. We had to wait for batch reports, which consumed valuable time and led to delays in critical decision-making across many of our teams. This lag prevented us from capitalizing on real-time business insights and responding quickly to operational issues or economic changes. Without immediate data visibility, our managers couldn’t make timely, data-driven decisions, resulting in missed opportunities for improvement and competitive advantage. Our IT team also struggled with constant report requests but lacked a scalable system to deliver them efficiently, further compounding the productivity loss across our organization.

Solution overview

AWS Solutions Architects worked closely with our application team to build a comprehensive analytics and AI solution to address these challenges. The architecture uses Infor Data Fabric Stream Pipelines to deliver real-time data to AWS. It powers operational dashboards, artificial intelligence and machine learning (AI/ML) models, and intelligent search capabilities. This approach aligns with Infor’s broader strategy of integrating ERP data, data lake information, machine learning (ML) predictions, and documentation to provide comprehensive end-to-end business solutions.

Real-time data streaming architecture

The foundation of our solution is Infor’s Data Fabric Stream Pipelines, an add-on feature that provides real-time streaming data processing. When data events are ingested into Data Fabric, Stream Pipelines processes them immediately and continuously without waiting for storage in the data lake. This approach minimizes the data journey and accelerates operations, helping us extract insights from our data in real time.

The end-to-end workflow consists of the following components:

Data ingestion – Infor Data Fabric tables stream changes in real-time. We enabled Stream Pipelines as an add-on feature within our Infor Cloud ERP environment. We Configure the specific ERP table that we want to stream (such as sales orders, inventory, financial transactions) to publish change events immediately upon data modification. Stream Pipelines captures insert, update, and delete operations with metadata about the operation type and timestamp.

Load distribution – Because Infor can’t reach our private VPC directly, we use Elastic Load Balancing (ELB) to distribute traffic and provide secure database access. We implemented a Network Load Balancer (NLB) with static Elastic IP addresses in public subnets, giving us stable, allowlisted IP addresses for Infor’s outbound connections. We configured an Amazon Relational Database Service (Amazon RDS) router with Amazon Elastic Compute Cloud (Amazon EC2) instances as NLB targets. These routers forward traffic from the NLB to our Amazon Aurora database in the private subnet using iptables NAT rules. This makes sure that even if the IP of Aurora changes during failover, our static Elastic IPs remain constant. We configured security groups to accept HTTPS traffic (port 443) only from Infor’s IP ranges on the NLB and allow traffic only from the NLB to the RDS routers.

Connection management – We use Amazon RDS Proxy to manage database connections and provide automatic failover. We deployed RDS Proxy in the private subnet between our RDS router instances and Aurora cluster to pool and reuse connections. This is critical for handling our high-frequency streaming data. We configured the proxy with IAM authentication for secure credentials and set connection pool parameters based on our expected concurrent stream volume to handle burst traffic without overwhelming the database. With automatic failover enabled, if our primary Aurora instance fails, RDS Proxy automatically redirects traffic to the promoted replica, maintaining continuous data flow.

Data storage – We store our operational data in Amazon Aurora PostgreSQL- Compatible Edition with multi-Availability Zone deployment for high availability. We provisioned an Aurora PostgreSQL cluster with one writer instance and multiple reader instances across different Availability Zones. We designed our database schema to handle the incoming streaming data, storing it in JSONB columns for flexible querying while using the native JSON functions of Aurora PostgreSQL when we need to parse and normalize specific fields. We created indexes on frequently queried fields to maintain query performance as our data volume grows. We also configured automated backups with point-in-time recovery and set up the automatic storage of Aurora scaling to accommodate our data growth.

Analytics and visualization – Amazon Quick Sight delivers the interactive dashboards and pixel-perfect reports our teams need. We created a Quick Sight account and established a connection to our Aurora PostgreSQL database using VPC connectivity with credentials stored in AWS Secrets Manager. We identified which datasets benefit from SPICE (Super-fast, Parallel, In-memory Calculation Engine) caching—typically aggregated or frequently accessed data—and configured incremental refresh schedules to keep them current. We built our dashboards using the visual interface of Quick Sight, using calculated fields for business logic, parameters for user interactivity, and row-level security rules to make sure that users only see data that they’re authorized to access. For pixel-perfect reports, we use the pixel-perfect report feature of Quick Sight to create formatted documents suitable for printing or regulatory compliance.

Embedded integration – We securely embedded Amazon Quick Sight dashboards within Infor OS through Amazon API Gateway, which generates dynamic URLs for seamless user access. We enabled Quick Sight embedding in our AWS account and registered our Infor domain. We created an API Gateway REST API with Lambda functions that authenticate users, validate Infor session tokens, and call QuickSight’s GenerateEmbedUrlForRegisteredUser API to produce time-limited, signed URLs with row-level security. Our Lambda function maps Infor user roles to Quick Sight permissions and applies dashboard filters based on the user’s organizational context. We configured CORS settings in API Gateway to allow requests from our Infor domain and implemented rate limiting. On the Infor side, we embedded the Quick Sight dashboards using iframe elements that call our API Gateway endpoint, providing a seamless experience where our users access analytics without leaving the ERP interface.

The following diagram illustrates the real-time analytics architecture:

Architecture diagram showing Infor ERP data flowing via Postgres streaming through Amazon Route 53, a Network Load Balancer, and RDS Proxy to Amazon Aurora PostgreSQL inside a VPC, with Amazon QuickSight for visualization and Amazon API Gateway plus Lambda generating embedded dashboard URLs for a reporting application. Amazon CloudWatch and IAM provide monitoring and access control.

This embedded experience aligns with Infor’s broader system strategy of integrating insights seamlessly into workflows.

Results and business impact

The implementation using this architecture on AWS brought substantial benefits, directly addressing the critical challenges that we faced and demonstrating measurable value in employee productivity and core business process optimization.

Business process improvement

The solution successfully addressed Oldcastle’s key operational challenges:

Challenge: Limited visibility into real-time operations

– Solution delivered: Deployed over 50 complex dashboards and reports in eight months, providing immediate visibility across customer service, finance, logistics, and manufacturing.- Technical achievement: Used Infor Data Fabric Stream Pipelines to process data events immediately upon ingestion, alleviating delays from traditional batch processing.- Impact: Real-time streaming architecture using the NDJSON format makes sure decision-makers have access to current operational data when they need it most.

Challenge: Fragmented user experience requiring multiple systems

– Solution delivered: Dashboards embedded directly into the Infor environment through Amazon API Gateway.- Technical achievement: Generated dynamic URLs for secure embedding with single sign-on capabilities.- Impact: Users access insights without leaving their familiar interface, with personalized views based on roles and permissions that maintain context across the application.

Screenshot of an Amazon QuickSight dashboard embedded in Infor M3, showing a Cash Desk Reconciliation report with a transaction table on the left, pie charts breaking down payment types (Visa, Amex, Mastercard, Cash) by cash desk location, and a detailed transaction grid below with customer order numbers, invoice dates, and payment amounts.

Challenge: Inflexible reporting limiting business agility

– Solution delivered: Both interactive dashboards and pixel-perfect reports are available to meet diverse business needs.- Technical achievement: Quick Sight SPICE caching enables subsecond response times on complex analytics across large datasets.- Impact: On-demand access for immediate insights, scheduled distribution, custom formatting aligned with corporate standards, and multiple export formats (PDF, CSV, Excel).

Screenshot of a JIT Report embedded in an ERP application showing two sections: Demand By Day and Projected On Hand by Day. Each section displays a grid of inventory items with daily quantity columns spanning multiple weeks, with backorder and on-hand counts highlighted in blue for items requiring attention.

Challenge: Delayed decision-making due to outdated data

– Solution delivered: Real-time visibility into operations enabled faster, data-driven decisions.- Technical achievement: The Multi-AZ deployment of Amazon Aurora PostgreSQL maintains high availability and continuous data access.- Impact: Decision-makers can respond immediately to operational issues and economic changes with current, reliable data.

Scalability and performance

The architecture delivered exceptional scale and performance, addressing concerns about future growth: – High concurrency: Supports over 100 concurrent users without performance degradation- Data volume handling: Processes millions of transactions daily in real-time- Elastic scaling: Aurora read replicas automatically scale based on demand- Future-ready: Architecture designed to expand to additional regions and use cases- Cost efficiency: AWS services avoided complex third-party integrations, with infrastructure costs scaling efficiently with business growth- API capabilities: Ability to expose data using AWS technologies enables integration with third-party and internal applications

Conclusion

Our journey proves that cloud ERP migrations don’t require sacrificing real-time operational reporting capabilities. By combining Infor Data Fabric Stream Pipelines with AWS analytics and AI services, we’ve maintained real-time data access, accelerated innovation, improved user experience, and built a system that scales efficiently as our business needs evolve. The combination of Infor’s enterprise-grade ERP system with the comprehensive analytics capabilities of AWS has given us the best of both worlds. As we expand our AWS analytics and AI capabilities, we’re not just maintaining parity with on-premises systems, we’re unlocking new sources of business value that weren’t possible before.

Further reading

For more information on the services mentioned in the post, see the following resources:

AWS services:

  • Amazon Aurora PostgreSQL Features – Learn more about the high-availability database that powers Oldcastle’s real-time data storage and Multi-AZ deployment strategy
  • Amazon Quick Sight Embedded Analytics – Explore how to embed interactive dashboards and pixel-perfect reports directly into your enterprise applications, as demonstrated in Oldcastle’s Infor OS integration
  • Amazon Bedrock for Generative AI – Discover opportunities to enhance your analytics system with AI-powered insights and intelligent search capabilities
  • Elastic Load Balancing – Understand how to distribute traffic and secure database connections when integrating cloud ERP systems with AWS services
  • Amazon API Gateway – Learn how to create secure, dynamic URLs for embedding analytics and exposing data through APIs to internal and external applications

Infor Resources:


About the authors

AWS Weekly Roundup: AWS AI/ML Scholars program, Agent Plugin for AWS Serverless, and more (March 30, 2026)

Post Syndicated from Prasad Rao original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-aws-ai-ml-scholars-program-agent-plugin-for-aws-serverless-and-more-march-30-2026/

Last week, what excited me most was the launch of the 2026 AWS AI & ML Scholars program by Swami Sivasubramanian, VP of AWS Agentic AI, to provide free AI education to up to 100,000 learners worldwide. The program has two phases: a Challenge phase where you’ll learn foundational generative AI skills, followed by a fully funded three-month Udacity Nanodegree for the top 4,500 performers. Anyone 18 or older can apply, with no prior AI or ML experience required. Applications close on June 24, 2026. Visit the AWS AI & ML Scholars webpage to learn more and apply.

The AWS AI & ML Scholars Program is back

I’m also excited about the start of AWS Summit season, kicking off with AWS Summit Paris on April 1, followed by London on April 22. AWS Summits are free in-person events where builders and innovators can learn about Cloud and AI, think big, and make new connections. Explore the AWS Summits near you and join us in person.

Now, let’s dive into this week’s AWS news…

Last week’s launches
Here are last week’s launches that caught my attention:

  • Announcing Amazon Aurora PostgreSQL serverless database creation in seconds — Amazon Aurora PostgreSQL now offers express configuration, a streamlined setup with preconfigured defaults that supports creating and connecting to a database in seconds. With just two clicks, you can launch an Aurora PostgreSQL serverless database. You can modify certain settings during or after creation.
  • Amazon Aurora PostgreSQL now available with the AWS Free Tier — Amazon Aurora PostgreSQL is now available on the AWS Free Tier. If you’re new to AWS, you receive $100 in AWS credits upon sign-up and can earn an additional $100 in credits by using services like Amazon Relational Database Service (Amazon RDS).
  • Announcing Agent Plugin for AWS Serverless — With the new Agent Plugin for AWS Serverless, you can easily build, deploy, troubleshoot, and manage serverless applications using AI coding assistants like Kiro, Claude Code, and Cursor. This plugin extends AI assistants with structured capabilities by packaging skills, sub-agents, and Model Context Protocol (MCP) servers into one modular unit. It automatically loads the guidance and expertise you need throughout development to build production-ready serverless applications on AWS.
  • Amazon SageMaker Studio now supports Kiro and Cursor IDEs as remote IDEs — You can now remotely connect from Kiro and Cursor IDEs to Amazon SageMaker Studio. This lets you use your existing Kiro and Cursor setup, including spec-driven development, conversational coding, and automated feature generation, while accessing the scalable compute resources of Amazon SageMaker Studio.
  • Introducing visual customization capability in AWS Management Console — You can now customize your AWS Management Console with visual settings like account color and control which regions and services you see. Hiding unused regions and services helps you focus better and work faster by reducing cognitive load and unnecessary scrolling.
  • Announcing Aurora DSQL connector to simplify building Ruby applications — You can now use the Aurora DSQL Connector for Ruby (pg gem) to easily build Ruby applications on Aurora DSQL. The Ruby Connector simplifies authentication and improves security by automatically generating tokens for each connection, eliminating the risks of traditional passwords while maintaining full compatibility with existing pg gem features.
  • AWS Lambda increases the file descriptor limit for functions running on Lambda Managed Instances — AWS Lambda increases the file descriptor limit from 1,024 to 4,096, a 4x increase, for functions running on Lambda Managed Instances (LMI). You can now run I/O intensive workloads such as high-concurrency web services and file-heavy data processing pipelines without running into file descriptor limits.
  • AWS Lambda now supports up to 32 GB of memory and 16 vCPUs for Lambda Managed Instances — AWS Lambda functions on Lambda Managed Instances now support up to 32 GB of memory and 16 vCPUs. You can run compute-intensive workloads like data processing, media transcoding, and scientific simulations without managing infrastructure. Plus, you can adjust the memory-to-vCPU ratio (2:1, 4:1, or 8:1) to fit your workload.
  • Announcing Bidirectional Streaming API for Amazon Polly — Traditional text-to-speech APIs use a request-response pattern. The new Bidirectional Streaming API for Amazon Polly is designed for conversational AI applications that generate text or audio incrementally, like large language model (LLM) responses. This lets you start synthesizing audio before the full text is available.

For a full list of AWS announcements, be sure to keep an eye on our News Blog channel and the What’s New with AWS page.

Upcoming AWS events
Check your calendar and sign up for upcoming AWS events:

  • AWS Summits — As I mentioned earlier, join AWS Summits in 2026 for free in-person events where you can explore emerging cloud and AI technologies, learn best practices, and network with industry peers and experts. Upcoming Summits include Paris (April 1), London (April 22), Bengaluru (April 23–24), Singapore (May 6), Tel Aviv (May 6), and Stockholm (May 7).
  • AWS Community Days — Community-led conferences where content is planned, sourced, and delivered by community leaders, featuring technical discussions, workshops, and hands-on labs. Upcoming events include San Francisco (April 10) and Romania (April 23–24).

Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development. Browse the AWS Events and Webinars for upcoming AWS-led in-person and virtual events and developer-focused events.

That’s all for this week. Check back next Monday for another Weekly Roundup!

— Prasad

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!

Announcing Amazon Aurora PostgreSQL serverless database creation in seconds

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/announcing-amazon-aurora-postgresql-serverless-database-creation-in-seconds/

At re:Invent 2025, Colin Lazier, vice president of databases at AWS, emphasized the importance of building at the speed of an idea—enabling rapid progress from concept to running application. Customers can already create production-ready Amazon DynamoDB tables and Amazon Aurora DSQL databases in seconds. He previewed creating an Amazon Aurora serverless database with the same speed, and customers have since requested quick access and speed to this capability.

Today, we’re announcing the general availability of a new express configuration for Amazon Aurora PostgreSQL, a streamlined database creation experience with preconfigured defaults designed to help you get started in seconds.

With only two clicks, you can have an Aurora PostgreSQL serverless database ready to use in seconds. You have the flexibility to modify certain settings during and after database creation in the new configuration. For example, you can change the capacity range for the serverless instance at the time of create or add read replicas, modify parameter groups after the database is created. Aurora clusters with express configuration are created without an Amazon Virtual Private Cloud (Amazon VPC) network and include an internet access gateway for secure connections from your favorite development tools – no VPN, or AWS Direct Connect required. Express configuration also sets up AWS Identity and Access Management (IAM) authentication for your administrator user by default, enabling passwordless database authentication from the beginning without additional configuration.

After it’s created, you have access to features available for Aurora PostgreSQL serverless, such as deploying additional read replicas for high availability and automated failover capabilities. This launch also introduces a new internet access gateway routing layer for Aurora. Your new serverless instance comes enabled by default with this feature, which allows your applications to connect securely from anywhere in the world through the internet using the PostgreSQL wire protocol from a wide range of developer tools. This gateway is distributed across multiple Availability Zones, offering the same level of high availability as your Aurora cluster.

Creating and connecting to Aurora in seconds means fundamentally rethinking how you get started. We launched multiple capabilities that work together to help you onboard and run your application with Aurora. Aurora is now available on AWS Free Tier, which you gain hands-on experience with Aurora at no upfront cost. After it’s created, you can directly query an Aurora database in AWS CloudShell or using programming languages and developer tools through a new internet accessible routing component for Aurora. With integrations such as v0 by Vercel, you can use natural language to start building your application with the features and benefits of Aurora.

Create an Aurora PostgreSQL serverless database in seconds
To get started, go to the Aurora and RDS console and in the navigation pane, choose Dashboard. Then, choose Create with a rocket icon.

Review pre-configured settings in the Create with express configuration dialog box. You can modify the DB cluster identifier or the capacity range as needed. Choose Create database.

You can also use the AWS Command Line Interface (AWS CLI) or AWS SDKs with the parameter --express-configuration to create both a cluster and an instance within the cluster with a single API call which makes it ready for running queries in seconds.To learn more, visit Creating an Aurora PostgreSQL DB cluster with express configuration.

Here is a CLI command to create the cluster:

$ aws rds create-db-cluster --db-cluster-identifier channy-express-db \
    --engine aurora-postgresql \
    –with-express-configuration

Your Aurora PostgreSQL serverless database should be ready in seconds. A success banner confirms the creation, and the database status changes to Available.

After your database is ready, go to the Connectivity & security tab to access three connection options. When connecting through SDKs, APIs, or third-party tools including agents, choose Code snippets. You can choose various programming languages such as .NET, Golang, JDBC, Node.js, PHP, PSQL, Python, and TypeScript. You can paste the code from each step into your tool and run the commands.

For example, the following Python code is dynamically generated to reflect the authentication configuration:

import psycopg2
import boto3

auth_token = boto3.client('rds', region_name='ap-south-1').generate_db_auth_token(DBHostname='channy-express-db-instance-1.abcdef.ap-south-1.rds.amazonaws.com', Port=5432, DBUsername='postgres', Region='ap-south-1')

conn = None
try:
    conn = psycopg2.connect(
        host='channy-express-db-instance-1.abcdef.ap-south-1.rds.amazonaws.com',
        port=5432,
        database='postgres',
        user='postgres',
        password=auth_token,
        sslmode='require'
    )
    cur = conn.cursor()
    cur.execute('SELECT version();')
    print(cur.fetchone()[0])
    cur.close()
except Exception as e:
    print(f"Database error: {e}")
    raise
finally:
    if conn:
        conn.close()

const { Client } = require('pg');
const AWS = require('aws-sdk');
AWS.config.update({ region: 'ap-south-1' });

async function main() {
  let password = '';
  const signer = new AWS.RDS.Signer({ region: 'ap-south-1', hostname: 'channy-express-db-instance-1.abcdef.ap-south-1.rds.amazonaws.com', port: 5432, username: 'postgres' });
  password = signer.getAuthToken({});

  const client = new Client({
    host: 'channy-express-db-instance-1.abcdef.ap-south-1.rds.amazonaws.com',
    port: 5432,
    database: 'postgres',
    user: 'postgres',
    password,
    ssl: { rejectUnauthorized: false }
  });

  try {
    await client.connect();
    const res = await client.query('SELECT version()');
    console.log(res.rows[0].version);
  } catch (error) {
    console.error('Database error:', error);
    throw error;
  } finally {
    await client.end();
  }
}
main().catch(console.error);

Choose CloudShell for quick access to the AWS CLI which launches directly from the console. When you choose Launch CloudShell, you can see the command is pre-populated with relevant information to connect to your specific cluster. After connecting to the shell, you should see the psql login and the postgres => prompt to run SQL commands.

You can also choose Endpoints to use tools that only support username and password credentials, such as pgAdmin. When you choose Get token, you use an AWS Identity and Access Management (IAM) authentication token generated by the utility in the password field. The token is generated for the master username that you set up at the time of creating the database. The token is valid for 15 minutes at a time. If the tool you’re using terminates the connection, you will need to generate the token again.

Building your application faster with Aurora databases
At re:Invent 2025, we announced enhancements to the AWS Free Tier program, offering up to $200 in AWS credits that can be used across AWS services. You’ll receive $100 in AWS credits upon sign-up and can earn an additional $100 in credits by using services such as Amazon Relational Database Service (Amazon RDS), AWS Lambda, and Amazon Bedrock. In addition, Amazon Aurora is now available across a broad set of eligible Free Tier database services.

Developers are embracing platforms such as Vercel, where natural language is all it takes to build production-ready applications. We announced integrations with Vercel Marketplace to create and connect to an AWS database directly from Vercel in seconds and v0 by Vercel, an AI-powered tool that transforms your ideas into production-ready, full-stack web applications in minutes. It includes Aurora PostgreSQL, Aurora DSQL, and DynamoDB databases. You can also connect your existing databases created through express configuration with Vercel. To learn more, visit AWS for Vercel.

Like Vercel, we’re bringing our databases seamlessly into their experiences and are integrating directly with widely adopted frameworks, AI assistant coding tools, environments, and developer tools, all to unlock your ability to build at the speed of an idea.

We introduced Aurora PostgreSQL integration with Kiro powers, which developers can use to build Aurora PostgreSQL backed applications faster with AI agent-assisted development through Kiro. You can use Kiro power for Aurora PostgreSQL within Kiro IDE and from the Kiro powers webpage for one-click installation. To learn more about this Kiro Power, read Introducing Amazon Aurora powers for Kiro and Amazon Aurora Postgres MCP Server.

Now available
You can create an Aurora PostgreSQL serverless database in seconds today in all AWS commercial Regions. For Regional availability and a future roadmap, visit the AWS Capabilities by Region.

You pay only for capacity consumed based on Aurora Capacity Units (ACUs) billed per second from zero capacity, which automatically starts up, shuts down, and scales capacity up or down based on your application’s needs. To learn more, visit the Amazon Aurora Pricing page.

Give it a try in the Aurora and RDS console and send feedback to AWS re:Post for Aurora PostgreSQL or through your usual AWS Support contacts.

— Channy

Extract data from Amazon Aurora MySQL to Amazon S3 Tables in Apache Iceberg format

Post Syndicated from Kunal Ghosh original https://aws.amazon.com/blogs/big-data/extract-data-from-amazon-aurora-mysql-to-amazon-s3-tables-in-apache-iceberg-format/

If you manage data in Amazon Aurora MySQL-Compatible Edition and want to make it available for analytics, machine learning (ML), or cross-service querying in a modern lakehouse format, you’re not alone.

Organizations often need to run analytics, build ML models, or join data across multiple sources. These are examples of workloads that can be resource-intensive and impractical to run directly against a transactional database. By extracting your Aurora MySQL data into Amazon S3 Tables in Apache Iceberg format, you can offload analytical queries from your production database without impacting its performance, while storing data in a fully managed Iceberg table store optimized for analytics. Built on the open Apache Iceberg standard, Amazon Simple Storage Service (Amazon S3) Table data is queryable from engines like Amazon Athena, Amazon Redshift Spectrum, and Apache Spark without additional data copies. You can also combine relational data with other datasets already in your data lake, enabling richer cross-domain insights.

Apache Iceberg and Amazon S3 Tables

Apache Iceberg is a widely adopted open table format that offers Atomicity, Consistency, Isolation, Durability (ACID) transactions, schema evolution, and time travel capabilities. It enables multiple engines to work concurrently on the same dataset, making it a popular choice for building open lakehouse architectures.

Amazon S3 Tables is a purpose-built, fully managed Apache Iceberg table store designed for analytics workloads. It delivers up to 3x faster query performance and up to 10x more transactions per second compared to self-managed Iceberg tables. It also automatically compacts data and removes unreferenced files to optimize storage and performance.

In this post, you learn how to set up an automated, end-to-end solution that extracts tables from Amazon Aurora MySQL Serverless v2 and writes them to Amazon S3 Tables in Apache Iceberg format using AWS Glue. The entire infrastructure is deployed using a single AWS CloudFormation stack.

Requirements

AWS offers zero-ETL integrations from Amazon Aurora to Amazon Redshift and Amazon SageMaker AI, enabling seamless data flow for analytics and machine learning workloads.

However, there isn’t yet a native zero-ETL integration between Amazon Aurora and Amazon S3 Tables. This means that organizations looking to use Amazon S3 Tables for their Lakehouse architecture currently face several requirements:

  • Setting up ETL pipelines to extract data from Amazon Aurora and transform it into Apache Iceberg format
  • Configuring networking and security for AWS Glue jobs to access Amazon Aurora databases in private subnets
  • Coordinating the provisioning of source databases, ETL pipelines, and target table stores
  • Managing the end-to-end workflow without native automation

Solution overview

In this solution, you automate the extraction of relational database tables from Amazon Aurora MySQL Serverless v2 to Amazon S3 Tables in Apache Iceberg format using AWS Glue 5.0. To help you get started and test this solution, a CloudFormation template is provided. This template provisions the required infrastructure, loads sample data, and configures the Extract, Transform, Load (ETL) pipeline. You can adapt this template for your own scenario.

Solution overview

Sample data

This solution uses the TICKIT sample database, a well-known dataset used in Amazon Redshift documentation. The TICKIT data models a fictional ticket sales system with seven interrelated tables: users, venue, category, date, event, listing, and sales. The dataset is publicly available as mentioned in the Amazon Redshift Getting Started Guide.

Solution flow

The solution flow as shown in the previous architecture diagram:

  1. An AWS Lambda function downloads the TICKIT sample dataset (a fictional ticket sales system used in Amazon Redshift documentation) from a public Amazon S3 bucket to a staging S3 bucket.
  2. A second Lambda function, using PyMySQL (a Python MySQL client library), loads the staged data files into the Aurora MySQL Serverless v2 database using LOAD DATA LOCAL INFILE.
  3. The AWS Glue job reads seven TICKIT tables from Aurora MySQL through a native MySQL connection and writes them to Amazon S3 Tables in Apache Iceberg format using the S3 Tables REST catalog endpoint with SigV4 authentication.
  4. You can query the migrated data in S3 Tables using Amazon Athena.

The solution consists of the following key components:

  1. Amazon Aurora MySQL Serverless v2 as the source relational database containing the TICKIT sample dataset (users, venue, category, date, event, listing, and sales tables)
  2. AWS Secrets Manager to store the Aurora MySQL database credentials securely
  3. Amazon S3 staging bucket for the TICKIT sample data files downloaded from the public redshift-downloads S3 bucket
  4. AWS Lambda functions using PyMySQL to load data into Aurora MySQL
  5. AWS Glue 5.0 job (PySpark) to read tables from Aurora MySQL and write them to S3 Tables in Apache Iceberg format
  6. Amazon S3 Tables as the target storage for the migrated Iceberg tables
  7. Amazon VPC with private subnets and VPC endpoints for Amazon S3, S3 Tables, AWS Glue, Secrets Manager, AWS Security Token Service (AWS STS), CloudWatch Logs, and CloudFormation

Here are some advantages of this architecture:

  • Fully automated setup: A single CloudFormation stack provisions the required infrastructure, loads sample data, and configures the ETL pipeline.
  • Serverless and cost-efficient: Aurora MySQL Serverless v2 and AWS Glue both scale based on demand, minimizing idle costs.
  • Apache Iceberg table format: Data is stored in Apache Iceberg format, enabling ACID transactions, schema evolution, and time travel queries.
  • Network isolation and credential management: The resources run within private subnets with Virtual Private Cloud (VPC) endpoints, and database credentials are managed through AWS Secrets Manager.
  • Extensible pattern: The same approach can be adapted for other relational databases (PostgreSQL, SQL Server) and other target formats supported by AWS Glue.

Prerequisites

To follow along, you need an AWS account. If you don’t yet have an AWS account, you must create one. The CloudFormation stack deployment takes approximately 30-45 minutes to complete and requires familiarity with Amazon S3 Tables, AWS CloudFormation, Apache Iceberg, AWS Glue, Amazon Aurora. This solution will incur AWS costs. The main cost drivers are AWS Glue ETL job runs (billed per DPU-hour, proportional to data volume) and Amazon S3 Tables storage and request charges. Remember to clean up resources when you are done to avoid unnecessary charges.

CloudFormation parameters

You can configure the following parameters before deploying the CloudFormation stack:

Parameter Description Default Required
S3TableBucketName Name of the S3 Tables bucket to create (or use existing) Yes
DatabaseName Name of the initial Aurora MySQL database tickit No
MasterUsername Master username for Aurora MySQL admin No
VpcCidr CIDR block for the VPC 10.1.0.0/16 No
S3TableNamespace Namespace for S3 Tables tickit No

Implementation walkthrough

The following steps walk you through the implementation. These steps are to deploy and test an end-to-end solution from scratch. If you are already running some of these components, you may skip to the relevant step. You can also refer to the aws-samples repository, sample-to-write-aurora-mysql-to-s3tables-using-glue for the entire solution.

Step 1: Deploy the CloudFormation stack

Deploy the CloudFormation template scripts/aurora-mysql-to-s3tables-stack.yaml using the AWS Console or the AWS Command Line Interface (AWS CLI). Provide a name for the S3 Tables bucket; the stack will create it automatically (or use an existing one if it already exists).

To deploy using the AWS Console (recommended), navigate to the AWS CloudFormation Console and use the CloudFormation template. Alternatively, to deploy using the AWS CLI first upload the template to an S3 bucket (the template exceeds the 51,200 byte limit for inline –template-body), then create the stack.

# Upload the template to S3
aws s3 cp scripts/aurora-mysql-to-s3tables-stack.yaml \
  s3://<your-s3-bucket>/aurora-mysql-to-s3tables-stack.yaml \
  --region <your-region>
# Create the stack using the S3 template URL
aws cloudformation create-stack \
  --stack-name aurora-mysql-tickit-stack \
  --template-url https://<your-s3-bucket>.s3.<your-region>.amazonaws.com/aurora-mysql-to-s3tables-stack.yaml \
  --parameters \
    ParameterKey=S3TableBucketName,ParameterValue=<your-s3-table-bucket-name> \
  --capabilities CAPABILITY_NAMED_IAM \
  --region <your-region>

The stack will automatically:

  • Create the S3 Tables bucket (or use existing if it already exists)
  • Create a VPC with private subnets and VPC endpoints
  • Provision an Aurora MySQL Serverless v2 cluster
  • Download TICKIT sample data from the public Amazon S3 bucket
  • Load the sample data into Aurora MySQL via a Lambda function using PyMySQL
  • Create a Glue job configured to migrate data to S3 Tables in Iceberg format

Note: The S3 Tables bucket is retained when the stack is deleted to preserve your data.

Step 2: Verify the Aurora MySQL data

Retrieve the AuroraClusterEndpoint, DatabaseName, and SecretArn values from the CloudFormation stack, make a note of the AuroraClusterEndpoint, DatabaseName, and SecretArn. You can navigate to the Amazon Aurora Console, choose the Query Editor, and enter the values from the CloudFormation stack to connect. You can also choose your preferred method of connecting to an Amazon Aurora DB cluster.

Use the AWS CLI to retrieve the stack outputs: –

aws cloudformation describe-stacks --stack-name aurora-mysql-tickit-stack --region <your-region> --query "Stacks[0].Outputs"

Then run the following SQL commands to verify the data load:

-- Verify if the tables are created
SELECT * FROM information_schema.tables WHERE table_schema = 'tickit';

-- Verify if the data is loaded
SELECT 'users' AS table_name, COUNT(*) AS record_count FROM tickit.users
UNION ALL SELECT 'venue', COUNT(*) FROM tickit.venue
UNION ALL SELECT 'category', COUNT(*) FROM tickit.category
UNION ALL SELECT 'date', COUNT(*) FROM tickit.date
UNION ALL SELECT 'event', COUNT(*) FROM tickit.event
UNION ALL SELECT 'listing', COUNT(*) FROM tickit.listing
UNION ALL SELECT 'sales', COUNT(*) FROM tickit.sales;

Step 3: Run the Glue job

Navigate to the AWS Glue Console, choose ETL jobs under Data Integration and ETL from the left panel. Select the AWS Glue job mysql-tickit-to-iceberg-job and choose Run job to start execution. You can also start the ETL job using the AWS CLI:

aws glue start-job-run --job-name mysql-tickit-to-iceberg-job --region <your-region>

The AWS Glue job performs the following operations for each of the seven TICKIT tables:

  • Reads the table from Aurora MySQL through the native MYSQL Glue connection
  • Converts the data to a Spark DataFrame
  • Creates the Iceberg table in the S3 Tables namespace using CREATE TABLE IF NOT EXISTS with the USING ICEBERG clause
  • Inserts the data using INSERT INTO (or INSERT OVERWRITE if the table already exists)
  • Verifies the record count and displays sample data

Step 4: Verify the results

After the AWS Glue job completes, verify that the tables have been created in your S3 Table bucket by navigating to the Amazon S3 Console. Choose Table buckets under Buckets and select your S3 Table bucket. You can also verify using the AWS CLI:

aws s3tables list-tables \
  --table-bucket-arn arn:aws:s3tables:<your-region>:<your-account-id>:bucket/<your-s3-table-bucket-name> \
  --namespace tickit \
  --region <your-region>

Select a table from the tickit namespace and choose Preview to inspect the data.

Preview S3 data

You can also query the migrated tables using Amazon Athena to validate the data.

Clean up resources

Remember to clean up resources when you no longer need them to avoid unnecessary charges.

Navigate to the CloudFormation console, search for your stack and choose Delete. Alternatively, use the AWS CLI:

aws cloudformation delete-stack --stack-name aurora-mysql-tickit-stack --region <your-region>

The S3 Tables bucket is retained by default. To delete it, use the Amazon S3 console or the AWS CLI to remove the table bucket separately. The staging S3 bucket will be automatically emptied and deleted as part of the stack deletion.

aws s3tables delete-table-bucket --table-bucket-arn arn:aws:s3tables:<your-region>:<your-account-id>:bucket/<your-s3-table-bucket-name> --region <your-region>

Summary

In this post, we showed you how to extract data from Amazon Aurora MySQL Serverless v2 and write it to Amazon S3 Tables in Apache Iceberg format using AWS Glue 5.0. By using the native Iceberg support of AWS Glue and the S3 Tables REST catalog endpoint, you can bridge the gap between relational databases and modern lakehouse storage formats. By automating the entire pipeline through CloudFormation, you can quickly set up and replicate this pattern across multiple environments.

As AWS Glue and Amazon S3 Tables continue to evolve, you can take advantage of future enhancements while maintaining this automated migration pattern.

If you have questions or suggestions, leave us a comment.


About the authors

Kunal Ghosh

Kunal Ghosh

Kunal is a Sr. Solutions Architect at AWS. He is passionate about building efficient and effective solutions on AWS, especially involving generative AI, analytics, data science, and machine learning. Besides family time, he likes reading, swimming, biking, and watching movies.

Arghya Banerjee

Arghya Banerjee

Arghya is a Sr. Solutions Architect at AWS in the San Francisco Bay Area, focused on helping customers adopt and use the AWS Cloud. He is focused on big data, data lakes, streaming and batch analytics services, and generative AI technologies.

Indranil Banerjee

Indranil Banerjee

Indranil is a Sr. Solutions Architect at AWS in the San Francisco Bay Area, focused on helping customers in the hi-tech and semi-conductor sectors solve complex business problems using the AWS Cloud. His special interests are in the areas of legacy modernization and migration, building analytics platforms and helping customers adopt cutting edge technologies such as generative AI.

Vipan Kumar

Vipan Kumar

Vipan is a Sr. Solutions Architect at AWS, where he works with strategic customers. He has extensive experience in machine learning and generative AI. With a background in application development, he is passionate about designing and building enterprise applications for the cloud.

AWS Weekly Roundup: Claude Sonnet 4.6 in Amazon Bedrock, Kiro in GovCloud Regions, new Agent Plugins, and more (February 23, 2026)

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-claude-sonnet-4-6-in-amazon-bedrock-kiro-in-govcloud-regions-new-agent-plugins-and-more-february-23-2026/

Last week, my team met many developers at Developer Week in San Jose. My colleague, Vinicius Senger delivered a great keynote about renascent software—a new way of building and evolving applications where humans and AI collaborate as co-developers using Kiro. Other colleagues spoke about building and deploying production-ready AI agents. Everyone stayed to ask and hear the questions related to agent memory, multi-agent patterns, meta-tooling and hooks. It was interesting how many developers were actually building agents.

We are continuing to meet developers and hear their feedback at third-party developer conferences. You can meet us at the dev/nexus, the largest and longest-running Java ecosystem conference on March 4-6 in Atlanta. My colleague, James Ward will speak about building AI Agents with Spring and MCP, and Vinicius Senger and Jonathan Vogel will speak about 10 tools and tips to upgrade your Java code with AI. I’ll keep sharing places for you to connect with us.

Last week’s launches
Here are some of the other announcements from last week:

  • Claude Sonnet 4.6 model in Amazon Bedrock – You can now use Claude Sonnet 4.6 which offers frontier performance across coding, agents, and professional work at scale. Claude Sonnet 4.6 approaches Opus 4.6 intelligence at a lower cost. It enables faster, high-quality task completion, making it ideal for high-volume coding and knowledge work use cases.
  • Amazon EC2 Hpc8a instances powered by 5th Gen AMD EPYC processors – You can use new Hpc8a instances delivering up to 40% higher performance, increased memory bandwidth, and 300 Gbps Elastic Fabric Adapter networking. You can accelerate compute-intensive simulations, engineering workloads, and tightly coupled HPC applications.
  • Amazon SageMaker Inference for custom Amazon Nova models – You can now configure the instance types, auto-scaling policies, and concurrency settings for custom Nova model deployments with Amazon SageMaker Inference to best meet your needs.
  • Nested virtualization on virtual Amazon EC2 instances – You can create nested virtual machines by running KVM or Hyper-V on virtual EC2 instances. You can leverage this capability for use cases such as running emulators for mobile applications, simulating in-vehicle hardware for automobiles, and running Windows Subsystem for Linux on Windows workstations.
  • Server-Side Encryption by default in Amazon Aurora – Amazon Aurora further strengthens your security posture by automatically applying server-side encryption by default to all new databases clusters using AWS-owned keys. This encryption is fully managed, transparent to users, and with no cost or performance impact.
  • Kiro in AWS GovCloud (US) Regions – You can use Kiro for the development teams behind government missions. Developers in regulated environments can now leverage Kiro’s agentic AI tool with the rigorous security controls required.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Additional updates
Here are some additional news items that you might find interesting:

  • Introducing Agent Plugins for AWS – You can see how new open-source Agent Plugins for AWS extend coding agents with skills for deploying applications to AWS. Using the deploy-on-aws plugin, you can generate architecture recommendations, cost estimates, and infrastructure-as-code directly from your coding agent.
  • A chat with Byron Cook on automated reasoning and trust in AI systems – You can hear how to verify AI systems doing the right thing using automated reasoning when they generate code or manage critical decisions. Byron Cook’s team has spent a decade proving correctness in AWS and apply those techniques to agentic systems.
  • Best practices for deploying AWS DevOps Agent in production – You can read best practices for setting up DevOps Agent Spaces that balance investigation capability with operational efficiency. According to Swami Sivasubramanian, AWS DevOps Agent, a frontier agent that resolves and proactively prevents incidents, has handled thousands of escalations, with an estimated root cause identification rate of over 86% within Amazon.

From AWS community
Here are my personal favorite posts from AWS community:

Join the AWS Builder Center to connect with community, share knowledge, and access content that supports your development.

Upcoming AWS events
Check your calendar and sign up for upcoming AWS events:

  • AWS Summits – Join AWS Summits in 2026, free in-person events where you can explore emerging cloud and AI technologies, learn best practices, and network with industry peers and experts. Upcoming Summits include Paris (April 1), London (April 22), and Bengaluru (April 23–24).
  • Amazon Nova AI Hackathon – Join developers worldwide to build innovative generative AI solutions using frontier foundation models and compete for $40,000 in prizes across five categories including agentic AI, multimodal understanding, UI automation, and voice experiences during this six-week challenge from February 2nd to March 16th, 2026.
  • AWS Community Days – Community-led conferences where content is planned, sourced, and delivered by community leaders, featuring technical discussions, workshops, and hands-on labs. Upcoming events include Ahmedabad (February 28), JAWS Days in Tokyo (March 7), Chennai (March 7), Slovakia (March 11), and Pune (March 21).

Browse here for upcoming AWS led in-person and virtual events, startup events, and developer-focused events.

That’s all for this week. Check back next Monday for another Weekly Roundup!

— Channy