Amazon Prime Day 2026 was exclusively for Prime members and ran June 23–26, 2026 with millions of deals across more than 35 categories.
As part of our annual tradition of sharing how AWS powered Prime Day (2016, 2017, 2019, 2020, 2021, 2022, 2023, 2024, and 2025), I want to share the services and chart-topping metrics from AWS that made your amazing shopping experience possible.
Prime Day 2026 – all the numbers
As always, Prime Day was powered by AWS. Here are some of the most interesting and/or mind-blowing metrics:
Amazon Elastic Compute Cloud (Amazon EC2) and AWS Graviton – During Amazon Prime Day 2026, AWS Graviton powered up to 49% of the Amazon EC2 compute used by Amazon.com.
Amazon Elastic Block Store (Amazon EBS) – During Prime Day 2026, Amazon EBS, our high-performance block storage service, peaked at over 24.8 trillion I/O operations, moving over an exabyte of data daily.
AWS Lambda – AWS Lambda handled over 2.3 trillion invocations per day during Prime Day 2026.
Amazon Elastic Container Service (ECS) and Fargate – During Prime Day 2026, Amazon ECS launched an average of 158.3 million tasks per day on AWS Fargate, representing a 47.7 percent increase from the previous year’s Prime Day average.
Amazon CloudFront – Amazon CloudFront delivered over 2.1 trillion HTTP requests during the global week of Prime Day 2026, a 5 percent increase in total requests compared to Prime Day 2025.
Amazon DynamoDB – Amazon DynamoDB, a serverless, fully managed, distributed NoSQL database, powers multiple high-traffic Amazon properties and systems including Alexa, the Amazon.com sites, and all Amazon fulfillment centers. Over the course of Prime Day 2026, between June 23 and June 26, 2026, DynamoDB processed over 59 trillion requests. DynamoDB maintained high availability while delivering single-digit millisecond responses and peaking at 192 million requests per second.
Amazon Aurora – On Prime Day, Amazon Aurora, a relational database management system (RDBMS) built for high performance and availability at global scale for PostgreSQL, MySQL, and DSQL, processed hundreds of billions of transactions, stored 5,491 terabytes of data, and transferred 1,194 terabytes of data.
Amazon ElastiCache – During Prime Day, Amazon ElastiCache peaked at serving over 2.3 quadrillion daily requests and 2.1 trillion requests in a minute.
Amazon Kinesis Data Streams – Amazon Kinesis Data Streams, a fully managed serverless data streaming service, processed a peak of 988 million records per second during Prime Day 2026.
Amazon Simple Notification Service (SNS) – Amazon SNS, a fully managed pub/sub messaging service for application-to-application and application-to-person communication, delivered 5 trillion messages in a single day during Prime Day 2026.
Amazon Simple Queue Service (SQS) – Amazon SQS, a fully managed message queuing service for microservices, distributed systems, and serverless applications, received a peak of 213 million messages per second during Prime Day 2026
AWS CloudTrail – AWS CloudTrail processes billions of API activity events per day for governance, compliance, and operational auditing. During Prime Day 2026, that volume surged to 3.6 trillion events in just four days, a 44% increase over Prime Day 2025.
AWS CloudWatch – Amazon CloudWatch processed over 2.15 quadrillion metric observations per day during Prime Day 2026.
Amazon GuardDuty – During Prime Day 2026, Amazon GuardDuty monitored an average of 14.08 trillion log events per hour, a 59% increase from last year’s Prime Day.
AWS Fault Injection Service (FIS) – We ran over 44,000 AWS FIS experiments – more than six times what we conducted in 2025 – to help ensure Amazon.com remains highly available on Prime Day.
Prepare to scale
If you’re preparing for similar business-critical events, product launches, and migrations, I recommend that you take advantage of AWS Countdown Premium. From retail peak seasons to major sporting events, elections, and healthcare enrollment periods, we help you deliver flawless experiences when it matters most. Our experts help you manage your infrastructure to handle massive traffic spikes while maintaining security and performance. We work alongside your team to scale infrastructure, optimize costs during demand surges, strengthen security measures, and monitor real-time demands. To learn more, visit AWS Countdown Premium for business critical events.
I look forward to seeing what other records will be broken next year!
Today, we’re announcing a new capability for Amazon Aurora PostgreSQL that you can use to directly query operational data together with data stored in your data lake in Apache Iceberg and Apache Parquet formats, using your existing PostgreSQL applications and tools. By eliminating the need to extract, transform, and load (ETL) structured data from data lakes into your operational database, you can reduce operational complexity and simplify application development. You can also use Aurora PostgreSQL to query data from data lakes managed in Iceberg REST Catalog (IRC)-compatible catalogs, giving you access to data across a breadth of analytics systems without moving or duplicating it. Whether you’re powering real-time dashboards, enriching transactions with historical context, or building AI agents that reason over both live and archived data, you can now do it all through a single, familiar interface.
Previously, if your application needed to combine recent transactional data in Aurora with historical records stored in Amazon S3, a common approach was to build reverse ETL pipelines that duplicated data, increased infrastructure costs, and required ongoing engineering effort to keep everything synchronized. This challenge only grows as you increasingly embed AI agents into your applications, where it is impractical to predict and pre-replicate every dataset an agent might need.
DuckLabs, the team that maintains the DuckDB project, recently joined Amazon, and this capability is an example of how the efficiency of DuckDB is being integrated into our services. DuckDB is now embedded directly within Aurora PostgreSQL, so you can query live operational data (including uncommitted writes) alongside your data lake in a single query. Query processing stays within Aurora, with no additional network hops and no ETL pipelines that duplicate data. You can query Apache Iceberg tables managed through the AWS Glue Data Catalog, as well as Parquet and Iceberg data stored in Amazon S3 and S3 Tables. You do all of this using familiar PostgreSQL syntax and your existing applications and tools.
We’re excited to bring the speed and simplicity of DuckDB directly into Aurora PostgreSQL, so you and your agents can query and combine operational and Iceberg data using the familiar PostgreSQL applications, tools, and endpoints already in use. By building this capability around DuckDB, future improvements to the open source engine can continue to bring performance and functionality gains to Aurora and other AWS services.
What is new
This capability is supported on two Aurora PostgreSQL major versions: 17 (starting with 17.11) and 18 (starting with 18.6). To use it, you create an Aurora PostgreSQL cluster, attach an IAM role with the AuroraAnalytics feature, and enable the aurora_analytics extension. The IAM role is what gives Aurora access to your data in Amazon S3 and the AWS Glue Data Catalog. You then create foreign tables that point to your Iceberg or Parquet data in the data lake, and query them using familiar PostgreSQL syntax. You can complete this setup through the Amazon RDS console, or with any PostgreSQL client such as psql. The process is well documented in the Aurora PostgreSQL documentation.
You can query data across external IRC-compatible catalogs through AWS Glue Data Catalog federation. You register the external catalog once with Glue, and then create foreign tables for the tables you want to query, the same way you would for any Glue-native table. A single query can then join data stored in Aurora with Iceberg tables registered across multiple catalogs, so applications get a unified view without moving data or replacing your existing catalog investments.
Aurora also applies optimizations such as predicate pushdown and column pruning so that only the relevant data is read. This keeps queries efficient even as the underlying data grows. Frequently accessed data is also cached in your Aurora instance, so subsequent queries against the same data return faster. You can inspect this behavior per query using aurora_analytics_stat_statements(), which reports metrics such as rows scanned, bytes read from Amazon S3, and cache hits.
To see how direct querying works, I connected to my Aurora PostgreSQL database using psql and created the extension:
CREATE EXTENSION aurora_analytics;
For my walkthrough, I set up a simple financial scenario. I have a recent_transactions table in Aurora with the last 7 days of customer transactions, and a Parquet file in Amazon S3 containing 5 years of historical transaction data. To make Aurora aware of the historical data, I created a foreign table pointing at the Parquet file in S3:
CREATE FOREIGN TABLE transaction_history ()
SERVER aurora_analytics_server
OPTIONS (
location 's3://<my-bucket>/finance/transaction_history.parquet',
format 'parquet'
);
Notice the empty parentheses in the CREATE FOREIGN TABLE statement. Aurora automatically reads the schema from the Parquet file metadata, so you do not need to define columns manually. For workloads with many tables, you can skip creating them one at a time: a single IMPORT FOREIGN SCHEMA statement bulk-creates foreign tables for every Iceberg or Parquet table in an AWS Glue Data Catalog database, inferring schemas automatically.
With both tables in place, I ran a single query that combines the recent operational data in Aurora with the historical data in S3:
SELECT merchant, category, amount, transaction_date, 'recent' AS source
FROM recent_transactions
WHERE customer_id = 'C-1001'
UNION ALL
SELECT merchant, category, amount, transaction_date, 'historical' AS source
FROM transaction_history
WHERE customer_id = 'C-1001'
AND transaction_date >= CURRENT_DATE - INTERVAL '5 years'
ORDER BY transaction_date DESC
LIMIT 15;
The result shows both recent and historical transactions in a single result set. The 7 most recent rows come from Aurora, and the rest come directly from the Parquet file in S3. DuckDB handles the analytical scan of the Parquet data under the hood, while Aurora handles the operational data. That single query would have previously required a pipeline to move the historical data into the database first.
If a query pattern needs single-digit-millisecond latency, you can materialize data from the data lake into a native Aurora PostgreSQL table using familiar commands such as CREATE TABLE AS SELECT, INSERT INTO ... SELECT, or MERGE INTO. The materialized table lives in Aurora and is queried like any other PostgreSQL table, giving you a low-latency path for hot data without operating a separate ingestion pipeline. The read queries can run on any Aurora PostgreSQL instance in your cluster, whether the writer or a read replica, so you can offload analytical scans from your operational workload. The materialization commands write data into Aurora, so they run on the writer instance.
Get started today
Direct querying of Apache Iceberg and Parquet data from Amazon Aurora PostgreSQL is available today in all commercial AWS Regions and AWS GovCloud (US) Regions, at no additional charge. You pay only for the incremental Aurora compute the queries consume and Amazon S3 request costs for reading data lake files.
When you need quick insights from your Amazon Aurora PostgreSQL operational data, traditional analytics approaches force you to build complex extract, transform, and load (ETL) pipelines. These pipelines introduce latency, operational overhead, and data silos, which slow down decision making and increase cost. AWS introduced the support for Amazon Aurora PostgreSQL zero-ETL integration with Amazon SageMaker, providing near real-time data availability for analytics workloads.
The zero-ETL integration automatically replicates the data from your Amazon Aurora PostgreSQL database into a target AWS Glue managed catalog, where it’s available as Apache Iceberg tables. You can then analyze this data through Amazon SageMaker alongside data from other sources using your preferred analytics and machine learning (ML) tools. The data is compatible with Apache Iceberg open standards, so you can use SQL, Apache Spark, business intelligence, and artificial intelligence and machine learning (AI/ML) tools.
In this post, you explore the benefits of this integration, the architectural concepts, and the underlying change data capture (CDC) mechanics. You also go through the setup process and learn how to query your Aurora PostgreSQL data in Amazon SageMaker AI.
Zero-ETL in the lakehouse architecture
The lakehouse architecture of Amazon SageMaker AI brings together data across Amazon Simple Storage Service (Amazon S3) data lakes and Amazon Redshift data warehouses. Because it’s built on open standards, you can build analytics and AI/ML applications on a single copy of data, without moving it between systems.
Amazon SageMaker AI uses AWS Glue Data Catalog and AWS Lake Formation to provide integrated access controls across S3 data lakes and Amazon Redshift data warehouses from a single governance plane.
Understanding change data capture mechanics
At its core, Aurora PostgreSQL zero-ETL integration is powered by CDC. CDC continuously monitors the database transaction log and streams every insert, update, and delete to a downstream target in near real time.
Aurora PostgreSQL uses enhanced logical replication as its CDC engine. Standard PostgreSQL logical replication publishes row-level changes from the write-ahead log (WAL). The enhanced logical replication in Aurora offers added capabilities that make it well-suited for zero-ETL integrations, including automatic DDL propagation and continuous streaming of transactional changes.
Solution overview
With Amazon Aurora PostgreSQL zero-ETL integration with Amazon SageMaker AI, you can:
Remove ETL complexity – Automatically replicate data without building custom ETL pipelines.
Near real-time analytics – Access operational data in Amazon SageMaker AI within seconds of changes in Aurora PostgreSQL.
Unify data analysis – Combine Aurora PostgreSQL data with data from other sources in a single lakehouse architecture.
Reduce costs – Minimize operational overhead and infrastructure costs associated with maintaining ETL pipelines.
Accelerate insights – Query data using familiar SQL tools and integrate with ML workflows in Amazon SageMaker AI.
The following diagram illustrates the architecture of this solution:
Figure 1: Architecture of the Aurora PostgreSQL zero-ETL integration with Amazon SageMaker
The workflow includes the following steps:
Your application writes data to an Amazon Aurora PostgreSQL database cluster.
The zero-ETL integration automatically captures changes from the Aurora PostgreSQL database.
Data is replicated to the target AWS Glue managed catalog in near real time.
You can query and analyze the data using Amazon Athena, Amazon Redshift, or other analytics tools integrated with Amazon SageMaker AI.
Data scientists can build and train ML models using Amazon SageMaker AI with direct access to the Apache Iceberg tables in the target AWS Glue managed catalog.
Prerequisites
Before setting up the zero-ETL integration, verify that you have the following:
An Amazon Virtual Private Cloud (Amazon VPC) setup with the proper networking configurations for database connectivity.
If you’re creating a new Aurora PostgreSQL cluster, wait for your DB instance(s) to be in an “Available” status. You can verify DB instance status by using the describe-db-instances API call:
Wait until the cluster and the primary instance are back in Available status. For more information, see reboot-db-instance.
Create a target AWS Glue managed catalog
With your source PostgreSQL database configured for enhanced logical replication, the next step is setting up your target Amazon SageMaker AI. Zero-ETL integration uses AWS Glue Data Catalog backed by Amazon Redshift managed storage as its target. To have this functionality, you need to create a managed catalog, configure IAM permissions for Amazon SageMaker AI to access and query the managed catalog, and set up authorization for incoming integration requests from your source database.
Create an AWS Glue managed catalog
You must create a new catalog (if it doesn’t exist already) managed by AWS Glue to store table metadata and serve as the landing zone for your replicated datasets. Zero-ETL integration streams the data into Amazon Redshift managed storage, and AWS Glue keeps track of table definitions so that tools such as SageMaker AI, Athena, and Amazon Redshift Spectrum can query the data.
Create an IAM role for AWS Glue and Amazon Redshift to access the AWS Glue managed catalog
Now, use the following command to create an IAM role so that AWS Glue and Amazon Redshift can interact with the catalog. This role serves two key functions: It allows AWS Glue and Amazon Redshift to perform catalog operations, and it authorizes incoming integration requests from your source database.
Next, attach a policy to this IAM role that provides the minimum required permissions for AWS Glue and Amazon Redshift. This policy should also include the necessary permissions for encryption key actions to help maintain secure data handling throughout the integration process:
Before using the managed catalog for zero-ETL integration, you must configure data lake administrators in AWS Lake Formation who have administrative or read-only permissions on the managed resources. Additionally, you need to grant ReadOnlyAdmin permissions to the Amazon Redshift service-linked role, AWSServiceRoleForRedshift, in your account. If this role doesn’t exist in your account or you need to verify its permissions, see Using service-linked roles for Amazon Redshift.
Register the catalog as a zero-ETL integration target
To prepare your target AWS Glue managed catalog for zero-ETL integration, use the create-integration-resource-property command with these required parameters:
The –resource-arn parameter specifies the Amazon Resource Name (ARN) of your AWS Glue managed catalog that will serve as the integration target.
The –target-processing-properties parameter requires the ARN of an IAM role that has describe permissions on the target AWS Glue managed catalog.
You can use the GlueDataCatalogDataTransferRole created in the earlier step because it already includes the minimal describe permissions needed for this integration. Alternatively, you can create a new IAM role specifically for this purpose and attach the necessary minimal permissions to meet your company’s security requirements.
Configure authorization for inbound integration requests
The last step in creating a target managed catalog is to define a resource-based access policy that authorizes zero-ETL integration to push data into your catalog. This policy grants AWS Glue the necessary permissions to create and authorize incoming integration requests from your source database. Apply this resource policy by using the AWS Glue put-resource-policy API call to complete the catalog configuration for your zero-ETL integration:
Your AWS Glue managed catalog is now ready to receive data from the zero-ETL integration.
Load data in the source Aurora PostgreSQL database
Now that your Aurora PostgreSQL database is configured and ready, you must populate it with sample data that serves as the historical baseline for your zero-ETL integration. This first dataset provides the foundation for testing and demonstrating the integration capabilities. After you set up the zero-ETL integration, subsequent database changes stream automatically in near real time to your target AWS Glue managed catalog.
Connect to the source Aurora PostgreSQL cluster
Use the following commands to create a connection to your source Aurora PostgreSQL cluster:
Create a table named products to store product information:
CREATE TABLE products (product_id SERIAL PRIMARY KEY,product_name VARCHAR(100) NOT NULL, description TEXT,category VARCHAR(50),price NUMERIC(10,2) NOT NULL,stock_quantity INTEGER DEFAULT 0,is_active BOOLEAN DEFAULT TRUE,created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP );
Insert historical data
Use the following code to insert a row:
INSERT INTO products (product_name, description, category, price, stock_quantity) VALUES ('Laptop', 'High-performance laptop with 16GB RAM and 512GB SSD', 'Electronics', 1299.99, 50);
This table serves as a representative dataset to demonstrate the data capture and streaming capabilities of the zero-ETL integration. After your zero-ETL integration is active, all database changes, including inserts, updates, and deletes, are automatically captured and streamed to your AWS Glue managed catalog. This creates a data pipeline from your Aurora PostgreSQL database to your Amazon SageMaker for real-time analytics on your operational data.
Create a zero-ETL integration
Because your Aurora PostgreSQL database is now populated with historical data, you can set up the zero-ETL integration that continuously streams database changes to your AWS Glue managed catalog backed by Amazon Redshift managed storage.
Create the integration
Create the integration between your source PostgreSQL database and target AWS Glue catalog by using the aws rds create-integration AWS CLI command. You can customize the integration by specifying added configurations, such as data filters, to control which data gets replicated to your target environment:
When you run the command, the zero-ETL integration begins provisioning and enters a ‘creating’ state. The AWS CLI response provides key details about the integration configuration.
When the integration status changes to “active”, your zero-ETL integration pipeline is fully operational.
Monitor the integration
Before generating new live data, verify that the integration has reached an “active” state by running the describe-integrations AWS CLI command. This monitoring step is important to confirm that changes from your source Aurora cluster are successfully streaming to the AWS Glue managed catalog without errors:
Now that your historical data is loaded and the zero-ETL integration is “active”, you must confirm that the data has been successfully replicated.
Grant Lake Formation permissions
Before you can query the AWS Glue managed catalog by using the Amazon Redshift Data API, you must make sure the IAM user or role has the right permissions to create and manage tables within the catalog. Use the Lake Formation grant-permissions API to provide these necessary permissions so that Amazon Redshift can access your AWS Glue managed catalog for the zero-ETL integration. For more information, see Creating an Amazon Redshift managed catalog in the AWS Glue Data Catalog.
These permissions allow for query execution and metadata inspection on the managed catalog.
Query historical data by using the Amazon Redshift Data API
With the necessary permissions in place, you can now verify your historical data by querying the AWS Glue managed catalog through the Amazon Redshift execute-statement Data API. Begin this verification process by running a SELECT statement against the catalog:
Monitor your query’s progress by using the describe-statement API with the query ID. Continue checking until the status shows that your query has completed successfully:
//use the Id to make the describe-statement API call to verify execution status is Started
aws redshift-data describe-statement --id <ce1ff0el-xxxx>
To complete the verification process and view your historical data now available in Amazon SageMaker AI, retrieve the query results by using the get-statement-result API call:
With your zero-ETL integration now active, you can demonstrate real-time data streaming by adding new data to your source Aurora PostgreSQL instance. Run the following INSERT query to add a new row, which shows how changes are automatically replicated in near real time:
INSERT INTO products (product_name, description, category, price, stock_quantity)
VALUES ('Wireless Mouse', 'Ergonomic wireless mouse with USB receiver and long battery life', 'Electronics', 29.99, 150);
You can verify that the recent changes from your source database have been replicated to the target environment within seconds. Use the same Amazon Redshift Data API workflow you used earlier to confirm the real-time replication:
Use the describe-statement API call to monitor the query execution and confirm that the status shows ‘FINISHED’ before proceeding to retrieve the results:
This verification process confirms that your zero-ETL integration from Aurora PostgreSQL to Amazon SageMaker AI is working and continuously replicating both historical and real-time data. Although zero-ETL integration significantly simplifies data replication, it’s important to understand certain limitations on supported data types, schema change handling, and data filtering capabilities. For more details about these considerations and best practices, see Aurora zero-ETL integrations and Amazon RDS zero-ETL integrations.
Clean up
This section guides you through the cleanup process to remove the resources and components you created during this walkthrough. When you delete a zero-ETL integration, Amazon Aurora removes it from the source Aurora DB cluster. Your transactional data isn’t removed from Amazon Aurora or the analytics destination, but Aurora doesn’t send new data to Amazon SageMaker AI.
Delete the zero-ETL integration: Begin the cleanup process by removing the integration between your source Amazon Relational Database Service (Amazon RDS) database and the AWS Glue managed catalog. Run the following command to delete the integration:
Delete the AWS Glue managed catalog: After you successfully delete the integration, delete the AWS Glue managed catalog that served as your zero-ETL target destination. Use the following command to remove the catalog:
This permanently removes all associated table metadata and Amazon Redshift managed storage references.
Delete the Aurora DB cluster: If you created the source Aurora DB cluster for this demonstration and you no longer need it, you can complete the cleanup by deleting the entire DB cluster. By skipping the final snapshot option, you avoid retaining any test data and confirm complete resource removal:
In this post, you learned how to configure zero-ETL integration between Aurora PostgreSQL and your Amazon SageMaker AI using AWS CLI. This integration automatically replicates your PostgreSQL data to a lakehouse in near real time, removing the need for custom ETL pipelines.
As you move forward, consider expanding this zero-ETL approach to more supported data sources, such as Amazon RDS for MySQL and Amazon DynamoDB. This creates a centralized data access strategy across your company. You can also explore advanced analytics scenarios by combining zero-ETL integrations with Amazon Redshift capabilities. These include large-scale SQL analytics, Amazon Redshift ML for in-database ML, and federated queries that span multiple data lakes and warehouses. These integrations provide the foundation for building a near real-time data platform that scales with your business needs.
To get started, see the AWS zero-ETL documentation for setup guidance, supported configurations, troubleshooting integrations, and architectural best practices.
As AI applications scale from reactive bots to autonomous agents, their reliability is bound to the speed and accuracy of the data layer beneath them.
The integrity crisis nobody is talking about
There’s a quiet assumption baked into most AI architectures today regarding data layer consistency, and it’s costing companies more than they realize. The assumption is that the data your AI agent reads is the current state of reality.
In a world of distributed systems, cross-region replication, and autonomous agents making millisecond decisions, this assumption breaks down.
I’ve spent extensive time working with enterprise teams building agentic AI, and a recurring failure pattern emerges.
The breakdown isn’t in the model or the prompts. It’s in how we manage replication consistency when an agent performs the reading.
The context window is the new database row
In a modern agentic Retrieval-Augmented Generation (RAG) architecture, the database is the active memory of your AI. When an agent performs a task, it retrieves data to build its context window, forming the foundation of the large language model’s (LLM) reasoning.
If that data is even slightly out of date, the agent’s entire reasoning chain is invalidated. We must shift from simply managing data availability to strictly verifying contextual integrity.
The silent poison of asynchronous lag
In traditional web applications, asynchronous replication scales global reads with minimal write impact. If a user sees a post 500ms late, nobody notices.
For an autonomous AI agent, a 500ms delay is silent poison. If an agent writes a decision to a primary node and immediately reads from a lagging replica, it treats stale data as ground truth. It then executes a logically coherent, multi-step plan based on factually incorrect inputs.
In the age of AI, a fast answer that is wrong is more expensive than a slightly slower answer that is right.
The anatomy of a stale-read failure: When memory betrays logic
Consider an autonomous Inventory Reconciliation Agent managing a flash sale:
The write: The agent updates available_stock to 500 units on the primary database in us-east-1.
The lag: Network congestion causes a 2-second replication lag to the ap-south-1 (Mumbai) replica.
The read: A secondary agent instance in Mumbai queries the replica and retrieves the old value: 0 units.
The failure: The agent triggers a “Sold Out” notification and halts the sale, despite having 500 units in the warehouse.
The agent didn’t make a reasoning error. It performed logical operations on poisoned context.
Figure 1: The stale read cascade, showing how replication lag poisons an AI agent’s context
The hallucination debt problem
When an agent writes an incorrect conclusion back to the database, that error becomes long-term memory. Future retrievals pull this poisoned history, creating a self-reinforcing cycle of “Hallucination Debt.”
LLMs amplify this because they lack a temporal compass. They cooperatively treat retrieved database results as current facts without hesitation. The burden of verifying contextual integrity falls entirely on the architecture.
The replication trinity: Choosing your truth
Not all AI tasks have the same consistency requirements. You must match your replication model to the specific “truth requirement” of the task.
Here are three architectural patterns I’ve found most effective.
Pattern A: Precision through global consistency
When an agent manages high-stakes data (user permissions, security policies, financial records, core system instructions), the cost of a stale read is unacceptable. You need strong consistency.
For many workloads, Amazon Aurora Global Database provides the necessary foundation. While its cross-region storage replication is asynchronous by default, you can close the consistency gap by turning on Global Write Forwarding with a GLOBAL consistency level.
To verify Read-Your-Own-Writes integrity, you configure the SESSION consistency level, which makes an agent wait for its own forwarded writes to replicate back before reading.
For the strongest consistency, the GLOBAL level makes a read query wait for replication to catch up to the exact point in time when the read started.
For the next generation of globally distributed AI, Amazon Aurora DSQL addresses this need. Aurora DSQL offers native synchronous strong consistency across multiple regions, so multi-agent systems can scale globally without compromising accuracy.
Every agent, regardless of location, operates on the exact same ground truth.
Best for: Identity metadata, financial ledgers, immutable system prompts.
Why it matters: Eliminates “mid-thought” state changes that cause contradictory behavior between agent instances.
Pattern B: Global availability at scale
For global AI agents that need ultra-low latency at massive scale, Amazon DynamoDB Global Tables offer a multi-leader architecture where data replicates across regions. For replication details, refer to the DynamoDB documentation.
The key technique here is Conditional Writes. By using a ConditionExpression that checks a version timestamp or whether an attribute exists, an agent updates a record only if the data hasn’t changed since it was last retrieved.
If the condition fails, DynamoDB returns a ConditionalCheckFailedException. This is a critical signal: it tells the agent to re-read the current state and reconsider its decision, rather than blindly overwriting another agent’s work.
This pattern prevents the “Lost Update” anomaly (where two agents running in parallel overwrite each other’s reasoning) without requiring synchronous global coordination.
Best for: Conversational history, user session state, personalized agent memory.
Why it matters: Handles concurrent updates from distributed agents while maintaining a shared memory that’s resilient to race conditions.
Pattern C: High-velocity intake
Some AI agents perform real-time anomaly detection or trend analysis on massive streams of telemetry data. In these cases, you need unthrottled ingestion above all else.
Keyspaces provides highly available, predictable performance by automatically replicating data across three Availability Zones.
Every write is durably committed using LOCAL_QUORUM.
To make sure your AI agent doesn’t miss a critical spike in telemetry, you enforce strong consistency by setting its read operations to LOCAL_QUORUM rather than the eventually consistent LOCAL_ONE.
This quorum overlap means the agent retrieves the latest data without slowing down the high-speed ingestion pipeline.
It transforms a noisy, high-frequency data stream into a reliable foundation for real-time AI decision-making.
Best for: Internet of Things (IoT) telemetry, real-time log analysis, high-frequency sensor data.
Why it matters: Throughput is the priority, but you still need a safety valve to confirm the agent doesn’t miss critical spike data.
Conclusion: Becoming a context architect
Our role as architects has evolved.
We can no longer treat database replication as a background infrastructure concern, something to configure once and forget. In the era of autonomous agents, the stability of the data layer is the direct prerequisite for the trustworthiness of the AI. The two are inseparable.
By matching your replication model to your agent’s reasoning requirements, you move beyond simply managing data. You become a Context Architect, someone who works to confirm that every decision your AI makes is grounded in a synchronized version of the truth.
Because in the end, an AI is only as good as the context it operates in. And context is only as good as the data it’s built on.
Get the database layer right, and everything else follows.
When you deploy Eclipse Dataspace Components (EDC) connectors on AWS, one of the first challenges you face is predicting and controlling the cost of the required infrastructure. Without clear benchmarks, it is difficult to make informed decisions about workload sizing, environment configuration, and long-term investment.
Part 1 of this 3-part blog series covered the fundamentals of data space architectures and the EDC per the International Data Space Association’s (IDSA) standards. Part 2 explored production-ready architecture patterns for deploying EDC connectors on Amazon Web Services (AWS), discussing operational excellence, security, and reliability principles. This final post covers the remaining three AWS Well-Architected Framework pillars: Performance Efficiency, Cost Optimization, and Sustainability.
In this post, you will learn which AWS services drive cost in an EDC connector deployment, how to estimate monthly costs for business-critical and non-critical workloads, and how to apply optimization strategies that can reduce your spending by up to 58%.
Understanding cost drivers in data space deployments
Data spaces are secure and sovereign data environments that enable data sharing across independent organizations. With these architectures, you can collaborate with external organizations while maintaining full control over your data and compliance with data sovereignty principles. Your infrastructure costs can vary significantly. The main factors are your performance and reliability requirements, along with data volume and velocity across the network. It’s also important to distinguish between two types of infrastructure. A Dataspace Governance Authority (DSGA) centrally establishes components such as management, identity, and discovery functions. Participants host other components themselves, including the connector. This blog post focuses only on costs associated with EDC connector deployment on the participant, that is, the data provider and consumer sides.
Fictional usage assumptions
Before diving into the numbers, here are technical and operational assumptions you can use as a baseline for your own estimates.
Technical assumptions
Category
Assumption
Justification
Data Volume
5 GB per participant
Includes 6 months of historical data, and backups
Network Traffic
20 GB/month per participant
Data transfers between participants
API Calls
100,000/month per participant
Catalog queries, contract negotiations, and data transfers
OAuth Token Requests
1,000/month per participant
Machine-to-machine authentication for data plane operations
Table 1: Technical assumptions for EDC connector cost estimation
Growth rate: Not considered in baseline estimates.
Disaster recovery: Automated backups only (no cross-region replication)
Deployment architecture and scenarios
Figure 1 shows the reference architecture for deploying production-ready EDC connectors on AWS, covered in depth in Part 2 of this series.
Figure 1: Production-ready EDC connector deployment
This post considers two cost scenarios depending on the criticality of the workload:
Business-critical workloads: Designed for high availability, performance, and reliability of use cases supporting critical business functions.
Non-critical workloads: Designed for use cases that tolerate interruptions, testing environments, or production workloads where brief disruptions are acceptable.
Both scenarios follow the architecture patterns described in Part 2 of this post series, with the primary differences being in sizing of compute and database resources.
Cost estimation: Business-critical workloads
Note: These estimates use the assumptions above and illustrate the relative cost contribution of each service. Your actual costs may vary based on your specific usage patterns, data volumes and regional pricing. This post highlights which components represent the highest cost drivers and therefore come with the highest potential for optimization.
Table 2: Estimated monthly cost for business-critical EDC connector deployment
These estimates help identify where your budget goes and where optimization has the most impact. In the business-critical scenario, the main cost driver is Amazon Aurora PostgreSQL. The db.r6g.large configuration is selected for constant workloads that require reliability and speed with high memory and performance. Amazon ECS with AWS Fargate is the second largest cost contributor as it runs containers continuously to maintain environment availability. Network Load Balancer represents a third notable cost component, while the remaining services contribute only a small portion of the total cost.
Cost estimation: Non-critical workloads
If you are running development, testing, or experimentation environments, you can reduce costs by up to 58% through rightsizing and use of Amazon EC2 Spot capacity.
Table 3: Estimated monthly cost for non-critical EDC connector deployment
These figures show that a non-critical configuration can cut costs significantly while maintaining the same data throughput and API capacity. Costs are reduced through the use of smaller and more flexible resources. Amazon Aurora PostgreSQL remains the main cost driver, but the smaller instance type (db.t4g.medium) reduces cost significantly. From a compute perspective, using Amazon ECS with AWS Fargate Spot capacity cuts costs by almost 70% compared to the business-critical setup. In total, this configuration reduces the monthly cost by approximately 58%, while maintaining identical assumptions for data throughput, API calls, and storage.
Key takeaways on cost optimization
This comparison shows that the primary cost contributors in both scenarios are database, compute and load balancing resources, which represent baseline infrastructure costs rather than usage-based charges. Services like Amazon S3, API Gateway, and data transfer charges contribute marginally to overall costs at these volumes. This cost structure indicates that the architecture scales efficiently with increased usage. As you onboard more use cases and increase data volume and velocity, you get more value from your existing infrastructure investment without proportional cost increases.
Well-Architected pillars: Performance efficiency, cost optimization, and sustainability
Part 2 of this series covered EDC best practices along the Operational Excellence, Security, and Reliability pillars of the AWS Well-Architected Framework. This section covers the remaining three pillars as they apply to EDC deployments.
Performance efficiency
Right-size compute resources: Match your Amazon ECS task definitions to actual workload requirements. Start with smaller configurations and scale up based on observed metrics rather than over-provisioning from the start. Amazon CloudWatch Container Insights provides the visibility needed to make informed sizing decisions.
Use the flexibility of Amazon Aurora: For workloads with variable demand patterns, consider Amazon Aurora Serverless v2 which automatically scales database capacity based on application needs. This eliminates the need to provision for peak capacity while maintaining performance during high-demand periods.
Optimize data transfer patterns: Design your data plane operations to minimize unnecessary data movement. Use Amazon S3 Transfer Acceleration for large transfers across geographic distances and consider data compression where appropriate to reduce both transfer times and costs.
Cost optimization
Reduce compute costs for fault-tolerant workloads: With AWS Fargate Spot, you can save up to 70% for workloads that can tolerate interruptions. Non-critical environments, batch processing, and development workloads are ideal candidates. Implement graceful shutdown handling to manage Spot interruptions effectively.
Lower storage costs over time: Configure Amazon S3 Lifecycle policies to automatically transition infrequently accessed data to lower-cost storage classes such as S3 Intelligent-Tiering or S3 Glacier Instant Retrieval. For EDC connector deployments, historical transfer logs and archived assets are good candidates for tiered storage.
Monitor for unexpected cost increases: Use AWS Cost Explorer and set up AWS Budgets with alerts to help detect unexpected cost increases. Tag EDC-related AWS resources consistently so you can accurately allocate costs and identify optimization opportunities.
Lock in lower rates for predictable workloads: For business-critical connectors with predictable, steady-state usage, Savings Plans for Amazon Aurora and AWS Fargate can provide significant discounts compared to On-Demand pricing.
Sustainability
Optimize resource utilization: Higher utilization of provisioned resources means less waste. Use automatic scaling policies to match capacity with demand and shut down non-production environments outside of business hours when possible.
Select efficient instance types:AWS Graviton-based instances (such as the r6g and t4g families used in our example) deliver better price-performance and energy efficiency compared to equivalent x86 instances. AWS Graviton processors offer improved performance per watt of energy use.
Minimize data movement: Each data transfer consumes energy. Design your data space integrations to avoid redundant transfers, cache frequently accessed catalog data of peers locally using the Federated Catalog, and batch operations where possible to reduce the total number of network round trips.
Summary
By rightsizing AWS infrastructure to match actual compute and database capacity needs, data space participants can achieve significant cost savings without compromising on data security and sovereignty aspects that make data spaces valuable. The comparison between business-critical and non-critical workload configurations demonstrates how AWS services like Amazon Aurora, AWS Fargate Spot, and Amazon S3 can be combined effectively to balance data sovereignty, performance, and cost efficiency.
As data spaces grow in adoption across industries and geographies, understanding these cost dynamics becomes increasingly important as you plan your network participation. The patterns and estimates in this post series offer a foundation for planning your cross-organizational data strategy and data spaces journey on AWS.
To get started, assess your workload criticality to determine whether a business-critical or non-critical configuration fits your needs. Then use the AWS Pricing Calculator to estimate costs for your specific data volumes, regions, and usage patterns. For an end-to-end reference implementation, explore the Dataspace Connector on AWS project which combines Infrastructure-as-Code with custom EDC extensions and AI tooling integration.
Running Eclipse Dataspace Components (EDC) connectors in production on AWS requires deliberate architecture decisions around isolation, managed services, and security layering. In Part 1 of this series, we covered the fundamentals of data space architectures and EDC per the International Data Space Association’s (IDSA) standards. If you are new to EDC, we recommend starting there. We showed how connector functionality can be customized to support native integration with Amazon Web Services (AWS) Cloud services. Examples include Amazon Simple Storage Service (Amazon S3) for data storage and AWS Secrets Manager for credentials management. In this post, we dive deeper into the connector deployment architecture on AWS and present patterns and practices for production-grade deployments.
Fundamental architecture building blocks
The EDC connector consists of a control plane and a data plane that customers typically ship and deploy as containers. Depending on data integration requirements and support for specific protocols and capabilities, a custom EDC build process may need to be implemented as described in Part 1 of this series. For example, you may need OAuth 2.0 client credentials for the data plane to connect to backend systems. You store the resulting EDC container images in a container registry, such as Amazon Elastic Container Registry (Amazon ECR). Figure 1 shows an example architecture for EDC connector deployments on AWS following best practices for production use.
Figure 1: Production-ready EDC connector deployment on AWS
You can split the architecture into four sub-components:
Amazon Elastic Container Service (Amazon ECS) and AWS Fargate provide serverless container orchestration. This allows for scalable EDC deployment without managing any of the underlying infrastructure.
EDC requires persistence to store secrets and relational control plane data, and a means of vending OAuth 2.0 client credentials. AWS Secrets Manager, Amazon Aurora and Amazon Cognito can provide these capabilities as managed services.
Amazon S3 provides durable data storage for handling both inbound and outbound data that is shared and received through the data space.
With this approach, all cloud resources belonging to a single EDC connector instance form an isolated architecture cell. You access this cell through the S3 bucket to move in data that is to be shared as part of an EDC asset, or to retrieve data received from a third party as part of an EDC data transfer. Secondly, the API Gateway can be configured to expose selected EDC API resources from its management API, data plane API and Dataspace Protocol (DSP) API. You can protect both means of interacting with the EDC architecture cell using AWS Identity and Access Management (AWS IAM) and the AWS Signature Version 4 (SigV4) protocol.
Larger enterprises participating in data spaces may decide to operate multiple EDCs depending on their requirements on failure isolation, data governance, and separation of shared and received data. A common pattern is to deploy separate connector instances per use case. Infrastructure-as-code such as AWS Cloud Development Kit (CDK) allows for automated, templatized deployment and management of EDC connectors over time while keeping operational effort at bay. Using the Dataspace Connector on AWS reference implementation, a full connector cell deploys from a single CDK command, ready to negotiate contracts and transfer data. Amazon API Gateway also comes with Model Context Protocol (MCP) proxy support. This allows EDC APIs to be consumed by authorized AI agents and MCP clients for autonomous data collection and sharing. Besides integration with agentic systems, customers often follow a workflow-based approach for connecting EDCs with their cloud-based data environments. They interact with both APIs and the peripheral S3 bucket to securely expose and retrieve external information.
Real-world validation of these architecture patterns can be seen in production deployments like the Prometheus-X Data Space Connector, for education sector use cases. This implementation uses the same core architecture we recommend: Amazon ECS with AWS Fargate for container orchestration, S3 for data storage, and event-driven processing with AWS Lambda and Amazon EventBridge. This demonstrates how these patterns work effectively in production environments across different industry sectors.
Key principles for production-readiness
We discuss some of the architecture principles that inform the best practices diagram highlighted in Figure 1 along three of the AWS Well-Architected Framework’s pillars.
Operational Excellence
Infrastructure as Code for Consistency: Define all infrastructure declaratively to support repeatable, version-controlled, and testable deployments. Automated validation, for example using CDK Nag, helps catch misconfigurations and security issues before deployment, shifting security left in the development lifecycle. The code itself serves as living documentation of the architecture.
Observability as a First-Class Concern: Treat monitoring and logging as core infrastructure components. Amazon CloudWatch Container Insights, Amazon CloudWatch Logs, and EDC’s structured health check endpoints provide visibility into system behavior, supporting proactive issue detection and faster troubleshooting. EDC APIs for health checks can be similarly exposed with restricted access through API Gateway and IAM.
Managed Services Over Self-Managed Infrastructure: Use AWS managed services (Aurora, Secrets Manager, Fargate, Cognito) instead of deploying and maintaining compatible self-managed solutions. This shifts undifferentiated heavy lifting to AWS and reduces operational burden. You gain high availability, built-in security best practices, compliance certifications, and automatic updates.
Security
Defense in Depth: Implement security through multiple independent layers rather than relying on a single control. Network isolation (VPC private subnets), security group segmentation (restricting traffic between components), IAM least privilege (scoped permissions per service), and encryption (at rest and in transit) each provide independent controls. These layers work together so that if one layer is bypassed, others continue to provide protection.
Principle of Least Privilege: Every component receives only the minimum permissions required for its specific function. Scope IAM roles to individual services (control plane, data plane) and restrict security groups to necessary ports and sources. The internal-only Network Load Balancer fronted by API Gateway prevents unintended public exposure of EDC APIs and data. This may also support security review and approval of EDC as open-source software, since APIs can be allowlisted and validated individually.
Encryption Everywhere: Encrypt data by default at every stage: at rest (Aurora, S3, Secrets Manager), in transit (TLS enforcement, HTTPS-only egress), and during processing (encrypted environment variables). This provides comprehensive data protection regardless of where information resides in the system.
Reliability
Fail Fast, Recover Automatically: Systems detect failures quickly and recover without manual intervention. ECS circuit breakers can automatically roll back failed deployments, automated health checks remove unhealthy targets, and point-in-time recovery supports rapid database restoration. This minimizes mean time to recovery (MTTR) and reduces the scope of failures, even within a single EDC architecture cell.
Design for Regional Resilience: Cross-zone load balancing distributes traffic across multiple Availability Zones (AZs), Aurora automatically replicates data across AZs, and Fargate tasks can be scheduled in any AZ. The highlighted architecture can tolerate Availability Zone failures in an AWS Region without service disruption. For more information about Availability Zones and Regions, see AWS Global Infrastructure.
Decoupled Components with Clear Boundaries: Deploy the control plane and data plane as independent services with distinct responsibilities, security contexts, and scaling characteristics. This separation enables independent updates, targeted scaling, and failure isolation between coordination logic and data transfer operations.
The remaining three Well-Architected Framework pillars of Performance Efficiency, Cost Optimization, and Sustainability are covered in the third post of this series where we discuss cost optimization strategies for running EDC connectors on AWS.
Conclusion
With the growing popularity of data spaces and EDC as a data space connector, it is important to distinguish between a setup suitable for testing and experimentation and one that is ready for production. Production environments require that business-critical processes depend on timely, successful transmission of confidential information between participants. The architecture defined in this post combines EDC deployment best practices from the community with AWS recommendations to achieve fault tolerance, scalability, and security while keeping operational complexity at a minimum.
In part 3, you will learn about cost optimization strategies to run your production-ready connector efficiently and maximize the value it returns by supporting business use cases for data sharing along your supply network. In the meantime, explore the Dataspace Connector on AWS project and see how the patterns and best practices covered in this post come together in an end-to-end reference implementation.
This three-part blog series guides you through implementing Eclipse Dataspace Components (EDC) on AWS, from foundational concept to production deployment. Part 1 establishes the theoretical foundation with IDSA standards, the Dataspace Protocol (DSP), and core EDC architecture. Part 2 provides production-ready AWS deployment patterns using services like Amazon Elastic Container Service (Amazon ECS), Amazon Aurora, and Amazon API Gateway. Part 3 completes the journey with cost optimization strategies and practical guidance for running efficient, scalable data space infrastructure on AWS.
The International Data Spaces Association (IDSA) is a non-profit organization focused on establishing and promoting standards for data spaces. A data space is defined as a “set of technical services that facilitate interoperable dataset sharing between distinct entities”. At a technical level, a data space has participants: data consumers and data providers. Centrally, there is a Dataspace Governance Authority (DSGA). The DSGA manages the data space, enforces rules and policies and issues membership credentials to participants.
IDSA rules and specifications are implemented in the Dataspace Protocol (DSP). The DSP has become an Eclipse Specification project and can be found in the Dataspace Protocol GitHub repository. Standardization is under development and is currently in the approval phase as ISO/IEC DIS 20151. The Eclipse Dataspace Components (EDC) provide the technical components to implement data spaces according to IDSA requirements.
These components include the federated catalog (FC), the connector, and the identity hub. The FC contains an aggregated repository of catalogs gathered from multiple participants in the data space. These are obtained by periodically crawling participants’ data assets and storing them in a local cache, to eliminate the need to query each participant individually on demand. The connector is the software that enables data to be shared between participants, and identity hub manages a participant’s credentials in the data space. Amazon Web Services (AWS) provides a comprehensive cloud infrastructure that supports the deployment and operation of data space components like the EDC connector. AWS offers scalable compute, storage, and networking services that help you build secure, compliant, and interoperable data spaces aligned with IDSA standards and the ISO/IEC DIS 20151 specifications.
To verify identities in a data space in a decentralized manner, the Decentralized Claims Protocol (DCP) is used. DCP represents an overlay on top of DSP to establish trust between network participants. When an issuer needs to prove their identity to a verifier, they first generate a Decentralized Identifier (DID), which contains information about their identity. The issuer stores their Verifiable Credential (VC) in their identity hub. A verifiable credential is similar to a certificate: while a certificate authority validates a public key, the credential issuer validates the DID with specific attributes. The verifier looks up the DID of the issuer and with the information obtained from the DID document verifies the VC.
Figure 1: High-level diagram of the Decentralized Claims Protocol (DCP)
Figure 2: Simplified overview of DID document processing
The final element of the EDC are the different types of policies, including membership, access, contract and usage policies.
The role of the connector
The EDC connector is divided into two parts: the control plane and the data plane. The control plane handles contract negotiation and sends messages to the data plane to initiate a data transfer. The data plane is responsible for transferring data from a provider’s to a consumer’s EDC connector across distinct legal entities.
Figure 3: Overview of the connector control and data plane
Structure of the EDC connector open-source project
The Connector repository as part of the EDC project defines how the connector control and data planes are implemented. We provide an overview of how the repository is structured:
/spi – The Service Provider Interface (SPI) is the architectural foundation that defines how modules communicate within the EDC connector. It contains foundational interfaces and contracts that every component must implement, establishing standardized integration patterns. This layer also acts as a blueprint for developers to extend EDC connector functionality while maintaining clean separation of concerns and ensuring compatibility between core and custom components.
/core – The Core module represents the core SPI implementation. It houses the actual working code for the connector’s essential operations, including default implementations of key services and business logic for data sharing.
/extensions – The Extensions layer showcases the connector’s modular plugin architecture through real-world integrations, including connections to major cloud providers like AWS. These extensions serve both as reference implementations and ready-to-use components for common enterprise integration scenarios.
SPI defines the contracts, Core implements the basics, and Extensions add specialized functionality. These three layers work together to create a flexible, extensible data space connector system. The “vanilla” Eclipse EDC connector does not bundle the AWS extensions by default. To add AWS service-specific functionality, such as integration with Amazon Simple Storage Service (Amazon S3), AWS Secrets Manager, or Amazon DynamoDB, you need to include the respective AWS extensions into a custom EDC control and data plane build.
High-level EDC customization guide
Customizing the EDC connector for native AWS service integration creates a purpose-built solution that can use AWS managed services for storage, security, and scalability. The following steps help your connector natively integrate with services like Amazon S3 for data storage (used for example as Asset Administration Shell “Submodel Server”) and AWS Secrets Manager for credentials management, rather than relying on operations of self-managed components.
EDC uses a modular, plugin-based architecture built on Gradle. The vanilla connector ships only core functionality. To integrate with specific cloud services or persistence backends, you assemble a custom build that combines EDC’s core modules with the extensions you need. The customization revolves around three concepts: a version catalog, launcher modules, and a project settings file.
Step 1: Version catalog
The version catalog (gradle/libs.versions.toml) is the centralized registry of all dependencies and their versions. Here you declare which EDC modules, cloud provider extensions, and third-party libraries your connector needs to use. EDC publishes its AWS extensions under the org.eclipse.edc.aws Maven group. You reference these alongside the core EDC artifacts (org.eclipse.edc) at compatible version numbers. This single file ensures all modules in your project share consistent dependency versions.
Step 2: Launcher modules
The launcher modules are the actual deployable units: one for the control plane and one for the data plane. Each launcher is a small Gradle subproject whose build.gradle.kts lists the extensions to bundle at runtime. A typical control plane launcher might include the core control plane module, a metadata persistence extension (such as PostgreSQL or DynamoDB), a vault extension (such as HashiCorp Vault or AWS Secrets Manager), and any provisioning extensions for your target storage system. The data plane launcher follows the same pattern with data plane-specific modules. Both launchers typically use Gradle plugins to produce a single executable JAR. You only include what you actually use, keeping the connector lightweight and tailored to your unique requirements.
Step 3: Project settings
The project settings file (settings.gradle.kts) registers all modules with Gradle: your launcher subprojects and any custom extensions you may develop locally. If you write your own extension, for example, a DynamoDB-backed asset store, you place it in an extensions/ directory and register it here so your launchers can reference it as a project dependency.
The resulting project structure typically looks like this:
An open-source reference implementation for EDC customization on AWS is provided as part of the Dataspace Connector on AWS project. In this post, you learned about the fundamentals behind secure, cross-organizational data sharing using emerging data space architectures. We covered the two protocols included in ISO/IEC DIS 20151 data space standardization, DSP and DCP, and discussed the EDC connector, the main software required for data space participants, and how it can be customized.
In part 2, we explore production-ready deployment patterns for EDC connectors on AWS, examining architecture best practices that support your data space infrastructure’s security, reliability, and operational efficiency. To get started, explore the Dataspace Connector on AWS project and see how its Gradle build settings are configured to support Amazon DynamoDB for serverless, inexpensive metadata persistence.
AWS Builder Center turned one year old last week. Launched on July 9, 2025, the platform has grown from a community hub with Wishlist voting, community profiles, and a toolbox into a full ecosystem with sandbox environments, workshops, Spaces, and a Builders’ Library. To mark the anniversary, Rick Suttles published a full feature timeline covering everything shipped over the past year: AWS Capabilities by Region (1,500+ services across 37 Regions), Spaces for community-created groups, workshops with category and complexity filters, badges and streaks, article series, view counts, saved items, student status, availability notifications, sign-in with GitHub and Amazon, and sandbox environments.
Jeff Barr published a retrospective summarizing Builder Center’s first year. Since launch, 5,548 authors have published 6,448 articles with more than 10.4 million page views combined. Builders have earned 99,226 badges since the badge system launched in March 2026. Community members have submitted 565 wishes, 10 of which have shipped with another 20 on the near-term roadmap.
The week’s headline addition is Sandbox Environments by Rick Suttles. Sandboxes give you a free, pre-provisioned AWS account to complete a workshop exercise. Each environment is active for 8 hours, after which the account and all its resources are automatically de-provisioned. You can have one active sandbox at a time and request one per week. No personal AWS account, credit card, or manual cleanup required.
Last week’s launches Here’s what else happened this week.
AWS Security Hub introduces Network Scanning – Security Hub introduced Network Scanning, a capability that identifies resources in your environment that are reachable from the public internet. Network Scanning probes your resources from the internet to detect actual reachability, complementing the existing network reachability findings in Security Hub that identify configurations that could make a resource reachable. It discovers public IP addresses, virtual machines, and load balancers across your AWS and Azure environments, identifies reachable ports, and determines what services are running behind them. Each reachable port generates a Security Hub finding with evidence of the port and service discovered. Security Hub Exposures then automatically correlates these findings with other findings and resource configurations to determine broader risk. Existing customers can enable Network Scanning in individual accounts and Regions, or across an organization through a configuration policy. For new customers, Network Scanning is on by default. It is included with Security Hub Essentials at no additional cost.
Security Hub also extends unified security management to Microsoft Azure – Security Hub now monitors Microsoft Azure resources, providing unified posture management, vulnerability management, and security response across both clouds. It automatically discovers Azure VMs, container images, Function Apps, and identities, and evaluates them for misconfigurations, internet exposure, and software vulnerabilities. AWS and Azure findings appear in the same prioritized view with the same formats and automation workflows.
Amazon SageMaker Studio integrates with Hugging Face for one-click model deployment and customization – You can now go from discovering a model on Hugging Face to working with it in SageMaker Studio in a single click. Select any supported model on Hugging Face and choose “Customize on SageMaker AI” or “Deploy on SageMaker AI” to land directly on the corresponding workflow page with the model pre-loaded. New customers receive a Studio environment created in seconds with pre-configured permissions for serverless model customization (including fine-tuning with custom reward functions for reinforcement learning), model evaluation, and deployment to SageMaker or Bedrock endpoints. Verified customers receive default GPU access to G5, G6, and G4dn instances without requesting quota increases, and quota utilization is visible directly inside the Studio environment.
Amazon EKS Auto Mode and Amazon ECS Managed Instances reduce GPU management fees by up to 60% – Beginning July 1, 2026, EKS Auto Mode and ECS Managed Instances reduce management fees for accelerated instance types: G-series fees are down 35%, and P-series and AWS Trainium fees are down 60%. The reductions apply automatically to existing clusters and require no action from customers. Both services include capabilities built for accelerated workloads. EKS Auto Mode provides automatic parallel image pulling on GPU instances with local NVMe storage and accelerator-aware node repair. ECS Managed Instances provides GPU metrics through Amazon CloudWatch Container Insights and automatic health monitoring for GPU hardware failures.
Amazon Aurora DSQL change data capture (CDC) is now generally available – Aurora DSQL CDC streams the results of insert, update, and delete operations as change events to Amazon Kinesis Data Streams. You can use it to synchronize data across microservices, trigger Lambda functions, or deliver changes to S3, Redshift, and OpenSearch Service through Amazon Data Firehose. CDC streaming is designed to have zero impact on database workload performance and requires no infrastructure to manage.
For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.
Other AWS news Here are some additional posts you may find useful:
Building secure AI agents at scale: Introducing Loom for AWS – Loom is an open-source enterprise platform for building agents with AWS Strands Agents and deploying them on Amazon Bedrock AgentCore Runtime. It provides a unified management UI and backend API with identity provider integration, scope-based authorization, multi-persona navigation, and full lifecycle management for agents, memory, MCP servers, and agent-to-agent integrations. Loom enforces automated resource tagging for cost attribution, implements RBAC and ABAC for multi-tenant security, uses paved-path blueprints for agent deployments, manages identity propagation through delegated actor chains, integrates with AWS Agent Registry for discovery and governance, and supports human-in-the-loop review before sensitive actions. The project is available in AWS Labs on GitHub.
Introducing Claude apps gateway for AWS – The Claude apps gateway is a self-hosted control plane that gives organizations centralized control over access, cost, and policy for Claude Code and Claude Desktop. It connects to any OIDC-compliant identity provider, enforces managed settings on every request, routes inference to Amazon Bedrock or Claude Platform on AWS, and supports per-user and per-group spend caps. The gateway runs as a stateless container in your private network, backed by a PostgreSQL database for short-lived sign-in state. No long-lived secrets are stored on developer machines. Deploy it through Amazon Bedrock to keep data within the AWS security boundary, or through Claude Platform on AWS for the native Claude platform experience.
Introducing OAuth support for AWS MCP Server – You can now connect agents to the AWS MCP Server using browser-based OAuth with the same credentials you use for the AWS Console or CLI. The new sign-in path supports IAM federation, AWS IAM Identity Center, and root or IAM users. AWS Sign-In issues short-lived access tokens and refresh tokens, with automatic token management so developers stay authenticated across restarts. For headless use cases, a non-interactive flow lets applications with existing AWS credentials obtain OAuth access tokens through the create-oauth2-token-with-iam API. New governance controls include OAuth-specific IAM condition keys, token introspection and revocation, dynamic client registration, and CloudTrail audit elements.
For a full list of AWS blog posts, be sure to keep an eye on the AWS Blogs page.
Upcoming AWS events Check your calendar and sign up for upcoming AWS events:
AWS Summits – Free in-person events for builders and innovators to learn, think big, and make new connections. Coming up: Taipei (July 15), Bogotá (July 30), Jakarta (August 6), Ciudad de México (August 12), Johannesburg (August 19), and Zurich (September 2).
Visit the AWS Builder Center to meet other builders, contribute solutions, and find resources that help you keep building.
Wishing everyone a restful and enjoyable summer. Whether you’re building, learning, or recharging, I hope you find time for all three. I’ll be heading to Scandinavia for a few weeks to trade the heat for some cooler weather and longer evenings. Come back next week for more news!
In my last Week in Review post, I shared what I’d been hearing from customers in the AI-Driven Development Lifecycle (AI-DLC) workshops I’ve been delivering. Last week I was back at it, this time in Denver for a two-day AI-DLC workshop, where I helped facilitate 17 teams to deliver nearly 20 separate use cases in just two days. The pace of acceleration that AI-DLC unlocks—especially when paired with tools like Claude Code on Amazon Bedrock—is fundamentally changing how businesses operate. Traditional roles within software development teams are collapsing into smaller, AI-augmented squads, and the paradigm shift is beginning to take place right in front of us. To learn more about how to utilize various AI tools, visit the GitHub repository of AI-DLC workflow.
This shift is also reshaping how AWS account teams (solutions architects, customer solutions managers, and technical account managers) collaborate with customers. It’s becoming less about handing off advisory design documents and more about building alongside them in real time. It’s a genuinely exciting moment to be in the middle of the change, and this week’s headline launch — Anthropic’s most capable model yet, now on AWS — is going to push that pace even further.
Now, let’s get into this week’s AWS news…
Headlines Claude Opus 4.8 on AWS — Anthropic’s most capable generally available model is now accessible through both Amazon Bedrock and the Claude Platform on AWS. Opus 4.8 is built for agentic coding, knowledge work, and extended autonomous task execution — it sustains longer autonomous sessions with deeper reasoning, recovers from errors, and synthesizes information across lengthy documents. For coding workloads, it reads codebases like an engineer, plans before it edits, and holds context across long sessions. On Amazon Bedrock, you get AWS-managed features like Guardrails, Knowledge Bases, and data residency; on the Claude Platform on AWS, you get Anthropic’s native APIs unified with AWS billing. To learn more, visit the deep-dive blog post.
Last week’s launches Here are some launches and updates from this past week that caught my attention:
Introducing the next generation of AWS Resilience Hub — A reimagined Resilience Hub gives SREs and developers a unified framework to define resilience standards, evaluate applications against them, and demonstrate compliance across an entire portfolio. It introduces modular resilience policies (covering service-level objectives (SLOs), multi-AZ/Region DR, and data recovery), business-oriented application modeling, generative AI-powered assessments aligned with the Well-Architected and Resilience Analysis Frameworks, and automatic dependency discovery via DNS query log analysis. Integration with AWS Organizations enables organization-wide resilience management from a single delegated administrator account.
Introducing the next generation of Amazon OpenSearch Serverless for building agentic AI applications — Amazon OpenSearch Serverless is now a fully managed search and vector engine purpose-built for agentic AI applications. It scales from zero to thousands of requests per second—roughly 20x faster than the prior generation—delivers up to 60% cost savings versus peak-provisioned clusters, and adds GPU acceleration plus new SEARCH and VECTORSEARCH collection types. Native integrations with Vercel, Kiro, Claude Code, and Cursor through OpenSearch Agent Skills make it straightforward to plug into your agent stack.
New assessment capabilities in AWS Transform — AWS Transform expands with new tools to help you build migration business cases and evaluate TCO before moving workloads to AWS. You can ingest data from RVTools exports, CMDB data, the AWS Transform discovery tool, and third-party discovery tools, then run what-if scenarios across region, utilization, and service mapping for EC2, FSx, S3, SQL Server on EC2, and virtual desktops. The release also adds Agentic Readiness Analysis (ARA) and Modernization Analysis (MODA), which scan code repositories in 5 to 30 minutes per repo to surface severity-tagged findings with file-level evidence and AWS-mapped remediation guidance.
Amazon Aurora MySQL with Kiro Powers — Aurora MySQL now integrates with Kiro Powers, drawing from a curated repository of pre-packaged MCP servers, steering files, and hooks validated by Kiro partners. Developers can execute both data plane tasks (queries, schema management) and control plane tasks (cluster management) in natural language, with dynamic guidance for Aurora MySQL Serverless scaling, RDS-to-Aurora migration, and replication setup. The companion Database Blog post explains how the agent produces the API calls, SQL, and configuration for you to review and run — available via one-click install from the Kiro IDE or webpage.
Amazon WorkSpaces Applications now supports Windows Desktop OS — You can now bring your own Windows Desktop licenses to Amazon WorkSpaces Applications and stream full Windows desktops and applications from AWS-hosted dedicated hardware. BYOL eliminates OS fees (you pay only for compute and streaming infrastructure), supports eligible Microsoft 365 Apps for enterprise, and gives users a matching experience between local and remote environments — same workflows, shortcuts, and navigation in both.
For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.
Other AWS news Here are some additional posts and resources that you might find interesting:
Meet our newest AWS Heroes — May 2026 — An AWS Builder Center post introducing the latest cohort of AWS Heroes recognized for their contributions to the global AWS community.
AI-native, full-stack web apps with Vercel and AWS databases — Database blog post on the new Vercel and AWS Databases integration, which lets you provision Aurora PostgreSQL, DynamoDB, or Aurora DSQL directly from the Vercel dashboard or via v0. The post also highlights the H0 hackathon with $160,000 in prizes for building full-stack apps on this stack.
Late March took me to Seattle for the Specialist Tech Conference, one of the most energizing gatherings of AWS specialists from around the world. It was an incredible opportunity to connect with peers, exchange experiences, and go deep on the latest advancements in Generative AI and Amazon Bedrock — and a powerful reminder of something I truly believe in: when specialists come together to challenge each other, explore edge cases, and co-create solutions, the impact goes far beyond the meeting room. In a fast-moving space like AI, having a strong internal community isn’t a nice-to-have — it’s a competitive advantage.
Now, let’s get into this week’s AWS news…
Headlines
Anthropic partnership: Claude on AWS Trainium and Graviton, and Claude Cowork in Amazon Bedrock – This week, AWS and Anthropic deepened their product collaboration in meaningful ways for builders. Anthropic is now training its most advanced foundation models on AWS Trainium and Graviton infrastructure, co-engineering directly at the silicon level with Annapurna Labs to maximize computational efficiency from the hardware up through the full stack.
Claude Cowork is now available in Amazon Bedrock — Claude Cowork brings Anthropic’s collaborative AI capabilities directly to enterprise builders within the AWS ecosystem, enabling teams to work alongside Claude as a true collaborator, not just a tool. You can now deploy Claude Cowork within your existing Amazon Bedrock environment, keeping your data secure within AWS while leveraging the full power of Claude for team-based AI workflows.
Claude Platform on AWS (Coming soon) — A unified developer experience to build, deploy, and scale Claude-powered applications without leaving AWS. If you’re building with Generative AI on AWS, this is a significant step forward in what you’ll be able to do with Claude directly through Amazon Bedrock.
Meta signs agreement with AWS to power agentic AI on Amazon’s Graviton chips — Meta has signed an agreement to deploy AWS Graviton processors at scale, starting with tens of millions of Graviton cores to power CPU-intensive agentic AI workloads — including real-time reasoning, code generation, search, and multi-step task orchestration.
Last week’s launches
Here are some launches and updates from this past week that caught my attention:
AWS Lambda functions can now mount Amazon S3 buckets as file systems with S3 Files — You can now mount Amazon S3 buckets as file systems in AWS Lambda using S3 Files, enabling your functions to perform standard file operations without downloading data for processing. Built on Amazon EFS, S3 Files provides the simplicity of a file system with the scalability, durability, and cost-effectiveness of S3 — and multiple Lambda functions can connect to the same file system simultaneously, sharing data through a common workspace. This is particularly valuable for AI and machine learning workloads where agents need to persist memory and share state across pipeline steps.
Amazon EKS Hybrid Nodes gateway for hybrid Kubernetes networking — Amazon Elastic Kubernetes Service now offers the Amazon EKS Hybrid Nodes gateway, which automates networking between your EKS cluster VPC and Kubernetes Pods running on EKS Hybrid Nodes. You can now eliminate the need to make on-premises pod networks routable or coordinate network infrastructure changes, greatly simplifying hybrid Kubernetes environments. The gateway automatically enables pod-to-pod traffic across cloud and on-premises environments, control plane-to-webhook communication, and connectivity for AWS services like Application Load Balancers, and is available at no additional charge.
Amazon Aurora Serverless: Up to 30% better performance, smarter scaling, and still scales to zero — Amazon Aurora Serverless just got faster and smarter, with up to 30% better performance than the previous version and an enhanced scaling algorithm designed to handle workloads where multiple tasks compete for resources — like busy APIs and agentic AI applications with bursts of activity and long idle windows. You can now run even more demanding workloads serverlessly, paying only for what you use, and automatically scaling to zero when not in use. All improvements are available in platform version 4 at no additional cost.
Amazon Bedrock AgentCore adds new features to help developers build agents faster — Amazon Bedrock AgentCore introduces a managed harness (preview), the AgentCore CLI, and AgentCore skills for coding assistants, helping developers go from idea to working agent prototype faster. The managed harness lets you define an agent by specifying a model, system prompt, and tools and run it immediately with no orchestration code required — and when you’re ready for full control, you can export the harness orchestration as Strands-based code. The AgentCore CLI deploys your agents with the governance and auditability of infrastructure-as-code (AWS CDK today, Terraform coming soon), and is available in 14 AWS Regions at no additional charge.
For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.
Other AWS news
Here are some additional posts and resources that you might find interesting:
Introducing granular cost attribution for Amazon Bedrock — This post walks through how Amazon Bedrock’s granular cost attribution works and covers practical example cost tracking scenarios. You can now tag and track Bedrock usage costs at a finer level of detail — useful for organizations running multiple teams or projects on Bedrock who need precise cost visibility and chargeback capabilities.
Automating Incident Investigation with AWS DevOps Agent and Salesforce MCP Server — This post (co-written with Salesforce) shows how AWS DevOps Agent, integrated with the Salesforce MCP Server, automates the full lifecycle of infrastructure incident investigation — from identifying issues and diagnosing root causes to notifying customers through Salesforce Service Cloud. It’s a compelling real-world example of how AI agents and MCP-based tool connectivity are reshaping DevOps workflows in production, dramatically reducing mean time to resolution.
Microcredentials from AWS are now free — Here’s why that matters — You can now access AWS microcredentials at no cost through AWS Skill Builder in all countries where the platform is offered. Unlike traditional multiple-choice certifications, microcredentials are hands-on assessments that place builders in simulated business scenarios where they configure, troubleshoot, and optimize directly in a live AWS environment — the same way they would on the job. A great opportunity to validate real cloud skills without a cost barrier.
Amazon SageMaker AI now supports optimized generative AI inference recommendations — You can now use Amazon SageMaker AI to automatically identify optimized deployment configurations for your generative AI models, including instance type, container, and inference parameters. This new capability takes the guesswork out of tuning inference infrastructure, helping you reduce costs and improve latency for your AI applications in production.
Upcoming AWS events
Check your calendar and sign up for upcoming AWS events:
What’s Next with AWS — Tune in on April 28 for What’s Next with AWS, a virtual event featuring the latest announcements and product updates directly from AWS teams. A great opportunity to get up to speed on what’s new before diving into the week’s launches.
AWS Summits — AWS Summits are free in-person events where you can explore the latest in cloud and AI innovation, learn best practices, and network with builders and experts. Coming up in May: Singapore (May 6), Tel Aviv (May 6), Warsaw (May 6), Stockholm (May 7), Sydney (May 13–14), Hamburg (May 20), Seoul (May 20), Amsterdam (May 27), Bangkok (May 28), Milan (May 28), and Mumbai (May 28). And in June, join us in Los Angeles (June 10). Check the full schedule and register at the link above.
AWS Community Days — Community-led conferences where content is planned, sourced, and delivered by community leaders, featuring technical discussions, workshops, and hands-on labs. Upcoming events include Athens, Greece (April 28), Vancouver, Canada (May 1), İstanbul, Türkiye (May 9), and Panama City, Panama (May 23). If you’re in Latin America, mark your calendar for the AWS Community Day Belo Horizonte (August 22) — registration is open at awscommunityday.com.br.
Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development. Browse here for upcoming AWS-led in-person and virtual events and developer-focused events.
That’s all for this week. Check back next Monday for another Weekly Roundup!
— Daniel Abib
This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!
This post is cowritten with Avdhesh Paliwal from Oldcastle.
Oldcastle APG is one of the largest suppliers of construction materials in North America, including asphalt and concrete. The company also provides construction and paving services across more than 150 facilities. As the company migrated from on-premises systems to Infor Cloud ERP hosted on Amazon Web Services (AWS), they faced a critical challenge: maintaining the real-time operational reporting capabilities that hundreds of users across customer service, finance, logistics, and manufacturing depended on daily.
This post explores how Oldcastle used AWS services to transform their analytics and AI capabilities by integrating Infor ERP with Amazon Aurora and Amazon Quick Sight. We discuss how they overcame the limitations of traditional cloud ERP reporting to deploy real-time dashboards and build a scalable analytics system. This practical, enterprise-grade approach offers a blueprint that organizations can adapt when extending ERP capabilities with cloud-native analytics and AI.
Challenges with cloud ERP reporting
The primary challenge that we faced was finding a solution that could accomplish the following:
Maintain real-time data access – Our on-premises environment supported hundreds of complex real-time reports, but Infor ERP Cloud’s configuration-based reporting covered minimal reports of our operational needs.
Support complex reporting requirements – Users needed multi-dimensional analysis across customer service, finance, logistics, and manufacturing functions.
Provide seamless user experience – Business users demanded integrated reporting within the ERP network without switching between multiple systems.
Enable advanced analytics –We needed capabilities for demand forecasting, machine learning (ML) capabilities, and intelligent search across real-time data.
Scale efficiently – The solution needed to support over 100 concurrent users and process millions of transactions while maintaining performance.
Expose Data using API – The solution needed to expose data through APIs, allowing both external and internal applications to access and consume the data securely and efficiently.
Our existing batch reporting process created significant operational challenges across our organization. We had to wait for batch reports, which consumed valuable time and led to delays in critical decision-making across many of our teams. This lag prevented us from capitalizing on real-time business insights and responding quickly to operational issues or economic changes. Without immediate data visibility, our managers couldn’t make timely, data-driven decisions, resulting in missed opportunities for improvement and competitive advantage. Our IT team also struggled with constant report requests but lacked a scalable system to deliver them efficiently, further compounding the productivity loss across our organization.
Solution overview
AWS Solutions Architects worked closely with our application team to build a comprehensive analytics and AI solution to address these challenges. The architecture uses Infor Data Fabric Stream Pipelines to deliver real-time data to AWS. It powers operational dashboards, artificial intelligence and machine learning (AI/ML) models, and intelligent search capabilities. This approach aligns with Infor’s broader strategy of integrating ERP data, data lake information, machine learning (ML) predictions, and documentation to provide comprehensive end-to-end business solutions.
Real-time data streaming architecture
The foundation of our solution is Infor’s Data Fabric Stream Pipelines, an add-on feature that provides real-time streaming data processing. When data events are ingested into Data Fabric, Stream Pipelines processes them immediately and continuously without waiting for storage in the data lake. This approach minimizes the data journey and accelerates operations, helping us extract insights from our data in real time.
The end-to-end workflow consists of the following components:
Data ingestion – Infor Data Fabric tables stream changes in real-time. We enabled Stream Pipelines as an add-on feature within our Infor Cloud ERP environment. We Configure the specific ERP table that we want to stream (such as sales orders, inventory, financial transactions) to publish change events immediately upon data modification. Stream Pipelines captures insert, update, and delete operations with metadata about the operation type and timestamp.
Load distribution – Because Infor can’t reach our private VPC directly, we use Elastic Load Balancing (ELB) to distribute traffic and provide secure database access. We implemented a Network Load Balancer (NLB) with static Elastic IP addresses in public subnets, giving us stable, allowlisted IP addresses for Infor’s outbound connections. We configured an Amazon Relational Database Service (Amazon RDS) router with Amazon Elastic Compute Cloud (Amazon EC2) instances as NLB targets. These routers forward traffic from the NLB to our Amazon Aurora database in the private subnet using iptables NAT rules. This makes sure that even if the IP of Aurora changes during failover, our static Elastic IPs remain constant. We configured security groups to accept HTTPS traffic (port 443) only from Infor’s IP ranges on the NLB and allow traffic only from the NLB to the RDS routers.
Connection management – We use Amazon RDS Proxy to manage database connections and provide automatic failover. We deployed RDS Proxy in the private subnet between our RDS router instances and Aurora cluster to pool and reuse connections. This is critical for handling our high-frequency streaming data. We configured the proxy with IAM authentication for secure credentials and set connection pool parameters based on our expected concurrent stream volume to handle burst traffic without overwhelming the database. With automatic failover enabled, if our primary Aurora instance fails, RDS Proxy automatically redirects traffic to the promoted replica, maintaining continuous data flow.
Data storage – We store our operational data in Amazon Aurora PostgreSQL- Compatible Edition with multi-Availability Zone deployment for high availability. We provisioned an Aurora PostgreSQL cluster with one writer instance and multiple reader instances across different Availability Zones. We designed our database schema to handle the incoming streaming data, storing it in JSONB columns for flexible querying while using the native JSON functions of Aurora PostgreSQL when we need to parse and normalize specific fields. We created indexes on frequently queried fields to maintain query performance as our data volume grows. We also configured automated backups with point-in-time recovery and set up the automatic storage of Aurora scaling to accommodate our data growth.
Analytics and visualization – Amazon Quick Sight delivers the interactive dashboards and pixel-perfect reports our teams need. We created a Quick Sight account and established a connection to our Aurora PostgreSQL database using VPC connectivity with credentials stored in AWS Secrets Manager. We identified which datasets benefit from SPICE (Super-fast, Parallel, In-memory Calculation Engine) caching—typically aggregated or frequently accessed data—and configured incremental refresh schedules to keep them current. We built our dashboards using the visual interface of Quick Sight, using calculated fields for business logic, parameters for user interactivity, and row-level security rules to make sure that users only see data that they’re authorized to access. For pixel-perfect reports, we use the pixel-perfect report feature of Quick Sight to create formatted documents suitable for printing or regulatory compliance.
Embedded integration – We securely embedded Amazon Quick Sight dashboards within Infor OS through Amazon API Gateway, which generates dynamic URLs for seamless user access. We enabled Quick Sight embedding in our AWS account and registered our Infor domain. We created an API Gateway REST API with Lambda functions that authenticate users, validate Infor session tokens, and call QuickSight’s GenerateEmbedUrlForRegisteredUser API to produce time-limited, signed URLs with row-level security. Our Lambda function maps Infor user roles to Quick Sight permissions and applies dashboard filters based on the user’s organizational context. We configured CORS settings in API Gateway to allow requests from our Infor domain and implemented rate limiting. On the Infor side, we embedded the Quick Sight dashboards using iframe elements that call our API Gateway endpoint, providing a seamless experience where our users access analytics without leaving the ERP interface.
The following diagram illustrates the real-time analytics architecture:
This embedded experience aligns with Infor’s broader system strategy of integrating insights seamlessly into workflows.
Results and business impact
The implementation using this architecture on AWS brought substantial benefits, directly addressing the critical challenges that we faced and demonstrating measurable value in employee productivity and core business process optimization.
Business process improvement
The solution successfully addressed Oldcastle’s key operational challenges:
Challenge: Limited visibility into real-time operations
– Solution delivered: Deployed over 50 complex dashboards and reports in eight months, providing immediate visibility across customer service, finance, logistics, and manufacturing.- Technical achievement: Used Infor Data Fabric Stream Pipelines to process data events immediately upon ingestion, alleviating delays from traditional batch processing.- Impact: Real-time streaming architecture using the NDJSON format makes sure decision-makers have access to current operational data when they need it most.
Challenge: Fragmented user experience requiring multiple systems
– Solution delivered: Dashboards embedded directly into the Infor environment through Amazon API Gateway.- Technical achievement: Generated dynamic URLs for secure embedding with single sign-on capabilities.- Impact: Users access insights without leaving their familiar interface, with personalized views based on roles and permissions that maintain context across the application.
Challenge: Inflexible reporting limiting business agility
– Solution delivered: Both interactive dashboards and pixel-perfect reports are available to meet diverse business needs.- Technical achievement: Quick Sight SPICE caching enables subsecond response times on complex analytics across large datasets.- Impact: On-demand access for immediate insights, scheduled distribution, custom formatting aligned with corporate standards, and multiple export formats (PDF, CSV, Excel).
Challenge: Delayed decision-making due to outdated data
– Solution delivered: Real-time visibility into operations enabled faster, data-driven decisions.- Technical achievement: The Multi-AZ deployment of Amazon Aurora PostgreSQL maintains high availability and continuous data access.- Impact: Decision-makers can respond immediately to operational issues and economic changes with current, reliable data.
Scalability and performance
The architecture delivered exceptional scale and performance, addressing concerns about future growth: – High concurrency: Supports over 100 concurrent users without performance degradation- Data volume handling: Processes millions of transactions daily in real-time- Elastic scaling: Aurora read replicas automatically scale based on demand- Future-ready: Architecture designed to expand to additional regions and use cases- Cost efficiency: AWS services avoided complex third-party integrations, with infrastructure costs scaling efficiently with business growth- API capabilities: Ability to expose data using AWS technologies enables integration with third-party and internal applications
Conclusion
Our journey proves that cloud ERP migrations don’t require sacrificing real-time operational reporting capabilities. By combining Infor Data Fabric Stream Pipelines with AWS analytics and AI services, we’ve maintained real-time data access, accelerated innovation, improved user experience, and built a system that scales efficiently as our business needs evolve. The combination of Infor’s enterprise-grade ERP system with the comprehensive analytics capabilities of AWS has given us the best of both worlds. As we expand our AWS analytics and AI capabilities, we’re not just maintaining parity with on-premises systems, we’re unlocking new sources of business value that weren’t possible before.
Further reading
For more information on the services mentioned in the post, see the following resources:
AWS services:
Amazon Aurora PostgreSQL Features – Learn more about the high-availability database that powers Oldcastle’s real-time data storage and Multi-AZ deployment strategy
Amazon Quick Sight Embedded Analytics – Explore how to embed interactive dashboards and pixel-perfect reports directly into your enterprise applications, as demonstrated in Oldcastle’s Infor OS integration
Amazon Bedrock for Generative AI – Discover opportunities to enhance your analytics system with AI-powered insights and intelligent search capabilities
Elastic Load Balancing – Understand how to distribute traffic and secure database connections when integrating cloud ERP systems with AWS services
Amazon API Gateway – Learn how to create secure, dynamic URLs for embedding analytics and exposing data through APIs to internal and external applications
Infor Resources:
Infor Data Fabric Documentation – Deep dive into Stream Pipelines configuration and real-time data streaming capabilities that enable immediate operational insights
Last week, what excited me most was the launch of the 2026 AWS AI & ML Scholars program by Swami Sivasubramanian, VP of AWS Agentic AI, to provide free AI education to up to 100,000 learners worldwide. The program has two phases: a Challenge phase where you’ll learn foundational generative AI skills, followed by a fully funded three-month Udacity Nanodegree for the top 4,500 performers. Anyone 18 or older can apply, with no prior AI or ML experience required. Applications close on June 24, 2026. Visit the AWS AI & ML Scholars webpage to learn more and apply.
I’m also excited about the start of AWS Summit season, kicking off with AWS Summit Paris on April 1, followed by London on April 22. AWS Summits are free in-person events where builders and innovators can learn about Cloud and AI, think big, and make new connections. Explore the AWS Summits near you and join us in person.
Now, let’s dive into this week’s AWS news…
Last week’s launches Here are last week’s launches that caught my attention:
Announcing Amazon Aurora PostgreSQL serverless database creation in seconds — Amazon Aurora PostgreSQL now offers express configuration, a streamlined setup with preconfigured defaults that supports creating and connecting to a database in seconds. With just two clicks, you can launch an Aurora PostgreSQL serverless database. You can modify certain settings during or after creation.
Amazon Aurora PostgreSQL now available with the AWS Free Tier — Amazon Aurora PostgreSQL is now available on the AWS Free Tier. If you’re new to AWS, you receive $100 in AWS credits upon sign-up and can earn an additional $100 in credits by using services like Amazon Relational Database Service (Amazon RDS).
Announcing Agent Plugin for AWS Serverless — With the new Agent Plugin for AWS Serverless, you can easily build, deploy, troubleshoot, and manage serverless applications using AI coding assistants like Kiro, Claude Code, and Cursor. This plugin extends AI assistants with structured capabilities by packaging skills, sub-agents, and Model Context Protocol (MCP) servers into one modular unit. It automatically loads the guidance and expertise you need throughout development to build production-ready serverless applications on AWS.
Amazon SageMaker Studio now supports Kiro and Cursor IDEs as remote IDEs — You can now remotely connect from Kiro and Cursor IDEs to Amazon SageMaker Studio. This lets you use your existing Kiro and Cursor setup, including spec-driven development, conversational coding, and automated feature generation, while accessing the scalable compute resources of Amazon SageMaker Studio.
Introducing visual customization capability in AWS Management Console — You can now customize your AWS Management Console with visual settings like account color and control which regions and services you see. Hiding unused regions and services helps you focus better and work faster by reducing cognitive load and unnecessary scrolling.
Announcing Aurora DSQL connector to simplify building Ruby applications — You can now use the Aurora DSQL Connector for Ruby (pg gem) to easily build Ruby applications on Aurora DSQL. The Ruby Connector simplifies authentication and improves security by automatically generating tokens for each connection, eliminating the risks of traditional passwords while maintaining full compatibility with existing pg gem features.
AWS Lambda increases the file descriptor limit for functions running on Lambda Managed Instances — AWS Lambda increases the file descriptor limit from 1,024 to 4,096, a 4x increase, for functions running on Lambda Managed Instances (LMI). You can now run I/O intensive workloads such as high-concurrency web services and file-heavy data processing pipelines without running into file descriptor limits.
AWS Lambda now supports up to 32 GB of memory and 16 vCPUs for Lambda Managed Instances — AWS Lambda functions on Lambda Managed Instances now support up to 32 GB of memory and 16 vCPUs. You can run compute-intensive workloads like data processing, media transcoding, and scientific simulations without managing infrastructure. Plus, you can adjust the memory-to-vCPU ratio (2:1, 4:1, or 8:1) to fit your workload.
Announcing Bidirectional Streaming API for Amazon Polly — Traditional text-to-speech APIs use a request-response pattern. The new Bidirectional Streaming API for Amazon Polly is designed for conversational AI applications that generate text or audio incrementally, like large language model (LLM) responses. This lets you start synthesizing audio before the full text is available.
For a full list of AWS announcements, be sure to keep an eye on our News Blog channel and the What’s New with AWS page.
Upcoming AWS events Check your calendar and sign up for upcoming AWS events:
AWS Summits — As I mentioned earlier, join AWS Summits in 2026 for free in-person events where you can explore emerging cloud and AI technologies, learn best practices, and network with industry peers and experts. Upcoming Summits include Paris (April 1), London (April 22), Bengaluru (April 23–24), Singapore (May 6), Tel Aviv (May 6), and Stockholm (May 7).
AWS Community Days — Community-led conferences where content is planned, sourced, and delivered by community leaders, featuring technical discussions, workshops, and hands-on labs. Upcoming events include San Francisco (April 10) and Romania (April 23–24).
Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development. Browse the AWS Events and Webinars for upcoming AWS-led in-person and virtual events and developer-focused events.
That’s all for this week. Check back next Monday for another Weekly Roundup!
At re:Invent 2025, Colin Lazier, vice president of databases at AWS, emphasized the importance of building at the speed of an idea—enabling rapid progress from concept to running application. Customers can already create production-ready Amazon DynamoDB tables and Amazon Aurora DSQL databases in seconds. He previewed creating an Amazon Aurora serverless database with the same speed, and customers have since requested quick access and speed to this capability.
Today, we’re announcing the general availability of a new express configuration for Amazon Aurora PostgreSQL, a streamlined database creation experience with preconfigured defaults designed to help you get started in seconds.
With only two clicks, you can have an Aurora PostgreSQL serverless database ready to use in seconds. You have the flexibility to modify certain settings during and after database creation in the new configuration. For example, you can change the capacity range for the serverless instance at the time of create or add read replicas, modify parameter groups after the database is created. Aurora clusters with express configuration are created without an Amazon Virtual Private Cloud (Amazon VPC) network and include an internet access gateway for secure connections from your favorite development tools – no VPN, or AWS Direct Connect required. Express configuration also sets up AWS Identity and Access Management (IAM) authentication for your administrator user by default, enabling passwordless database authentication from the beginning without additional configuration.
After it’s created, you have access to features available for Aurora PostgreSQL serverless, such as deploying additional read replicas for high availability and automated failover capabilities. This launch also introduces a new internet access gateway routing layer for Aurora. Your new serverless instance comes enabled by default with this feature, which allows your applications to connect securely from anywhere in the world through the internet using the PostgreSQL wire protocol from a wide range of developer tools. This gateway is distributed across multiple Availability Zones, offering the same level of high availability as your Aurora cluster.
Creating and connecting to Aurora in seconds means fundamentally rethinking how you get started. We launched multiple capabilities that work together to help you onboard and run your application with Aurora. Aurora is now available on AWS Free Tier, which you gain hands-on experience with Aurora at no upfront cost. After it’s created, you can directly query an Aurora database in AWS CloudShell or using programming languages and developer tools through a new internet accessible routing component for Aurora. With integrations such as v0 by Vercel, you can use natural language to start building your application with the features and benefits of Aurora.
Create an Aurora PostgreSQL serverless database in seconds To get started, go to the Aurora and RDS console and in the navigation pane, choose Dashboard. Then, choose Create with a rocket icon.
Review pre-configured settings in the Create with express configuration dialog box. You can modify the DB cluster identifier or the capacity range as needed. Choose Create database.
Your Aurora PostgreSQL serverless database should be ready in seconds. A success banner confirms the creation, and the database status changes to Available.
After your database is ready, go to the Connectivity & security tab to access three connection options. When connecting through SDKs, APIs, or third-party tools including agents, choose Code snippets. You can choose various programming languages such as .NET, Golang, JDBC, Node.js, PHP, PSQL, Python, and TypeScript. You can paste the code from each step into your tool and run the commands.
For example, the following Python code is dynamically generated to reflect the authentication configuration:
Choose CloudShell for quick access to the AWS CLI which launches directly from the console. When you choose Launch CloudShell, you can see the command is pre-populated with relevant information to connect to your specific cluster. After connecting to the shell, you should see the psql login and the postgres => prompt to run SQL commands.
You can also choose Endpoints to use tools that only support username and password credentials, such as pgAdmin. When you choose Get token, you use an AWS Identity and Access Management (IAM) authentication token generated by the utility in the password field. The token is generated for the master username that you set up at the time of creating the database. The token is valid for 15 minutes at a time. If the tool you’re using terminates the connection, you will need to generate the token again.
Building your application faster with Aurora databases At re:Invent 2025, we announced enhancements to the AWS Free Tier program, offering up to $200 in AWS credits that can be used across AWS services. You’ll receive $100 in AWS credits upon sign-up and can earn an additional $100 in credits by using services such as Amazon Relational Database Service (Amazon RDS), AWS Lambda, and Amazon Bedrock. In addition, Amazon Aurora is now available across a broad set of eligible Free Tier database services.
Developers are embracing platforms such as Vercel, where natural language is all it takes to build production-ready applications. We announced integrations with Vercel Marketplace to create and connect to an AWS database directly from Vercel in seconds and v0 by Vercel, an AI-powered tool that transforms your ideas into production-ready, full-stack web applications in minutes. It includes Aurora PostgreSQL, Aurora DSQL, and DynamoDB databases. You can also connect your existing databases created through express configuration with Vercel. To learn more, visit AWS for Vercel.
Like Vercel, we’re bringing our databases seamlessly into their experiences and are integrating directly with widely adopted frameworks, AI assistant coding tools, environments, and developer tools, all to unlock your ability to build at the speed of an idea.
Now available You can create an Aurora PostgreSQL serverless database in seconds today in all AWS commercial Regions. For Regional availability and a future roadmap, visit the AWS Capabilities by Region.
You pay only for capacity consumed based on Aurora Capacity Units (ACUs) billed per second from zero capacity, which automatically starts up, shuts down, and scales capacity up or down based on your application’s needs. To learn more, visit the Amazon Aurora Pricing page.
If you manage data in Amazon Aurora MySQL-Compatible Edition and want to make it available for analytics, machine learning (ML), or cross-service querying in a modern lakehouse format, you’re not alone.
Organizations often need to run analytics, build ML models, or join data across multiple sources. These are examples of workloads that can be resource-intensive and impractical to run directly against a transactional database. By extracting your Aurora MySQL data into Amazon S3 Tables in Apache Iceberg format, you can offload analytical queries from your production database without impacting its performance, while storing data in a fully managed Iceberg table store optimized for analytics. Built on the open Apache Iceberg standard, Amazon Simple Storage Service (Amazon S3) Table data is queryable from engines like Amazon Athena, Amazon Redshift Spectrum, and Apache Spark without additional data copies. You can also combine relational data with other datasets already in your data lake, enabling richer cross-domain insights.
Apache Iceberg and Amazon S3 Tables
Apache Iceberg is a widely adopted open table format that offers Atomicity, Consistency, Isolation, Durability (ACID) transactions, schema evolution, and time travel capabilities. It enables multiple engines to work concurrently on the same dataset, making it a popular choice for building open lakehouse architectures.
Amazon S3 Tables is a purpose-built, fully managed Apache Iceberg table store designed for analytics workloads. It delivers up to 3x faster query performance and up to 10x more transactions per second compared to self-managed Iceberg tables. It also automatically compacts data and removes unreferenced files to optimize storage and performance.
In this post, you learn how to set up an automated, end-to-end solution that extracts tables from Amazon Aurora MySQL Serverless v2 and writes them to Amazon S3 Tables in Apache Iceberg format using AWS Glue. The entire infrastructure is deployed using a single AWS CloudFormation stack.
Requirements
AWS offers zero-ETL integrations from Amazon Aurora to Amazon Redshift and Amazon SageMaker AI, enabling seamless data flow for analytics and machine learning workloads.
However, there isn’t yet a native zero-ETL integration between Amazon Aurora and Amazon S3 Tables. This means that organizations looking to use Amazon S3 Tables for their Lakehouse architecture currently face several requirements:
Setting up ETL pipelines to extract data from Amazon Aurora and transform it into Apache Iceberg format
Configuring networking and security for AWS Glue jobs to access Amazon Aurora databases in private subnets
Coordinating the provisioning of source databases, ETL pipelines, and target table stores
Managing the end-to-end workflow without native automation
Solution overview
In this solution, you automate the extraction of relational database tables from Amazon Aurora MySQL Serverless v2 to Amazon S3 Tables in Apache Iceberg format using AWS Glue 5.0. To help you get started and test this solution, a CloudFormation template is provided. This template provisions the required infrastructure, loads sample data, and configures the Extract, Transform, Load (ETL) pipeline. You can adapt this template for your own scenario.
Sample data
This solution uses the TICKIT sample database, a well-known dataset used in Amazon Redshift documentation. The TICKIT data models a fictional ticket sales system with seven interrelated tables: users, venue, category, date, event, listing, and sales. The dataset is publicly available as mentioned in the Amazon Redshift Getting Started Guide.
Solution flow
The solution flow as shown in the previous architecture diagram:
An AWS Lambda function downloads the TICKIT sample dataset (a fictional ticket sales system used in Amazon Redshift documentation) from a public Amazon S3 bucket to a staging S3 bucket.
A second Lambda function, using PyMySQL (a Python MySQL client library), loads the staged data files into the Aurora MySQL Serverless v2 database using LOAD DATA LOCAL INFILE.
The AWS Glue job reads seven TICKIT tables from Aurora MySQL through a native MySQL connection and writes them to Amazon S3 Tables in Apache Iceberg format using the S3 Tables REST catalog endpoint with SigV4 authentication.
You can query the migrated data in S3 Tables using Amazon Athena.
The solution consists of the following key components:
Amazon Aurora MySQL Serverless v2 as the source relational database containing the TICKIT sample dataset (users, venue, category, date, event, listing, and sales tables)
Amazon S3 staging bucket for the TICKIT sample data files downloaded from the public redshift-downloads S3 bucket
AWS Lambda functions using PyMySQL to load data into Aurora MySQL
AWS Glue 5.0 job (PySpark) to read tables from Aurora MySQL and write them to S3 Tables in Apache Iceberg format
Amazon S3 Tables as the target storage for the migrated Iceberg tables
Amazon VPC with private subnets and VPC endpoints for Amazon S3, S3 Tables, AWS Glue, Secrets Manager, AWS Security Token Service (AWS STS), CloudWatch Logs, and CloudFormation
Here are some advantages of this architecture:
Fully automated setup: A single CloudFormation stack provisions the required infrastructure, loads sample data, and configures the ETL pipeline.
Serverless and cost-efficient: Aurora MySQL Serverless v2 and AWS Glue both scale based on demand, minimizing idle costs.
Apache Iceberg table format: Data is stored in Apache Iceberg format, enabling ACID transactions, schema evolution, and time travel queries.
Network isolation and credential management: The resources run within private subnets with Virtual Private Cloud (VPC) endpoints, and database credentials are managed through AWS Secrets Manager.
Extensible pattern: The same approach can be adapted for other relational databases (PostgreSQL, SQL Server) and other target formats supported by AWS Glue.
Prerequisites
To follow along, you need an AWS account. If you don’t yet have an AWS account, you must create one. The CloudFormation stack deployment takes approximately 30-45 minutes to complete and requires familiarity with Amazon S3 Tables, AWS CloudFormation, Apache Iceberg, AWS Glue, Amazon Aurora. This solution will incur AWS costs. The main cost drivers are AWS Glue ETL job runs (billed per DPU-hour, proportional to data volume) and Amazon S3 Tables storage and request charges. Remember to clean up resources when you are done to avoid unnecessary charges.
CloudFormation parameters
You can configure the following parameters before deploying the CloudFormation stack:
Parameter
Description
Default
Required
S3TableBucketName
Name of the S3 Tables bucket to create (or use existing)
Yes
DatabaseName
Name of the initial Aurora MySQL database
tickit
No
MasterUsername
Master username for Aurora MySQL
admin
No
VpcCidr
CIDR block for the VPC
10.1.0.0/16
No
S3TableNamespace
Namespace for S3 Tables
tickit
No
Implementation walkthrough
The following steps walk you through the implementation. These steps are to deploy and test an end-to-end solution from scratch. If you are already running some of these components, you may skip to the relevant step. You can also refer to the aws-samples repository, sample-to-write-aurora-mysql-to-s3tables-using-glue for the entire solution.
Step 1: Deploy the CloudFormation stack
Deploy the CloudFormation template scripts/aurora-mysql-to-s3tables-stack.yaml using the AWS Console or the AWS Command Line Interface (AWS CLI). Provide a name for the S3 Tables bucket; the stack will create it automatically (or use an existing one if it already exists).
To deploy using the AWS Console (recommended), navigate to the AWS CloudFormation Console and use the CloudFormation template. Alternatively, to deploy using the AWS CLI first upload the template to an S3 bucket (the template exceeds the 51,200 byte limit for inline –template-body), then create the stack.
# Upload the template to S3
aws s3 cp scripts/aurora-mysql-to-s3tables-stack.yaml \
s3://<your-s3-bucket>/aurora-mysql-to-s3tables-stack.yaml \
--region <your-region>
# Create the stack using the S3 template URL
aws cloudformation create-stack \
--stack-name aurora-mysql-tickit-stack \
--template-url https://<your-s3-bucket>.s3.<your-region>.amazonaws.com/aurora-mysql-to-s3tables-stack.yaml \
--parameters \
ParameterKey=S3TableBucketName,ParameterValue=<your-s3-table-bucket-name> \
--capabilities CAPABILITY_NAMED_IAM \
--region <your-region>
The stack will automatically:
Create the S3 Tables bucket (or use existing if it already exists)
Create a VPC with private subnets and VPC endpoints
Provision an Aurora MySQL Serverless v2 cluster
Download TICKIT sample data from the public Amazon S3 bucket
Load the sample data into Aurora MySQL via a Lambda function using PyMySQL
Create a Glue job configured to migrate data to S3 Tables in Iceberg format
Note: The S3 Tables bucket is retained when the stack is deleted to preserve your data.
Step 2: Verify the Aurora MySQL data
Retrieve the AuroraClusterEndpoint, DatabaseName, and SecretArn values from the CloudFormation stack, make a note of the AuroraClusterEndpoint, DatabaseName, and SecretArn. You can navigate to the Amazon Aurora Console, choose the Query Editor, and enter the values from the CloudFormation stack to connect. You can also choose your preferred method of connecting to an Amazon Aurora DB cluster.
Then run the following SQL commands to verify the data load:
-- Verify if the tables are created
SELECT * FROM information_schema.tables WHERE table_schema = 'tickit';
-- Verify if the data is loaded
SELECT 'users' AS table_name, COUNT(*) AS record_count FROM tickit.users
UNION ALL SELECT 'venue', COUNT(*) FROM tickit.venue
UNION ALL SELECT 'category', COUNT(*) FROM tickit.category
UNION ALL SELECT 'date', COUNT(*) FROM tickit.date
UNION ALL SELECT 'event', COUNT(*) FROM tickit.event
UNION ALL SELECT 'listing', COUNT(*) FROM tickit.listing
UNION ALL SELECT 'sales', COUNT(*) FROM tickit.sales;
Step 3: Run the Glue job
Navigate to the AWS Glue Console, choose ETL jobs under Data Integration and ETL from the left panel. Select the AWS Glue job mysql-tickit-to-iceberg-job and choose Run job to start execution. You can also start the ETL job using the AWS CLI:
The AWS Glue job performs the following operations for each of the seven TICKIT tables:
Reads the table from Aurora MySQL through the native MYSQL Glue connection
Converts the data to a Spark DataFrame
Creates the Iceberg table in the S3 Tables namespace using CREATE TABLE IF NOT EXISTS with the USING ICEBERG clause
Inserts the data using INSERT INTO (or INSERT OVERWRITE if the table already exists)
Verifies the record count and displays sample data
Step 4: Verify the results
After the AWS Glue job completes, verify that the tables have been created in your S3 Table bucket by navigating to the Amazon S3 Console. Choose Table buckets under Buckets and select your S3 Table bucket. You can also verify using the AWS CLI:
The S3 Tables bucket is retained by default. To delete it, use the Amazon S3 console or the AWS CLI to remove the table bucket separately. The staging S3 bucket will be automatically emptied and deleted as part of the stack deletion.
In this post, we showed you how to extract data from Amazon Aurora MySQL Serverless v2 and write it to Amazon S3 Tables in Apache Iceberg format using AWS Glue 5.0. By using the native Iceberg support of AWS Glue and the S3 Tables REST catalog endpoint, you can bridge the gap between relational databases and modern lakehouse storage formats. By automating the entire pipeline through CloudFormation, you can quickly set up and replicate this pattern across multiple environments.
As AWS Glue and Amazon S3 Tables continue to evolve, you can take advantage of future enhancements while maintaining this automated migration pattern.
If you have questions or suggestions, leave us a comment.
Last week, my team met many developers at Developer Week in San Jose. My colleague, Vinicius Senger delivered a great keynote about renascent software—a new way of building and evolving applications where humans and AI collaborate as co-developers using Kiro. Other colleagues spoke about building and deploying production-ready AI agents. Everyone stayed to ask and hear the questions related to agent memory, multi-agent patterns, meta-tooling and hooks. It was interesting how many developers were actually building agents.
We are continuing to meet developers and hear their feedback at third-party developer conferences. You can meet us at the dev/nexus, the largest and longest-running Java ecosystem conference on March 4-6 in Atlanta. My colleague, James Ward will speak about building AI Agents with Spring and MCP, and Vinicius Senger and Jonathan Vogel will speak about 10 tools and tips to upgrade your Java code with AI. I’ll keep sharing places for you to connect with us.
Last week’s launches Here are some of the other announcements from last week:
Claude Sonnet 4.6 model in Amazon Bedrock – You can now use Claude Sonnet 4.6 which offers frontier performance across coding, agents, and professional work at scale. Claude Sonnet 4.6 approaches Opus 4.6 intelligence at a lower cost. It enables faster, high-quality task completion, making it ideal for high-volume coding and knowledge work use cases.
Amazon EC2 Hpc8a instances powered by 5th Gen AMD EPYC processors – You can use new Hpc8a instances delivering up to 40% higher performance, increased memory bandwidth, and 300 Gbps Elastic Fabric Adapter networking. You can accelerate compute-intensive simulations, engineering workloads, and tightly coupled HPC applications.
Amazon SageMaker Inference for custom Amazon Nova models – You can now configure the instance types, auto-scaling policies, and concurrency settings for custom Nova model deployments with Amazon SageMaker Inference to best meet your needs.
Nested virtualization on virtual Amazon EC2 instances – You can create nested virtual machines by running KVM or Hyper-V on virtual EC2 instances. You can leverage this capability for use cases such as running emulators for mobile applications, simulating in-vehicle hardware for automobiles, and running Windows Subsystem for Linux on Windows workstations.
Server-Side Encryption by default in Amazon Aurora – Amazon Aurora further strengthens your security posture by automatically applying server-side encryption by default to all new databases clusters using AWS-owned keys. This encryption is fully managed, transparent to users, and with no cost or performance impact.
Kiro in AWS GovCloud (US) Regions – You can use Kiro for the development teams behind government missions. Developers in regulated environments can now leverage Kiro’s agentic AI tool with the rigorous security controls required.
For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.
Additional updates Here are some additional news items that you might find interesting:
Introducing Agent Plugins for AWS – You can see how new open-source Agent Plugins for AWS extend coding agents with skills for deploying applications to AWS. Using the deploy-on-aws plugin, you can generate architecture recommendations, cost estimates, and infrastructure-as-code directly from your coding agent.
A chat with Byron Cook on automated reasoning and trust in AI systems – You can hear how to verify AI systems doing the right thing using automated reasoning when they generate code or manage critical decisions. Byron Cook’s team has spent a decade proving correctness in AWS and apply those techniques to agentic systems.
Best practices for deploying AWS DevOps Agent in production – You can read best practices for setting up DevOps Agent Spaces that balance investigation capability with operational efficiency. According to Swami Sivasubramanian, AWS DevOps Agent, a frontier agent that resolves and proactively prevents incidents, has handled thousands of escalations, with an estimated root cause identification rate of over 86% within Amazon.
From AWS community Here are my personal favorite posts from AWS community:
Let an AI Agent Do Your Job Searching – Still manually checking career pages during a job search? AWS Hero Danielle H. built an AI agent that does the work for you.
Building the AWS Serverless Power for Kiro – A former AWS Serverless Hero, Gunnar Grosch built a Kiro Power to integrate 25 MCP tools, ten steering guides, and structured decision guidance for the full development lifecycle.
Join the AWS Builder Center to connect with community, share knowledge, and access content that supports your development.
Upcoming AWS events Check your calendar and sign up for upcoming AWS events:
AWS Summits – Join AWS Summits in 2026, free in-person events where you can explore emerging cloud and AI technologies, learn best practices, and network with industry peers and experts. Upcoming Summits include Paris (April 1), London (April 22), and Bengaluru (April 23–24).
Amazon Nova AI Hackathon – Join developers worldwide to build innovative generative AI solutions using frontier foundation models and compete for $40,000 in prizes across five categories including agentic AI, multimodal understanding, UI automation, and voice experiences during this six-week challenge from February 2nd to March 16th, 2026.
AWS Community Days – Community-led conferences where content is planned, sourced, and delivered by community leaders, featuring technical discussions, workshops, and hands-on labs. Upcoming events include Ahmedabad (February 28), JAWS Days in Tokyo (March 7), Chennai (March 7), Slovakia (March 11), and Pune (March 21).
Customers migrating from on-premises Oracle databases to AWS face a challenge: efficiently relocating large object data types (LOBs) to object storage while maintaining data integrity and performance. This challenge originates from the traditional enterprise database design where LOBs are stored alongside structured data, leading to storage capacity constraints, backup complexity, and performance bottlenecks during data retrieval and processing. LOBs, which can include images, videos, and other large files, often cause traditional data migrations to suffer from slow speeds and LOB truncation issues. These issues are particularly problematic for long-running migrations that can span several years.
In this post, we present a scalable solution that uses Amazon Managed Streaming for Apache Kafka (Amazon MSK), Amazon Aurora PostgreSQL-Compatible Edition, and Amazon MSK Connect. The data streaming enables data replication where modifications are sent and received in a continuous flow, allowing the target database to access and apply the changes in real time. This solution generates events for database actions such as insert, update, and delete, triggering AWS Lambda functions to download LOBs from the source Oracle database and upload them to Amazon Simple Storage Service (Amazon S3) buckets. Simultaneously, the streaming events migrate the structured data from the Oracle database to the target database while maintaining proper linking with their respective LOBs.
The complete implementation is available on GitHub, including AWS Cloud Development Kit (AWS CDK) deployment code, configuration files, and setup instructions.
Solution overview
Although traditional Oracle database migrations handle structured data effectively, they struggle with LOBs that can include images, videos, and documents. These migrations often fail due to size limitations and truncation issues, creating significant business risks, including data loss, extended downtime, and project delays that can force you to delay your cloud transformation initiatives. The problem becomes more acute during long-running migrations spanning several years, where maintaining operational continuity is critical. This solution addresses the key challenges of LOB migration, enabling continuous, long-term operations without compromising performance or reliability.
By removing the size limitations associated with traditional migration technologies, our solution provides a robust framework that helps you seamlessly relocate LOBs while facilitating data integrity throughout the process.
Our approach uses a modern streaming architecture to alleviate the traditional constraints of Oracle LOB migration. The solution includes the following core components:
Amazon MSK – Provides the streaming infrastructure.
Amazon MSK Connect – Using two connectors:
Debezium Connector for Oracle as a source connector to capture row-level changes that occur in Oracle database. The connector emits change events and publishes to a Kafka source topic.
Debezium Connector for JDBC as a sink connector to consume events from Kafka source topic and then write those events to Aurora PostgreSQL-Compatible by using a JDBC driver.
Lambda function – Triggered by an event source mapping to Amazon MSK. The function processes events from the Kafka source topic, extracting the Oracle row primary key from each event payload. It uses this key to download the corresponding BLOB data from the source Oracle database and uploads it to Amazon S3, organizing files by primary key folders to maintain simple linking with the relational database records.
Aurora PostgreSQL-Compatible – Used as the target database for migrated data.
Amazon S3 – Used as object storage for storing the BLOB data from source database.
The following diagram shows the Oracle LOB data migration architecture solution.
Message flow
When data changes occur in the source Amazon RDS for Oracle database, the solution executes the following sequence, moving through event detection and publication, BLOB processing with Lambda, and structured data processing:
The Oracle source connector captures the change data capture (CDC) events, including the change to BLOB data column. This connector configures the BLOB data column to exclude from the Kafka event to optimize the Kafka payload.
The connector publishes this event to an MSK topic.
The MSK event triggers the BLOB Downloader Lambda function for the CDC events.
The Lambda function examines two key conditions: the Debezium event code (specifically checking for create (c) or update(u)) and the configured list of Oracle BLOB table names along with their column names. When a Kafka message matches both the configured table list and valid Debezium events, the Lambda function initiates the BLOB data download from the Oracle source using the primary key and table name; otherwise, the function bypasses the BLOB download process. This selective approach makes sure the Lambda function only executes SQL queries when processing Kafka messages for tables containing BLOB data, optimizing database interactions.
The Lambda function uploads the BLOB to Amazon S3, organizing by primary key folders with unique object names, which enables linking between structured database records and their corresponding BLOB data in Amazon S3.
The PostgreSQL sink connector receives the event from the MSK topic.
The connector applies these changes to the Aurora PostgreSQL database for the Oracle database changes except the BLOB data column. The BLOB data column is excluded by the Oracle source connector.
Key benefits
The solution offers the following key advantages:
Cost optimization and licensing – Our approach offers significant cost optimization benefits by reducing the overall size of your database and alleviating your need for expensive licenses associated with traditional databases and replication technologies. By decoupling LOB storage from the database and using Amazon S3, you can reduce your overall database footprint and reduce costs associated with traditional licensing and replication technologies. The streaming architecture also minimizes your infrastructure overhead during long-running migrations.
Avoids size constraints and migration failures – Traditional migration tools often impose size limitations on LOB transfers, leading to truncation issues and failed migrations. This solution removes those constraints entirely, so you can migrate LOBs of different sizes while maintaining data integrity. The event-driven architecture enables near real-time data replication, allowing your source systems to remain operational during migration.
Business continuity and operational excellence – Changes flow continuously to your target environment, allowing for business continuity. The solution preserves relationships between structured database records and their corresponding LOBs through primary key-based organization in Amazon S3, allowing for referential integrity while providing the flexibility of object storage for large files.
Architectural advantages – Storing LOBs in Amazon S3 while maintaining structured data in Aurora PostgreSQL-Compatible creates a clear separation. This architecture simplifies your backup and recovery operations, improves query performance on structured data, and provides flexible access patterns for binary objects through Amazon S3.
Implementation best practices
Consider the following best practices when implementing this solution:
Start small and scale gradually – To implement this solution, start with a pilot project using non-production data to validate your approach before committing to full-scale migration. This gives you a chance to work out issues in a controlled environment and refine your configuration without impacting production systems.
Monitoring – Set up comprehensive monitoring through Amazon CloudWatch to track key metrics like Kafka lag, Lambda function errors, and replication latency. Establish alerting thresholds early so you can catch and resolve issues quickly before they impact your migration timeline. Size your MSK cluster based on expected CDC volume and configure Lambda reserved concurrency to handle peak loads during initial data synchronization.
Security – For security, use encryption in transit and at rest for both structured data and LOBs, and follow the principle of least privilege when setting up AWS Identity and Access Management (IAM) roles and policies for your MSK cluster, Lambda functions, S3 buckets, and database instances. Document your schema mappings between Oracle and Aurora PostgreSQL-Compatible, including how database records link to their corresponding LOBs in Amazon S3.
Testing and preparation – Before you go live, test your failover and recovery procedures thoroughly. Validate scenarios like Lambda function failures, MSK cluster issues, and network connectivity problems to ensure you’re prepared for potential issues. Finally, remember that this streaming architecture maintains eventual consistency between your source and target systems, so there might be brief lag times during high-volume periods. Plan your cutover strategy with this in mind.
Limitations and considerations
Although this solution provides a robust approach for migrating Oracle databases with LOBs to AWS, there are several inherent constraints to understand before implementation.
This solution requires network connectivity between your source Oracle database and AWS environment. For on-premises Oracle databases, you must establish AWS Direct Connect or VPN connectivity before deployment. Network bandwidth directly impacts replication speed and overall migration performance, so your connection must be able to handle the expected volume of CDC events and LOB transfers.
The solution uses Debezium Connector for Oracle as the source connector and Debezium Connector for JDBC as the sink connector. This architecture is specifically designed for your Oracle-to-PostgreSQL migrations. Other database combinations require different connector configurations or might not be supported by the current implementation. Migration throughput is also constrained by your MSK cluster capacity and Lambda concurrency limits. You can also exceed AWS service quotas for large-scale migrations and you might need to request quota increases through AWS Enterprise Support.
Conclusion
In this post, we presented a solution that addresses the critical challenge of migrating your large binary objects from Oracle to AWS by using a streaming architecture that separates LOB storage from structured data. This approach avoids size constraints, reduces Oracle licensing costs, and preserves data integrity throughout extended migration periods.
Ready to transform your Oracle migration strategy? Visit the GitHub repository, where you will find the complete AWS CDK deployment code, configuration files, and step-by-step instructions to get started.
Earlier this year in May, we announced the general availability of AWS Transform for .NET, the first agentic AI service for modernizing .NET applications at scale. During the early adoption period of the service, we received valuable feedback indicating that, in addition to .NET application modernization, you would like to modernize SQL Server and legacy UI frameworks. Your applications typically follow a three-tier architecture—presentation tier, application tier, and database tier—and you need a comprehensive solution that can transform all of these tiers in a coordinated way.
Today, based on your feedback, we’re excited to announce AWS Transform for full-stack Windows modernization, to offload complex, tedious modernization work across the Windows application stack. You can now identify application and database dependencies and modernize them in an orchestrated way through a centralized experience.
AWS Transform accelerates full-stack Windows modernization by up to five times across application, UI, database, and deployment layers. Along with porting .NET Framework applications to cross-platform .NET, it migrates SQL Server databases to Amazon Aurora PostgreSQL-Compatible Edition with intelligent stored procedure conversion and dependent application code refactoring. For validation and testing, AWS Transform deploys applications to Amazon Elastic Compute Cloud (Amazon EC2) Linux or Amazon Elastic Container Service (Amazon ECS), and provides customizable AWS CloudFormation templates and deployment configurations for production use. AWS Transform has also added capabilities to modernize ASP.NET Web Forms UI to Blazor.
There is much to explore, so in this post I’ll provide the first look at AWS Transform for full-stack Windows modernization capabilities across all layers.
Create a full-stack Windows modernization transformation job AWS Transform connects to your source code repositories and database servers, analyzes application and database dependencies, creates modernization waves, and orchestrates full-stack transformations for each wave.
To get started with AWS Transform, I first complete the onboarding steps outlined in the getting started with AWS Transform user guide. After onboarding, I sign in to the AWS Transform console using my credentials and create a job for full-stack Windows modernization.
After creating the job, I complete the prerequisites. Then, I configure the database connector for AWS Transform to securely access SQL Server databases running on Amazon EC2 and Amazon Relational Database Service (Amazon RDS). The connector can connect to multiple databases within the same SQL Server instance.
Furthermore, I have the option to choose if I would like AWS Transform to deploy the transformed applications. I choose Yes and provide the target AWS account ID and AWS Region for deploying the applications. The deployment option can be configured later as well.
After the connectors are set up, AWS Transform connects to the resources and runs the validation to verify IAM roles, network settings, and related AWS resources.
After the successful validation, AWS Transform discovers databases and their associated source code repositories. It identifies dependencies between databases and applications to create waves for transforming related components together. Based on this analysis, AWS Transform creates a wave-based transformation plan.
Assessing database and dependent applications For the assessment, I review the databases and source code repositories discovered by AWS Transform and choose the appropriate branches for code repositories. AWS Transform scans these databases and source code repositories, then presents a list of databases along with their dependent .NET applications and transformation complexity.
I choose the target databases and repositories for modernization. AWS Transform analyzes these selections and generates a comprehensive SQL Modernization Assessment Report with a detailed wave plan. I download the report to review the proposed modernization plan. The report includes an executive summary, wave plan, dependencies between databases and code repositories, and complexity analysis.
Wave transformation at scale The wave plan generated by AWS Transform consists of four steps for each wave. First, it converts the SQL Server schema to PostgreSQL. Second, it migrates the data. Third, it transforms the dependent .NET application code to make it PostgreSQL compatible. Finally, it deploys the application for testing.
Before converting the SQL Server schema, I can either create a new PostgreSQL database or choose an existing one as the target database.
After I choose the source and target databases, AWS Transform generates conversion reports for my review. AWS Transform converts the SQL Server schema to PostgreSQL-compatible structures, including tables, indexes, constraints, and stored procedures.
For any schema that AWS Transform can’t automatically convert, I can manually address them in the AWS Database Migration Service (AWS DMS) console. Alternatively, I can fix them in my preferred SQL editor and update the target database instance.
After completing schema conversion, I have the option to proceed with data migration, which is an optional step. AWS Transform uses AWS DMS to migrate data from my SQL Server instance to the PostgreSQL database instance. I can choose to perform data migration later, after completing all transformations, or work with test data by loading it into my target database.
The next step is code transformation. I specify a target branch for AWS Transform to upload the transformed code artifacts. AWS Transform updates the codebase to make the application compatible with the converted PostgreSQL database.
With this release, AWS Transform for full-stack Windows modernization supports only codebases in .NET 6 or later. For codebases in .NET Framework 3.1+, I first use AWS Transform for .NET to port them to cross-platform .NET. I’ll expand on this in a following section.
After the conversion is completed, I can view the source and target branches along with their code transformation status. I can also download and review the transformation report.
Modernizing .NET Framework applications with UI layer One major feature we’re releasing today is the modernization of UI frameworks from ASP.NET Web Forms to Blazor. This is added to existing support for modernizing model-view-controller (MVC) Razor views to ASP.NET Core Razor views.
As mentioned previously, if I have a .NET application in legacy .NET Framework, then I continue using AWS Transform for .NET to port it to cross-platform .NET. For legacy applications with UIs built on ASP.NET Web Forms, AWS Transform now modernizes the UI layer to Blazor along with porting the backend code.
AWS Transform for .NET converts ASP.NET Web Forms projects to Blazor on ASP.NET Core, facilitating the migration of ASP.NET websites to Linux. The UI modernization feature is enabled by default in AWS Transform for .NET on both the AWS Transform web console and Visual Studio extension.
During the modernization process, AWS Transform handles the conversion of ASPX pages, ASCX custom controls, and code-behind files, implementing them as server-side Blazor components rather than web assembly. The following project and file changes are made during the transformation:
From
To
Description
*.aspx, *.ascx
*.razor
.aspx pages and .ascx custom controls become .razor files
Web.config
appsettings.json
Web.config settings become appsettings.json settings
Global.asax
Program.cs
Global .asax code becomes Program.cs code
*.master
*layout.razor
Master files become layout.razor files
Other new features in AWS Transform for .NET Along with UI porting, AWS Transform for .NET has added support for more transformation capabilities and enhanced developer experience. These new features include the following:
Port to .NET 10 and .NET Standard – AWS Transform now supports porting to .NET 10, the latest Long-Term Support (LTS) release, which was released on November 11, 2025. It also supports porting class libraries to .NET Standard, a formal specification for a set of APIs that are common across all .NET implementations. Furthermore, AWS Transform is now available with AWS Toolkit for Visual Studio 2026.
Editable transformation report – After the assessment is complete, you can now view and customize the transformation plan based on your specific requirements and preferences. For example, you can update package replacement details.
Real-time transformation updates with estimated remaining time – Depending on the size and complexity of the codebase, AWS Transform can take some time to complete the porting. You can now track transformation updates in real-time along with the estimated remaining time.
Next steps markdown – After the transformation is complete, AWS Transform now generates a next steps markdown file with the remaining tasks to complete the porting. You can use this as a revised plan to repeat the transformation with AWS Transform or use AI code-companions to complete the porting.
Things to know Some more things to know are:
AWS Regions – AWS Transform for full-stack Windows modernization is generally available today in the US East (N. Virginia) Region. For Regional availability and future roadmap, visit the AWS Capabilities by Region.
Pricing – Currently, there is no added charge for Windows modernization features of AWS Transform. Any resources you create or continue to use in your AWS account using the output of AWS Transform are billed according to their standard pricing. For limits and quotas, refer to the AWS Transform User Guide.
SQL Server versions supported – AWS Transform supports the transformation of SQL Server versions from 2008 R2 through 2022, including all editions (Express, Standard, and Enterprise). SQL Server must be hosted on Amazon RDS or Amazon EC2 in the same Region as AWS Transform.
Entity Framework versions supported – AWS Transform supports the modernization of Entity Framework versions 6.3 through 6.5 and Entity Framework Core 1.0 through 8.0.
Getting started – To get started, visit AWS Transform for full-stack Windows modernization User Guide.
Next week, don’t miss AWS re:Invent, Dec. 1-5, 2025, for the latest AWS news, expert insights, and global cloud community connections! Our News Blog team is finalizing posts to introduce the most exciting launches from our service teams. If you’re joining us in person in Las Vegas, review the agenda, session catalog, and attendee guides before arriving. Can’t attend in person? Watch our Keynotes and Innovation Talks via livestream.
AWS CloudFormation StackSets offers deployment ordering for auto-deployment mode. You can define the sequence in which your stack instances automatically deploy across accounts and Regions.
AWS NAT Gateway supports Regional availability to create a single NAT Gateway that automatically expands and contracts across availability zones (AZs).
As I was preparing for this week’s roundup, I couldn’t help but reflect on how database technology has evolved over the past decade. It’s fascinating to see how architectural decisions made years ago continue to shape the way we build modern applications. This week brings a special milestone that perfectly captures this evolution in cloud database innovation as Amazon Aurora celebrated 10 years of database innovation.
Amazon Web Services (AWS) Vice President Swami Sivasubramanian reflected on LinkedIn about his journey with Amazon Aurora, calling it “one of the most interesting products” he’s worked on. When Aurora launched in 2015, it shifted the database landscape by separating compute and storage. Now trusted by hundreds of thousands of customers across industries, Aurora has grown from a MySQL-compatible database to a comprehensive platform featuring innovations such as Aurora DSQL, serverless capabilities, I/O-Optimized pricing, zero-ETL integrations, and generative AI support. Last week’s celebration on August 21 highlighted this decade-long transformation that continues to simplify database scaling for customers.
Last week’s launches
In addition to the inspiring celebrations, here are some AWS launches that caught my attention:
AWS Billing and Cost Management introduces customizable Dashboards — This new feature consolidates cost data into visual dashboards with multiple widget types and visualization options, combining information from Cost Explorer, Savings Plans, and Reserved Instance reports to help organizations track spending patterns and share standardized cost reporting across accounts.
Amazon Bedrock simplifies access to OpenAI open weight models — AWS has streamlined access to OpenAI’s open weight models (gpt-oss-120b and gpt-oss-20b), making them automatically available to all users without manual activation while maintaining administrator control through IAM policies and service control policies.
Amazon Bedrock adds batch inference support for Claude Sonnet 4 and GPT-OSS models —This feature provides asynchronous processing of multiple inference requests with 50 percent lower pricing compared to on-demand inference, optimizing high-volume AI tasks such as document analysis, content generation, and data extraction with Amazon CloudWatch metrics for tracking batch workload progress
AWS launching Amazon EC2 R8i and R8i-flex memory-optimized instances — Powered by custom Intel Xeon 6 processors, these new instances deliver up to 20 percent better performance and 2.5 times higher memory throughput than R7i instances, making them ideal for memory-intensive workloads like databases and big data analytics, with R8i-flex offering additional cost savings for applications that don’t fully utilize compute resources.
Amazon S3 introduces batch data verification feature — A new capability in S3 Batch Operations that offers efficient verification of billions of objects using multiple checksum algorithms without downloading or restoring data, generating detailed integrity reports for compliance and audit purposes regardless of storage class or object size.
Other AWS news
Here are some additional projects and blog posts that you might find interesting:
Amazon introduces DeepFleet foundation models for multirobot coordination — Trained on millions of hours of data from Amazon fulfillment and sortation centers, these pioneering models predict future traffic patterns for robot fleets, representing the first foundation models specifically designed for coordinating multiple robots in complex environments.
Building Strands Agents with a few lines of code — A new blog demonstrates how to build multi-agent AI systems with a few lines of code, enabling specialized agents to collaborate seamlessly, handle complex workflows, and share information through standardized protocols for creating distributed AI systems beyond individual agent capabilities.
AWS Security Incident Response introduces ITSM integrations — New integrations with Jira and ServiceNow provide bidirectional synchronization of security incidents, comments, and attachments, streamlining response while maintaining existing processes, with open source code available on GitHub for customization and extension to additional IT service management (ITSM) platforms.
Finding root-causes using a network digital twin graph and agentic AI — A detailed blog post shows how AWS collaborated with NTT DOCOMO to build a network digital twin using graph databases and autonomous AI agents, helping telecom operators to move beyond correlation to identify true root causes of complex network issues, predict future problems, and improve overall service reliability.
Upcoming AWS events Check your calendars and sign up for these upcoming AWS events:
AWS Summits — Join free online and in-person events that bring the cloud computing community together to connect, collaborate, and learn about AWS. Register in your nearest city: Toronto (September 4), Los Angeles (September 17), and Bogotá (October 9).
AWS re:Invent 2025 — This flagship annual conference is coming to Las Vegas from December 1–5. The event catalog is now available. Mark your calendars for this not to be missed gathering of the AWS community.
AWS Community Days — Join community-led conferences that feature technical discussions, workshops, and hands-on labs led by expert AWS users and industry leaders from around the world: Adria (September 5), Baltic (September 10), Aotearoa (September 18), South Africa (September 20), Bolivia (September 20), Portugal (September 27).
Let me start this week’s update with something I’m especially excited about – the upcoming BeSA (Become a Solutions Architect) cohort. BeSA is a free mentoring program that I host along with a few other AWS employees on a volunteer basis to help people excel in their cloud careers. Last week, the instructors’ lineup was finalized for the 6-week cohort starting September 6. The cohort will focus on migration and modernization on AWS. Visit the BeSA website to learn more.
Another highlight for me last week was the announcement of six new AWS Heroes for their technical leadership and exceptional contributions to the AWS community. Read the full announcement to learn more about these community leaders.
Last week’s launches Here are some launches from last week that got my attention:
Amazon EC2 Single GPU P5 instances are now generally available — You can right-size your machine learning (ML) and high performance computing (HPC) resources cost-effectively with the new Amazon Elastic Compute Cloud (Amazon EC2) P5 instance size with one NVIDIA H100 GPU.
AWS Advanced Go Driver is generally available — You can now use the AWS Advanced Go Driver with Amazon Relational Database Service (Amazon RDS) and Amazon Aurora PostgreSQL-Compatible and MySQL-Compatible database clusters for faster switchover and failover times, Federated Authentication, and authentication with AWS Secrets Manager or AWS Identity and Access Management (IAM). You can install the PostgreSQL and MySQL packages for Windows, Mac, or Linux, by following the installation guides in GitHub.
Expanded support for Cilium with Amazon EKS Hybrid Nodes — Cilium is a Cloud Native Computing Foundation (CNCF) graduated project that provides core networking capabilities for Kubernetes workloads. Now, you can receive support from AWS for a broader set of Cilium features when using Cilium with Amazon EKS Hybrid Nodes including application ingress, in-cluster load balancing, Kubernetes network policies, and kube-proxy replacement mode.
Amazon SageMaker AI now supports P6e-GB200 UltraServers — You can accelerate training and deployment of foundational models (FMs) at trillion-parameter scale by using up to 72 NVIDIA Blackwell GPUs under one NVLink domain with the new P6e-GB200 UltraServer support in Amazon SageMaker HyperPod and Model Training.
Amazon SageMaker HyperPod now supports fine-grained quota allocation of compute resources, topology-aware-scheduling of LLM tasks and custom Amazon Machine Images (AMIs) — You can allocate fine-grained compute quota for GPU, Trainium accelerator, vCPU, and vCPU memory within an instance to optimize compute resource distribution. With topology-aware scheduling, you can schedule your large language model (LLM) tasks on an optimal network topology to minimize network communication and enhance training efficiency. Using custom AMIs, you can deploy clusters with pre-configured, security-hardened environments that meet your specific organizational requirements.
Additional updates Here are some additional news items and blog posts that I found interesting:
Introducing AWS Cloud Control API (CCAPI) MCP Server — You can now use natural language to managing cloud infrastructure using CCAPI MCP Server. You can create, read, update, delete, and list resources using natural language.
Introducing Amazon Bedrock AgentCore Identity — AgentCore Identity provides a centralized capability to manage agent identities, securing credentials, and supporting seamless integration with AWS and third-party services through Sigv4, standardized OAuth 2.0 flows, and API key.
Introducing Amazon Bedrock AgentCore Gateway — A fully managed service to connect AI agents with tools and services. It serves as a centralized tool server, providing a unified interface where agents can discover, access, and invoke tools.
Upcoming AWS events Check your calendars and sign up for upcoming AWS and AWS Community events:
AWS re:Invent 2025 (December 1-5, 2025, Las Vegas) — The AWS flagship annual conference offering collaborative innovation through peer-to-peer learning, expert-led discussions, and invaluable networking opportunities.
AWS Summits — Join free online and in-person events that bring the cloud computing community together to connect, collaborate, and learn about AWS. Coming up soon are summits in Johannesburg (August 20) and Toronto (September 4).
AWS Community Days — Join community-led conferences that feature technical discussions, workshops, and hands-on labs led by expert AWS users and industry leaders from around the world: Adria (September 5), Baltic (September 10), Aotearoa (September 18), and South Africa (September 20).
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
