Post Syndicated from Ryan Smith original https://www.servethehome.com/nvidia-gtc-berlin-2026-nvidia-intros-dsx-pods-and-more/
NVIDIA kicked off GTC 2026 with its lead keynote. Among the highlights for this year’s European NVIDIA tradeshow are the DSX POD, a server-in-a-container, as well as a slew of software updates for the NVIDIA ecosystem
The post NVIDIA GTC Berlin 2026 NVIDIA Intros DSX PODs and More appeared first on ServeTheHome.
Post Syndicated from Vic A original https://www.servethehome.com/intel-moves-forward-at-ohio-one-with-first-major-structural-truss-installation/
Intel Ohio One is making progress with truss installation as part of its construction and future wafer capacity
The post Intel Moves Forward at Ohio One With First Major Structural Truss Installation Arizona Fab 62 Tooling appeared first on ServeTheHome.
Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/introducing-amazon-ecs-gateway-a-fully-managed-ingress-solution-for-ecs-services/
When platform teams manage containerized applications at scale on Amazon Elastic Container Service (Amazon ECS), configuring ingress networking for each service often becomes a bottleneck. For every new service, teams must provision and connect Application Load Balancers (ALBs), AWS Certificate Manager (ACM) certificates, AWS WAF web ACLs, and security groups, then keep those resources in sync as services scale, update, or operate. This per-service overhead slows down deployment velocity and increases the risk of configuration drift across an organization.
Today, we’re announcing Amazon ECS Gateway, a new account-level resource that simplifies ingress networking for containerized workloads by dynamically provisioning and managing the necessary infrastructure on your behalf, with all resources created directly in your account.
ECS Gateway introduces a clear separation of responsibilities between platform teams and application teams.
ECS Gateway offers host-based routing using ALBs and enables up to 25 services to share a single ALB, optimizing infrastructure costs. ECS manages the full lifecycle of the underlying infrastructure, provisioning the first ALB when your first service connects, adding more ALBs as capacity is reached, and removing them when no longer needed. Each service receives a unique DNS hostname, giving your teams a ready-to-use endpoint the moment provisioning completes.
A first look at ECS Gateway
In the ECS console, platform team can create a gateway and sets its configuration. The infrastructure role is an IAM role that ECS assumes to provision and manage networking resources such as ALBs, security groups, and certificates in your account. To create a gateway, provide a gateway name and select an infrastructure role. You can optionally associate the gateway with ECS clusters during creation or later.
Choose a load balancing scheme, either public or VPC-internal traffic routing. Under Additional Settings, you can also set up a listener, port, maximum count of load balancers the gateway can scale to, and a web ACL lets you protect your web applications. Some settings, including the name, scheme, VPC, subnets, IP address type, listener protocol and port cannot be changed after gateway creation.
Using AWS Command Line Interface (AWS CLI), call CreateGateway with a name (unique per account), an infrastructure role, and a configuration object. No infrastructure is provisioned until a service connects with a gateway connector.
aws ecs create-gateway \
--name my-team-gateway \
--infrastructure-role-arn arn:aws:iam::123456789012:role/ECSGatewayRole \
--configuration '{
"HostSharedAlb": {
"networkConfiguration": {
"subnets": [
"subnet-080779f9bd887757a",
"subnet-03d5b2fead6bc6258",
"subnet-0861210a397d31252",
"subnet-06539e194023cb77a"
],
"ipAddressType": "ipv4"
},
"scheme": "internet-facing"
}
}'
Once the gateway is created, you can take a look at the detailed configuration, gateway routes, load balancers, and associated clusters.
Now associate the gateway with one or more clusters (many-to-many), to make it available to ECS services on those clusters.
Use AssociateGatewayWithCluster to make the gateway available to services on a cluster:
aws ecs associate-gateway-with-cluster \
--gateway my-team-gateway \
--cluster arn:aws:ecs:us-east-1:123456789012:cluster/my-clusterTo create and manage the ECS Gateway programmatically, including calling APIs and searching documentation, try the AWS MCP Server and plugins with your preferred AI coding tool. To learn more, visit the Amazon ECS Gateway documentation.
ECS Gateway:Behind the scenes
A gateway can be associated with up to 10,000 clusters, and a single cluster supports up to five gateway associations. This many-to-many relationship gives platform teams the flexibility to share a gateway across environments or assign multiple gateways (for example, public and private) to a single cluster.
Platform teams can control access by choosing which gateways to associate with which clusters. For additional control, use IAM policies to restrict who can deploy services to specific clusters.
Application teams add the gateway connector to each service definition in CreateService or UpdateService. If you have an existing ECS service with its own load balancer, you can adopt ECS Gateway incrementally by calling UpdateService to add a gateway connector. Both ingress paths remain available, allowing you to validate the gateway connection before removing the existing load balancer configuration.
aws ecs create-service \
--cluster my-cluster \
--service my-api-service \
--task-definition my-api-service:1 \
--gateway-connectors '[{ "portName": "api", "filters": {"accessTypes": ["PUBLIC"]}, "healthCheck": {...} }]'At deployment time, Amazon ECS evaluates the gateways associated with the cluster and connects the service to every compatible gateway. A gateway is compatible when its access type matches the service’s connector filter, it is in the same VPC as the service, and it covers all Availability Zones used by the service’s configured subnets.
For each connection, Amazon ECS assigns the service to an ALB with available capacity and provisions the per-service resources: an ACM certificate, a unique DNS hostname, a listener rule routing traffic for that hostname to the service target group, and a service security group allowing inbound traffic only from the ALB security group. If no existing ALB has capacity, ECS creates a new one (up to maxAlbCount referring to quota limit). Once provisioning completes, the gateway route becomes ACTIVE, and the service URL is ready to receive traffic.
ECS Gateway handles common operational scenarios automatically:
Things to Know
Here are a few important details to keep in mind:
awsvpc network mode across all launch types (Amazon ECS EC2, AWS Fargate, and ECS Managed Instances). Your service Availability Zones (AZs) must be a subset of each gateway’s AZs on the cluster. A gateway ALB only routes traffic to tasks in AZs where it has subnets. We recommend creating gateways and services for a given cluster with the same set of AZs.maxAlbCount is reached and no ALB has available capacity, the deployment fails. Default quota limit of maxAlbCount is 10. Plan your maxAlbCount setting according to the number of services you expect to connect.TargetGroup dimension to monitor per-service traffic, and LoadBalancer dimension to monitor traffic across the shared ALB. Use describe-target-health to check real-time target status. Configure CloudWatch alarms for metrics such as HealthyHostCount, TargetResponseTime, and HTTPCode_Target_5XX_Count, using gateway ARN-based tagging so monitoring scales automatically with your maxAlbCount.Now available
Amazon ECS Gateway is available now in all commercial AWS Regions where ECS is available at no additional charge. You pay only for the underlying infrastructure provisioned in your account when services connect to a gateway, including applicable ALB charges, CloudWatch logs and metrics, ACM certificates, and data transfer charges. To learn more, visit the Amazon ECS Pricing page.
To get started, create your first gateway today in the ECS console, and send feedback to AWS re:Post for ECS or through your usual AWS Support contacts.
— Channy
Post Syndicated from Vic A original https://www.servethehome.com/hpe-spaceborne-computer-4-will-bring-ai-and-hpc-to-the-moon-on-astrolab-flip/
The HPE Spaceborne Computer 4 will head to the moon on Astrolab’s Flip to test HPC benchmarks and AI on the lunar surface
The post HPE Spaceborne Computer-4 Will Bring AI and HPC to the Moon on Astrolab FLIP appeared first on ServeTheHome.
Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-aws-lambda-web-functions-amazon-s3-on-the-vercel-marketplace-and-more-october-19-2026/
Nothing last week added a new kind of compute. What it did instead was shorten the distance between the code you already have and that code being live. AWS Lambda learned to run a plain HTTP server with no adapter in front of it, and Amazon S3 became something you can provision from inside the tool you were already using, without opening the AWS Management Console. Different services, same direction.
The counterweight arrived on Thursday. As it gets easier to put things into production, it matters more that you can say what those things are allowed to do, and this week’s most interesting read is about exactly that. Now, let’s get into this week’s AWS news…
Headlines
AWS Lambda Web Functions is a new form factor that runs a standard HTTP web application on Lambda without rewriting it into an event handler. You take a port-listening server built with Express, Hono, Fastify, or Next.js, run aws lambda-web deploy, and get back a live HTTPS URL in under a minute. No adapters, no infrastructure to set up first, no framework lock-in. Lambda now has three form factors on the same Firecracker isolation, which compose inside one application: Event Functions (the model you already know) for async work, Web Functions for the user-facing layer, and MicroVMs for stateful sandboxes running untrusted code.
Two details make this more than repackaging. Each execution environment handles up to 128 concurrent requests, so a burst of traffic shares warm environments instead of mapping one request to one environment, and billing meters the CPU time your code actually uses rather than wall-clock time. For server-side rendering, API backends, and inference endpoints streaming tokens, where most of the elapsed time is spent waiting, that changes the arithmetic. Adding --regions to the same deploy command replicates your code and configures latency-based routing behind a single global URL, and you can split traffic across revisions by weight for a canary rollout. It’s available in five Regions, running Node.js 24 on Arm64 with VPC connectivity. The AWS Compute Blog has a longer walkthrough, and Amazon CloudFront can now use a Web Function endpoint as an origin with origin access control.
The week’s other headline isn’t a launch. Trust in Depth: giving agents more autonomy without losing control makes an argument I keep running into in practice: putting a person in the loop for every agent action works at small scale and then quietly stops working, because people asked to check everything stop catching what matters. The alternative is four questions answered at four moments, which AWS calls scope, know, bound, and observe. The part worth sitting with is that an agent’s limits belong outside its own reasoning, in Amazon Bedrock AgentCore policies enforced at the gateway, because an agent cannot turn off a control it does not run. AWS has run automated reasoning in production for more than a decade, including AWS Identity and Access Management at more than 2.1 billion requests per second, and the case here is for pointing it at the rules you write for agents.
Last week’s launches
Here are some launches and updates from this past week that caught my attention:
t8i.micro and t8i.small are available under the AWS Free Tier.go get to add it, deploy a single static binary on the provided.al2023 runtime, and test a full workflow locally with the durabletest package, no network access or credentials required.For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.
Other AWS news
Here are some additional posts and resources that you might find interesting:
Upcoming AWS events
Check your calendar and sign up for upcoming AWS events:
Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development. Browse here for upcoming AWS-led in-person and virtual events and developer-focused events.
That’s all for this week. Check back next Monday for another Weekly Roundup!
— Daniel Abib
This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!
Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-prime-day-2026-by-the-numbers-claude-haiku-5-5-amazon-nova-2-5-sonic-and-more-october-13-2026/
Some numbers are hard to picture until you sit with them for a moment. Last week we published the Prime Day 2026 metrics, and the one that stuck with me is 2.3 quadrillion: the daily requests Amazon ElastiCache peaked at during the event. Prime Day ran back in June, so this post always lands as a retrospective, and that gap is part of what makes it worth reading. By the time the numbers reach you, the only evidence that any of it happened is that nothing went wrong.
The rest of the week pulled in a different direction: smaller, cheaper, and more constrained. Claude Haiku 5.5 arrived on Amazon Bedrock at roughly 75 percent less than Haiku 4.5, Amazon Nova 2.5 Sonic made voice agents quicker on their feet, and Strands Box showed up to put real boundaries around what an agent is allowed to do. Now, let’s get into this week’s AWS news…
Headlines
All the numbers: Amazon Prime Day 2026 powered by AWS is the latest entry in a tradition that goes back to 2016, and the scale is still difficult to hold in your head. Over the four days of Prime Day, Amazon DynamoDB processed more than 59 trillion requests while peaking at 192 million requests per second, and kept single-digit millisecond responses throughout. AWS Lambda handled over 2.3 trillion invocations per day. Amazon EBS peaked above 24.8 trillion I/O operations, moving more than an exabyte of data daily. Amazon ECS launched an average of 158.3 million tasks per day on AWS Fargate, a 47.7 percent jump over last year, and AWS Graviton powered up to 49 percent of the Amazon EC2 compute that Amazon.com used.
The number I keep coming back to, though, isn’t a throughput record. The team ran more than 44,000 AWS Fault Injection Service experiments ahead of the event, more than six times what they ran in 2025. Those are deliberate failures injected on purpose, months in advance, so that the trillions of requests above turn into an ordinary shopping afternoon. If you’re preparing for your own peak moment, AWS Countdown Premium is built for exactly that kind of planning.
Last week’s launches
Here are some launches and updates from this past week that caught my attention:
For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.
Other AWS news
Here are some additional posts and resources that you might find interesting:
Upcoming AWS events
Check your calendar and sign up for upcoming AWS events:
Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development. Browse here for upcoming AWS-led in-person and virtual events and developer-focused events.
That’s all for this week. Check back next Monday for another Weekly Roundup!
– Daniel Abib
This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!
Post Syndicated from Micah Walter original https://aws.amazon.com/blogs/aws/announcing-aws-lambda-web-functions-for-fast-consistent-web-app-performance-anywhere/
Today, we’re announcing AWS Lambda Web Functions, a new form factor that lets you run standard HTTP web applications on Lambda without rewriting your code into an event handler. You can take a standard port-listening server built with Express, Hono, Fastify, Next.js, or any Node.js framework, deploy it with a single command, and Lambda automatically handles scaling, multi-region distribution, and active CPU billing. No adapters, no infrastructure to manage, and no framework lock-in.
With this launch, AWS Lambda now offers three distinct form factors, each built on the same Firecracker isolation technology and designed for a different programming model.
Lambda Event Functions are what you already know as Lambda functions. They use the event-driven, single-concurrent handler model with wall-clock billing. We’re introducing the “Event Functions” name to highlight that programming model: your function receives an event, processes it, and returns a response. The event-driven programming model is ideal for event processing, queue consumers, and invocations triggered by other AWS services.
Lambda Web Functions use an HTTP-native programming model with multi-concurrent execution and active CPU billing. Your code listens on a port and handles standard HTTP requests. That programming model is ideal for web applications, API backends, and interactive HTTP workloads where multiple concurrent requests and I/O-heavy processing are the norm.
Lambda MicroVMs provide isolated, stateful sandbox environments with customer-managed lifecycles for running untrusted or user-generated code.
The three complement each other. An application might use Event Functions for asynchronous backend processing, Web Functions for the user-facing API and server-side rendering layer, and MicroVMs for interactive code execution features, all within the same AWS account and sharing the same operational model.
Why Web Functions
Modern web applications need to serve global audiences with fast, consistent response times while giving developers the freedom to use standard frameworks and iterate quickly. Building this today typically requires assembling multiple infrastructure components, configuring multi-region deployments, setting up latency-based routing, and tuning billing models to avoid paying for I/O wait. Developers often find themselves adapting their application code to fit a specific compute model rather than deploying the HTTP server they already have.
Lambda Web Functions removes this complexity. Each execution environment handles multiple requests simultaneously (up to 128 concurrent requests per environment), which means a burst of incoming traffic can share warm environments instead of mapping each request to a new one. Web Functions meter the CPU time your code uses, so time spent waiting on a database query or an API call consumes little CPU. Memory is metered for as long as the execution environment exists. For I/O-heavy web workloads like server-side rendering, API backends, and AI inference endpoints streaming token responses, this can reduce compute costs.
Let’s try it out
To get started, I wrote a simple Express.js server. This is the same code I run locally with node index.js, no Lambda-specific adapters or wrappers required.
import express from 'express';
const app = express();
const [host, port] = process.env.AWS_LAMBDA_HTTP_ENDPOINT.split(':');
app.get('/', (req, res) => {
res.json({ message: 'Hello from Lambda Web Functions!' });
});
app.listen(port, host, () => {
console.log(`Server running on port ${port}`);
});
The only difference from running locally is the AWS_LAMBDA_HTTP_ENDPOINT environment variable, which Lambda sets to the host and port your server must listen on. To run locally, set it yourself, for example AWS_LAMBDA_HTTP_ENDPOINT=127.0.0.1:3000 node index.js.
With my code ready, I deployed using the AWS Command Line Interface (AWS CLI):
aws lambda-web deploy --name my-web-app --code ./my-app
One command. No prior resource setup. The CLI bundled my source, created the Web Function, published its first revision, attached a Web Function endpoint, and returned a live HTTPS URL in under a minute.
A revision is an immutable snapshot of the code I deployed. The endpoint is the HTTPS URL that routes traffic to a revision, or splits traffic across revisions. Those are the resources I use to iterate and to control how traffic moves to new code.
I tested it immediately:
$ curl https://<domain-id>.<shard-id>.lambda-web.us-east-1.on.aws/
{"message":"Hello from Lambda Web Functions!"}
Next, I wanted my application to serve users in Europe and Asia with lower latency. I redeployed with multiple regions:
aws lambda-web deploy --name my-web-app --code ./my-app \
--regions us-east-1,eu-west-1,ap-northeast-1
Lambda Web Functions replicated my code to all three regions and configured latency-based routing automatically. Each request now routes to the Region with the lowest latency for the caller, on a best-effort basis, through one global URL (<domain-id>.lambda-web.global.on.aws).
When I updated my code and ran the same deploy command again, the CLI detected the existing function, uploaded the new revision, and pointed the endpoint at it. Total turnaround from code change to live: seconds.
For production workflows that require a more gradual rollout, I can shift traffic between revisions with a single API call:
aws lambda-web update-web-function-endpoint \
--function-name my-web-app \
--endpoint-name dev \
--auto-deployment-mode Disabled \
--revision-weights '[{"revisionId": "<current-revision-id>", "weight": 90}, {"revisionId": "<new-revision-id>", "weight": 10}]'
This sends 90% of endpoint traffic to the current revision and 10% to the new one, giving me a canary deployment without additional tooling.
Better together with Amazon CloudFront
For applications that combine static and dynamic content, Web Functions works well alongside Amazon CloudFront. You can select a Web Function endpoint as the origin for your CloudFront distribution. Static assets are served from the cache while dynamic requests route to your Web Function, giving you CDN performance for static content with serverless compute for the dynamic parts.
Web Functions improves the private-origin experience with CloudFront. With this launch, you attach CloudFront origin access control (OAC) to a Web Function endpoint instead. CloudFront signs requests to the origin, and the endpoint allows only your distribution to invoke it. You get the same private-origin pattern CloudFront already provides for Amazon Simple Storage Service (Amazon S3), without exposing the endpoint or adding custom authorization to your app.
Now available
AWS Lambda Web Functions is available today in US East (N. Virginia), US East (Ohio), US West (Oregon), Asia Pacific (Tokyo), and Europe (Ireland). Web Functions run Node.js 24 (nodejs24.x) on Arm64 at launch and support Amazon Virtual Private Cloud (VPC) connectivity, allowing your function to access resources that exist within a VPC.
To get started, deploy your first Web Function in under a minute:
aws lambda-web deploy --name hello-world --hello-world
For pricing details, visit the AWS Lambda pricing page. To learn more, see the Lambda Web Functions documentation.
Post Syndicated from Ryan Smith original https://www.servethehome.com/intels-crescent-island-ai-server-gpu-gets-a-name-and-specs-meet-the-intel-gpu-d-series/
Intel today has unveiled its official performance specifications and branding for its upcoming “Crescent Island” data center AI inference cards. Going under the name Intel GPU D-Series, the D270 will offer 1.3 PLFOPs of dense FP8 and 1.5TB/second of memory bandwidth
The post Intel’s Crescent Island AI Server GPU Gets a Name and Specs: Meet the Intel GPU D-Series appeared first on ServeTheHome.
Post Syndicated from Ryan Smith original https://www.servethehome.com/microsoft-launches-surface-laptop-ultra-surface-rtx-spark-dev-box/
Microsoft today launched its first two NVIDIA RTX Spark-powered systems, the Surface Laptop Ultra and Surface RTX Spark Dev box. Aimed at developers and other professionals, the high-end systems offer a fresh face to NVIDIA and Microsoft’s Arm ecosystem efforts
The post Microsoft Launches Surface Laptop Ultra, Surface RTX Spark Dev Box appeared first on ServeTheHome.
Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/announcing-aws-well-architected-agent-an-ai-powered-intelligence-to-optimize-your-cloud-environment-preview/
Today, we’re announcing the public preview of AWS Well-Architected Agent, an AI-powered service that analyzes your AWS environment to deliver targeted, contextual recommendations for improving your applications’ cost, security, performance, and resilience. The AWS Well-Architected Agent analyzes your infrastructure, understands unique business goals, and delivers contextual recommendations with ready-to-implement fixes. It delivers context-aware optimization without relying on manual audits or generic checklists.
The agent evaluates your environment as an experienced cloud architect would. It automatically correlates utilization metrics, resource configurations, and application topology, and analyzes against Well-Architected best practices across 65+ AWS services. It generates recommendations aligned to your declared business goals, delivers implementation packages with every finding, and surfaces cross-pillar trade-offs making it simpler to remediate the findings.
Here are the three main features of this service:
AWS Well-Architected Agent in action
To get started, create an agent profile to define the scope of what Well-Architected Agent can access and provide recommendations on, complete the IAM role setup to access resources, conduct architecture review, and remediate recommendations.
Create an agent profile
In the AWS Well-Architected console, choose Get started with Well-Architected Agent. You can define an agent profile that specifies which AWS accounts and applications to monitor, which optimization pillars to focus on, and the permissions required.
You can choose AWS accounts or AWS Regions to monitor and optimization pillars that matter most to your business. You can also set goals for each pillar: cost optimization, performance, resilience, and security.
To give access to the agent for your AWS environment, provision customer-managed IAM roles the agent uses to read resource configurations, utilization metrics, and application topology. To learn more, visit the IAM prerequisite for AWS Well-Architected Agent.
When you choose Get Started, the agent creates your agent profile. Resource and application recommendations will be generated within 24 hours after profile creation.
You can conduct an architecture review on pre-deployment workloads by uploading an IaC project in Terraform, AWS CloudFormation, or AWS Cloud Development Kit (CDK) to be analyzed. Choose Conduct architecture review in the dashboard, upload a.zip file containing IaC project or repository file, and select which Well-Architected lens to use for reviewing your infrastructure.
You can define your applications to add context which will enhance the relevancy and further contextualize recommendations. Choose Add application context in the dashboard, add your applications with AWS accounts, AWS Regions, AWS services, tags if you want to narrow the scope to specific resources, and the details of applications.
Review prioritized recommendations and start remediating
Now you can see generated prioritized recommendations generated by the agent across your resources and applications, selected pillars, ranked against your declared goals, with automation-ready remediation included.
When you choose the specific recommendation, you can see the details, insights into why the agent are suggesting the recommendation, impacts and trade-off, and recommended fixes across affected AWS resources.
Choose Start remediation to address recommended fixes. You can choose the console, updated IaC template, CLI commands to remediate by the resolution type. It provides detailed step-by-step instructions and you can roll out this instruction and verify the result.
When you choose Using updated IaC template, the agent provides the code changes needed to update your existing IaC templates such as the CDK function shown above which you can copy directly into your codebase.
You can also configure API access to integrate recommendations directly into your existing development and operations workflows. To interact with the agent programmatically, including calling APIs and searching documentation, try the AWS MCP Server and plugins with your preferred AI coding tool. To learn more, visit the AWS Well-Architected Agent documentation.
Things to know
Here are some things that you should know about the Well-Architected Agent.
You can still use existing AWS Well-Architected Tool to manually evaluate your cloud architecture with user-defined lenses that measure your workload using your own best practices.
Join the preview
Access to the AWS Well-Architected Agent and its recommendations is available in US East (N. Virginia), US East (Ohio), and US West (Oregon). You can onboard workloads from any AWS commercial Region. AWS Well-Architected Agent is delivered by AWS Support and available to AWS customers with an AWS Support plan.
Give it a try today in the AWS Well-Architected console and send feedback through your usual AWS Support contacts.
— Channy
Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/amazon-s3-tables-now-support-all-apache-iceberg-v3-data-types/
Amazon S3 Tables now support all data types in the Apache Iceberg V3 specification. You can create V3 tables or upgrade existing V2 tables to take advantage of V3 features like deletion vectors, row lineage, and new data types such as variant, nanosecond timestamps, unknown, geometry, and geography.
Apache Iceberg has become the open standard for managing large analytics datasets. It lets you manage petabyte-scale tables with features like schema evolution, hidden partitioning, and time travel queries, while keeping your data in open Parquet files in data lakes on object storage like Amazon S3. Amazon S3 Tables offer storage purpose-built to keep Iceberg tables performant and cost-effective as they grow, with fully managed features like automatic compaction, maintenance, replication, and Intelligent-Tiering.
Teams running analytics on Apache Iceberg V2 tables often hit the same limits as their data grows. A compliance request to delete 50,000 user records from a 2-billion-row table leaves behind positional delete files that slow queries until compaction runs. Semi-structured events land as JSON strings that every query has to parse. Geospatial coordinates and nanosecond-precision timestamps get encoded as strings or integers. Each workaround adds storage cost, query latency, and pipeline code. With V3, Iceberg solves these challenges by offering native support for semi-structured and geospatial data, faster row-level operations, and built-in row lineage for data governance.
Starting today, Amazon S3 Tables support all V3 data types, including variant, nanosecond timestamps, geometry, geography, and unknown, along with deletion vectors and row lineage. You can create new V3 tables or upgrade existing V2 tables in place, and S3 Tables continue to run compaction and maintenance for you.
Apache Iceberg V3
V3 is the latest version of the Iceberg specification. Among its many improvements, V3 introduces capabilities that address the most common pain points in V2. This includes:
Deletion vectors replace V2’s positional delete files with a compact binary format. That 50,000-row compliance delete now writes a single deletion vector file instead of thousands of small deletes, significantly reducing compaction time and delete file overhead.
Row lineage adds _row_id and _last_updated_sequence_number to each record automatically. Your downstream pipelines can query these fields to find changed rows without scanning the full table.
New data types let you store semi-structured, geospatial, and nanosecond-precision data natively instead of encoding it as strings or integers:
Variant data type stores semi-structured data in columnar format. During writes, the engine shreds variant data into hidden columns and collects statistics. At query time, those statistics enable file pruning that significantly reduces I/O compared to parsing JSON strings.
The following sections walk through how to use these V3 capabilities in practice, with examples that show how to create tables, work with the new data types, and manage data at scale.
Getting started
A retail analytics team tracks user behavior across web and mobile apps. Each event has a different structure: page views include URLs and duration, purchases include items and amounts, and searches include query terms and result counts. With V3’s variant type, you store all event shapes in one table without predefined schemas:
CREATE TABLE my_catalog.namespace.clickstream (
event_id bigint,
event_time timestamp,
user_id string,
payload variant
)
USING iceberg
TBLPROPERTIES ('format-version' = '3')
Insert events with different payload shapes without worrying about schema evolution:
INSERT INTO my_catalog.namespace.clickstream VALUES
(1, current_timestamp(), 'user-42',
PARSE_JSON('{"action": "purchase", "amount": 99.99, "items": ["laptop_stand"]}')),
(2, current_timestamp(), 'user-17',
PARSE_JSON('{"action": "page_view", "url": "/products/webcam", "duration_ms": 4200}'));
Now query the variant column directly, without PARSE_JSON at read time. With Amazon EMR Spark, use variant_get:
SELECT
event_id,
user_id,
variant_get(payload, '$.action', 'string') AS action,
variant_get(payload, '$.amount', 'double') AS amount
FROM my_catalog.namespace.clickstream
WHERE variant_get(payload, '$.action', 'string') = 'purchase'
AND variant_get(payload, '$.amount', 'double') > 50.00
To enable deletion vectors for write operations, configure merge-on-read mode:
ALTER TABLE my_catalog.namespace.clickstream
SET TBLPROPERTIES (
'write.delete.mode' = 'merge-on-read',
'write.update.mode' = 'merge-on-read',
'write.merge.mode' = 'merge-on-read'
)
Now when you run a compliance delete, V3 writes a small deletion vector instead of rewriting data files:
DELETE FROM my_catalog.namespace.clickstream
WHERE user_id = 'user-42'
S3 Tables compaction handles these deletion vector files automatically on the next maintenance cycle.
Upgrading from V2
AWS provides backwards compatibility for both versions to minimize disruption during migration to V3. Existing V2 readers continue to work on upgraded tables until you’re ready to fully adopt V3 features. For more details, see the S3 Tables Iceberg V3 documentation.
Upgrade an existing table atomically without rewriting data:
ALTER TABLE my_catalog.namespace.existing_table
SET TBLPROPERTIES ('format-version' = '3')
On the next compaction cycle, S3 Tables remove old V2 delete files. New modifications use deletion vectors automatically. Row lineage fields initialize on the first data modification after the upgrade.
This is a one-way operation. The Apache Iceberg specification does not support downgrading from V3 to V2. Verify that all engines accessing the table support V3 before upgrading.
Using row lineage for incremental pipelines
After your table has V3 data, use row lineage to build efficient incremental pipelines:
SELECT *, _row_id, _last_updated_sequence_number
FROM my_catalog.namespace.clickstream
WHERE _last_updated_sequence_number > 42
This returns only rows modified after sequence number 42. Your downstream jobs can checkpoint this value and process only new changes on each run, instead of scanning the full table.
Compatibility across AWS analytics services
AWS offers the broadest native Apache Iceberg support of any major cloud provider, with Iceberg-compatible services at every layer of the data stack: ingestion, storage, catalog, and analytics. You can store and automatically optimize V3 tables in Amazon S3 Tables, write data with Amazon EMR Spark, integrate and manage data with AWS Glue, and run analytics with Amazon Redshift. To learn more about AWS analytics support for V3, see the Apache Iceberg on AWS prescriptive guidance.
Both S3 Tables and AWS Glue Data Catalog support the Iceberg REST Catalog (IRC) API, enabling interoperability across engines regardless of the catalog endpoint.
Things to know
Now available
Amazon S3 Tables support for all Apache Iceberg V3 data types is now available in all AWS Regions where S3 Tables are supported. Apache Iceberg V3 support is available at no additional charge; standard S3 Tables pricing applies.
To get started, visit the Amazon S3 Tables documentation or create a table bucket from the Amazon S3 console. If you want to call APIs, search documentation, find regional availability, and check troubleshooting about this feature, try using the AWS MCP Server and plugins with your preferred AI tool. Send feedback to AWS re:Post or through your usual AWS Support contacts.
– Daniel Abib
Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/amazon-s3-vectors-now-supports-metadata-pre-filtering-for-higher-recall-on-filtered-searches/
Today, we’re announcing metadata pre-filtering for Amazon S3 Vectors, which delivers higher recall on filtered queries by evaluating your metadata filter before the similarity search. You can filter on attributes such as tenant, category, status, or time, and pre-filtering adds prefix matching with $startsWith for paths, URLs, and hierarchical keys. Each vector carries up to 2 KB of filterable metadata, and a single query supports up to 100 filter constraints. There is no additional cost, no re-ingestion, and no change to your queries.
Most applications never search a whole index. They search the part of it that belongs to a particular user, account, or category, and they express that scope as a metadata filter. Semantic search, retrieval-augmented generation (RAG), and agentic applications all need the same thing from a filtered query: a similarity search that covers the vectors matching the filter, and returns the closest of them. With pre-filtering, a filtered query returns more of the relevant matches your index contains, giving you higher recall on filtered searches.
Common use cases
Pre-filtering applies wherever results have to be both relevant and correctly scoped:
$startsWith narrows further by matter number, folder path, or document ID prefix. A single client is a small share of a firm-wide archive, and filters this narrow are where pre-filtering improves recall most.How pre-filtering works
Each vector in an S3 Vectors index can carry application-defined metadata, and a query can filter on those fields.
Every vector index has an index mode. On an index whose index mode is ENHANCED, S3 Vectors resolves your filter first, then searches only the vectors that match. On an index whose index mode is CLASSIC, S3 Vectors performs the vector search and filter evaluation in tandem, validating each candidate vector against your filter as it searches. Existing indexes use CLASSIC until you update them.
Consider a support knowledge base of 8 million tickets, where an agent searches one customer’s history for a recurring error. If that customer accounts for 400 of those tickets, resolving customer_id first means the similarity search runs across all 400 of them, so the agent sees that customer’s prior occurrences. Before the index was updated, the same query drew its candidates from the full 8 million, and the result set contained fewer of that customer’s matching tickets.
On highly selective filters, pre-filtering returns up to 5x more of the matching vectors than the same query returned before on CLASSIC indexes.
Getting started
Before you start, make sure your IAM policy grants permissions for the new actions.
You can get started in three steps. The walkthrough below builds a small product-catalog index and runs a selective filter against it, the same pattern you would use for a multi-tenant RAG store or a document search scoped to one client.
First, create a vector index:
aws s3vectors create-index \
--index-name product-catalog \
--vector-bucket-name my-vector-bucket \
--dimension 1536 \
--distance-metric cosine
The dimension must match the output size of your embedding model, and distance-metric should match how that model was trained (cosine is common for text embeddings). Second, write vectors with the PutVectors API, attaching up to 2 KB of filterable metadata to each vector:
aws s3vectors put-vectors \
--index-name product-catalog \
--vector-bucket-name my-vector-bucket \
--vectors '[{
"key": "doc-001",
"data": {"float32": [0.1, 0.2, 0.3, ...]},
"metadata": {
"tenant_id": "t-10428",
"category": "legal",
"created_date": "2026-03-15",
"active": true
}
}]'
Each vector carries the attributes your application filters on. In this example, tenant_id scopes results to a single customer, category narrows by document type, created_date records when the document was created, and active is a boolean flag. By default every metadata field is filterable, so you can query on any of them without declaring a schema up front.
Third, run a filtered similarity query with the QueryVectors API. The filter uses a compact JSON syntax where a bare key-value pair is an equality match, and operators such as $and, $or, and $gt combine or refine conditions. Pass --return-metadata so the query returns each vector’s metadata:
aws s3vectors query-vectors \
--index-name product-catalog \
--vector-bucket-name my-vector-bucket \
--query-vector '{"float32": [0.1, 0.2, 0.3, ...]}' \
--top-k 50 \
--return-metadata \
--filter '{"$and": [
{"tenant_id": "t-10428"},
{"category": "legal"},
{"active": true}
]}'
The expected result is a single vector, doc-001, the only one matching all three filter conditions (tenant_id, category, and active):
{
"vectors": [
{
"distance": 0.9717477560043335,
"key": "doc-001",
"metadata": {
"tenant_id": "t-10428",
"category": "legal",
"created_date": "2026-03-15",
"active": true
}
}
],
"distanceMetric": "cosine"
}
S3 Vectors first narrows the search space to vectors matching all three filter conditions, then returns the 50 most similar vectors from that subset. Because the filter is applied before the search, those results are drawn from across all the vectors that match it.
Prefix matching with $startsWith
Pre-filtering adds a prefix match operator for filtering on paths, URLs, and hierarchical keys. A document store that encodes case and folder structure into a document ID can scope a search to a subtree in one condition:
--filter '{"$startsWith": {"document_id": "matter-4417/exhibits/"}}'
$startsWith joins the existing operators: equality, numeric range, set membership, existence checks, and boolean logic with $and and $or.
Turning on pre-filtering for existing indexes
Call UpdateIndexMode on an existing index to turn on pre-filtering:
aws s3vectors update-index-mode \
--vector-bucket-name my-vector-bucket \
--index-name product-catalog \
--index-mode ENHANCED
Pre-filtering takes effect in place. Your existing vectors are not re-ingested, your queries do not change, and the new filter operators are available immediately.
Here is the difference on the same index and the same query. Before the update, a query scoped to one tenant returns two of the ten results requested:
aws s3vectors query-vectors \
--vector-bucket-name my-vector-bucket \
--index-name product-catalog \
--query-vector '{"float32": [0.1, 0.2, 0.3, ...]}' \
--top-k 10 \
--return-metadata \
--filter '{"tenant_id": "t-10428"}'
{
"vectors": [
{ "key": "doc-114", "distance": 0.41 },
{ "key": "doc-322", "distance": 0.55 }
],
"distanceMetric": "cosine"
}
After the update, the same query returns a full result set drawn from across that tenant’s documents:
{
"vectors": [
{ "key": "doc-018", "distance": 0.09 },
{ "key": "doc-207", "distance": 0.13 },
{ "key": "doc-114", "distance": 0.41 },
... 7 more
],
"distanceMetric": "cosine"
}
Rolling out across your indexes
Once you have validated pre-filtering on an index, set the default index mode on the vector bucket so that new indexes use ENHANCED without a follow-up call:
aws s3vectors put-vector-bucket-default-index-mode \
--vector-bucket-name my-vector-bucket \
--default-index-mode ENHANCED
To bring the rest of your existing indexes across, list them and check the index mode on each one, then call UpdateIndexMode on the ones still using CLASSIC:
aws s3vectors list-indexes \
--vector-bucket-name my-vector-bucket
aws s3vectors get-index \
--vector-bucket-name my-vector-bucket \
--index-name product-catalog
Things to know
Get started today
Metadata pre-filtering is available at no additional cost in all commercial AWS Regions where Amazon S3 Vectors is available, and in the AWS China Regions. You pay standard S3 Vectors pricing for storage, PUT requests, and queries. For full pricing details, visit the Amazon S3 pricing page. For regional availability, visit Amazon S3 Vectors Regions and quotas.
Whether you’re scoping a RAG application to one tenant, scoping an agent’s searches to one user’s documents, or narrowing a catalog search to a licensing window, pre-filtering lets you apply those filters without trading away recall. To learn more and get started, visit the Amazon S3 Vectors documentation. Send feedback to AWS re:Post for S3 or through your usual AWS Support contacts.
— Daniel Abib
Post Syndicated from Esra Kayabali original https://aws.amazon.com/blogs/aws/amazon-aurora-postgresql-now-supports-direct-querying-of-apache-iceberg-and-parquet-data-in-your-data-lake/
Today, we’re announcing a new capability for Amazon Aurora PostgreSQL that you can use to directly query operational data together with data stored in your data lake in Apache Iceberg and Apache Parquet formats, using your existing PostgreSQL applications and tools. By eliminating the need to extract, transform, and load (ETL) structured data from data lakes into your operational database, you can reduce operational complexity and simplify application development. You can also use Aurora PostgreSQL to query data from data lakes managed in Iceberg REST Catalog (IRC)-compatible catalogs, giving you access to data across a breadth of analytics systems without moving or duplicating it. Whether you’re powering real-time dashboards, enriching transactions with historical context, or building AI agents that reason over both live and archived data, you can now do it all through a single, familiar interface.
Previously, if your application needed to combine recent transactional data in Aurora with historical records stored in Amazon S3, a common approach was to build reverse ETL pipelines that duplicated data, increased infrastructure costs, and required ongoing engineering effort to keep everything synchronized. This challenge only grows as you increasingly embed AI agents into your applications, where it is impractical to predict and pre-replicate every dataset an agent might need.
DuckLabs, the team that maintains the DuckDB project, recently joined Amazon, and this capability is an example of how the efficiency of DuckDB is being integrated into our services. DuckDB is now embedded directly within Aurora PostgreSQL, so you can query live operational data (including uncommitted writes) alongside your data lake in a single query. Query processing stays within Aurora, with no additional network hops and no ETL pipelines that duplicate data. You can query Apache Iceberg tables managed through the AWS Glue Data Catalog, as well as Parquet and Iceberg data stored in Amazon S3 and S3 Tables. You do all of this using familiar PostgreSQL syntax and your existing applications and tools.
We’re excited to bring the speed and simplicity of DuckDB directly into Aurora PostgreSQL, so you and your agents can query and combine operational and Iceberg data using the familiar PostgreSQL applications, tools, and endpoints already in use. By building this capability around DuckDB, future improvements to the open source engine can continue to bring performance and functionality gains to Aurora and other AWS services.
What is new
This capability is supported on two Aurora PostgreSQL major versions: 17 (starting with 17.11) and 18 (starting with 18.6). To use it, you create an Aurora PostgreSQL cluster, attach an IAM role with the AuroraAnalytics feature, and enable the aurora_analytics extension. The IAM role is what gives Aurora access to your data in Amazon S3 and the AWS Glue Data Catalog. You then create foreign tables that point to your Iceberg or Parquet data in the data lake, and query them using familiar PostgreSQL syntax. You can complete this setup through the Amazon RDS console, or with any PostgreSQL client such as psql. The process is well documented in the Aurora PostgreSQL documentation.
You can query data across external IRC-compatible catalogs through AWS Glue Data Catalog federation. You register the external catalog once with Glue, and then create foreign tables for the tables you want to query, the same way you would for any Glue-native table. A single query can then join data stored in Aurora with Iceberg tables registered across multiple catalogs, so applications get a unified view without moving data or replacing your existing catalog investments.
Aurora also applies optimizations such as predicate pushdown and column pruning so that only the relevant data is read. This keeps queries efficient even as the underlying data grows. Frequently accessed data is also cached in your Aurora instance, so subsequent queries against the same data return faster. You can inspect this behavior per query using aurora_analytics_stat_statements(), which reports metrics such as rows scanned, bytes read from Amazon S3, and cache hits.
To see how direct querying works, I connected to my Aurora PostgreSQL database using psql and created the extension:
CREATE EXTENSION aurora_analytics;For my walkthrough, I set up a simple financial scenario. I have a recent_transactions table in Aurora with the last 7 days of customer transactions, and a Parquet file in Amazon S3 containing 5 years of historical transaction data. To make Aurora aware of the historical data, I created a foreign table pointing at the Parquet file in S3:
CREATE FOREIGN TABLE transaction_history ()
SERVER aurora_analytics_server
OPTIONS (
location 's3://<my-bucket>/finance/transaction_history.parquet',
format 'parquet'
);Notice the empty parentheses in the CREATE FOREIGN TABLE statement. Aurora automatically reads the schema from the Parquet file metadata, so you do not need to define columns manually. For workloads with many tables, you can skip creating them one at a time: a single IMPORT FOREIGN SCHEMA statement bulk-creates foreign tables for every Iceberg or Parquet table in an AWS Glue Data Catalog database, inferring schemas automatically.
With both tables in place, I ran a single query that combines the recent operational data in Aurora with the historical data in S3:
SELECT merchant, category, amount, transaction_date, 'recent' AS source
FROM recent_transactions
WHERE customer_id = 'C-1001'
UNION ALL
SELECT merchant, category, amount, transaction_date, 'historical' AS source
FROM transaction_history
WHERE customer_id = 'C-1001'
AND transaction_date >= CURRENT_DATE - INTERVAL '5 years'
ORDER BY transaction_date DESC
LIMIT 15;
The result shows both recent and historical transactions in a single result set. The 7 most recent rows come from Aurora, and the rest come directly from the Parquet file in S3. DuckDB handles the analytical scan of the Parquet data under the hood, while Aurora handles the operational data. That single query would have previously required a pipeline to move the historical data into the database first.
If a query pattern needs single-digit-millisecond latency, you can materialize data from the data lake into a native Aurora PostgreSQL table using familiar commands such as CREATE TABLE AS SELECT, INSERT INTO ... SELECT, or MERGE INTO. The materialized table lives in Aurora and is queried like any other PostgreSQL table, giving you a low-latency path for hot data without operating a separate ingestion pipeline. The read queries can run on any Aurora PostgreSQL instance in your cluster, whether the writer or a read replica, so you can offload analytical scans from your operational workload. The materialization commands write data into Aurora, so they run on the writer instance.
Get started today
Direct querying of Apache Iceberg and Parquet data from Amazon Aurora PostgreSQL is available today in all commercial AWS Regions and AWS GovCloud (US) Regions, at no additional charge. You pay only for the incremental Aurora compute the queries consume and Amazon S3 request costs for reading data lake files.
To learn more, visit the Amazon Aurora features page, read the Aurora PostgreSQL documentation, or try it in the Amazon RDS console. We welcome your feedback through AWS re:Post or through your usual AWS Support contacts.
Post Syndicated from Taylor Jacobsen original https://aws.amazon.com/blogs/aws/celebrating-our-newest-aws-heroes-september-2026/
Today, we’re excited to introduce the newest members of the AWS Heroes program. AWS Heroes are a vibrant, worldwide group of AWS experts who go above and beyond to share knowledge, mentor others, and build thriving communities. These individuals make a real difference in helping developers and organizations succeed with AWS.
This month, we welcome three exceptional community leaders from across the globe, each bringing unique expertise and a deep commitment to empowering builders everywhere.
Serverless Hero Avinash Shashikant Dalvi is a tech architect and co-organizer of AWS User Group Bengaluru who is focused on serverless, containers, and production-ready applications on AWS. He has delivered over 40 community talks, publishes the AWS for Product Builders newsletter, and creates technical content covering Amazon ECS, AWS Fargate, AWS Lambda, and AWS Amplify.
Serverless Hero Joanne Skiles is an engineering leader and educator with over 16 years of experience building full-stack systems, including serverless architecture and AI systems on AWS. She organizes the Orlando AWS User Group and teaches cloud and AI concepts through her YouTube channel, conference talks, and her podcasts Chaotic Commits and Her Career Unplugged. Joanne is also a professor in the Computer Science department at Rollins College, where she runs the Transparent Systems lab.
Community Hero Xiaofei Li is an AWS Golden Jacket holder and is an active community leader in the Greater China Region, leading the Kiro, Amazon Quick, and Tokyo Chinese AWS communities. He founded the Kiro Chinese User Community (5,000+ members) and initiated the Chinese localization of AWS Builder Cards across 15 cities and 3,000+ participants. Xiaofei also mentors underserved students and supports Women in Tech initiatives.
Visit the AWS Heroes webpage if you’d like to learn more about the AWS Heroes program, or to connect with a Hero near you. To learn more about how to get involved with the AWS community, visit our AWS Builder Center.
— Taylor
Post Syndicated from Vic A original https://www.servethehome.com/nvidia-open-agent-safety-platform-launched/
NVIDIA is taking on agentic AI security with the new Open Agent Safety Platform and OpenShell 0.1.0 frameworks
The post NVIDIA Open Agent Safety Platform Launched appeared first on ServeTheHome.
Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-gpt-6-sol-and-luna-claude-opus-5-5-on-amazon-bedrock-strands-harness-and-more-september-28-2026/
If there’s one theme that defined last week, it’s choice. The frontier models keep arriving, and the interesting question is no longer just “how smart is it?” but “which model fits this step, at this cost, at this latency?” That’s exactly what landed on Amazon Bedrock over the past few days: GPT-6 Sol and GPT-6 Luna from OpenAI, giving you two new points on the intelligence-versus-efficiency curve, and Claude Opus 5.5 from Anthropic, the first of the Claude 5.5 family.
GPT-6 Sol is built for the demanding, recurring work of development and operations, while GPT-6 Luna makes focused, repeatable tasks practical at high volume, and both ship at significantly lower pricing than their GPT-5.6 predecessors. Claude Opus 5.5, meanwhile, does more with fewer tokens than Opus 5 and is tuned for agentic coding and long-running tasks. What I like about all three is that they push toward the same idea: match the model to the job instead of reaching for the biggest one every time. The other thread was observability catching up to this agentic world, including a launch I had the pleasure of writing about myself.
Now, let’s get into this week’s AWS news…
Last week’s launches
Here are some launches and updates from this past week that caught my attention:
For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.
Other AWS news
Here are some additional posts and resources that you might find interesting:
For a full list of AWS blog posts, be sure to keep an eye on the AWS Blogs page.
Upcoming AWS events
Check your calendar and sign up for upcoming AWS events:
Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development. Browse here for upcoming AWS-led in-person and virtual events and developer-focused events. That’s all for this week. Check back next Monday for another Weekly Roundup!
— Daniel Abib
Post Syndicated from Patrick Kennedy original https://www.servethehome.com/asus-ai-tech-2026-seoul-south-korea/
At the ASUS AI Tech 2026 day in Seoul, South Korea, the company brought together its partners, customers, and suppliers and I hosted a panel
The post ASUS AI Tech 2026 Seoul South Korea appeared first on ServeTheHome.
Post Syndicated from Micah Walter original https://aws.amazon.com/blogs/aws/introducing-enhanced-custom-event-buses-in-amazon-eventbridge-for-enterprise-scale-event-driven-applications/
Organizations building event-driven applications on Amazon EventBridge typically start with a single custom event bus in one account. This works well when a single team owns the architecture. As adoption grows across the organization, though, things get complicated. AWS best practices recommend a multi-account structure, which means each team runs in its own account. To route events between them, teams create multiple event buses connected through cross-account rules or bus-to-bus configurations. This workaround reintroduces the operational complexity that serverless architectures are meant to eliminate. Platform teams lose visibility into who is subscribing to which events, cross-account and bus-to-bus routing charges compound quickly, and teams that need capabilities like event ordering are forced to build complex workarounds or adopt entirely different technologies.
Today, we are announcing an enhanced custom event bus in Amazon EventBridge, purpose-built for organizations scaling event-driven applications across teams and accounts. With the new enhanced custom event bus, you can deploy a single, centralized event bus shared across all AWS accounts in your organization, with ordering guarantees, a simplified Subscriber resource, and a new pricing model that delivers improved economics at scale and cost allocation for publishers and subscribers.
Let’s try it out
To get started with an enhanced custom event bus, I navigated to the EventBridge console in the AWS Management Console and opened the Create custom event bus page. I selected Custom event bus, the recommended option labeled New. The page also offered Custom event bus – classic, which continues to receive events and route them with rules and targets. Below the selection, EventBridge showed how the new bus works. One shared bus serves every team in the organization. Publishers send events, subscribers consume only what they need, and EventBridge handles ordering, retention, routing, and delivery.
The Create custom event bus page. Custom event bus is the recommended new option, with ordered delivery, filter patterns, event replay, and sharing across your AWS organization. Custom event bus – classic remains available for existing workloads.
Next, I configured resource sharing. I turned on Enable event bus sharing and selected Allow sharing only within your organization. I chose AWS account ID as the principal type. I could also share with an organization, an organizational unit, or an AWS Identity and Access Management (IAM) role or user. Sharing uses AWS Resource Access Manager (AWS RAM), so I did not have to set up cross-account permissions or bus-to-bus routing myself.
Resource sharing on the new custom event bus. I enabled sharing within my organization through AWS RAM and selected an AWS account as the principal, which is how teams publish and subscribe on the same bus without extra routing.
Organization-wide sharing
With the new enhanced custom event bus, you can create a single event bus and share it across all AWS accounts in your organization. Platform teams deploy one bus and establish it as the central event backbone, eliminating the need to configure cross-account permissions or bus-to-bus routing. Application teams across your organization can publish and subscribe to events on the same bus without waiting for infrastructure provisioning.
Publishers send events without needing to know which teams consume them, and subscribers create their own Subscriptions independently. Platform teams maintain visibility into all event flows and fine-grained control over who can publish and consume events. The new enhanced custom event bus has a default quota of 10,000 Subscribers per bus, and you can request a higher quota. That reduces the fragmentation that occurs when subscriber limits force you to split across multiple buses.
Event ordering
Event-driven architectures work best when consumers are designed around asynchronous patterns, where the order of events does not matter. There are a few cases where order does matter. In a logistics application, driver location updates must arrive in sequence. Out-of-sequence events cause routing algorithms to make decisions based on stale data.
The enhanced custom event bus supports both patterns on the same bus. Publishers can include an EventGroupId when sending events. EventBridge delivers events that share the same EventGroupId in sequence to Subscribers that chose ordered delivery. Other subscribers on that bus can receive the same events without ordering. You can keep events for each driver in the correct order without building complex workarounds, while the rest of your consumers stay fully asynchronous.
To support ordered processing, the enhanced custom event bus includes synchronous invocation for targets like AWS Lambda. Synchronous mode confirms successful processing before acknowledging the event, eliminating the common pattern of placing Amazon Simple Queue Service (Amazon SQS) between an event bus and Lambda to ensure reliability.
Subscriptions
The enhanced custom event bus introduces the Subscriber resource, which combines event filtering, target configuration, retry policies, and dead-letter destinations into a single, manageable unit. Today, achieving the same outcome with EventBridge requires configuring separate rules, targets, and retry settings across multiple resources. Subscribers simplify this by giving each consumer one resource that defines what events they want, where to deliver them, and how to handle failures.
Subscribers also include variable start time options, making it easier for teams to onboard new consumers or replay events to recover from application errors or hydrate new applications.
Event evaluation
Publishers can turn on content-based deduplication so EventBridge detects and drops retries of the same event from the payload itself. You do not have to generate and track a deduplication ID when a timeout or a partial failure sends the same event twice. EventBridge hashes the meaningful parts of the event and collapses matches that arrive within five minutes, which gives those retries exactly-once delivery semantics instead of EventBridge’s usual at-least-once model. If you already stamp your own idempotency token, keep using it. Content-based deduplication is for sources that cannot reliably identify the same event on a retry.
Subscribers can use JSONata expressions to reshape an event before it reaches a target, extracting fields, renaming them, or computing new values when a downstream API expects a different shape. If you already produce Apache Avro or Protocol Buffers events, EventBridge can deserialize those payloads to JSON, allowing subscribers fine grained filtering and routing on the full event payload without having to consume, deserialize, and match or discard on their own.
New pricing model
The enhanced custom event bus uses a new ingress and egress throughput pricing model. Publishers pay for events ingested, and subscribers pay for events delivered. This replaces the per-event model where cross-account and bus-to-bus routing charges compound in multi-bus architectures. For pricing details, visit the EventBridge pricing page.
Existing EventBridge custom event buses continue to work as they do today with no changes required. They now appear as Custom event bus – classic. The enhanced custom event bus is a new resource that you adopt at your own pace. In the console, it appears as Custom event bus.
Now available
The enhanced custom event bus is available today in the US East (N. Virginia, Ohio), US West (Oregon), Europe (Ireland, Frankfurt, Stockholm, Spain), and Asia Pacific (Hong Kong, Malaysia, Mumbai, Singapore, Sydney, Thailand, Tokyo) Regions. You can create your first enhanced custom event bus through the AWS Management Console, AWS Command Line Interface (AWS CLI), or EventBridge APIs. To get started, visit the EventBridge documentation or try it out directly in the EventBridge console.
Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/introducing-amazon-cloudwatch-omni-collaborative-ai-powered-observability-for-your-applications/
Amazon CloudWatch now offers CloudWatch Omni, an AI-powered observability experience for the applications and AI agents you run together. You reach Omni through a dedicated URL for your organization and sign in with the identities you already manage, so working in Omni does not require access to the AWS Management Console. Omni is built on OpenTelemetry: the telemetry you already send to CloudWatch appears in Omni with nothing to reconfigure, and any other workload you instrument with OpenTelemetry sends its telemetry to an OpenTelemetry Protocol (OTLP) endpoint.
CloudWatch Omni offers both agent observability and application observability in a single experience. In our companion post, we introduced the agent observability capabilities of Omni for generative AI and agentic workloads. In this post, we present the application observability experience.
Engineering teams spend a significant portion of their observability time maintaining dashboards, tuning thresholds, and switching between tools to piece together what happened during an incident. When an issue crosses team boundaries, context gets lost in Slack threads and screenshots rather than flowing naturally to the next engineer. CloudWatch Omni changes this by organizing observability around your applications rather than individual signals, and bringing your whole team into the same workspace.
What CloudWatch Omni brings
CloudWatch Omni addresses three problems that engineering teams told us they face today.
One collaborative experience for your whole team. Every engineer accesses CloudWatch Omni through a single URL with enterprise SSO (via IAM Identity Center, supporting Okta, Azure AD, and other providers). No AWS Console access is required. SREs, developers, database engineers, and managers share the same data and investigation context. When an investigation escalates, the next person joins the same session with full context already in front of them.
The system adapts as your applications evolve. CloudWatch Omni discovers your services, maps dependencies, and adjusts alarms automatically. Instead of manually curating dashboards and tuning thresholds, you declare what matters (availability targets, latency budgets, error rate thresholds) and Omni adapts as your system changes. When you deploy new services, Omni updates the application topology automatically.
AI-powered investigation with Amazon DevOps Agent. Amazon DevOps Agent participates alongside your team in investigation sessions, correlating signals and suggesting next steps. The agent works from the same telemetry your engineers see, so its suggestions are grounded in the actual state of your application. It identifies correlated events across services, traces root cause paths through your dependency graph, and maintains investigation history for post-incident review.
How an investigation works
When something breaks, CloudWatch Omni opens an investigation session pre-loaded with context. Here is a typical incident workflow:
An alarm fires on elevated error rates in your checkout service. Omni opens a session showing the service topology, correlated signals (a deployment 10 minutes earlier, increased latency from a downstream payment API), and DevOps Agent’s initial analysis.
Your on-call SRE confirms the deployment correlation, pulls in the trace view to identify failing endpoints, and checks if the payment API latency correlates with a capacity limit.
The SRE escalates to the payments team. The payments engineer joins the same session and sees everything found so far, plus DevOps Agent’s correlation with a configuration change in the payment provider’s API gateway. They identify the root cause and roll back.
The entire investigation history is captured automatically. No separate incident report needed.
Walkthrough: setting up your first Space
To set up CloudWatch Omni for your team, open the CloudWatch console and click “Try CloudWatch Omni.”
Next, connect your identity provider through IAM Identity Center (supporting Okta, Azure AD, and other SAML 2.0 providers). Once connected, your team members access Omni directly at your dedicated URL without needing AWS Console credentials.
Create a Space for your team. A Space groups the applications your team owns and the telemetry associated with them.
Once created, Omni discovers your services automatically and maps the dependencies between them. You see your application topology immediately.
You can ask CloudWatch Omni any question about your applications in plain English, and Omni will analyze your telemetry data and surface insights.
You can also set up service health alerts, configure what matters to your team, and trigger an AWS DevOps agent investigation to identify the root cause and develop a mitigation plan.
Application-centric organization
CloudWatch Omni organizes telemetry by application rather than by infrastructure component. The system automatically discovers services from the telemetry data and AWS Config resource discovery, maps dependencies, and lets you see your application as a connected system rather than a collection of isolated resources.
Each team gets a Space that contains the applications they own. A Space points at existing CloudWatch data (logs, metrics, traces, and alarms) with no additional data movement required. Dynamic views replace the maintenance burden of static dashboards, providing ongoing visibility into SLOs and application health.
Getting started
Getting started takes minutes and doesn’t require reconfiguration of your existing CloudWatch setup.
If you’re an existing CloudWatch customer: Click “Try CloudWatch Omni” in the CloudWatch console. All your existing telemetry (logs, metrics, traces, and alarms) is immediately available. Workloads are discovered automatically, and you can start an investigation or browse your application topology right away.
For organization-wide deployment: An administrator configures a domain, connects your identity provider via IAM Identity Center, defines Spaces for teams and environments, and invites users. Each Space points at existing CloudWatch data with no additional data movement required.
For applications in other environments: CloudWatch Omni provides connectors that make it easy to bring in telemetry from additional environments. All ingested telemetry appears alongside your AWS data in the same Spaces and investigation sessions.
For generative AI and agentic workloads: The same CloudWatch Omni experience delivers purpose-built observability for AI agents, including trace exploration, evaluation frameworks, and real-time monitoring. In our companion post, we introduced the agent observability capabilities of Omni; for that walkthrough, see Introducing Amazon CloudWatch Omni: AI-powered observability for generative AI and agentic workloads.
Things to know
Pricing and availability
Amazon CloudWatch Omni is now available. Existing CloudWatch customers can try it directly from the CloudWatch console. For pricing details, visit the Amazon CloudWatch pricing page.
To get started, visit Amazon CloudWatch Omni or click “Try CloudWatch Omni” in the Amazon CloudWatch console.
If you want to call APIs, search documentation, find regional availability, and check troubleshooting about this feature, try using the AWS MCP Server and plugins with your preferred AI tool. Share your feedback on AWS re:Post or reach out through your usual AWS Support contacts.
— Daniel Abib
Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/introducing-amazon-cloudwatch-omni-ai-powered-observability-for-generative-ai-and-agentic-workloads/
Today, Amazon CloudWatch introduces CloudWatch Omni, a unified observability experience for application and AI workloads that is app-centric, AI-powered, built on open standards, and delivered off-console. CloudWatch Omni is a purpose-built observability, evaluation, and experimentation solution for AI agents. It helps teams design, evaluate, and operate AI agents across any model provider, framework, or runtime, with an eval-driven workflow, support for the tools you already use, and observability delivered where you work: directly in your IDE and through a standalone web experience, separate from the AWS Management Console.
Organizations deploying agentic AI systems face observability challenges that traditional monitoring can’t address. Agent behavior is non-deterministic: a prompt change can degrade response quality even when standard metrics show no errors. Teams spend hours manually reviewing logs across multiple systems, unable to pinpoint what changed or why. Existing tools force teams to choose between siloed generative AI monitoring or fragmented solutions requiring constant context-switching between their coding environment and browser-based dashboards.
CloudWatch Omni captures every trace and includes built-in evaluators for correctness, coherence, retrieval quality, and tool selection, among others. You can compare prompt versions side by side in the playground, build test datasets from production traffic, run experiments across different configurations, and detect regressions automatically.
Two surfaces for development and operations
CloudWatch Omni delivers observability through two complementary surfaces. Developers get a native extension inside VS Code and Kiro (the currently supported IDEs), where traces appear as you run your agent with a playground and evaluators a click away. Operators get a standalone web experience, separate from the AWS Management Console to monitor the fleet, accessible through SSO with no AWS console needed. Both share the same data: the trace a developer debugs is the trace an operator investigates.
The Cloud Login feature connects your local IDE environment to your AWS account, enabling you to send telemetry data to Amazon CloudWatch for persistent storage, share traces with your team, and access production dashboards. This connection is optional. You can use CloudWatch Omni entirely locally during development, then connect to the cloud when you are ready to monitor agents in production.
Getting started
CloudWatch Omni offers two ways to get started: through the IDE extension (for VS Code and Kiro) or directly through the cloud experience, where you can start sending telemetry data to CloudWatch without installing any IDE extension. In this walkthrough, I install the extension, create an agent, run it, and explore the traces and evaluation tools from my IDE.
After installing the CloudWatch Omni extension from the VS Code Marketplace, the CloudWatch Omni icon appears in the Activity Bar. From the welcome screen, I selected Get started with Sample Project to load a pre-configured agent with sample trace data or use shortcut to Command Palette using Command + Shift + P (on macOS) or Ctrl + Shift + P (on Windows/Linux) and select Omni: Create a new Project
The sample project comes with an agent implementation and example datasets. Part of the getting-started experience is adding OpenTelemetry instrumentation, and CloudWatch Omni guides you through each step. You can also create a new agent from scratch. CloudWatch Omni walks you through the process using an interactive chat where you define the agent’s purpose, select a model provider, and configure tools. All data is stored locally by default. You can optionally connect to AWS to send data to Amazon CloudWatch.
After verifying the configuration, I started the local dev server and sent a question to the agent. What makes this different from a typical chatbot interface is what happens next: selecting View Trace shows exactly how the agent processed the request.
CloudWatch Omni integrates with AI code assistants such as Kiro, Claude Code, and Codex to streamline the setup process. These assistants can configure the Dev Server, install dependencies, and set up instrumentation on your behalf, so you can go from installation to running your first traced agent session in minutes without manual configuration.
Traces are essential for understanding AI agent behavior. Unlike traditional request-response systems, agents make multiple decisions per invocation: choosing tools, composing prompts, and chaining sub-calls. Without full trace visibility, diagnosing why an agent produced an incorrect answer or took an unexpected path becomes guesswork. CloudWatch Omni records every step in a structured timeline so you can pinpoint exactly where behavior diverged.
The Trace Explorer shows a detailed breakdown of every step the agent took (LLM calls, tool invocations, and reasoning steps) in a structured, hierarchical timeline. I could drill into any span to inspect inputs, outputs, token usage, and latency.
The Trace Explorer also supports Compare mode, which places two traces side by side to see how different prompts or configurations affect behavior. Compare mode is especially helpful when debugging regressions. And with Ask Assistant, an AI agent analyzes your traces to surface patterns and anomalies, answering questions like “Why did the agent call this tool twice?”
Evaluation is what turns observability into actionable quality improvement for generative AI. Traditional metrics like latency and error rate cannot tell you whether an agent’s response was helpful, coherent, or factually correct. Evaluators score each response against quality dimensions, letting you measure what users actually experience and catch regressions that standard monitoring misses entirely.
CloudWatch Omni includes 17 built-in evaluators for metrics like coherence, helpfulness, faithfulness, and routing correctness. I selected traces from the Trace Explorer, chose evaluators, and ran an evaluation, getting per-example scores and aggregate metrics without building any custom evaluation framework.
From there, I used the Playground to test different system prompts side by side, comparing multiple model and prompt configurations in real time to see how each variation affects output quality before committing changes. With the Experiments view, I could run the same dataset against two agent variants and compare their evaluation scores, latency, and token usage side by side to pick the best-performing configuration.
Figure 7. Comparing evaluations across agent variants in the Omni Experiments console
With Prompt Management, you can version and track prompt configurations over time, making it easy to roll back when a new version underperforms.
CloudWatch Omni also provides a Session Explorer to review full conversation histories and understand how agents handle multi-turn interactions, along with an Agent Topology view that visualizes the architecture of your agent system, including sub-agents, tools, and their interconnections. You can drill into any node to inspect performance and identify bottlenecks.
CloudWatch Omni also offers a dedicated web experience accessible from any browser without an IDE. Teams can access all capabilities collaboratively, including application monitoring, analytics, agent observability, and AI-powered investigations.
I curated traces into golden datasets for structured experimentation. The Experiment function runs the agent against a dataset and automatically scores results, creating benchmarks for regression testing whenever prompts or agent logic change.
If you already have an agent built with a supported framework, CloudWatch Omni provides two paths to add instrumentation: Auto-instrument with Kiro, which detects your framework and configures tracing automatically, or manual instrumentation with ready-to-use code snippets for Python and TypeScript. For detailed instrumentation guides, see the CloudWatch Omni documentation.
Supported frameworks and open standards
The walkthrough above uses the sample project, but CloudWatch Omni works with the agent frameworks teams are already using: LangChain, LangGraph, CrewAI, OpenAI SDK, Strands, Vercel AI SDK, and more, in both Python and TypeScript. It also provides native observability for agents built with Amazon Bedrock AgentCore, and uses AgentCore’s evaluation capabilities to assess agent quality directly within the Omni workflow.
Instrumentation uses open standards (OpenInference and ADOT), whether your agents run on Lambda, ECS, EKS, or other clouds. For evaluation, Omni integrates with third-party evaluators including Braintrust, DeepEval, and Ragas, alongside built-in datasets, a playground, and batch experiments. No re-platforming required.
CloudWatch Omni brings agent observability and application observability together in a single experience. For the application observability experience, read the companion post Introducing Amazon CloudWatch Omni: collaborative AI-powered observability for your applications.
Pricing and availability
Amazon CloudWatch Omni is now generally available. The IDE extension is free to use. You don’t need an AWS account to get started. You only need AWS credentials for Amazon Bedrock models, or API keys for other providers like OpenAI or Anthropic. Get started today by installing the extension from the VS Code Marketplace.
To explore all capabilities and get started quickly, visit CloudWatch on AWS Builder Center.
If you want to call APIs, search documentation, find regional availability, and check troubleshooting about this feature, try using the AWS MCP Server and plugins with your preferred AI tool. Share your feedback on AWS re:Post or reach out through your usual AWS Support contacts.
Happy building!
— Daniel Abib