Tag Archives: Amazon EC2

AWS Weekly Roundup: AWS Lambda Web Functions, Amazon S3 on the Vercel Marketplace, and more (October 19, 2026)

2026-10-19 Daniel Abib

Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-aws-lambda-web-functions-amazon-s3-on-the-vercel-marketplace-and-more-october-19-2026/

Nothing last week added a new kind of compute. What it did instead was shorten the distance between the code you already have and that code being live. AWS Lambda learned to run a plain HTTP server with no adapter in front of it, and Amazon S3 became something you can provision from inside the tool you were already using, without opening the AWS Management Console. Different services, same direction.

The counterweight arrived on Thursday. As it gets easier to put things into production, it matters more that you can say what those things are allowed to do, and this week’s most interesting read is about exactly that. Now, let’s get into this week’s AWS news…

Headlines

AWS Lambda Web Functions is a new form factor that runs a standard HTTP web application on Lambda without rewriting it into an event handler. You take a port-listening server built with Express, Hono, Fastify, or Next.js, run aws lambda-web deploy, and get back a live HTTPS URL in under a minute. No adapters, no infrastructure to set up first, no framework lock-in. Lambda now has three form factors on the same Firecracker isolation, which compose inside one application: Event Functions (the model you already know) for async work, Web Functions for the user-facing layer, and MicroVMs for stateful sandboxes running untrusted code.

Two details make this more than repackaging. Each execution environment handles up to 128 concurrent requests, so a burst of traffic shares warm environments instead of mapping one request to one environment, and billing meters the CPU time your code actually uses rather than wall-clock time. For server-side rendering, API backends, and inference endpoints streaming tokens, where most of the elapsed time is spent waiting, that changes the arithmetic. Adding --regions to the same deploy command replicates your code and configures latency-based routing behind a single global URL, and you can split traffic across revisions by weight for a canary rollout. It’s available in five Regions, running Node.js 24 on Arm64 with VPC connectivity. The AWS Compute Blog has a longer walkthrough, and Amazon CloudFront can now use a Web Function endpoint as an origin with origin access control.

The week’s other headline isn’t a launch. Trust in Depth: giving agents more autonomy without losing control makes an argument I keep running into in practice: putting a person in the loop for every agent action works at small scale and then quietly stops working, because people asked to check everything stop catching what matters. The alternative is four questions answered at four moments, which AWS calls scope, know, bound, and observe. The part worth sitting with is that an agent’s limits belong outside its own reasoning, in Amazon Bedrock AgentCore policies enforced at the gateway, because an agent cannot turn off a control it does not run. AWS has run automated reasoning in production for more than a decade, including AWS Identity and Access Management at more than 2.1 billion requests per second, and the case here is for pointing it at the rules you write for agents.

Last week’s launches

Here are some launches and updates from this past week that caught my attention:

Amazon S3 is now available on the Vercel Marketplace and v0 by Vercel – You can now provision S3 general purpose buckets, S3 Tables, and S3 Vectors from the Vercel dashboard, or by describing what you want to build in v0. It joins existing integrations for Amazon Aurora, Amazon DynamoDB, and Amazon OpenSearch Serverless, so the whole stack sits in one AWS account with one bill. The authentication detail is the good part: a short-lived OIDC token is exchanged at runtime with AWS Security Token Service for credentials scoped to that resource, so no static keys are stored anywhere.
Amazon EC2 T8i instances are now available in South America (São Paulo) – The newest burstable generation reached the São Paulo Region, which matters for the unglamorous half of a workload: login gateways, small databases, CI/CD runners, low-traffic sites. Amazon EC2 T8i delivers up to 30 percent better price performance than T3, with up to 70 percent higher compute performance and 2.4 times the Amazon EBS bandwidth. It keeps the same CPU credit system as T3, so moving over is a size change rather than a redesign, and t8i.micro and t8i.small are available under the AWS Free Tier.
Reflection Beam, an open-weight model for agentic coding, is now on AWS – You can now deploy Beam, Reflection AI’s open-weight model for agentic coding and long-horizon tasks, from Amazon SageMaker JumpStart. It’s a sparse mixture-of-experts model with 501 billion total and 23 billion active parameters, released on Hugging Face under Apache 2.0, and Reflection reports an effective context length of 1 million tokens. Start with a JumpStart real-time endpoint, or use SageMaker HyperPod Inference with SGLang or vLLM when you need Kubernetes-native control. It was also ported and optimized for AWS Trainium3, and launches in N. Virginia and Ohio.
AWS Lambda Durable Execution SDK for Go is now generally available – You can now write multi-step Go workflows that checkpoint their own progress and pause for up to a year while waiting on an external event, without building progress tracking yourself. That covers deployment rollouts with approval gates and pipelines that call a model and then stop for human review. It behaves like ordinary Go tooling: go get to add it, deploy a single static binary on the provided.al2023 runtime, and test a full workflow locally with the durabletest package, no network access or credentials required.
Amazon CloudWatch announces log routing and chained log parsers – You can now define routes in an Amazon CloudWatch pipeline that send each event to a different log group based on conditions evaluated against processed fields, so severity, source, or event type can each land somewhere with its own retention and access controls. Chained parsers handle the other half: parsers run in sequence, with later ones consuming fields that earlier ones produced, so you can parse a JSON envelope first and then run a Grok parser over a field it extracted.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news

Here are some additional posts and resources that you might find interesting:

AWS Security Agent exploited 33 of 37 new real-world CVEs – The team pointed their autonomous penetration-testing agent, now part of AWS Continuum, at 37 recently disclosed CVEs packaged as live containerized targets, starting from nothing but an endpoint URL and a one-line hint. It captured 35 flags, and after auditing every win against the challenge’s canonical proof of concept, 33 of 37 (89.2 percent) held up as genuine end-to-end exploitation. The two misses are what make the post worth your time: in both, the agent found and exploited the bug, then took the target down before it could read the flag. A capable autonomous attacker is also a destructive one.
Your serverless attendee’s guide to AWS re:Invent 2026 – A curated path through a very large catalog, worth having before the good sessions fill up.
How Dynata scaled AI across the development lifecycle with AI-DLC and Kiro IDE – What changes when agentic coding stops being an individual habit and becomes how a team works.

Upcoming AWS events

Check your calendar and sign up for upcoming AWS events:

AWS re:Invent – re:Invent returns to Las Vegas from November 30 to December 4, with more than 2,200 sessions. The event catalog is live and reserved seating is open, so register and claim your spots before they fill up.
AWS Community Days – Community-led conferences planned and delivered by community leaders. Here in Brazil, the 2026 season closes with AWS Community Day Brasil Norte in Belém on October 31, the final stop in a year that covered every region of the country for the first time.

Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development. Browse here for upcoming AWS-led in-person and virtual events and developer-focused events.

That’s all for this week. Check back next Monday for another Weekly Roundup!

— Daniel Abib

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!

Scaling RL rollouts with Firecracker on Amazon EC2 metal

2026-10-09 Karsten Ploesser

Post Syndicated from Karsten Ploesser original https://aws.amazon.com/blogs/compute/scaling-rl-rollouts-with-firecracker-on-amazon-ec2-metal/

Modern reinforcement learning (RL) systems rely on thousands of rollout environments running in parallel. For large language model (LLM) training using Verifiable Rewards (RLVR) each rollout environment runs in a tightly isolated sandbox so state never leaks between environments.

The faster you can recycle sandboxes, the more training steps you can run per hour. That means higher rollout density per host which directly lowers your cost per step.

But density is a tuning knob with two failure modes. Push it too high and you hit a tail latency cliff. Play it too safe and you overprovision, leaving money on the table. The sweet spot depends on your instance type, workload profile, and how your training algorithm amplifies stragglers.

This post shows how to find that setting. We walk through the tunables that control density on Amazon EC2 metal. We then introduce labsweep, a measurement harness for mapping your latency and density curve. Throughout the post, we draw on lessons learned from production deployments with customers. The post is written for readers familiar with Kernel-based Virtual Machine (KVM), Firecracker, and container orchestration.

Background: EC2 metal, Nitro, and Firecracker

The AWS Nitro System offloads networking, storage, and security to dedicated hardware. Amazon EC2 instances expose a stable CPU topology you can inspect and pin workloads against. On Amazon EC2 metal, applications run directly on the underlying hardware.

Firecracker is a lightweight virtual machine monitor (VMM) designed for high density workloads. Each microVM boots quickly, runs its own Linux kernel, and has a small memory footprint.

Running Firecracker on Amazon EC2 metal combines direct access to underlying hardware with lightweight virtualization to support high density sandboxing. The key question is how far you can increase density before tail latency hurts the RL training loop.

Solution overview

The following diagram shows the reference architecture of a production RL sandbox node. This architecture serves as the foundation for the rest of the post. An Amazon EC2 metal instance runs a host agent that manages Firecracker microVMs, CPU pinning, non-uniform memory access (NUMA)-aware memory placement, and the warm pool used for dispatch.

Reference architecture of a production RL sandbox node showing host agent orchestration of Firecracker microVMs, resource preparation, warm pool management, workload dispatch, and telemetry collection on Amazon EC2 metal

The host agent runs as a DaemonSet on each Amazon EC2 metal node in the Kubernetes cluster. At startup, it discovers the CPU topology, creates per-microVM cgroup hierarchies, and orchestrates Firecracker jailer processes throughout the microVM lifecycle.

To reduce dispatch latency, the system maintains a warm pool of pre-booted microVMs. The host agent applies CPU pinning, NUMA-aware memory placement, and memory sizing on a per-microVM basis. labsweep identifies the combination of these settings that best balances dispatch latency, density, and tail latency for a given workload.

Prerequisites

Amazon EC2 metal instance (for example, we tested m8i.metal-48xl, m8a.metal-48xl, and m8g.metal-48xl).
Linux kernel 6.2+ (required for cgroup partition isolated mode).
Firecracker 1.7+ and the Firecracker jailer.
Go 1.22+ (to build labsweep).
AWS CDK v2 with Python 3.11+ (to deploy the accompanying stack).
Guest rootfs image (a builder script is included in the accompanying repository).

Scope note: this post focuses on CPU and memory placement for Firecracker microVMs. Other system variables including storage architecture (Amazon Elastic Block Store (Amazon EBS), local SSD) and network latency to the LLM endpoint were held constant to isolate the effects of CPU and memory placement.

Density tunables on EC2 metal

Your density ceiling depends on tunables whose availability varies by instance type. Not every knob exists on every instance type, and where a knob does exist its underlying mechanism can differ. The following table shows which tunables are available across the three metal instance types this post considers: m8i, m8a, and m8g. You can derive availability and mechanism from each instance type’s published CPU topology. The measured latency and throughput results later in the post currently cover m8i and m8a, with m8g results to follow.

Tunable	m8i.metal-48xl	m8a.metal-48xl	m8g.metal-48xl	What it does
Core pinning (`cpuset.cpus`)	✓	✓	✓	Dedicates a physical core to each microVM. Eliminates scheduler-induced tail latency
NUMA node confinement (`cpuset.mems`)	✓	✓	✓ (2 NUMA nodes on the 48xl)	Keeps memory allocations local. The remote-node latency penalty varies by instance type. Check the NUMA distance matrix for your instance before pinning
SMT state (smt/control)	✓ (on/off)	N/A (no SMT)	N/A (no SMT)	Doubles vCPUs when on. Workload-dependent: helps I/O-bound sandboxes, hurts compute-bound loops
L3 cache domain pinning	✓ (place workloads within NUMA-aligned cache domains)	✓ (24 domains, 8 cores each)	✓ (single shared L3 per socket. Pin on NUMA/socket boundary)	Reduces cache contention for working sets above 4 MiB per microVM
`cgroup` partition (isolated)	✓	✓	✓	Removes kernel workqueue interference from pinned cores. Requires kernel 6.2+
Guest memory size	✓	✓	✓	Primary cost driver. Determines how much of your memory-to-core budget each guest consumes; M-series gives 4 GiB per vCPU

Key observation: different instance types expose different optimization opportunities. Among the instances evaluated, simultaneous multithreading (SMT) is available on m8i but not on m8a or m8g. Cache-local placement also differs by instance type: some present many discrete L3 domains, some a shared L3 that is effectively partitioned along NUMA boundaries, and some a single mesh-shared L3 per socket where the meaningful locality boundary is the NUMA/socket rather than a cache-domain count. Each instance type’s L3 layout is described in its own row of the table above and in its published CPU topology. As a result, the set of tunables and their impact on density vary by instance type. This is why measuring your own configuration matters more than following generic engineering best practice lists.

To make this concrete, here is what configuring one tunable looks like in practice: a cpuset pin that dedicates a single physical core to one microVM.

# Create an isolated cgroup for one microVM on core 12
mkdir -p /sys/fs/cgroup/microvm/vm-012
echo "12" > /sys/fs/cgroup/microvm/vm-012/cpuset.cpus
echo "0" > /sys/fs/cgroup/microvm/vm-012/cpuset.mems
# Launch the jailer into this cgroup
firecracker-jailer --id vm-012 \
  --exec-file /usr/bin/firecracker \
  --parent-cgroup microvm/vm-012 \
  --uid 1000 --gid 1000

From tunables to measurement: labsweep

These tunables interact in ways that depend on your workload and instance type. You cannot predict their combined effect from the specification sheet alone. You must measure the resulting system behavior.

labsweep is the test harness we built to do exactly that. It is a Go tool that:

Discovers your host topology through sysfs (cores, NUMA nodes, L3 domains).
Plans a matrix of configurations and densities.
Launches jailed Firecracker microVMs at each density with bounded concurrency.
Runs the workload in every guest concurrently and records per-microVM latency.
Outputs raw samples, percentile aggregates, and variance data with full host provenance.

Methodology note: each experiment is performed at fixed microVM density. Dynamic churn is outside the scope of this evaluation.

The following command launches a typical labsweep run on an m8a.metal-48xl:

labsweep -configs A,B \
  -densities 90,120,150,186 \
  -repeats 5 \
  -resident-kib 131072 \
  -steps 14 \
  -write-kib 4096

Each run produces percentile latencies for every configuration, variance across repeated trials, and raw samples for recomputing any statistics. Results include full provenance (instance type, CPU model, kernel version, and Firecracker versions) making runs reproducible and comparable over time. If a microVM fails to respond, the harness reports failed samples instead of silently omitting them. This prevents failures from biasing the reported results.

The accompanying repository includes everything needed to reproduce these experiments: the labsweep harness, guest agent, rootfs builder, and the CDK application that deploys the complete stack on an Amazon EC2 metal instance (github.com/aws-samples/sample-firecracker-microvm-density-lab).

Results on representative workloads

Using labsweep we benchmarked workloads representative of the execution patterns in production RL systems: short-lived verification tasks (run a test suite, check the result), build-heavy tasks (compile and link), full 14-turn replayed RL episodes, and a synthetic compute loop as a control. Across all evaluated instance types, we collected more than 100,000 samples across 500+ configurations.

The 1.0x ratio rule

Tail latency remains flat as density increases until the workload reaches one microVM per vCPU. Beyond that threshold, p99 latency rises sharply over the next 15% increase in density before reaching a new plateau while median latency remains essentially unchanged. Expressing density as microVMs per vCPU rather than as an absolute microVM count makes this threshold portable across instance types. The latency cliff occurs at the same 1.0x density threshold regardless of instance size. Only the maximum number of microVMs scales with available vCPUs. The complete per-density measurements supporting this figure are provided in the appendix.

Rollout latency versus density (microVMs per vCPU): the p99 latency cliff occurs at approximately 1.0x density while median latency remains stable.

Core pinning reduces the tail latency for most real workloads

On the synthetic compute loop, core pinning made no measurable difference at equal density. On real sandbox workloads however, core pinning measurably tightened the tail: it reduced the p99/p50 ratio toward roughly 1.1x. The p99/p50 ratio measures tail spread by dividing the 99th-percentile latency by the median. Lower values indicate a tighter, more predictable latency distribution. The advantage grows with density, so pinning matters most where you are pushing the host hardest.

Why does this matter when choosing the right density? If you run Group Relative Policy Optimization (GRPO), every gradient step waits for the slowest rollout in the group. At a group size G=64, a p99 event affects ~47% of gradient steps. The effect becomes more pronounced as deployment density increases.

Density / cores	G=8	G=16	G=64
0.94x	1.01x	1.01x	1.04x
1.04x	1.01x	1.03x	1.13x
1.15x	1.01x	1.04x	1.32x

Past the 1.0x ceiling the amplification climbs steadily, and by the time you are well above it a p99 event can dominate the group’s completion time. Median-focused dashboards do not surface this because the median stays flat. Core pinning keeps the p99/p50 ratio low across density, holding tail latency amplification far closer to flat than an unpinned configuration even at a group size of 64. Oversubscription can appear to provide spare capacity until tail latency amplification dominates training time. Set the density to the highest value where the amplification factor remains flat.

SMT depends on your workload

The optimal SMT setting depends on the workload:

I/O-bound workloads: SMT-on provides higher throughput. At high density it delivers 1.49x throughput because the vCPU count stays above the 1.0x ratio.
Compute-bound workloads: SMT-off produces more predictable tail latency, with a markedly tighter p99 spread than SMT-on.

Most RL sandbox workloads are I/O-bound, spending much of their time waiting on file operations, process creation, or git operations. For these workloads, keep SMT enabled. With SMT enabled, you can place more microVMs before reaching the 1.0x ratio, for higher density without crossing the latency cliff.

Configuration matters as much as hardware

Hardware selection is only one factor affecting performance. Configuration choices produce measurable differences. To isolate the effect of configuration, we compare identical workloads on the same m8i instance with SMT enabled and disabled. The results show that changing a single configuration parameter measurably affects peak throughput per host.

The table below shows how enabling and disabling SMT affects peak throughput across a representative set of RL workloads, expressed as the SMT-on to SMT-off throughput ratio. The workloads were selected to represent the major execution patterns encountered in RL systems. Collectively, they cover short-lived verification tasks, interactive sessions, and full RL episodes.

Workload	Throughput gain from SMT
Verification task	1.14x
Build task	1.13x
Replayed RL episode	1.17x

The optimal SMT configuration depends on the density at which the workload is intended to run. With SMT enabled, the host exposes more vCPUs, which supports higher microVM densities before reaching the 1.0x microVM per vCPU threshold. This increases throughput at higher densities by delaying the onset of the tail latency cliff. At densities at or below one microVM per vCPU, SMT disabled delivers slightly better per-task performance and tighter tail latency. These results demonstrate that no single SMT configuration is universally optimal. The best choice depends on the target operating density. You should therefore determine the optimal configuration through measurement rather than assuming it in advance.

When memory becomes the limiting resource

We ran all experiments on M-series metal instances, which provide 4 GiB of memory per vCPU. At this memory-to-vCPU ratio, every workload we tested remained vCPU-bound. We exhausted vCPUs before memory. If your workloads require a higher memory-to-vCPU ratio, the bottleneck shifts from vCPUs to memory. In that case, a memory-optimized instance family may be the better choice. It provides more memory per vCPU but fewer vCPUs per host. As a result, the maximum sandbox density is lower. Consider this trade-off when selecting an instance family.

Reproduce these results on your own workloads

You do not need to take these numbers on faith. With labsweep, you can reproduce them on your own workloads. Clone the accompanying repository and deploy the CDK stack on the instance type you plan to run in production. Build a rootfs from your real workload. Sweep across densities below and above your expected level so you can see where expected performance drops off (the -densities flag in the preceding example is a reasonable start). Repeat each configuration a few times using the -repeats flag so you can see the variance, then read the p99/p50 ratio and the amplification table for your group size. Tune your system based on the curve you measure for your own workloads. That is more reliable than tuning to synthetic benchmarks.

Clean up

If you deployed the accompanying CDK stack for labsweep, tear down these resources when you are done:

Delete the CDK stack (cdk destroy).
Terminate any running Amazon EC2 metal instances.
Remove the guest rootfs images from Amazon Simple Storage Service (Amazon S3) (if uploaded).
Delete the Amazon CloudWatch log groups created by the harness.

Conclusion

Sandbox density on Amazon EC2 metal is determined by a small set of tunables whose optimal values depend on both the workload and the underlying instance type. Because those values vary across workloads, there is no single density setting or configuration that is optimal for every deployment. Instance type specifications give you a starting point but only measurement can identify the right operating point for your workload. labsweep is designed to make that measurement straightforward so you can optimize against your own production workloads.

Blog resources

Appendix

Full per-density measurements behind the tail-latency chart in “The 1.0x ratio rule.” Conditions: m8i, SMT off, 96 physical cores, config A (unpinned). Amplification columns give the GRPO group-completion factor (group max over rollout p50) at group sizes G=8, G=16, and G=64.

Density	Density / cores	Rollout p50	Rollout p99	G=8 amp	G=16 amp	G=64 amp
45	0.47x	7,768 us	7,857 us	1.01	1.01	1.01
90	0.94x	7,740 us	8,035 us	1.01	1.01	1.04
100	1.04x	7,740 us	8,601 us	1.01	1.03	1.13
110	1.15x	7,737 us	9,909 us	1.01	1.04	1.32
135	1.44x	7,743 us	15,601 us	1.03	1.20	2.01
186	1.94x	7,744 us	15,697 us	1.22	1.56	2.03

All the numbers: Amazon Prime Day 2026 powered by AWS

2026-10-06 Channy Yun (윤석찬)

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/all-the-numbers-amazon-prime-day-2026-powered-by-aws/

Amazon Prime Day 2026 was exclusively for Prime members and ran June 23–26, 2026 with millions of deals across more than 35 categories.

As part of our annual tradition of sharing how AWS powered Prime Day (2016, 2017, 2019, 2020, 2021, 2022, 2023, 2024, and 2025), I want to share the services and chart-topping metrics from AWS that made your amazing shopping experience possible.

Prime Day 2026 – all the numbers

As always, Prime Day was powered by AWS. Here are some of the most interesting and/or mind-blowing metrics:

Amazon Elastic Compute Cloud (Amazon EC2) and AWS Graviton – During Amazon Prime Day 2026, AWS Graviton powered up to 49% of the Amazon EC2 compute used by Amazon.com.

Amazon Elastic Block Store (Amazon EBS) – During Prime Day 2026, Amazon EBS, our high-performance block storage service, peaked at over 24.8 trillion I/O operations, moving over an exabyte of data daily.

AWS Lambda – AWS Lambda handled over 2.3 trillion invocations per day during Prime Day 2026.

Amazon Elastic Container Service (ECS) and Fargate – During Prime Day 2026, Amazon ECS launched an average of 158.3 million tasks per day on AWS Fargate, representing a 47.7 percent increase from the previous year’s Prime Day average.

Amazon CloudFront – Amazon CloudFront delivered over 2.1 trillion HTTP requests during the global week of Prime Day 2026, a 5 percent increase in total requests compared to Prime Day 2025.

Amazon DynamoDB – Amazon DynamoDB, a serverless, fully managed, distributed NoSQL database, powers multiple high-traffic Amazon properties and systems including Alexa, the Amazon.com sites, and all Amazon fulfillment centers. Over the course of Prime Day 2026, between June 23 and June 26, 2026, DynamoDB processed over 59 trillion requests. DynamoDB maintained high availability while delivering single-digit millisecond responses and peaking at 192 million requests per second.

Amazon Aurora – On Prime Day, Amazon Aurora, a relational database management system (RDBMS) built for high performance and availability at global scale for PostgreSQL, MySQL, and DSQL, processed hundreds of billions of transactions, stored 5,491 terabytes of data, and transferred 1,194 terabytes of data.

Amazon ElastiCache – During Prime Day, Amazon ElastiCache peaked at serving over 2.3 quadrillion daily requests and 2.1 trillion requests in a minute.

Amazon Kinesis Data Streams – Amazon Kinesis Data Streams, a fully managed serverless data streaming service, processed a peak of 988 million records per second during Prime Day 2026.

Amazon Simple Notification Service (SNS) – Amazon SNS, a fully managed pub/sub messaging service for application-to-application and application-to-person communication, delivered 5 trillion messages in a single day during Prime Day 2026.

Amazon Simple Queue Service (SQS) – Amazon SQS, a fully managed message queuing service for microservices, distributed systems, and serverless applications, received a peak of 213 million messages per second during Prime Day 2026

AWS CloudTrail – AWS CloudTrail processes billions of API activity events per day for governance, compliance, and operational auditing. During Prime Day 2026, that volume surged to 3.6 trillion events in just four days, a 44% increase over Prime Day 2025.

AWS CloudWatch – Amazon CloudWatch processed over 2.15 quadrillion metric observations per day during Prime Day 2026.

Amazon GuardDuty – During Prime Day 2026, Amazon GuardDuty monitored an average of 14.08 trillion log events per hour, a 59% increase from last year’s Prime Day.

AWS Fault Injection Service (FIS) – We ran over 44,000 AWS FIS experiments – more than six times what we conducted in 2025 – to help ensure Amazon.com remains highly available on Prime Day.

Prepare to scale

If you’re preparing for similar business-critical events, product launches, and migrations, I recommend that you take advantage of AWS Countdown Premium. From retail peak seasons to major sporting events, elections, and healthcare enrollment periods, we help you deliver flawless experiences when it matters most. Our experts help you manage your infrastructure to handle massive traffic spikes while maintaining security and performance. We work alongside your team to scale infrastructure, optimize costs during demand surges, strengthen security measures, and monitor real-time demands. To learn more, visit AWS Countdown Premium for business critical events.

I look forward to seeing what other records will be broken next year!

— Channy

Introducing EC2 AMI Tag Sharing: Share EC2 tags across AWS accounts

2026-10-05 Gena Gizzi

Post Syndicated from Gena Gizzi original https://aws.amazon.com/blogs/compute/introducing-ec2-ami-tag-sharing-share-ec2-tags-across-aws-accounts/

Tags are an important and versatile tool on AWS. FinOps teams track costs with them, security teams enforce compliance, and governance teams gate access through attribute-based access control (ABAC).

Tags become especially important as infrastructure management grows more complex across multiple accounts in an organization. Because compute is a common shared resource, many organizations tag their AMIs to track approval status, OS versions, and team. However, when you share an AMI with another AWS account, the tags you attached to it stay behind. The other account sees the image, but tag metadata used to describe it could not be shared.

Until today, the only way to solve this was to build a custom workflow to copy tags into every account the AMI is shared with. Typically created with services such as AWS Simple Notification Service (Amazon SNS) and AWS Lambda, this caused operational burden.

Today, we’re introducing EC2 Tag Sharing, a new feature that AMI owners can use to share tags with any account that has access to the image. Whether an AMI is shared with an account, across an organization, or made public, a shared tag is automatically shared alongside the image by adding the ec2:SharedTag/ prefix to the tag key. Updates also propagate automatically without any need for custom replication workflows. In this post, we walk through how tag sharing works, demonstrate a common use case, and cover considerations and best practices.

How it works

Any tag whose key starts with ec2:SharedTag/ is visible to all AWS accounts the AMI is shared with, and also when the AMI is shared publicly. Tags without the prefix remain private to the account that created them, exactly as they do today.

Only the resource owner can create, modify, or delete tags with the ec2:SharedTag/ prefix. Accounts you share the AMI with can view these tags but cannot change them. They can continue to add their own private tags to a shared AMI.

Shared tags count against the resource owner’s 50-tag-per-resource quota for both shared and private tags. They do not count against the quota of accounts you share the image with. When you copy an AMI that has shared tags, AWS Elastic Compute Cloud (AWS EC2) copies the shared tags to the new image when the --copy-image-tags parameter is included. At launch, tag sharing is supported specifically for AMI resources.

Creating a shared tag (CLI)

To share a tag, create it with the ec2:SharedTag/ prefix on the key name. For example, to share a tag indicating the AMI’s approval status:

aws ec2 create-tags \
    --resources ami-0abcdef1234567890 \
    --tags Key=ec2:SharedTag/status,Value=approved

You can add multiple shared tags alongside your private tags. In the following example, status and os-version are shared with other accounts. The team tag remains private to the owner.

aws ec2 create-tags \
    --resources ami-0abcdef1234567890 \
    --tags \
    Key=ec2:SharedTag/status,Value=approved \
    Key=ec2:SharedTag/os-version,Value=al2023-2026.09 \
    Key=team,Value=platform-engineering

You can also apply shared tags when you create an AMI from the start, using the --tag-specifications parameter. For example:

aws ec2 create-image \
    --instance-id i-1234567890abcdef0 \
    --name "My server" \
    --tag-specifications "ResourceType=image,Tags=[{Key=ec2:SharedTag/publicTag,Value=text}]"

Listing all resources with shared tags (CLI)

To see which of your AMIs have shared tags, you can run the following CLI command:

aws ec2 describe-tags \
    --filters "Name=key,Values=ec2:SharedTag/*"

And get an output such as:

{
    "Tags": [
        {
            "Key": "ec2:SharedTag/status",
            "ResourceId": "ami-0abcdef1234567890",
            "ResourceType": "image",
            "Value": "approved"
        },
        {
            "Key": "ec2:SharedTag/os-version",
            "ResourceId": "ami-0abcdef1234567890",
            "ResourceType": "image",
            "Value": "al2023-2026.09"
        }
    ]
}

You can also receive a similar output using the describe-images command as shown in the following:

aws ec2 describe-images --filters "Name=tag-key,Values=ec2:SharedTag/*"

To stop sharing a specific tag, delete the tag with the ec2:SharedTag/ prefix:

# Remove the shared tag
aws ec2 delete-tags \
    --resources ami-0abcdef1234567890 \
    --tags Key=ec2:SharedTag/status

If you want to keep the tag private, recreate it without the prefix:

# Optionally, recreate as a private tag
aws ec2 create-tags \
    --resources ami-0abcdef1234567890 \
    --tags Key=status,Value=approved

Viewing shared tags as a recipient

When an AMI is shared with your account, you can see the shared tags alongside any tags you’ve added yourself. Figure 1 shows an example of an AMI shared from the original account with both private and shared tags. Figure 2 shows that AMI in the recipient account. The ec2:SharedTag/ prefix in the key name makes it clear which tags came from the owner.

Amazon EC2 console in the owner account showing an AMI with both private tags and ec2:SharedTag/ prefixed shared tags

Figure 1: AMI tags in the originating account, showing both private and shared tags

Amazon EC2 console in the recipient account showing only the shared tags on the same AMI, with the private tags absent

Figure 2: The same AMI in the recipient account, where only the shared tags are visible

As you can see in the preceding figures, the shared tags are visible across both accounts. However, the private tags created in the original account are only visible in the original account.

A common use case: Golden AMIs

Many organizations maintain a central build factory account that produces hardened, security-approved AMIs. These golden images can be shared with hundreds or thousands of workload accounts across an organization. Tags like status=approved or patch-date=2026-09-18 are commonly used by recipient accounts to enforce policies that only allow launches from approved images.

Before tag sharing, the build factory had to replicate tags to every account the AMI is shared with after each AMI publish. A typical workflow looked like this:

The build factory creates a new AMI and tags it.
An SNS notification triggers a Lambda function in each account.
Each Lambda function reads the tag values from the AWS SNS message and calls CreateTags on the shared AMI in its own account.

Because this runs for every AMI, in every Region, across every account, the number of CreateTags API calls can grow rapidly in concentrated bursts. At that scale, teams risk throttling, failed Lambda invocations, and tag drift when calls silently fail.

With tag sharing, the build factory tags the AMI with the ec2:SharedTag/ prefix. Every account the AMI is shared with sees these tags, without the need for additional pipelines to replicate the tags. When the build factory updates a tag (for example, marking an older image as deprecated), the change is visible across all accounts without any additional API calls.

Considerations

Shared tags are visible to every account with access. We recommend that you do not include personally identifying, confidential, or sensitive information in these tags.

If consuming accounts have ABAC policies that evaluate tags, those policies will also apply to the shared AMI. For example, if a recipient account has a policy that allows ec2:RunInstances only when the AMI has the tag status=approved, and you share that tag by using ec2:SharedTag/status=approved, the policy will deny it.

When you adopt the shared tags model, any action that depends on those tags is controlled by the AMI owner, not by the recipient account. This includes tag-gated instance launches, AWS Identity and Access Management (IAM) and ABAC policy evaluations, and any downstream automation that reads tag values. Because only the owner can create, modify, or delete shared tags, the recipient account cannot change the values its own policies depend on. Before you rely on a shared tag in a policy or workflow, make sure you trust the owner account to set and maintain that value.

As a best practice, apply account-level governance so that you only consume AMIs from providers you trust. With Allowed AMIs, you can define criteria for which AMIs are allowed in your account, for example a list of trusted AMI provider account IDs. Only AMIs that meet the criteria are discoverable and available to launch in your account. To preview the impact before enforcing, start in audit mode.

Conclusion

EC2 Tag Sharing removes the need to build and maintain cross-account tag replication workflows. For organizations that share AMIs, this means less undifferentiated heavy lifting. To get started, add a tag with the ec2:SharedTag/ prefix to a shared AMI in the AWS EC2 console. To learn more, see Tag your AWS EC2 resources in the Amazon EC2 User Guide.

Restart EC2 and on-premises fleets faster with AWS CodeDeploy RESTART deployment mode

2026-10-05 Iskandar Anvarov

Post Syndicated from Iskandar Anvarov original https://aws.amazon.com/blogs/devops/restart-ec2-and-on-premises-fleets-faster-with-aws-codedeploy-restart-deployment-mode/

Operators often restart fleets when they need to pick up runtime configuration changes, recover unhealthy processes, or return hosts to a known state. Before RESTART, AWS CodeDeploy customers either redeployed the current revision, repeating completed work, or ran custom scripts outside of CodeDeploy production safeguards. Now there’s a purpose-built option: RESTART deployment mode. It keeps the operation inside CodeDeploy, so you get the same batch sizing, health checks, alarm monitoring, and rollback behavior as a standard deployment.

This new option reapplies the last successful revision to Amazon Elastic Compute Cloud (Amazon EC2) and on-premises in-place deployments. It retains your deployment configurations, lifecycle hooks, Amazon CloudWatch alarm monitoring, rollback settings, and deployment history.

In testing, a fleet restart completed up to 6.1x faster than a standard deployment, turning a multi-minute rollout into tens of seconds.

In this post, we explain how RESTART works, walk through starting a restart deployment from the command line and the CodeDeploy console, and review the safety controls that carry over from a standard deployment.

Why use a CodeDeploy restart?

A restart changes production even when the application revision stays the same. Hosts stop and start, failures reduce fleet capacity, and external configuration errors spread across restarted hosts. Fleet scripts must recreate batch sizing, minimum healthy capacity, validation, alarm monitoring, and audit history. One customer experienced this firsthand. Their restart script restarted hosts in batches with no health checks in between. When a bad configuration left the first batch unable to start, the script never noticed and continued. What should have been a routine restart took down their fleet.

This new deployment mode keeps the operation in CodeDeploy. You configure how the deployment runs with the familiar CreateDeployment API. CodeDeploy still runs your lifecycle hooks, validates the result, and stops after a health or alarm breach. A failed restart stays within the active batch instead of continuing through the fleet.

RESTART also works well as a building block for automated remediation. Point a CloudWatch alarm on a memory or resource-utilization metric at an Amazon EventBridge rule, and have that rule invoke an AWS Lambda function that calls CreateDeployment with deploymentMode: RESTART. Long-running or stateful workloads benefit from periodic recycling. Examples include self-managed Kafka brokers and JVM services with memory growth. This turns that recycling into a managed, self-healing loop instead of a cron job or a manual restart. The following diagram shows the flow:

CloudWatch alarm triggering an Amazon EventBridge rule that invokes a Lambda function to call CreateDeployment in RESTART mode

Figure 1: Automated remediation loop using an Amazon CloudWatch alarm, an Amazon EventBridge rule, and an AWS Lambda function to trigger a RESTART deployment

How RESTART works

Set deploymentMode to RESTART on CreateDeployment and provide no revision. CodeDeploy resolves the deployment group’s last successful revision and creates a deployment record.

Each selected host runs:

ApplicationStop → DownloadBundle → BeforeInstall → Install → AfterInstall → ApplicationStart → ValidateService

CodeDeploy agents that support local reuse (from version 2.1.0 onward) use the previous deployment’s archive. DownloadBundle remains in the signed workflow. If the archive is unavailable or invalid, the agent downloads the same pinned revision, covering added or replaced hosts.

Install reapplies the revision and corrects drift in managed files. BeforeInstall, AfterInstall, and ValidateService run as defined in the AppSpec file. Local reuse removes the network transfer, and end-to-end savings vary with revision size, agent version, hooks, fleet size, and deployment configuration.

Performance in feature testing

Feature tests peaked at 6.14x. Tests used m5.large instances, a 1 GB revision, and 10, 50, and 100-host fleets with and without an Application Load Balancer (ALB). Each scenario ran seven times. We discarded the minimum and maximum and report the median of five. Results combine local reuse with omitted traffic control and are not predictive of results for other applications.

At 75 percent minimum healthy, four or five waves saved ALB-backed fleets 397.1 to 424.8 seconds. The 50-host fleet reached 6.14x.

Host Count	ALB in Front?	Rollout Waves	Standard Deployment Time	Restart Deployment Time	Speedup	Time Saved
10	Yes	5	526.2s	129.1s	4.08x	397.1s
50	Yes	5	507.4s	82.6s	6.14x	424.8s
100	Yes	4	544.5s	144.7s	3.76x	399.8s
10	No	5	192.4s	167.8s	1.15x	24.6s
50	No	5	190.5s	117.2s	1.63x	73.3s
100	No	4	260.8s	139.6s	1.87x	121.2s

With two-wave HalfAtATime, ALB-backed fleets saved 168.6–185.7 seconds. No-ALB rows isolate local reuse.

Host Count	ALB in Front?	Standard Deployment Time	Restart Deployment Time	Speedup	Time Saved
10	Yes	230.2s	61.6s	3.74x	168.6s
50	Yes	276.2s	90.5s	3.05x	185.7s
100	Yes	275.2s	103.9s	2.65x	171.3s
10	No	133.8s	37.0s	3.62x	96.8s
50	No	152.4s	86.7s	1.76x	65.7s
100	No	162.7s	109.1s	1.49x	53.6s

Multi-wave ALB deployments repeated traffic control and saved the most time. These results carry a couple of safety implications to keep in mind.

The safety controls remain familiar

A RESTART deployment uses the controls already configured for the deployment group:

Deployment configuration: Use CodeDeployDefault.OneAtATime, CodeDeployDefault.HalfAtATime, CodeDeployDefault.AllAtOnce, or a custom minimum healthy host setting to bound concurrent restarts.
Lifecycle validation: CodeDeploy runs ValidateService on each host before it considers that host healthy.
CloudWatch alarms: CodeDeploy polls the alarms configured on the deployment group and stops the deployment when an alarm enters ALARM.
Automatic rollback: CodeDeploy applies the deployment group’s automatic rollback configuration for qualifying failures. A rollback can’t reverse an external configuration change, so correct the underlying configuration before retrying.
Deployment history: GetDeployment and ListDeployments expose the restart’s status, timestamps, revision, and result for monitoring and audit.

Traffic-control behavior

RESTART doesn’t run the load balancer BlockTraffic and AllowTraffic steps. The host remains registered while its application stops and starts. Treat this as an operational constraint, not as the reason to use RESTART.

For request-serving fleets, make ApplicationStop stop accepting new work and drain in-flight work before the process exits. Select a deployment configuration that preserves enough healthy capacity for the expected restart duration. Pull-based workers stop receiving work when the process stops, but their hooks still need to handle in-flight jobs safely.

Walk through a restart deployment

This section walks through starting a restart deployment from the command line and the CodeDeploy console, and then monitoring its progress.

Prerequisites

An existing Amazon EC2 or on-premises application and deployment group with at least one successful deployment.
An AWS Command Line Interface (AWS CLI) or SDK version that supports the deploymentMode request field.
CodeDeploy agent version 2.1.0 or newer on target hosts to benefit from local revision reuse. Earlier agents work but fall back to downloading the revision.

Start a restart deployment with the AWS CLI

The following command restarts the fleet with the deployment group’s default deployment configuration:

aws deploy create-deployment \
    --application-name MyApp \
    --deployment-group-name MyDeploymentGroup \
    --deployment-mode RESTART \
    --description "Restart processes after a runtime configuration update"

CodeDeploy returns a deployment ID:

{
  "deploymentId": "d-EXAMPLE123"
}

To restart one host at a time, provide a deployment configuration in the request:

aws deploy create-deployment \
    --application-name MyApp \
    --deployment-group-name MyDeploymentGroup \
    --deployment-mode RESTART \
    --deployment-config-name CodeDeployDefault.OneAtATime \
    --description "Restart one host at a time"

Omit --deployment-config-name to use the deployment group’s configured default.

Start a restart deployment on the console

You can also use this feature in the CodeDeploy console. Under Applications, select the deployment group you want to restart and choose Create deployment.

CodeDeploy console Applications page with a deployment group selected and the Create deployment button

Figure 2: Choosing Create deployment for a deployment group in the CodeDeploy console

For Deployment mode, select Restart, and configure any other settings or overrides on the page (the same options you would set with the AWS CLI).

CodeDeploy Create deployment page with Restart selected as the deployment mode

Figure 3: Selecting Restart as the deployment mode on the Create deployment page

Monitor the restart

You can see the status of the deployment on the console, or use the deployment ID with the GetDeployment API or CLI command. For example:

aws deploy get-deployment --deployment-id d-EXAMPLE123

The deployment moves through the standard Created, InProgress, and terminal states. If a lifecycle hook fails, the deployment breaches its minimum healthy host requirement, or a configured alarm enters ALARM, CodeDeploy stops the restart and applies the configured failure behavior.

You can distinguish restart deployments by the deploymentMode field in the GetDeployment API, or visually on the console under Deployment details.

CodeDeploy console Deployment details showing the deploymentMode field set to RESTART

Figure 4: Deployment details showing the deploymentMode field set to RESTART

Handle alarms during incident recovery

If an alarm is already in ALARM state, CreateDeployment still creates the deployment. CodeDeploy then stops it when the deployment workflow observes that alarm during polling. The ignorePollAlarmFailure setting does not ignore an alarm in ALARM. It only controls behavior when CodeDeploy cannot retrieve alarm state.

If an approved incident runbook requires a restart despite the current alarm state, override alarm monitoring for that deployment:

aws deploy create-deployment \
    --application-name MyApp \
    --deployment-group-name MyDeploymentGroup \
    --deployment-mode RESTART \
    --override-alarm-configuration enabled=false \
    --description "Restart under an approved incident runbook"

This override disables all deployment-group alarms for that deployment. It requires codedeploy:UpdateDeploymentGroup in addition to the permission to create a deployment. Use it only when your incident process provides another health signal and explicitly authorizes the override. The deployment configuration and lifecycle validation continue to apply.

Request constraints

RESTART has the following constraints:

It supports EC2 and on-premises in-place deployment groups. It doesn’t support Amazon Elastic Container Service (Amazon ECS) or AWS Lambda deployment groups.
The deployment group must have a successful revision for CodeDeploy to reapply.
Don’t provide revision, s3Location, gitHubLocation, or deploymentRevisions. CodeDeploy resolves the revision from deployment history.
Don’t combine RESTART with updateOutdatedInstancesOnly. That option selects hosts that aren’t running the target revision, which conflicts with restarting hosts on the current successful revision.

For exact request validation and error types, see the CreateDeployment API reference.

Conclusion

Restarting a fleet is routine, but it still changes production availability and exposes configuration or process failures. RESTART gives the operation a first-class CodeDeploy path instead of requiring a separate fleet script.

Set deploymentMode to RESTART to reapply the deployment group’s last successful revision. CodeDeploy controls the batch size, runs the lifecycle hooks, validates each host, monitors configured alarms, records the result, and reuses the local revision archive when possible. The operation is faster when the archive is already present, while hosts that need to download it use the normal fallback path.

To get started, review the AWS CodeDeploy documentation and create-deployment CLI reference.

About the authors

Simplify AMI discovery with Amazon EC2 and SSM Parameter Store

2026-09-29 Ashwani Tyagi

Post Syndicated from Ashwani Tyagi original https://aws.amazon.com/blogs/compute/simplify-ami-discovery-with-amazon-ec2-and-ssm-parameter-store/

If you manage Amazon Elastic Compute Cloud (Amazon EC2) infrastructure at scale, you have likely encountered the following situation. You release an infrastructure change with the correct Region, the correct instance type, and a launch template that has operated reliably for months. The deployment nevertheless comes up on an Amazon Machine Image (AMI) that is several patch cycles out of date, because the AMI ID hardcoded in the template had become stale weeks earlier. The condition goes unnoticed until a security scan flags the instance, at which point you must reconcile AMI IDs across Regions rather than close out the week.

That scenario is rarely a one-time event. It is one example of a broader pattern that quietly taxes teams running Amazon EC2 at scale: stale AMI IDs, manual parameter lookups, inconsistent Region mappings, and pipelines that silently fail to update. The following section examines four variations of this pattern in detail.

The common thread across all of these is the same. Locating the correct image is not the hard part. The difficulty lies in wiring that image into your infrastructure as code (IaC) in a manner that remains current. You identify the appropriate AMI on the console, then search AWS Systems Manager (SSM) Parameter Store paths to obtain the dynamic reference that maps to it. The workflow spans two tools and two mental models, with a gap in between where errors accumulate. Because the authoritative link between an AMI and its SSM parameter lived outside the API, teams had to reconstruct it by hand, and hands make mistakes.

A recent enhancement to the Amazon EC2 DescribeImages API closes that gap. When you call DescribeImages on a public AMI, the response now contains a PublicSsmParameterName field: the SSM parameter that resolves to the latest AMI in that lineage. A single API call replaces manual correlation.

In this post, we examine the operational friction that makes AMI management harder than it should be and show how this enhancement addresses it. We walk through practical examples using the AWS Command Line Interface (AWS CLI), AWS CloudFormation, Terraform, and Amazon EC2 Auto Scaling launch templates. We conclude with best practices for golden AMI pipelines, including operational considerations to review before adopting the feature in production.

Prerequisites

To follow the examples in this post, you will need the following:

An AWS account.
The AWS CLI v2 installed and configured with appropriate permissions (ec2:DescribeImages, ssm:GetParameters).
Basic familiarity with AMIs, SSM Parameter Store, and at least one IaC tool (CloudFormation or Terraform).

Understanding the operational challenges

Before addressing the solution, it is worth examining the problem in detail, because the problem seldom manifests as a single, dramatic failure. It is instead a gradual accumulation of minor frictions that, in aggregate, impose a measurable cost on teams responsible for compute.

AMI IDs are Region-specific, version-specific, and change frequently. The workflow of finding an AMI, locating its SSM parameter, and referencing it in templates spans multiple tools, and the boundaries between steps are where errors accumulate.

Challenge 1: Silent image aging

Scenario: An engineer copies an AMI ID into a Terraform module as an interim measure. Several months later, that identifier is embedded across four environments. New instances launch on an image that predates numerous patches. There is no error and no alert, only drift that remains invisible until an audit or a review brings it to light.

Impact: Hardcoded AMI IDs do not fail conspicuously. They fail quietly, by launching a prior image at a later date. The distance between “this was correct when written” and “this remains correct” widens continuously, and no owner is assigned to monitor it.

Challenge 2: The multi-region maintenance burden

Scenario: An application operates across three Regions. The same logical image (for example, the latest Amazon Linux 2023) carries a different AMI ID in each Region. Templates therefore accrue region-to-AMI mapping blocks, lookup logic, or both. Each additional Region introduces another entry to maintain, and each AMI refresh requires updating all of them.

Impact: The team ends up maintaining a translation table that AWS already maintains on its behalf. The mapping logic becomes load-bearing infrastructure in its own right, and a single stale entry in one Region produces inconsistent fleets that are difficult to diagnose.

Challenge 3: Barriers to onboarding

Scenario: A new engineer joins the team and poses a reasonable question: which SSM parameter corresponds to a given AMI? The answer resides in an internal knowledge-base page that was accurate eighteen months earlier. The engineer copies a path that appears correct, deploys, and inadvertently references the wrong lineage.

Impact: When the relationship between an AMI and its parameter is not discoverable from the API, it must be documented manually. Manually maintained mappings degrade over time. Each new team member re-learns the same institutional knowledge, and each instance of degradation introduces an opportunity to reference an incorrect value.

Challenge 4: Uncertainty about update success

Scenario: A golden AMI pipeline completes a build and updates a parameter. The command returns a success response, and the team assumes the new image is in effect. However, for certain parameter data types, a success response does not always indicate that the value was accepted. This specific behavior is examined in the best-practices section, as it is particularly relevant to golden AMI pipelines.

Impact: Confidence without confirmation carries substantial risk. A pipeline that presumes success can propagate a stale image across a fleet before the discrepancy is identified.

Considered individually, none of these situations constitutes a crisis. Considered collectively, they explain why “launch the latest image” is never, in fact, a single step. The common root cause is consistent across all four: the authoritative link between an AMI and its SSM parameter existed outside the API, requiring teams to reconstruct it manually, a process inherently prone to error.

What’s new: DescribeImages returns the associated SSM parameter

The new feature addresses precisely this boundary.

As of July 16, 2026, the Amazon EC2 DescribeImages API response includes a new field, PublicSsmParameterName, for public AMIs that have an associated SSM parameter. This capability is available at no additional cost in supported AWS Regions, including AWS GovCloud (US) Regions and the China Regions.

In place of the previous three-step correlation exercise, the workflow reduces to a single call:

Before	After
Find AMI → manually search SSM paths → confirm the correct match	Find AMI → PublicSsmParameterName returns the SSM path immediately
Two separate API calls or console workflows	A single DescribeImages call provides the complete mapping
Prone to mapping an incorrect parameter to an AMI	Authoritative mapping obtained directly from the API

The change introduces neither a new service nor a new pricing dimension. It relocates information that previously lived in knowledge bases into the API response.

In addition, you can now use the public-ssm-parameter-name filter in DescribeImages to identify all AMIs associated with a specific SSM parameter, making the relationship queryable in either direction.

How it works: API response walkthrough

Call DescribeImages on a public AMI with an associated SSM parameter. The response includes PublicSsmParameterName:

{
  "Images": [
    {
      "ImageId": "ami-0abcdef1234567890",
      "Name": "al2023-ami-2023.7.20260601.0-kernel-6.1-arm64",
      "Description": "Amazon Linux 2023 AMI 2023.7.20260601.0 arm64 HVM kernel-6.1",
      "Architecture": "arm64",
      "PlatformDetails": "Linux/UNIX",
      "State": "available",
      "Public": true,
      "OwnerId": "111122223333",
      "ImageType": "machine",
      "RootDeviceType": "ebs",
      "VirtualizationType": "hvm",
      "EnaSupport": true,
      "PublicSsmParameterName": "aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64"
    }
  ]
}

The PublicSsmParameterName value (in this case, aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64) identifies the SSM parameter associated with this AMI lineage.

Tip: The field is returned under the aws/service/ namespace without a leading slash. When using this value in SSM API calls, resolve:ssm: references, or CloudFormation dynamic references, prepend a forward slash. For example, use /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64. The SSM parameter is intended to resolve to the latest AMI in the lineage, which can help you keep infrastructure current.

Note: Not every public AMI has an associated parameter. The field is present only for lineages for which AWS publishes parameters. The field is also populated only for public AMIs. If you query one of your own private AMIs and observe an empty field, this is expected behavior rather than a defect.

Practical examples

The following four examples show how to use the new PublicSsmParameterName field across common IaC tools.

Example 1: Discover the SSM parameter for an AMI using the AWS CLI

Suppose you have identified an AMI on the console and wish to determine its SSM parameter path for use in your templates:

aws ec2 describe-images \
  --image-ids ami-0abcdef1234567890 \
  --query "Images[0].PublicSsmParameterName" \
  --output text

Output:

aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64

You can also perform the inverse operation and determine which AMI a given SSM parameter currently references:

# Linux example
aws ssm get-parameter \
  --name "/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64" \
  --query "Parameter.Value" \
  --output text

# Windows example
aws ssm get-parameter \
  --name "/aws/service/ami-windows-latest/Windows_Server-2022-English-Full-Base" \
  --query "Parameter.Value" \
  --output text

Tip: Public SSM parameters are available for both Linux (/aws/service/ami-amazon-linux-latest) and Windows (/aws/service/ami-windows-latest) AMIs. You can list all available parameters under these paths using aws ssm get-parameters-by-path –path .

Alternatively, you can use the new filter to identify AMIs by their SSM parameter name:

Note: The public-ssm-parameter-name filter returns all AMIs that have ever been associated with the specified parameter, including previous versions. Use sorting or additional filters (such as –query with CreationDate) to identify the most recent AMI.

aws ec2 describe-images \
  --filters "Name=public-ssm-parameter-name,Values=/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64" \
  --query "Images[].{ImageId:ImageId,Name:Name,CreationDate:CreationDate}" \
  --output table

Example 2: CloudFormation with dynamic SSM references

Once the SSM parameter path is known from DescribeImages, you can use CloudFormation dynamic references to resolve to the latest AMI at deployment time:

AWSTemplateFormatVersion: '2010-09-09'
Description: EC2 instance using SSM parameter for latest Amazon Linux 2023 AMI

Parameters:
  InstanceType:
    Type: String
    Default: t4g.micro

Resources:
  MyInstance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: !Ref InstanceType
      ImageId: '{{resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64}}'
      Tags:
        - Key: Name
          Value: MyLatestAL2023Instance

Alternatively, you can use the AWS::SSM::Parameter::Value parameter type to permit users to override the SSM path at stack creation time:

AWSTemplateFormatVersion: '2010-09-09'
Description: EC2 instance with configurable SSM-based AMI lookup

Parameters:
  AmiSsmParameter:
    Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
    Default: '/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64'
    Description: SSM parameter path for the AMI (discovered via DescribeImages)
  InstanceType:
    Type: String
    Default: t4g.micro

Resources:
  MyInstance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: !Ref InstanceType
      ImageId: !Ref AmiSsmParameter
      Tags:
        - Key: Name
          Value: DynamicAMIInstance

CloudFormation resolves the AMI ID at deployment time, so the template never contains a hardcoded AMI ID, and any stack update adopts the latest AMI automatically. Two considerations warrant attention before relying on this approach. First, running instances are not affected. A stack update is required to roll out a newer AMI. Second, CloudFormation does not support drift detection on dynamic references, so if the underlying SSM parameter value changes between deployments, CloudFormation will not report it as drift. For ssm dynamic references in which a version has not been pinned, AWS recommends performing a stack update whenever the parameter changes, so that the stack retrieves the current value.

Example 3: Terraform with SSM parameter data source

Use the aws_ssm_parameter data source to resolve the SSM path to the latest AMI ID:

# Use the SSM parameter path discovered from DescribeImages
data "aws_ssm_parameter" "latest_al2023" {
  name = "/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64"
}

resource "aws_instance" "web" {
  ami           = data.aws_ssm_parameter.latest_al2023.value
  instance_type = "t4g.micro"

  tags = {
    Name = "LatestAL2023-Instance"
  }
}

Important: In Terraform, ami is a replacement-forcing argument on aws_instance. When the SSM parameter changes, Terraform proposes to destroy and recreate the instance. For stateful workloads, add lifecycle { ignore_changes = [ami] } or use launch templates with Auto Scaling (Example 4) instead.

Example 4: Auto Scaling launch templates with SSM parameters

For Auto Scaling groups, you can reference the SSM parameter directly in the launch template using the resolve:ssm: prefix:

# Create a launch template that uses the SSM parameter
aws ec2 create-launch-template \
  --launch-template-name al2023-auto-scaling \
  --launch-template-data '{
  "ImageId": "resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-arm64",
  "InstanceType": "t4g.micro"
}'

When EC2 Auto Scaling launches a new instance, it resolves the SSM parameter at launch time to obtain the current AMI ID. You can verify the AMI ID to which a launch template resolves:

aws ec2 describe-launch-template-versions \
  --launch-template-name al2023-auto-scaling \
  --versions '$Latest' \
  --resolve-alias

The response shows the resolved ImageId:

{
  "LaunchTemplateVersions": [
    {
      "LaunchTemplateId": "lt-089c023a30example",
      "LaunchTemplateName": "al2023-auto-scaling",
      "VersionNumber": 1,
      "LaunchTemplateData": {
        "ImageId": "ami-0ac394d6a3example",
        "InstanceType": "t4g.micro"
      }
    }
  ]
}

The parameter is stored in the launch template. When the Auto Scaling group scales out or replaces an instance, it uses the launch template to resolve the SSM parameter and determine the AMI to launch. This is the most direct of the four patterns: the parameter serves as the single source of truth, and the Auto Scaling group’s normal instance lifecycle effects the rollout.

Before-and-after workflow comparison

The following table summarizes how this feature improves common workflows, and relates each entry to the challenges described earlier.

Workflow	Before	After
Discover the SSM path for a known AMI	Search SSM parameter namespaces manually. Test multiple paths. Confirm a correct match	A single DescribeImages call returns PublicSsmParameterName
Validate that an SSM parameter maps to the expected AMI	Call GetParameter, then call DescribeImages on the returned ID to verify	Use the public-ssm-parameter-name filter to view all associated AMIs directly
Set up IaC templates	Find AMI → search for SSM path → copy path to template → verify correctness over time	Find AMI → read PublicSsmParameterName from the response → use directly in the template
Onboard new team members	Document AMI-to-parameter mappings in knowledge bases, which become stale	New members self-discover using standard API calls
Audit AMI usage across teams	Cross-reference AMI IDs with SSM parameters in separate calls	A single API call provides the complete picture

Best practices: Using SSM parameters for golden AMI pipelines

The following recommendations describe how to derive the greatest benefit from this feature, with operational considerations identified where they are material.

1. Discontinue hardcoding AMI IDs

With PublicSsmParameterName removing the discovery barrier, switch all templates to SSM parameter references. Use {{resolve:ssm:}} in CloudFormation, the aws_ssm_parameter data source in Terraform (note the replacement behavior in Example 3), or the resolve:ssm: prefix in launch templates.

2. Create custom SSM parameters for your golden AMIs

For internally built golden AMIs, create your own SSM parameters using the aws:ec2:image data type:

aws ssm put-parameter \
  --name "/my-org/golden-ami/amazon-linux-hardened" \
  --type "String" \
  --data-type "aws:ec2:image" \
  --value "ami-0abcdef1234567890"

When the pipeline produces a new golden AMI, update the parameter:

aws ssm put-parameter \
  --name "/my-org/golden-ami/amazon-linux-hardened" \
  --type "String" \
  --data-type "aws:ec2:image" \
  --value "ami-0fedcba9876543210" \
  --overwrite

Stacks, launch templates, or Terraform configurations that reference this parameter can adopt the new AMI on their next deployment, with no template edits required in most cases.

Operational consideration: Because PutParameter validates aws:ec2:image values asynchronously, an HTTP 200 does not confirm the value was accepted. Subscribe to Parameter Store change events in Amazon EventBridge and confirm the operation succeeded before considering the rollout complete.

3. Use parameter versions and labels for controlled rollouts

SSM Parameter Store supports versioning and labels, which provide control over rollouts:

# Label the current production version
aws ssm label-parameter-version \
  --name "/my-org/golden-ami/amazon-linux-hardened" \
  --parameter-version 5 \
  --labels "prod"

Production launch templates reference the labeled version:

ImageId: resolve:ssm:/my-org/golden-ami/amazon-linux-hardened:prod

With this approach, you can update the parameter with a new AMI without immediately affecting production. Promotion to production is accomplished by moving the prod label, a deliberate, auditable action rather than an automatic side effect.

4. Combine with Amazon EC2 Image Builder for end-to-end automation

Use Amazon EC2 Image Builder to automate AMI creation, then configure the distribution settings to update your SSM parameter automatically when a new AMI is built. Combined with the new discovery feature, this establishes a closed loop:

Image Builder creates a new AMI on a schedule.
Distribution settings update the SSM parameter to point to the new AMI.
Auto Scaling and IaC resolve the parameter to the latest AMI at launch time.
With DescribeImages, any authorized party can determine which SSM parameter an AMI maps to.

5. Scope IAM permissions appropriately

Two permission requirements apply.

To launch instances by using SSM-referenced AMIs, the launching principal requires ssm:GetParameters on the relevant parameter paths:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "ssm:GetParameters",
      "Resource": "arn:aws:ssm:*:*:parameter/aws/service/ami-amazon-linux-latest/*"
    }
  ]
}

Scope the Resource element to the paths actually in use. If you reference Windows parameters (/aws/service/ami-windows-latest/) or your own golden AMI paths (/my-org/golden-ami/), include those ARNs as well. Otherwise, launches will fail with an AccessDenied error.

To create a custom aws:ec2:image parameter, the pipeline principal also requires ssm:PutParameter and ec2:DescribeImages:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "ssm:PutParameter",
      "Resource": "arn:aws:ssm:*:*:parameter/my-org/golden-ami/*"
    },
    {
      "Effect": "Allow",
      "Action": "ec2:DescribeImages",
      "Resource": "*"
    }
  ]
}

For broader guidance on keeping infrastructure current and automating operational processes, see the Operational Excellence Pillar of the AWS Well-Architected Framework.

Clean up

The examples in this post use read-only API calls (DescribeImages, GetParameter) and do not create billable resources. If you created a launch template while following Example 4, you can delete it as follows:

aws ec2 delete-launch-template --launch-template-name al2023-auto-scaling

Conclusion

The difficulty of AMI management was never attributable to any single failure. It arose from the steady accumulation of stale identifiers, region-mapping tables, stale documentation, and pipelines that presumed success, all of which are minor frictions that together produced significant operational effort and risk. The common thread was that the authoritative link between an AMI and its SSM parameter existed outside the API, requiring teams to reconstruct it manually.

The new PublicSsmParameterName field in the Amazon EC2 DescribeImages API relocates that link into the response, where it appropriately belongs. With a single API call, you can determine the SSM parameter for any public AMI. You can then reference it directly in CloudFormation templates, Terraform configurations, or Auto Scaling launch templates for automatic AMI updates.

To begin, call DescribeImages on any public AMI and examine the PublicSsmParameterName field. For further detail, see Reference the latest AMIs using Systems Manager public parameters in the Amazon EC2 User Guide.

For additional learning resources on AMI management and IaC on AWS, explore Amazon EC2, AWS Systems Manager Parameter Store, and Amazon EC2 Image Builder.

Frozen package management for air-gapped RHEL-family AMIs

2026-09-29 Anand Krishna Varanasi

Post Syndicated from Anand Krishna Varanasi original https://aws.amazon.com/blogs/compute/frozen-package-management-for-air-gapped-rhel-family-amis/

If you run a regulated, air-gapped compute fleet on RHEL-family instances, you have probably felt three requirements pulling against each other. Your organization must configure the network to remove internet access from the instances. Your team must review and approve new packages or version upgrades before you adopt them. Your team removes public repository definitions, restricts network paths, and configures instances to use only the internal repository your team has approved. Teams in chip design, finance, healthcare, defense, and the public sector often face this combination while still needing operating system updates.

This post describes a two-account pattern that separates the connected package-ingestion path from the air-gapped fleet. You create an authorized initial baseline and approve later changes to form a versioned package snapshot in Amazon Simple Storage Service (Amazon S3). EC2 Image Builder uses the frozen snapshot to build Amazon Machine Images (AMIs). Your team configures AWS Systems Manager Patch Manager to patch the instances your organization runs from the same internal package source.

The accompanying reference implementation demonstrates the pattern for RPM-based RHEL-family systems (AlmaLinux for example). It is a reference, not a substitute for distribution of licensing, vulnerability analysis, testing, or an organization’s change-management process.

The challenge: Getting packages into an air-gapped approval-gated fleet

Common delivery models each assume something an air-gapped fleet might not provide:

Red Hat Update Infrastructure (RHUI) expects each instance to reach the service. A fleet with no internet egress needs a different content path.
Red Hat Satellite supports disconnected content management, but it is a separate product and operational footprint. Teams that need a custom package-level approval workflow must integrate that workflow with their content-management process.
The Red Hat CDN requires a connected, entitled content-management path. Centralizing that path changes the network architecture, not the customer’s Red Hat subscription obligations.

The objective is not to replace these products universally. It is to show a serverless AWS pattern for teams that need an authorized repository baseline, explicit approval for later package changes, and a fleet with no public package source.

How the pattern works

The pattern combines three controls:

A frozen package repository on Amazon S3: The pattern stores a deployment-authorized baseline and subsequent approved package changes in versioned, per-OS repository prefixes. The repository manifest records the package inventory for each state.
EC2 Image Builder Orchestration builds AMIs from that repository: The build helps remove upstream repository definitions and configures the internal frozen mirror to be used for all package operations.
The launched fleet has no internet egress: The dnf operations are configured to resolve the internal mirror. Patch Manager uses the same repository source, so image builds and in-place patching draw from one frozen snapshot.

Choosing the upstream source

Choose one package lineage end to end. The parent AMI, repository content, and trusted signing keys must belong to that same lineage.

The reference implementation defaults to AlmaLinux vault content plus EPEL and an AlmaLinux parent AMI. The AlmaLinux OS Foundation states that AlmaLinux aims for binary and application binary interface (ABI) compatibility with RHEL. This is an AlmaLinux compatibility goal, not a Red Hat certification, and it does not make repository mixing a supported practice.

For genuine RHEL systems, use a Red Hat parent AMI, entitled Red Hat repositories, and Red Hat signing keys. A connected content-management host can retrieve content for the isolated environment. This centralizes the network path but does not reduce or change the customer’s Red Hat subscription obligations. Confirm those obligations against the applicable Red Hat agreement.

Do not pair AlmaLinux repositories with genuine RHEL hosts, or Red Hat repositories with AlmaLinux hosts. Mixed-vendor package lineages can create support, stability, and maintainability problems even when the RPMs appear mechanically compatible.

Architecture and workflow

The account boundary provides a primary security boundary for this architecture. The following diagram shows the connected Distribution account, the read-only Workload account, and an example cross-Region layout.

Two-account architecture showing the connected Distribution account with the control plane and internet path, and the air-gapped read-only Workload account, spanning two Regions

Figure 1: Two-account, cross-Region architecture separating the connected Distribution account from the air-gapped Workload account

The Distribution account owns the writable control plane and the only internet path. It runs Amazon EventBridge, three AWS Lambda functions, Amazon DynamoDB, Amazon Simple Notification Service (Amazon SNS), and the AWS Fargate sync task. It also owns the frozen S3 repository and its AWS Key Management Service (AWS KMS) key.

The Workload account is air-gapped and read-only with respect to the repository. It runs the internal HTTPS mirror, EC2 Image Builder, Patch Manager, and the compute fleet. Its mirror task role can read and decrypt frozen content but cannot write it.

The sample repository places the Distribution control plane in US East (N. Virginia), the frozen store in US West (Oregon), and the Workload resources in US West (Oregon) to demonstrate API-only cross-account and cross-Region operation. This Region split is not required. In most deployments, place the Distribution control plane and frozen store in the same Region unless data residency, disaster recovery, or an existing regional footprint justifies the additional latency, transfer cost, and KMS policy complexity.

The two accounts do not need Amazon Virtual Private Cloud (VPC) peering or a transit gateway. Cross-account access uses S3, KMS, and IAM policies. The VPC address ranges can overlap because no VPC-to-VPC route is required.

Package baseline and scheduled upgrade workflow

Before the scheduled workflow begins, your organization must authorize and run a full sync to establish the initial repository baseline. This bootstrap does not provide package-by-package approval. If your organization requires individual approval for every initial RPM, your team should generate and review the baseline manifest before promotion instead of relying solely on deployment authorization.

After the baseline, the detector runs on a customer-defined schedule. The reference implementation defaults to monthly. The following diagram shows the bootstrap distinction and the selective approval flow.

Workflow diagram distinguishing the initial baseline bootstrap sync from the recurring detect, request approval, review, record, selective sync, and manifest update steps

Figure 2: Package baseline bootstrap and the scheduled selective approval workflow

Detect. Amazon EventBridge invokes the detector Lambda function on the configured schedule. The detector compares upstream repository metadata with manifest.json, which records the current frozen inventory. It classifies a newer version as an upgrade and an absent package as new.
Request approval. The detector writes candidates to S3, creates a KMS-protected review token carrying the request ID and expiry, and is designed to send a review link through SNS. The detector can use kms:Encrypt but not kms:Decrypt.
Review. A human opens the review page through an Amazon API Gateway HTTP API, reviews the proposed package versions, and chooses which changes to approve. The approver can use kms:Decrypt but not kms:Encrypt.
Record and start. A conditional DynamoDB update changes a request from pending to approved only once. The approver then starts the Fargate sync task and passes the request ID.
Selective sync. The task reads the approved package list, downloads those package versions, is designed to perform verification checks, and regenerates repository metadata.
Update the manifest. When the task stops, Amazon EventBridge invokes the manifest-updater Lambda function. It archives the outgoing manifest and records the resulting repository inventory.

The approval decision controls adoption. It does not prove that package code is safe. Advisory review, vulnerability scanning, testing, and staged rollout remain in separate controls.

Evidence from the approval workflow

The token ties a review action to a specific request and expiry. Separating kms:Encrypt from kms:Decrypt prevents either Lambda function from performing both token roles. The conditional DynamoDB write makes the approval transition single-use.

DynamoDB records request state, AWS CloudTrail records control-plane API activity, and manifest history records repository inventory changes. These service records can feed the organization’s existing audit and evidence-management workflow. Object-level S3 access auditing requires CloudTrail S3 data events. KMS activity alone is not a substitute for those events.

The frozen package repository on Amazon S3

The following diagram shows the per-OS, per-component prefix layout, and manifest objects.

Amazon S3 prefix layout with one prefix per operating system version, each holding BaseOS, AppStream, and EPEL components with Packages and repodata trees plus manifest objects

Figure 3: Per-OS, per-component prefix layout of the frozen repository on Amazon S3

Each pinned operating system version receives its own prefix. Repository components such as BaseOS, AppStream, and EPEL contain Packages/ and repodata/ trees. manifest.json records the active inventory, and archived manifests preserve historical evidence and comparison points.

Your organization configures the bucket with versioning and SSE-KMS. Public RPM content does not require a customer-managed KMS key for confidentiality, so your organization could instead configure SSE-S3 for encryption at rest. However, SSE-S3 would remove the separate cross-account authorization control provided by the customer-managed KMS key policy. The customer managed key is used here for explicit cross-account key-policy control and revocation, and the manifests reveal the fleet’s exact software inventory. S3 Bucket Keys reduce KMS request volume. If object-level access evidence is required, enable CloudTrail S3 data events.

A rollback must restore a coherent repository state, including metadata and any required object versions. Restoring only manifest.json does not roll back repository contents.

Building, patching, and running the fleet

At AMI build time, an Image Builder component installs the configured repository keys, moves existing repository definitions aside, and writes one frozen repository definition per component. It locks the package manager to the frozen repository directory, fetches metadata through the internal mirror, and fails the build if the mirror validation step fails. An optional curated package list demonstrates that the AMI can install real packages through the frozen path.

Patch Manager uses the same mirror for the running fleet. A host created from an older AMI and a newly built host are therefore patched toward the same frozen snapshot. Instances run without an Amazon VPC NAT gateway, public IP, or an Amazon VPC internet gateway route in the Workload VPC, and their repository configuration contains no public fallback.

The intended verification model is defense in depth: the sync task helps verify a vendor’s signature before content enters the trusted repository, and the system verifies it again at installation through dnf. The ingestion gate helps reject digest-only results and can be configured to help confirm that only valid package signatures from a trusted lineage key are accepted.

The package mirror

Nginx fronts aws-sigv4-proxy, which signs cross-account S3 GET requests using the mirror task role. To a client, the service appears as a standard HTTPS package repository behind an internal Application Load Balancer and private DNS name.

Use the latest version of aws-sigv4-proxy (current latest is v1.12). This version 1.12 contains the fix for signing S3 paths (or the OS package names) with special characters such as +. Earlier versions can return SignatureDoesNotMatch. The reference implementation pins the reviewed v1.12 release commit immutably. Keep it current through dependency-update reviews.

Security boundaries and limits

The design provides the following controls:

No automatic public-repository adoption: A new upstream version enters the selective path only after an explicit, recorded decision by the user.
Repository ingestion and installation checks: You configure strict sync-time signature validation to help validate content before it enters the trusted store. dnf verifies again during installation.
No package-channel egress: Workload instances are configured to prevent access to public package sources.
A read-only workload boundary: A Workload-account principal cannot modify the frozen repository.

Human approval is not a malware detection. A reviewer cannot reliably identify a backdoor in a legitimately signed package merely by seeing its name, version, or changelog. Use vulnerability intelligence, scanning, pre-production tests, and staged deployment as additional controls.

The approval state, manifests, and CloudTrail records can help support evidence for control frameworks such as SOC 2 change management, ISO 27001 patch-management controls, and FDA 21 CFR Part 11 electronic records. Applicability depends on the organization’s environment, audit scope, and assessor. Confirm it with the compliance team under the AWS shared responsibility model.

Cost and operations

Cost depends on the amount of repository content and the chosen networking and availability design. Components can include S3 storage and requests, KMS requests, Lambda invocations, DynamoDB, SNS, Fargate tasks, the internal load balancer, Distribution-account internet egress, Amazon VPC endpoints, and AMI snapshots. A three-task always-on mirror costs more than an S3 bucket alone. Estimate the target topology with current AWS pricing rather than applying a fixed monthly figure from the sample.

Run detection and review at an interval defined by patch policy and risk tolerance. The supplied default is monthly, but the Terraform input is configurable. If a package change must be reversed, restore a tested, coherent repository version and rebuild or patch affected hosts as appropriate.

Prerequisites

To set up the reference implementation, work through these in order:

Two AWS accounts: a connected Distribution account and an air-gapped Workload account.
Deployment tools: Terraform 1.5 or later, Terragrunt, Finch or Docker, and Python with pip.
AWS Command Line Interface (AWS CLI): one named profile per account.
Distribution networking: private subnets with internet egress that works without public IPs, security-group egress on port 443, and DNS resolution for public names.
Workload networking: VPC interface endpoints for ssm, ssmmessages, ec2messages, logs, kms, and imagebuilder, plus an S3 gateway endpoint.
Internal mirror identity: an AWS Certificate Manager (ACM) certificate and a private hosted zone. If the parent AMI does not trust the issuing CA, configure the CA file so the build installs the trust anchor.
Package lineage: a parent AMI, repositories, and signing keys from the same distribution lineage. A RHEL subscription is required when retrieving genuine entitled Red Hat content.
Optional deployment roles: otherwise, the stack uses each profile’s credentials.

The deployment creates state backend and Amazon Elastic Container Registry (ECR) repositories. Do not create those ECR repositories separately before applying their own Terraform units.

Reference implementation

The companion repository provides Terraform modules, Lambda handlers, two container images, a Terragrunt two-account layout, and Makefile targets for deployment and verification. The shipped alma810 example defaults to the AlmaLinux lineage (RHEL family).

Choose one distribution lineage before deployment:

AlmaLinux Parent Image default: use an AlmaLinux parent AMI, AlmaLinux vault repositories, EPEL, and the included AlmaLinux and EPEL signing keys. No Red Hat subscription is required.
Genuine RHEL Parent Image: use a Red Hat parent AMI, an entitled Red Hat content source, and Red Hat signing keys. The customer supplies the Red Hat subscription and content-access integration.

Note: The reference implementation only provides AlmaLinux lineage setup, not genuine RHEL. If you choose to use a genuine RHEL parent image lineage, only the reference implementation code needs to be updated to fetch the Red Hat credentials or subscription access, and the rest of the workflow remains the same.

Please follow the repository README for detailed setup instructions:

Fill in environments/config.hcl and both account files with the two accounts, networking, mirror certificate, package lineage, and parent AMI.
Create the Terraform state backend with make bootstrap DIST_PROFILE=<dist> WORK_PROFILE=<work>.
Review both account plans with make plan DIST_PROFILE=<dist> WORK_PROFILE=<work>.
Deploy in dependency order with make all BASELINE_APPROVED=true DIST_PROFILE=<dist> WORK_PROFILE=<work>. The baseline sync can take several hours.

Validate the internal package management workflow

Validation begins by running the AlmaLinux EC2 Image Builder pipeline. A successful build produces private, encrypted AMIs in the configured AWS Regions. To validate the configuration, launch a test instance from the generated AMI in a no-egress Workload subnet and verify that the instance is connected to the internal frozen repository for all package management workflow.

Terminal output listing only the internal frozen-baseos, frozen-appstream, and frozen-epel repositories enabled, with dnf makecache downloading metadata through the private mirror

Figure 4: Instance showing only the internal frozen repositories enabled, with no public repository configured

The output confirms that only the internal frozen AlmaLinux repositories are enabled: frozen-baseos, frozen-appstream, and frozen-epel. The dnf makecache command successfully downloads metadata for all three repositories through the private mirror. No public repository is configured or used.

Now try installing, upgrading, or installing a new package to test that package operations are served by the internal frozen repository.

Terminal output showing a package install and upgrade completing successfully through the internal frozen repository mirror

Figure 5: Package install and upgrade served by the internal frozen repository

Recommended practices

Fail closed on approval data. If the approved package list cannot be retrieved, stop the sync task and avoid substituting a full sync.
Govern the initial baseline. Record who authorized the bootstrap full sync, or require explicit review of its manifest before promotion.
Require vendor signatures at ingestion. Do not treat a valid package digest as equivalent to a trusted signature.
Keep package lineages consistent. Parent AMI, repository content, and signing keys must come from the same distribution lineage.
Use a customer-defined review cadence. Monthly is only the sample default.
Use immutable dependency references with active updates. Require aws-sigv4-proxy v1.12 or later, pin the reviewed artifact by digest or full SHA, and automate update proposals.
Test rollback as a repository operation. Restore metadata and objects together, then validate the mirror before using the restored state.

Clean up

The walkthrough deploys billable resources in both accounts. Please follow the repository README for detailed setup cleanup instructions.

Conclusion

This pattern separates connected package ingestion from an air-gapped fleet, provides an authorized package baseline and an explicit decision point for later package changes or upgrades, and keeps image builds and running hosts on one frozen repository snapshot. To learn more, visit the EC2 Image Builder service page, the EC2 Image Builder documentation, the Patch Manager documentation, and the Amazon S3 user guide. The reference implementation is available in aws-samples.

Announcing Spark Connect on Amazon EMR on EC2: Interactive PySpark anywhere

2026-09-23 Al MS

Post Syndicated from Al MS original https://aws.amazon.com/blogs/big-data/announcing-spark-connect-on-amazon-emr-on-ec2-interactive-pyspark-anywhere/

Today, we’re announcing support for Spark Connect on Amazon EMR on EC2 with the AWS runtime for Apache Spark (emr-spark-8.0, Apache Spark 4.0.2 and later). You can now develop and debug PySpark interactively from Amazon SageMaker Unified Studio Data Notebooks or your own IDE, such as Visual Studio Code, PyCharm, Kiro, or Jupyter. Spark runs on a dedicated Amazon EMR on EC2 cluster while your Python runs locally, so you can set breakpoints and inspect a DataFrame against full-size data from your IDE. In SageMaker Unified Studio Data Notebooks, you connect to your cluster, catalog, and AI tools. Production-scale PySpark and SQL run without leaving the studio. Because each session is isolated with its own permissions, your whole team can share one cluster at the same time. This post shows you how to get started with both SageMaker Unified Studio Data Notebooks and your own IDE.

Previously, developing Spark for an Amazon EMR on EC2 cluster meant working in a notebook tied to that cluster, or packaging your code as a job and submitting it before you could see a result. Local code often behaved differently on the cluster because of version and dependency mismatches, and the slow deploy-and-check loop made those differences hard to find. There was no way to attach your own IDE and debugger and inspect a DataFrame mid-transformation. Spark Connect closes that gap: your code runs against the cluster’s own Spark engine while you develop locally, so the environment you debug in is the one that runs your data.

How Spark Connect works on Amazon EMR on EC2

Spark Connect uses a client-server architecture that separates your application code from the Spark engine. The client is a lightweight PySpark library that runs in your notebook or IDE, and it sends DataFrame and SQL operations over a gRPC/TLS connection to a Spark Connect Server on your cluster. The server runs those operations and returns the results to your local session. Your machine does not need Spark installed and does not need to be sized for the workload.

Spark Connect client-server architecture connecting a local PySpark client to the Spark Connect Server on an Amazon EMR cluster

Figure 1: Spark Connect client-server architecture on Amazon EMR on EC2

When you start a session, Amazon EMR launches the Spark Connect Server as a YARN application on your cluster and hands back an endpoint and a short-lived token. There’s no server for you to stand up or manage. Because that server runs on a cluster you already operate, your session inherits the instance types, libraries, bootstrap actions, and Spark configuration you use in production. What you see while debugging is what runs when the same code is scheduled as a batch job, since both use the same cluster and its configuration.

Now that you can start sessions, a single dedicated cluster can serve your whole team, because each session is a separate resource with its own execution role, tags, and lifecycle. A single cluster supports up to 1,000 concurrent sessions and 1,000 concurrent execution roles. These values are service maximums, not sizing targets. Actual concurrency depends on cluster size and per-session workload. Because interactive sessions are bursty and rarely all active at once, one cluster typically serves a team larger than its peak concurrent-session count. Enable Amazon EMR managed scaling so that capacity tracks demand. If peak concurrency approaches these maximums, or to isolate cost and data access by group, use multiple clusters—for example, one per team, business unit, or environment. Sharing one cluster gives you:

On-demand Spark without extra clusters — Developers get interactive sessions without provisioning a cluster apiece, which keeps utilization high and removes the cost of idle per-person clusters.
Consistent environments — Everyone runs the same Spark version, libraries, and security configuration, so results stay consistent, and your platform team patches and monitors one cluster.
Isolation and attribution — Per-session execution roles and tags keep each person’s work separate, so you can scope data access by session, track cost by user, and stop one session without disturbing anyone else.
Full visibility and control — View active sessions in the Spark UI, review finished ones in the Spark History Server, and manage them from the Amazon EMR console, API, CLI, or SDK.

Getting started

Getting started with Spark Connect on Amazon EMR on EC2 takes three steps: Create an Amazon EMR cluster with Spark Connect session enabled, start a session, and connect from your IDE or SageMaker Unified Studio Data Notebooks.

Note: In SageMaker Unified Studio, on-demand cluster creation is available for domains that use AWS IAM Identity Center. For domains that use AWS Identity and Access Management (IAM), attach an existing cluster. If your cluster runs in a private subnet, make sure that your network configuration allows connectivity between SageMaker Unified Studio and the cluster endpoint.

Prerequisites

You must have the following prerequisites in place.

An Amazon EMR cluster running release emr-spark-8.0.0 or later with SessionEnabled set to true.
The Spark application is installed on the cluster.
Python 3.9 or later with pyspark[connect] installed locally. The PySpark version must match the Spark version on your cluster.
For clusters in private subnets, the Amazon EMR service role must include the AmazonEMRServicePolicyForSessions managed policy, which grants permissions to create Network Load Balancers and virtual private cloud (VPC) endpoint services in your account.
To use Spark Connect sessions, you need permissions to start and list sessions on the cluster (elasticmapreduce:StartSession, ListSessions), get session details and endpoints and terminate sessions (elasticmapreduce:GetSession, GetSessionEndpoint, TerminateSession), and pass the execution role to the Amazon EMR service (iam:PassRole).

Working with interactive sessions

To create a session-enabled cluster and connect to it, follow these steps.

To start a Spark Connect session

Create a cluster with sessions enabled, running emr-spark-8.0.0 or later. The following is a sample command that you can modify for your needs, such as the instance types and counts:

aws emr create-cluster \
  --name "spark-connect-cluster" \
  --release-label emr-spark-8.0.0 \
  --applications Name=Spark \
  --service-role EMR_DefaultRole \
  --ec2-attributes InstanceProfile=EMR_EC2_DefaultRole,SubnetId=subnet-id \
  --instance-groups '[
    {"InstanceCount":1,"InstanceGroupType":"MASTER","InstanceType":"m8g.xlarge"},
    {"InstanceCount":2,"InstanceGroupType":"CORE","InstanceType":"m8g.xlarge"}
  ]' \
  --session-enabled \
  --tags Key=for-use-with-amazon-emr-managed-policies,Value=true

Note: The following steps use the AWS Command Line Interface (AWS CLI) directly. If you develop in SageMaker Unified Studio (Option 1), cluster attachment and session creation are handled for you, so you can skip steps 2 through 6.

After the cluster reaches the WAITING state, start a session and wait for it to reach IDLE:
```
aws emr start-session --cluster-id j-XXXXXXXXXXXXX --name "my-session"
aws emr get-session --cluster-id j-XXXXXXXXXXXXX --session-id is-XXXXXXXXXXXXX
```
Note: For runtime role sessions, add the --execution-role-arn parameter to the start-session command.

Retrieve the endpoint and token, and build your connection string from the returned Endpoint value rather than hardcoding a host:

aws emr get-session-endpoint --cluster-id j-XXXXXXXXXXXXX --session-id is-XXXXXXXXXXXXX

The response includes the endpoint URL and an authentication token:

{
  "Endpoint": "https://session-id.emr-spark-connect.region.amazonaws.com",
  "AuthToken": "v2.local.xxx...",
  "AuthTokenExpirationTime": "2026-01-01T01:00:00Z"
}

Install the matching PySpark client and connect. GetSessionEndpoint returns an https:// URL with no port. Build the connection string by converting it to the sc:// scheme and appending :443. Without the port, the PySpark client defaults to 15002, which isn’t reachable. Your Python code runs locally. The SQL and DataFrame operations run on the cluster:

pip install 'pyspark[connect]==4.0.2' boto3

from pyspark.sql import SparkSession

session_id = "is-XXXXXXXXXXXXX"
auth_token = "<AuthToken from get-session-endpoint>"
host = "<Endpoint from get-session-endpoint, without https://>"

url = f"sc://{host}:443/;use_ssl=true;x-aws-proxy-auth={auth_token};authorization={session_id}"
spark = SparkSession.builder.remote(url).getOrCreate()
spark.sql("SELECT 'Hello from EMR on EC2' AS message").show()

Run a transformation against full-size data. This groups a DataFrame, writes the result to Amazon Simple Storage Service (Amazon S3), and reads it back:

import pyspark.sql.functions as F

df = spark.range(0, 1000).withColumn(
    "category", F.when(F.col("id") % 2 == 0, "even").otherwise("odd")
)
df.groupBy("category").count().show()
df.write.mode("overwrite").parquet("s3://amzn-s3-demo-bucket/demo/")
spark.read.parquet("s3://amzn-s3-demo-bucket/demo/").filter("id < 50").orderBy("id").show()

When you finish, terminate the session to release cluster resources. Calling spark.stop() only closes the local connection. The session keeps running until you terminate it or it reaches the idle timeout:
```
aws emr terminate-session --cluster-id j-XXXXXXXXXXXXX --session-id is-XXXXXXXXXXXXX
```
When you’re done with the walkthrough, terminate the cluster you created in step 1 so it stops incurring charges. Terminating the cluster also ends any sessions still running on it:
```
aws emr terminate-clusters --cluster-ids j-XXXXXXXXXXXXX
```

You can start a Spark Connect session in two ways: from SageMaker Unified Studio or from your own IDE client.

Option 1: Develop in SageMaker Unified Studio Data Notebooks

Amazon SageMaker Unified Studio brings your data, catalogs, and analytics and AI tools into one place, and Amazon EMR on EC2 is now one of the Spark runtimes a Data Notebook can use. When you choose that cluster as the notebook runtime, SageMaker Unified Studio connects to it over Spark Connect. The same runtime then drives both your PySpark and SQL cells, so a single notebook can query the AWS Glue Data Catalog and transform the data without switching tools. The built-in AI assistant generates code and execution plans from natural-language prompts, and the Spark UI shows running work alongside your other runtimes.

To start a session from SageMaker Unified Studio:

Open a Data Notebook in SageMaker Unified Studio.
In the Compute panel, do one of the following:
1. To create a new cluster, choose Create cluster and configure an Amazon EMR on EC2 cluster.
2. To use an existing cluster, choose Attach cluster and select a running Amazon EMR on EC2 cluster.
Select the cluster as the notebook’s runtime.
Begin writing PySpark or SQL code in the notebook cells.

For a complete example, open the SageMaker Unified Studio Spark Connect example notebook , which connects a Data Notebook to an Amazon EMR on EC2 cluster and runs PySpark and SQL cells against the AWS Glue Data Catalog.

Watch a walkthrough: Develop in a SageMaker Unified Studio Data Notebook. The preceding steps cover the same workflow, so you can complete it from the notebook without the video.

Option 2: Develop in your own IDE

Use the IDE of your choice, such as Visual Studio Code, PyCharm, Kiro, or a local Jupyter notebook. You debug Spark the way you debug any Python program: set a breakpoint, inspect a variable, and step through your code, all while the Spark work runs on the cluster. Your libraries, source control, and continuous integration and continuous delivery (CI/CD) stay on your local machine, and only your Spark operations are sent to the cluster.

To see this end to end, the following example attaches an IDE to a Spark Connect session and steps through a breakpoint against cluster data.

Open the local IDE Spark Connect example notebook then use the connection steps in the preceding Getting started section to attach your client.

Watch a walkthrough: Develop your own IDE with Spark Connect. The written connection steps in Getting started cover the same workflow, so you can complete it without the video.

Use cases

Spark Connect on Amazon EMR on EC2 supports the following interactive workflows:

Interactive extract, transform, and load (ETL) development: Build and test pipelines against full-size data on the cluster, then schedule the same transformations as a Spark step on that cluster, where the Spark version, libraries, and configuration already match what you validated.
Exploratory data analysis and feature engineering: Analyze production-scale data from your notebook or IDE instead of sampled subsets, so you catch data quality issues earlier.
Notebook-driven analytics in SageMaker Unified Studio: Run PySpark and SQL next to your catalogs and AI tools, switching runtimes per notebook.
Apache Iceberg lakehouse analytics: Query and manage Iceberg tables through the AWS Glue Data Catalog, with time travel, schema evolution, and partition management.
Compute standardization: Point interactive development at the same clusters that run your production batch jobs, so development and production share one engine and configuration.

Release information

Spark Connect on Amazon EMR on EC2 is available with the AWS runtime for Apache Spark (emr-spark-8.0, Apache Spark 4.0.2) and later. It’s available in all AWS Regions where Amazon EMR is available, except the AWS GovCloud (US) Regions and the China Regions. The SageMaker Unified Studio experience is available in its supported Regions. There’s no additional charge for Spark Connect. You pay for the Amazon Elastic Compute Cloud (Amazon EC2) instances in your cluster. Because these sessions run on your own clusters, they use the Amazon EMR on EC2 capabilities you already rely on, including AWS Graviton processors for price-performance and your choice of On-Demand, Reserved, AWS Savings Plans, or Spot capacity.

Considerations for the release are as follows:

The PySpark version that you install locally must match the Apache Spark version on your cluster.
Spark Connect supports the DataFrame and SQL APIs. RDD-based APIs aren’t supported.
Authentication tokens expire after 1 hour, and sessions end after a configurable idle timeout (60 minutes by default, up to 24 hours).
High-availability clusters with multiple primary nodes, Trusted Identity Propagation, and fine-grained access control through AWS Lake Formation aren’t supported for Spark Connect sessions in this release.

Conclusion

Spark Connect on Amazon EMR on EC2 brings interactive, debuggable PySpark development to the clusters you already run. Develop on a SageMaker Unified Studio Data Notebook or in your own IDE, debug against full-size data while the cluster runs the work and share a single cluster across your whole team. To get started, see the Interactive sessions with Spark Connect guide or open a Data Notebook in Amazon SageMaker Unified Studio. To learn more about the service, see the Amazon EMR detail page.

About the authors

AWS Weekly Roundup: AWS Builder Center mobile apps, Amazon Connect Talent GA, Amazon Corretto 27, and more (September 21, 2026)

2026-09-21 Esra Kayabali

Post Syndicated from Esra Kayabali original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-aws-builder-center-mobile-apps-amazon-connect-talent-ga-amazon-corretto-27-and-more-september-14-2026/

Living in the Netherlands, I spend a fair amount of time on trains, and that is usually where I catch up on what the builder community is writing. Until now, that meant opening a laptop or squinting at a browser tab on my phone. This week I found myself scrolling through trending articles and checking a workshop from the AWS Builder Center mobile app while waiting for a delayed train, and it made those spare twenty minutes very useful. That is why I am glad to open this week with the Builder Center mobile app.

AWS Builder Center is now available as a mobile app on iOS and Android, extending the experience beyond desktop and web. Using your AWS Builder ID, you stay signed in across sessions and can browse trending articles, access 600+ AWS Skill Builder courses, and manage hands-on workshops with free sandbox environments from your mobile device. You can follow AWS Heroes, Community Builders, and User Group Leaders, check Builder Loft event calendars on the go, and receive push notifications for subscribed topics and communities. The app also supports the Wishlist feature for submitting product feedback directly to AWS teams. It is available worldwide on the Apple App Store and Google Play Store.

Builder Center also added two features this week. Polls give you a way to ask the community a question from the Home feed: write a question, add 2 to 5 answer options, set a deadline, and people vote, with results updating live and discussion happening in the comments. Votes are anonymous, and creators see aggregate counts and percentages only. Separately, the Zero to Shipped hackathon is open from September 18 to October 2. You connect your coding agent to AWS, build a real application, and ship it live on AWS for a chance to win a share of a $28,000 prize pool. Five winning projects each receive $5,000 in AWS credits and an AWS Builder swag bundle.

Last week’s launches

Here is what else happened this week.

Amazon Connect Talent is now generally available – Amazon Connect Talent is an AI-powered hiring solution for talent acquisition teams managing hiring at scale. Informed by decades of Amazon hiring science, it uses AI agents to conduct structured voice interviews, administer evidence-based assessments, and score candidates consistently, so recruiters can focus on final decisions. Candidates interview 24/7 from any device, and recruiters review scores, transcripts, and detailed evaluations the next morning. All candidate data is anonymized during AI evaluation, each competency is scored against a rubric with every score tied to specific evidence from the interview, and recruiters keep final decision authority over every hire. General availability includes competency-based assessments, AI-led voice interviews with adaptive questioning, a brand-customizable mobile-first candidate portal, and admin onboarding tools.
Amazon Corretto 27 is now generally available – Amazon Corretto 27, a Feature Release version of the no-cost, multi-platform distribution of OpenJDK, is now available for download on Linux, Windows, and macOS, with support through April 2027. Notable features include G1 as the default garbage collector across all environments (JEP 523), post-quantum hybrid key exchange for TLS 1.3 (JEP 527), compact object headers by default for a smaller memory footprint (JEP 534), and JFR in-process data redaction to remove sensitive data from Java Flight Recorder recordings before they leave the JVM (JEP 536). It also continues previews of enhanced pattern matching, structured concurrency, and lazy constants, along with the Vector API incubator.
Kimi K3 by Moonshot AI is now generally available on Amazon Bedrock – Kimi K3 is now available on Amazon Bedrock for coding and knowledge work. According to Moonshot AI, Kimi K3 is its most capable model and the first open model to reach 2.8 trillion parameters. It combines native vision capabilities with a 1-million-token context window, making it well suited to long-running coding sessions across large repositories, multi-document analysis, and extended agent workflows. Moonshot AI reports an approximate 2.5x improvement in scaling efficiency over Kimi K2. Kimi K3 is the first open-weight model on Amazon Bedrock to support explicit prompt caching, which helps reduce latency and input costs when reusing context across model calls.
AWS reimagines the getting started experience – We announced a new simplified experience for builders starting a new project. Instead of completing configuration tasks first, you start with sensible defaults: sign up using an existing identity from providers including Google, GitHub, and Apple, and for most new customers no credit card is required, with $100 in free credits as part of the AWS Free Tier. AWS organizes your work in a project, which contains an AWS account and sharing settings, and applies security controls for you. You can invite collaborators by email without setting up IAM users, set a monthly spend limit starting at $20, and activate advanced AWS features later at no additional cost with no migration. The experience is gradually rolling out to new customers.
New low-cost burstable Amazon EC2 T8i instances are generally available – Amazon EC2 T8i instances, powered by custom sixth-generation Intel Xeon Scalable processors (Granite Rapids), are among the lowest-cost EC2 instances and deliver up to 30% better price performance over previous-generation T3 instances. They are designed for low-to-moderate CPU utilization workloads such as microservices, low-traffic websites, development and testing environments, and small databases. T8i instances deliver up to 70% higher compute performance, up to 1.25x higher network bandwidth, and up to 2.4x higher Amazon EBS bandwidth compared to T3, and they use the same CPU credit system, so upgrading from T3 is straightforward.
AWS Elastic Beanstalk introduces Cluster Mode – AWS Elastic Beanstalk Cluster Mode is a new fully managed option for teams running a portfolio of applications on shared infrastructure powered by Amazon EKS. Instead of operating each application in isolation, you run multiple applications through one experience with a single operational baseline, so per-application cost decreases as your portfolio grows. You can upload source code in Java, .NET, Python, Node.js, PHP, Ruby, or Go, and Elastic Beanstalk handles containerization automatically through Cloud Native Buildpacks when needed. Cluster Mode includes production-grade deployment strategies with automatic rollback, event-driven autoscaling, AWS Secrets Manager integration, native OpenTelemetry observability, and AI-powered troubleshooting. Standard and Cluster Mode environments run side by side within the same application, so teams can migrate one environment at a time.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news

Here are some additional posts you may find useful:

Building in the AWS European Sovereign Cloud – Two new posts cover building on the AWS European Sovereign Cloud, an independent cloud for Europe that runs as a distinct partition with its own control plane, IAM, billing, console, and service endpoints, and its first Region in Brandenburg, Germany. The first post walks through architecting a secure landing zone, covering account structure and governance, identity as infrastructure as code, centralized logging, data protection, and partition-aware ARN construction that works across AWS partitions. The second announces the general availability of Gemma 4 open-weight models on the Amazon Bedrock next-generation inference engine in the AWS European Sovereign Cloud, with inference staying entirely within eusc-de-east-1 under a zero data retention and zero operator access model.
The new AgentCore runtime: elastic, optimized, and consistently fast starts – We announced a new version of the Amazon Bedrock AgentCore runtime, the managed compute layer for running agents. The new runtime reclaims memory as a session releases it rather than holding it at the peak, so the bill tracks real usage over the life of a session. It also delivers consistent cold start times regardless of container image size or concurrency by preparing the environment once, snapshotting it, and restoring that snapshot for each new instance. In testing with an empty echo agent, the new runtime delivered a P75 cold start of about 2 seconds from a 200 MB image up to 2 GB, compared to roughly 5.4 to nearly 30 seconds for the original runtime.
Introducing the updated AWS Well-Architected Streaming Media Lens – We published a revised Streaming Media Lens, which provides architectural best practices for video streaming workloads. The revision expands from the original 2021 version to cover five streaming scenarios, including interactive live streaming with Amazon IVS Real-Time Streaming for up to 25,000 concurrent viewers, low-latency live streaming, and ad-supported content monetization, alongside enhanced video-on-demand and live streaming guidance. It also adds new sustainability best practices focused on reducing carbon footprint, expanded observability and incident-response frameworks, and advanced content protection with multi-layered DRM and forensic watermarking. The lens whitepaper and custom lens are available now.

For a full list of AWS blog posts, be sure to keep an eye on the AWS Blogs page.

Upcoming AWS events

Check your calendar and sign up for upcoming AWS events:

AWS re:Invent – AWS re:Invent returns to Las Vegas from November 30 to December 4. 2, 200+ session times, locations, and speakers are live. Reserved seating for AWS re:Invent opens October 6. Register now and be ready to claim your spot in chalk talks, workshops, and builders’ sessions when reserved seating opens.
AWS Summits – AWS Summits are free in-person events covering cloud and AI. With re:Invent on the horizon, the Summits are coming to an end for the year. The last Summit is Dubai (September 30) at the Dubai World Trade Center, with 60+ sessions, an AWS Village, and hands-on workshops.
AWS Community Days – Community-led conferences planned and delivered by community leaders. Upcoming events include Lebanon (September 26), Malaysia, Kuala Lumpur (September 26), Cebu, Philippines (September 26), Davao, Philippines (September 26), ComSum Manchester, UK (October 1), and Italy, Rome (October 2).

Summer has officially given way to September, but the weather where I am has not quite caught up. The days are still unusually warm, and I suspect these are the last mild afternoons before autumn settles in for good. I am making the most of them while they last. Come back next week for more!

— Esra

New low-cost burstable Amazon EC2 T8i instances are generally available

2026-09-18 Channy Yun (윤석찬)

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/new-low-cost-burstable-amazon-ec2-t8i-instances-are-generally-available/

Today, we’re announcing the general availability of new low-cost burstable Amazon EC2 T8i instances powered by custom sixth generation Intel Xeon Scalable Processors (Granite Rapids), available only on AWS. T8i instances are among the lowest-cost EC2 instances and deliver up to 30% better price performance over previous generation T3 instances. These instances are designed to run a variety of low-to-moderate CPU utilization workloads such as freemium services, training and demo environments, staging and development, data processing, microservices, low-traffic websites, and login gateways.

T8i instances
Thousands and thousands of customers run various lightweight workloads on T3 instances that require small, cost-effective compute configurations. These include microservices architectures, low-traffic websites, development and testing environments, small databases, data processing jobs, and short-duration compute tasks. Many of these customers like T family’s burstable performance model, which provides a baseline level of CPU performance with the ability to burst above the baseline when needed using CPU credits.

As customers modernize their infrastructure, migrate from on-premises environments, adopt event-driven and microservices architectures, and experiment with AI inference workloads, they have asked for newer generation cost-optimized small instances, better price performance to reduce their total cost of ownership, and a seamless migration path that leverages their existing knowledge and tooling.

T8i instances address each of these requests:

Up to 30% better price performance. Powered by the AWS Nitro System and custom sixth generation Intel Xeon Scalable Processors (Granite Rapids), T8i instances enable customers to lower their total cost of ownership with up to 30% better price performance.
Up to 70% higher compute performance. T8i instances deliver up to 70% higher compute performance, up to 1.25x higher network bandwidth, and up to 2.4x higher EBS bandwidth compared to T3 instances.
Seamless upgrade from T3. For existing T3 customers, upgrading to T8i is straightforward. The instances offer the same CPU credit system and the same familiar lightweight compute options customers already know. Customers simply select T8i instead of T3 and immediately benefit from improved price performance.
Cost-effective entry point for new customers. For customers new to AWS or migrating from on-premises, T8i instances provide one of the most cost-effective entry points to run workloads that need low-to-moderate CPU utilization or for running short-duration compute tasks such as batch processing, event-driven functions, or CI/CD pipelines.

Instance specifications
T8i instances offer four sizes, each with two vCPU offered as a single core. The following table summarizes the specifications.

Instance size	vCPUs	Memory (GiB)	Baseline Performance /vCPU (%)	CPU credits earned / hour	Network burst bandwidth (Gbps)
t8i.nano	2	0.25	5	3	Up to 6.25
t8i.micro	2	0.5	10	6	Up to 6.25
t8i.small	2	1	20	12	Up to 6.25
t8i.medium	2	2	20	12	Up to 6.25

Like T3, T8i instances offer unique vCPU-to-memory ratios such as 1:0.25, 1:0.5, and 1:1 that are not offered by other EC2 instances. Like T3, T8i instances utilize the CPU credit system along with the Standard and Unlimited credit configuration modes. Unlimited mode is the default on T8i.

For workloads that need larger instance sizes above T8i offerings (nano, micro, small, and medium), I recommend M8i Flex instances that offer up to 30% better price performance than equivalent previous generation T3 instances along with the flexibility to scale up to 16xlarge.

Now available
Amazon EC2 T8i instances are available today in the following AWS Regions: US East (N. Virginia, Ohio), US West (Oregon, N. California), Asia Pacific (Hyderabad, Malaysia, Mumbai, Seoul, Singapore, Sydney, Tokyo), Canada (Central), and Europe (Frankfurt, Ireland, London, Paris). For Regional availability and upcoming Region expansion, search the instance type in the CloudFormation resources tab of AWS Capabilities by Region.

You can purchase T8i instances via On-Demand instances, and Spot instances with Savings Plan option coming soon. T8i instances support shared tenancy only and do not support Dedicated tenancy or Dedicated Hosts. t8i.micro and t8i.small instances are also available under the AWS Free Tier. To learn more, visit the Amazon EC2 Pricing page.

Try T8i instances in the Amazon EC2 console and send feedback to AWS re:Post for EC2 or through your usual AWS Support contacts.

— Channy

Multi-modal autoscaling with Amazon EC2 Auto Scaling: adding signals for faster, more reliable scaling

2026-09-17 Shubhendu Dubey

Post Syndicated from Shubhendu Dubey original https://aws.amazon.com/blogs/compute/multi-modal-autoscaling-with-amazon-ec2-auto-scaling-adding-signals-for-faster-more-reliable-scaling/

How do you handle unpredictable workload patterns that spike during promotional events or seasonal peaks? Multi-modal autoscaling with Amazon EC2 Auto Scaling combines infrastructure metrics like CPU with application-level signals, so a group scales on the demand its users create and not only on how busy the servers look. Those signals track the load that drives your business outcomes, such as sales or sign-ups.

CPU-based autoscaling works well for many workloads, but some demand does not register as CPU right away. Adding signals such as request counts and application metrics lets a group respond to the load its users create. By publishing Amazon CloudWatch custom metrics and application-driven triggers, you give Auto Scaling more information to act on.

In our testing, a group that scaled only on CPU rejected about 7,000 checkout sessions during a demand spike, and a stronger baseline that added Application Load Balancer request count still rejected about 6,900. A group that added an application metric rejected none, and it held p99 latency to about 0.43 seconds against 2.25 seconds for the CPU-only group. Predictive scaling can add a forecasting layer for cyclical demand, but it needs days of history to be useful, so we treat it as a complement. In this post, we show you how to implement multi-modal autoscaling on EC2 Auto Scaling, with code samples and results from a controlled test.

Prerequisites

To follow along, you need access to the following AWS services with appropriate permissions:

EC2 Auto Scaling, for scaling policies and group management.
CloudWatch, for metrics, alarms, and dashboards.
AWS CloudFormation, for infrastructure deployment.

Expanding beyond single-metric scaling

The default target tracking policy in EC2 Auto Scaling uses average CPU utilization, a practical starting point because CPU usage is a universal characteristic of compute workloads. Adding complementary signals, such as application-level metrics or predictive forecasting, gives Auto Scaling more information to make timely capacity decisions.

For workloads that need a faster response from target tracking alone, see Faster scaling with Amazon EC2 Auto Scaling target tracking.

In distributed architectures, different components can have distinct scaling characteristics. An API gateway might correlate well with request rate, while a background processor scales better on queue depth. With multi-modal scaling, you can match each component’s policy to its actual workload pattern. For containerized workloads, consider event-driven autoscaling with KEDA on Amazon Elastic Kubernetes Service (Amazon EKS).

Multi-modal autoscaling combines three approaches to capacity management. Reactive scaling responds to current CloudWatch metrics, such as CPU utilization, memory, network throughput, response times, and custom application indicators. Application-metric scaling brings workload-specific signals into the decision, using custom CloudWatch metrics like active user sessions, queue depth, or transaction volume. These application metrics are often the closest measurable proxy for business activity such as orders or sign-ups. Predictive scaling uses machine learning in EC2 Auto Scaling to forecast capacity needs from historical patterns, so infrastructure scales before demand increases.

With application-metric scaling, applications can scale on signals that infrastructure metrics miss. An ecommerce platform might scale on active checkout sessions, while a streaming service scales on concurrent stream counts. In the test later in this post, we use active checkout sessions as the custom metric.

This section builds the configuration in layers. Start with CPU target tracking as a baseline that every group keeps, then add a custom application metric that reflects real user load. The test later in this post compares these signals against a request-count baseline. Predictive scaling is an optional forecasting layer described at the end.

Step 1: CPU target tracking

Start with the foundation that most workloads already use: a target tracking policy on average CPU utilization. Target tracking is a managed policy that adjusts capacity to keep a metric at or near a target value. It supports predefined metrics, including CPU utilization and request count per target, and custom CloudWatch metrics. When multiple target tracking policies are active, Auto Scaling coordinates them: it scales out if any policy requires it, but scales in only when all policies agree, which helps prevent oscillation.

# CPU target tracking scaling policy (ASG A)
CPUTargetTrackingPolicy:
  Type: AWS::AutoScaling::ScalingPolicy
  Properties:
    AutoScalingGroupName: !Ref AutoScalingGroupName
    PolicyType: TargetTrackingScaling
    TargetTrackingConfiguration:
      PredefinedMetricSpecification:
        PredefinedMetricType: ASGAverageCPUUtilization
      TargetValue: 70

Our test also included a second infrastructure baseline, a target tracking policy on the load balancer’s request count per target. It uses the same structure with a predefined metric:

# Request count target tracking (ASG B)
RequestCountPTTargetTrackingPolicy:
  Type: AWS::AutoScaling::ScalingPolicy
  Properties:
    AutoScalingGroupName: !Ref AutoScalingGroupName
    PolicyType: TargetTrackingScaling
    TargetTrackingConfiguration:
      PredefinedMetricSpecification:
        PredefinedMetricType: ALBRequestCountPerTarget
        ResourceLabel: !Sub "${Alb.LoadBalancerFullName}/${TargetGroupB.TargetGroupFullName}"
      TargetValue: 300
      DisableScaleIn: false

Step scaling is another option for spike handling. With step scaling, you can define different capacity increments for different alarm thresholds. It keeps evaluating the alarm during scaling activities, which can make it react faster than target tracking’s default evaluation window. Step scaling policies do not coordinate with each other.

Step 2: Add a custom application metric

Next, add a second target tracking policy on a custom CloudWatch metric that reflects application load. In our test, instances publish an active checkout sessions metric at a 10-second resolution. To act on that resolution, set a Period of 10 seconds on the policy. Without it, the policy waits for three 1-minute datapoints like any other and the high-resolution metric only adds publishing cost. With it, a scale-out can begin in about 30 seconds. The policy includes the Auto Scaling group dimension so it tracks the metric for the right group. We set the target to 100 active sessions per instance, about 75 percent of the measured per-instance capacity of 135. This leaves headroom to absorb a spike while new instances boot.

# Custom application metric target tracking (ASG C)
CustomMetricTargetTrackingPolicy:
  Type: AWS::AutoScaling::ScalingPolicy
  Properties:
    AutoScalingGroupName: !Ref AutoScalingGroupName
    PolicyType: TargetTrackingScaling
    TargetTrackingConfiguration:
      CustomizedMetricSpecification:
        MetricName: ActiveCheckoutSessions
        Namespace: ECommerce/CheckoutMetrics
        Dimensions:
          - Name: AutoScalingGroupName
            Value: !Ref AutoScalingGroupName
        Statistic: Average
        Period: 10
      TargetValue: 100
      DisableScaleIn: false

Step 3: Add predictive scaling

Predictive scaling is an optional forecasting layer. It uses machine learning in EC2 Auto Scaling to analyze historical load and scale ahead of recurring, cyclical demand, using customized metric specifications in ForecastAndScale mode. You need to provide several days of history for it to forecast well, so it complements reactive signals rather than replacing them. Start in ForecastOnly mode to watch the forecast before it drives any scaling.

Monitoring

Use CloudWatch dashboards to track how each policy contributes to scaling decisions, and set alarms on the metrics that matter for your workload, such as per-instance load or latency. Enable detailed monitoring on the launch template, with Monitoring set to true, so that the system publishes CPU metrics every minute. Without it, you cannot complete the CPU policy’s scale-in evaluation and your group will stop scaling in. Watching the policies side by side is what surfaced this scale-in behavior.

Performance results

We compared three Auto Scaling groups under an identical load profile in a single 75-minute test in the us-east-1 Region. Each group used c8g.large instances with a minimum of 6 and a maximum of 40 instances, and every group carried the same CPU target tracking policy at 70 percent as a fallback:

ASG A: CPU target tracking only. This is the single-signal infrastructure baseline.
ASG B: CPU target tracking plus an Application Load Balancer request-count policy. Request rate is a stronger infrastructure baseline than CPU alone.
ASG C: CPU target tracking plus the custom checkout-sessions metric at 10-second resolution, published with a Period of 10 seconds.

All the groups received the same load at the same time. During the shared ramp, arrival rate rose and every group scaled correctly, which makes the comparison fair. CPU crossed 70 percent on the CPU group, request count crossed its target of 300 on the request-count group, and all three converged to a similar size.

Ramp phase (arrivals 30% → 85%)	A: CPU only	B: A + ALB requests	C: A + app sessions
Instances (start → peak)	6 → 9	6 → 10	6 → 10
CPU	73.6%	74.0%	69.0%
Requests per target (target 300)	319	323	297

Then arrival rate was held flat while the number of concurrent checkout sessions kept rising, a shape that infrastructure signals cannot see. The next table reports that divergence phase, measured directly from CloudWatch and the load balancer.

Measured metric (divergence)	A: CPU only	B: A + ALB requests	C: A + app sessions
Instances (start → end)	11 → 11	11 → 11	11 → 22
Rejected checkouts	7,064	6,886	0
CPU (start → end)	67.2% → 45.1%	66.5% → 45.5%	66.2% → 36.8%
Peak sessions per instance	135	135	116
Requests per target	285 → 279	282 → 279	277 → 154

The difference is what each group could see. Arrival rate was held flat while the number of concurrent sessions rose, so CPU and request count stayed in range while the application saturated. The CPU-only and request-count groups held at 11 instances and rejected 7,064 and 6,886 checkouts. Their CPU even fell, from about 67 percent to about 45 percent, because a rejected request never reaches the work it would have done, so a policy targeting 70 percent saw spare capacity at the moment the application was failing users. The application-metric group read the rising sessions directly and scaled from 11 to 22 instances, rejecting none.

Effect on latency and errors

We measured latency and rejected checkouts on the load balancer during the test. At rest, all groups were identical. The gap opened only in the divergence phase, when concurrency rose without a matching change in arrival rate. Session slots are the scarce resource here, so sessions per instance is the causal driver of latency. The application group scales on sessions and we report latency as the outcome, rather than scaling on latency directly, which is not recommended for target tracking. The latency figures come from the load balancer’s TargetResponseTime at the end of the divergence phase. A client-side number measured over the internet would reflect network round-trip rather than the service.

Measured metric	A: CPU only	B: A + ALB requests	C: A + app sessions
TargetResponseTime (average), end of divergence	1.122 s	1.120 s	0.284 s
TargetResponseTime (p99), end of divergence	2.254 s	2.235 s	0.431 s
TargetResponseTime (average) at warm-up	0.283 s	0.283 s	0.284 s
Rejected checkouts, drain phase	4,115	3,631	0

The application-metric group, ASG C, kept per-instance load near its target and rejected no checkouts. Its average latency at the end of the divergence phase was 0.284 seconds against 1.122 for the CPU-only group, and its p99 was 0.431 seconds against 2.254. The request-count group, ASG B, tracked its own signal within range the whole time, which is exactly why it could not react: request rate was flat while concurrency climbed.

Once every group has enough capacity, they perform the same. The value of the application signal is in the transition, the gap between when demand arrives and when the fleet is ready, which the infrastructure signals here never detected.

Handling known high-traffic events

For planned events like flash sales, scheduled scaling can pre-scale capacity ahead of time. Multi-modal scaling complements scheduled scaling by handling unplanned spikes and organic traffic that does not follow a fixed schedule.

Understanding cost implications

Running the application signal requires more instances. During the spike it held about 22 instances, against 11 on the infrastructure-only groups. That extra capacity is what kept sessions per instance near the target and stopped the group from turning checkouts away. For your own workload, the question is whether a spike’s worth of extra instances costs less than the checkouts you would otherwise reject.

Conclusion

Multi-modal autoscaling combines infrastructure metrics with application-level signals so a group scales on the demand its users create, not only on how busy its servers look. In our test, a group that scaled only on CPU rejected about 7,000 checkout sessions during a demand spike, and a stronger baseline that added load balancer request count still rejected about 6,900. A group that added a custom application metric rejected none, and held p99 latency near 0.43 seconds against 2.25 seconds for the CPU-only group. Its CPU even fell while the infrastructure groups were failing requests, which shows why an infrastructure signal alone can miss the demand that matters.

Start with CPU target tracking as a fallback. Add a signal that reflects the load your users create, and pick the one that tracks closest to a business outcome like orders or active users. Set a Period on a high-resolution custom metric so the policy can act on it, and enable detailed monitoring so scale-in works. Predictive scaling is worth adding for demand you can forecast, once the group has days of history to learn from.

To implement multi-modal autoscaling, you can open Amazon EC2 Auto Scaling in the AWS Management Console and add a second scaling signal to one of your existing groups, following the configuration steps in this post. For a deeper look at target tracking behavior, see Faster scaling with Amazon EC2 Auto Scaling target tracking. The Amazon EC2 Auto Scaling User Guide covers predictive scaling policies, custom metrics, and scaling cooldowns in detail.

AWS Weekly Roundup: Claude Fable 5.1 on AWS, Amazon Linux 2027 preview, AWS Certified AI Business Strategist, and more (September 7, 2026)

2026-09-07 Channy Yun (윤석찬)

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-claude-fable-5-1-on-aws-amazon-linux-2027-preview-aws-certified-ai-business-strategist-and-more-september-7-2026/

Last week, Claude Fable 5.1 became available on AWS. According to Anthropic, Claude Fable 5.1 delivers frontier intelligence for ambitious tasks across coding, scientific research, and enterprise workflows. Claude Fable 5.1 is built for long-running, high-stakes work that runs for hours and spans many applications. It can own more of a software project on its own, handling features across an entire codebase, code review, and performance work over extended sessions.

Anthropic has designated Fable 5.1 a Covered Model, a category of Claude models that carry additional data retention, safety review, and access policies wherever they’re offered. Claude Fable 5.1 is subject to data retention for up to 30 days and human review by Amazon personnel, with a new aws_review data retention mode. In this mode, AWS retains your prompts and outputs for human safety review within the AWS boundary. The provider_data_share mode is legacy, and Amazon Bedrock does not share your data with the model provider. In addition, Enterprise Frontier Safeguards (EFS), built in partnership between AWS and Anthropic, will let eligible customers use Covered Models while keeping their data in a cloud environment they control.

You have two ways to access Claude Fable 5.1: Amazon Bedrock and Claude Platform on AWS. To learn more, see the Claude Fable 5.1 model card on Amazon Bedrock and Claude Platform on AWS.

Last week’s launches
Here are some launches that got my attention:

Amazon Linux 2027 (AL2027) in public preview: AL2027 is the next version of the Amazon Linux operating system. It runs on kernel 7.1+, purpose-built for cloud-native workloads on AWS with performance, scale, and security in mind. Built on AL2023’s baseline, AL2027 is designed for customers who need a secure, stable, and AWS-native operating system running web applications, databases, containerized microservices, AI/ML workloads, and large-scale infrastructure.
Amazon EC2 R9g and R9gd memory-optimized instances: These instances are powered by AWS Graviton5 processors, delivering the best price performance for memory-intensive workloads running on Amazon EC2. R9g and R9gd instances deliver up to 25% better compute performance compared to AWS Graviton4-based R8g and R8gd instances. They are up to 30% faster for databases, up to 35% faster for web applications, and up to 35% faster for machine learning. To learn more, read Daniel’s blog post.
AWS Lambda SnapStart for container image functions: Lambda SnapStart is an opt-in capability that makes it easier for you to build highly responsive and scalable applications without provisioning resources or implementing complex performance optimizations. Previously, SnapStart was only supported for managed runtimes (Python, .NET, and Java). You can now use SnapStart for container images to reduce startup times from several seconds to as low as sub-second for latency-sensitive workloads such as ML inference and interactive APIs.
AWS Agent Registry now generally available: AWS Agent Registry provides a private, governed catalog and discovery layer for agents, tools, skills, MCP servers, and custom resources within your organization. In addition to the capabilities launched in preview (manual and URL-based record creation, approval workflows, semantic and keyword search, and AWS CloudTrail audit trails), Registry now adds new enterprise features. To learn more, visit the AI Blog post.
Amazon Redshift now supports Apache Iceberg v3 tables: You can read from and write to Apache Iceberg v3 tables in your data lake of Amazon Redshift. With this launch, Amazon Redshift introduces support for default column values, row lineage, and deletion vectors. Amazon Redshift’s Graviton based provisioned and serverless clusters support the new v3 format. To learn more, visit Apache Iceberg v3 features in Redshift.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news
Here are some additional projects and news items you may find interesting:

AWS named a Leader in the 2026 Gartner Magic Quadrant for Strategic Cloud Platform Services: For the 16th consecutive year, Gartner has recognized AWS as a Leader in the 2026 Gartner Magic Quadrant for Strategic Cloud Platform Services, and once again placed AWS highest on the Ability to Execute axis. We believe this recognition reflects our commitment to delivering the broadest and deepest set of cloud capabilities from infrastructure and AI to security and operations, so you can build, innovate, and scale with confidence.
AWS Certified AI Business Strategist: This new certification targets professionals who evaluate, champion, and scale AI initiatives in their organizations: line-of-business leaders driving adoption across their teams, sales professionals articulating AI value to customers, consultants guiding client strategy from experimentation through production, program managers aligning AI investments to business outcomes. Beta exam registration opened September 1, 2026, with exam delivery beginning September 29.
Agentic Security: Detection and Response at Machine Speed: We believe security should evolve ahead of AI adoption, not behind it. That belief drove our team to collaborate with the SANS Institute on a new chapter in the 2026 Cloud Security Exchange eBook, where we lay out a practical framework for securing agentic workloads at enterprise scale. Our chapter goes deeper on securing agentic workloads, with specific architectural patterns, implementation guidance, and frameworks for security teams at every stage of agentic AI maturity, whether you’re evaluating, piloting, or operating at scale.

For a full list of AWS blog posts, be sure to keep an eye on the AWS Blogs page.

Learn more about AWS, browse and join upcoming AWS-led in-person and virtual events, startup events, and developer-focused events including AWS re:Invent, AWS Summits, and AWS Community Days. Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development.

That is all for this week. Check back next Monday for another Weekly Roundup!

— Channy

Amazon EC2 R9g and R9gd instances powered by AWS Graviton5 processors are now generally available

2026-08-31 Daniel Abib

Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/amazon-ec2-r9g-and-r9gd-instances-powered-by-aws-graviton5-processors-are-now-generally-available/

Today, Amazon EC2 R9g and R9gd instances are generally available, powered by AWS Graviton5 processors. R9g instances are memory-optimized and deliver up to 25% better compute performance compared to Graviton4-based R8g instances, powered by the most energy efficient processor AWS has ever built.

R9g instances are ideal for memory-intensive workloads including databases, in-memory caches (Valkey, Redis, MemCached), real-time big data analytics, Linux-based workloads including containerized and micro-service-based applications (e.g. Kubernetes, Docker, EKS, ECS), as well as applications written in popular programming languages such as C/C++, Rust, Go, Java, Python, .NET Core, Node.js, Ruby, and PHP.

R9gd instances include local NVMe-based SSD block-level storage, ideal for memory-intensive workloads requiring fast, low-latency local storage such as open-source databases, distributed real-time big data analytics, large in-memory databases, and large caching workloads.

If you’re running workloads on R8g instances today, R9g gives you more performance per vCPU with faster memory, higher network and Amazon EBS bandwidth, and a larger L3 cache, all while using less energy.

What makes R9g different
Graviton5 processors bring several hardware improvements over Graviton4:

Up to 25% higher compute performance per vCPU
DDR5 8800 MT/s memory (up from 5600 MT/s in Graviton4), the fastest memory available in the cloud
5x larger L3 cache for better data locality
Up to 2x higher network and EBS bandwidth for the largest instance sizes (up to 100 Gbps network, up to 72 Gbps EBS on the 48xlarge)
Up to 3x higher packet-processing performance

R9g and R9gd instances support Instance Bandwidth Configuration (IBC), which lets you adjust the allocation of bandwidth between Amazon EBS and Amazon VPC networking by 25%. This helps optimize performance for workloads with specific bandwidth requirements such as databases and caching.

All R9g and R9gd instances run on the AWS Nitro System, which offloads virtualization, storage, and networking to dedicated hardware. This gives your applications near-bare-metal performance while maintaining strong security isolation between instances.

R9g and R9gd instances feature the Nitro Isolation Engine (NIE), the same enhancement to the Nitro System introduced with C9g and M9g instances earlier this year, which enforces isolation of instances and harnesses formal verification to provide assurances of isolation with mathematical precision. Nitro Isolation Engine is a purpose-built component that is responsible for enforcing isolation between virtual machines, including mediation of all access to virtual machine memory, CPU register state, and I/O devices through a minimal set of APIs. Nitro Isolation Engine leverages formal verification, a technique to mathematically demonstrate that the hardware or software behaves as intended, and not just in specific test cases. This intensive verification technique establishes Nitro as the first formally verified cloud hypervisor, pioneering a new standard for mathematically proven cloud security. To learn more about the Nitro Isolation Engine, visit the blog post. For details on the formal verification results, including scope and assumptions, see the technical white paper.

EC2 R9g and R9gd instance specifications
R9g and R9gd instances are each available in 11 sizes, from medium to metal-48xl. The following tables show the full specifications for each size.

Instance size	vCPUs	Memory (GiB)	Instance Storage	Network Bandwidth (Gbps)	EBS Bandwidth (Gbps)
r9g.medium	1	8	EBS-Only	Up to 15	Up to 12
r9g.large	2	16	EBS-Only	Up to 15	Up to 12
r9g.xlarge	4	32	EBS-Only	Up to 15	Up to 12
r9g.2xlarge	8	64	EBS-Only	Up to 17	Up to 12
r9g.4xlarge	16	128	EBS-Only	Up to 17	Up to 12
r9g.8xlarge	32	256	EBS-Only	17	12
r9g.12xlarge	48	384	EBS-Only	25	18
r9g.16xlarge	64	512	EBS-Only	34	24
r9g.24xlarge	96	768	EBS-Only	50	36
r9g.48xlarge	192	1536	EBS-Only	100	72
r9g.metal‑48xl	192	1536	EBS-Only	100	72

R9gd instances offer the same compute and networking performance as R9g, with the addition of local NVMe-based SSD storage for workloads that need fast, low-latency scratch space or temporary caches.

Instance size	vCPUs	Memory (GiB)	Instance Storage (NVMe SSD)	Network Bandwidth (Gbps)	EBS Bandwidth (Gbps)
r9gd.medium	1	8	1 x 59 GB	Up to 15	Up to 12
r9gd.large	2	16	1 x 118 GB	Up to 15	Up to 12
r9gd.xlarge	4	32	1 x 237 GB	Up to 15	Up to 12
r9gd.2xlarge	8	64	1 x 474 GB	Up to 17	Up to 12
r9gd.4xlarge	16	128	1 x 950 GB	Up to 17	Up to 12
r9gd.8xlarge	32	256	1 x 1900 GB	17	12
r9gd.12xlarge	48	384	3 x 950 GB	25	18
r9gd.16xlarge	64	512	1 x 3800 GB	34	24
r9gd.24xlarge	96	768	3 x 1900 GB	50	36
r9gd.48xlarge	192	1536	3 x 3800 GB	100	72
r9gd.metal‑48xl	192	1536	3 x 3800 GB	100	72

Getting started
You can launch R9g and R9gd instances from the Amazon EC2 console using any supported Arm-based AMI. R9g instances support Amazon Linux 2023, Amazon Linux 2, Ubuntu 22.04+, RHEL 8.4+, SUSE Linux Enterprise Server 15 SP3+, Debian 12+, and other major Linux distributions.

If you’re migrating from R8g, no code changes are required for most applications. Select the equivalent R9g instance size and your application runs with better performance. For containerized workloads, R9g works with Amazon EKS, Amazon ECS, and standard Kubernetes deployments. Multi-arch container images built for Arm64 run without changes.

Several resources help you get started: the AWS Graviton Getting Started Guide covers how to build, run, and optimize workloads on Graviton-based instances. The Graviton Savings Dashboard helps you track cost savings. AWS Transform automates code transformations for migrating Java applications from x86 to Graviton. To learn more, visit AWS Graviton Processors or Level up your compute with AWS Graviton.

Pricing and availability
Amazon EC2 R9g and R9gd instances are available in US East (N. Virginia, Ohio), US West (Oregon), and Europe (Frankfurt) Regions.

R9g and R9gd instances are available for purchase through Savings Plans, On-Demand, Spot Instances, Dedicated Instances, or Dedicated Hosts. For detailed pricing, visit the Amazon EC2 pricing page.

Ready to get started? Launch R9g instances from the Amazon EC2 console. For more details, visit the Amazon EC2 R9g instances page.

If you want to call APIs, search documentation, find regional availability, and check troubleshooting about this feature, try using the AWS MCP Server and plugins with your preferred AI tool. Share your feedback on AWS re:Post for Amazon EC2 or reach out through your usual AWS Support contacts.

— Daniel Abib

AWS Weekly Roundup: Welcome DuckLabs to the team, Agentic Resource Discovery (ARD), and more (August 31, 2026)

2026-08-31 Daniel Abib

Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-welcome-ducklabs-to-the-team-agentic-resource-discovery-ard-and-more-august-31-2026/

The news that interested me the most last week was the DuckLabs acquisition. AWS has signed a definitive agreement to acquire DuckLabs, the Amsterdam-based company behind DuckDB, the popular open source analytical database that runs in-process and executes SQL directly against files like Parquet, CSV, and JSON. DuckDB stays open source under its independent foundation and the MIT license, and over time AWS plans to combine its speed at everyday queries with the enterprise scale of services like Amazon S3, Amazon Redshift, and Amazon Athena.

Co-founded by Hannes Mühleisen and Mark Raasveldt, DuckDB runs locally or on Amazon S3, which makes it remarkably fast for the everyday queries (a terabyte or less) that make up the bulk of real-world analytics. It also happens to pair beautifully with AI agents, which “poke” and experiment their way through data much like humans do. The co-founders will continue leading its technical direction while AWS combines DuckDB’s speed with analytics services like Amazon EMR, AWS Glue, and Amazon SageMaker. For the bigger picture on why this matters, Andy Warfield, Vice President and Distinguished Engineer shared his thoughts on the post DuckDB and the changing physics of analytics on All Things Distributed.

Now, let’s get into this week’s AWS news…

Last week’s launches
Here are some launches and updates from this past week that caught my attention:

Amazon ECS now automatically detects and recovers container instances that lose agent connectivity – Amazon ECS now continuously monitors agent connectivity to the control plane and surfaces a new AGENT_CONNECTIVITY health event across AWS Fargate, Amazon ECS Managed Instances, and Amazon ECS on EC2. On Fargate and Managed Instances, ECS handles recovery automatically, draining tasks, launching replacements, and deregistering the impaired instance. On EC2, you can wire the event into your own workflow. Available at no additional cost in all AWS Commercial and AWS GovCloud (US) Regions.
AWS Lambda introduces public preview runtimes, starting with Node.js 26 and Python 3.15 – You can now test upcoming Lambda runtimes before they reach general availability. Preview runtimes use the same identifier as the eventual GA version, so your functions graduate automatically with no action required. Third-party tools and deployment frameworks can also validate compatibility ahead of GA. Not meant for production yet (breaking changes are possible), but a great way to get ahead of your next upgrade. Available in all AWS commercial, AWS GovCloud (US), and China Regions.
AWS IoT Core adds a native InfluxDB rule action – You can now route time-series data from your IoT devices straight into InfluxDB (Amazon Timestream-managed or self-hosted) without writing custom code or standing up an intermediate service. IoT Core formats data into InfluxDB’s line protocol and supports device-side and server-side batching. Available in all AWS Regions where Amazon Timestream for InfluxDB is offered.
Amazon GameLift Servers now includes enhanced DDoS protection – Your game servers now get automatic protection against network and transport layer (layers 3 and 4) DDoS attacks – UDP reflection, SYN floods, and similar vectors – with nothing to enable or opt into. Built on top of AWS Shield Standard with gaming-optimized traffic shaping, it turns on the moment your servers start running (Server SDK 5) at no extra cost. It’s available in all supported GameLift Servers Regions except China (Beijing) and China (Ningxia).
Amazon SageMaker HyperPod expands support for Ray – You can now run Ray workloads on SageMaker HyperPod with built-in observability, resilient training, and accelerated inference. Create and manage Ray clusters from Amazon SageMaker Studio, attach JupyterLab or your local IDE so a multi-node cluster behaves like a local dev environment, and get auto-provisioned Grafana dashboards. Node auto recovery, hung job detection, and tiered checkpointing keep large training runs healthy, while Ray Serve adds a tiered KV cache for inference. Your existing open source Ray code runs unchanged. Available for HyperPod clusters orchestrated by Amazon EKS.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news
Here are some additional posts and resources that you might find interesting:

Happy 20th birthday, Amazon EC2! – Amazon EC2 turns 20. Channy Yun looks back at how EC2 grew from a single m1.small instance type in one Region to more than 1,200 instance types across 39 Regions, along with the custom silicon journey from the first Graviton to Graviton5 and Trainium3. A fun and worthwhile read on the service that still underpins so much of AWS – including Amazon ECS, Amazon EKS, AWS Lambda, Amazon SageMaker, and Amazon Bedrock.
Agentic Resource Discovery (ARD): an open specification for agent discovery – As organizations scale up agents, tools, and MCP servers, those resources end up scattered across clouds, on-premises infrastructure, and SaaS platforms – each with its own registry and metadata. ARD is a new open specification (Apache 2.0) that defines a common way to describe and discover agentic resources, so publishers “describe once” and consumers “discover everywhere” – think DNS, but for agents. AWS contributed feedback but doesn’t own the spec, and it complements the AWS Agent Registry by letting you federate across catalogs without migrating.
Get started with the Agent Toolkit for AWS in the AWS CLI – A single AWS CLI command (aws configure agent-toolkit) now equips AI coding agents like Kiro, Claude Code, Codex, and Cursor with curated, up-to-date AWS knowledge and a secure connection to thousands of AWS APIs through the AWS MCP Server. If you build with an AI coding assistant, this helps it choose the right services, use modern APIs, and follow security best practices – so it gets AWS code right more often the first time.

Upcoming AWS events
Check your calendar and sign up for upcoming AWS events:

AWS Summits – Free in-person events where builders come together to learn, connect, and explore the latest in cloud and AI. Upcoming stops include Zurich (September 2), São Paulo (September 3), Tel Aviv (September 10), and Dubai (September 30). Can’t attend in person? You can stream sessions through the Global Livestream and On-Demand Hub. I’ll be presenting two sessions on generative AI and Amazon Bedrock at the São Paulo Summit – if you’re there, come say hello.
AWS Community Days – Community-led conferences where content is planned, sourced, and delivered by community leaders. Upcoming events include JAWS SONIC 2026 in Tokyo (September 5) and Warsaw, Poland (September 8).

That’s all for this week. Check back next Monday for another Weekly Roundup!

— Daniel Abib

This post is part of our Weekly Roundup series. Check back each week for a quick roundup of interesting news and announcements from AWS!

GPU-accelerated Apache Spark with Amazon EMR and NVIDIA RTX PRO 4500 on Amazon EC2 G7 instances runs up to 3.7x faster

2026-08-25 McCall Peltier

Post Syndicated from McCall Peltier original https://aws.amazon.com/blogs/big-data/gpu-accelerated-apache-spark-with-amazon-emr-and-nvidia-rtx-pro-4500-on-amazon-ec2-g7-instances-runs-up-to-3-7x-faster/

For years, Apache Spark has been the backbone of large-scale data processing. However, as datasets grow and artificial intelligence and machine learning (AI/ML) pipelines become more complex, modern workloads demand more computational power. Feature engineering for machine learning models, large-scale extract, transform, and load (ETL) transformations, and real-time analytics workloads are computationally intensive by nature. GPU-accelerated instances improve performance and transform jobs that once took hours into minutes, so you can iterate on models faster and reduce operational costs. You can process larger datasets in single batches, make decisions in real time, and achieve strong performance without over-provisioning infrastructure.

We’re excited to share the benchmarking results on Amazon EMR with Amazon Elastic Compute Cloud (Amazon EC2) G7 instances, powered by NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs. For data engineers and data scientists running Apache Spark workloads, this means faster pipelines, shorter iteration cycles, and more time spent on insights.

Amazon EMR on EKS natively supports the NVIDIA cuDF plugin for Apache Spark. This support is the result of joint engineering between AWS and NVIDIA to qualify the cuDF plugin for Amazon EMR, co-optimize Spark execution paths for RTX PRO 4500, and validate performance at scale through shared TPC-DS benchmarking on Amazon EC2 G7 instances. Now, Apache Spark workloads on Amazon EMR on EKS run up to 3.7x faster with Amazon EC2 G7 GPU instances than with comparable CPU instances, and require no changes to existing Spark code.

In the TPC-DS 3 TB benchmark, at the 64 GB memory tier, EC2 G7 instances with RTX PRO finished in 4.7 minutes. If you run large-scale data processing pipelines, you can cut job run times by more than two-thirds while maintaining full compatibility with the applications you already have in production.

The use cases that benefit most are those where speed directly unlocks business value. In AI/ML feature engineering, faster Spark jobs mean data science teams can iterate on features more quickly, reducing the time from raw data to trained model. In complex ETL pipelines, like financial transactions, clickstream aggregation, or supply chain data consolidation, GPU acceleration compresses multi-hour batch windows into near-real-time processing. For real-time analytics, teams running fraud detection, personalization engines, or operational dashboards can process larger volumes of data within tighter latency windows, without redesigning their architecture.

Beyond data analytics, the G7 instances will support a broad range of AI and graphics workloads, including conversational AI, content generation, recommender systems, and video streaming and rendering. Built on the AWS Nitro System, they deliver the security and resource efficiency that production AI, analytics, and graphics workloads demand.

The following sections walk through the cluster configuration, benchmark methodology, and performance results.

Cluster configuration

We benchmarked four instance types to measure the real-world performance of G7 GPU instances against comparable CPU instances for Spark SQL performance. The g7.4xlarge also provides 80 Gbps network bandwidth (compared to 15–17 Gbps on the CPU baselines) and uses RapidsShuffleManager. However, CPU runs showed no evidence of being network- or shuffle-bound at this cluster scale. All tests used Amazon EMR on EKS 7.12.0 with Apache Spark 3.5.6 and cuDF plugin 26.04.2, running the full TPC-DS benchmark at 3 TB scale across 103 queries. Each experiment ran 5 iterations. We report the median. Data was stored as Parquet on Amazon Simple Storage Service (Amazon S3) (same-region gateway endpoint). All instances were launched in a single Availability Zone.

Instance specifications

All four instance types share the same compute footprint of 16 vCPUs and 64 GB system RAM. The g7.4xlarge additionally includes an NVIDIA RTX PRO 4500 Blackwell GPU with 32 GB of dedicated video memory (VRAM), which the cuDF plugin uses to accelerate Spark SQL operations. The baseline for all speedup and cost comparisons is m9gd.4xlarge (Graviton), the lowest-cost CPU instance in the group.

.	g7.4xlarge	m9gd.4xlarge	m8id.4xlarge	m8a.4xlarge
Architecture	x86_64	arm64 (Graviton)	x86_64	x86_64
vCPU	16	16	16	16
RAM	64 GB	64 GB	64 GB	64 GB
GPU	1× RTX PRO 4500 Blackwell (32 GB VRAM)	–	–	–
NVMe	875 GB	950 GB	950 GB	EBS only (GP3 16k IOPS and 2000 MB/s throughput to match NVMe
Network	80 Gbps	Up to 17 Gbps	Up to 15 Gbps	Up to 15 Gbps

The g7.4xlarge uses the RTX PRO 4500 Blackwell Server Edition GPU. The CPU baselines cover all three major architectures: m8id.4xlarge (Intel x86), m8a.4xlarge (AMD x86), and m9gd.4xlarge (Graviton arm64).

Spark configuration

All instances used eight executor nodes with the following configuration:

Configuration	GPU instances	CPU instances
Amazon EMR release	emr-7.12.0-spark-rapids-latest	emr-7.12.0-latest
executor.cores	14	14
executor.instances	8	8
executor.memory	20G	20G
executor.memoryOverhead	30G	30G
spark.plugins	com.nvidia.spark.SQLPlugin	–
rapids.memory.pinnedPool.size	8G	–
rapids.sql.concurrentGpuTasks	3	–
shuffle.manager	RapidsShuffleManager	default (sort)
sql.adaptive.enabled	true	true
io.compression.codec	zstd	zstd

CPU instances use the same 30 GB memoryOverhead as GPU to make sure that the memory comparison is apples-to-apples. This setting reserves off-heap memory for shuffle and caching on both sides.

For GPU instances, the cuDF plugin offloads Spark SQL operations to the GPU automatically. No code changes are required. The executor.memoryOverhead value is set higher on GPU instances to accommodate GPU memory management and the RAPIDS shuffle manager.

The cuDF plugin automatically falls back to CPU execution for unsupported operators and user-defined functions (UDFs). Your job still completes, but those stages run without GPU acceleration. To identify which operations run on GPU compared to CPU, set spark.rapids.sql.explain=NOT_ON_GPU in your Spark configuration. For a pre-migration assessment of your workloads, use the NVIDIA cuDF tool to estimate GPU acceleration potential before moving to G7 instances.

To tune settings like concurrentGpuTasks and pinnedPool.size, use the Spark History Server on Amazon EMR on EKS, which provides per-stage execution details to identify CPU fallback and shuffle bottlenecks.

Getting started

Reference the Using cuDF Accelerator for Apache Spark with Amazon EMR on EKS for detailed setup instructions.

Prerequisites

Before running GPU-accelerated Spark on Amazon EMR on EKS, make sure the following are in place:

Amazon EMR on EKS release version 6.9.0 or later (this post uses emr-7.12.0-spark-rapids-latest).

The -spark-rapids release variant ships the NVIDIA cuDF plugin pre-installed.

Amazon Elastic Kubernetes Service (Amazon EKS) cluster with a GPU-enabled node group using G7 instances.
Node AMI: AL2023_x86_64_NVIDIA (Amazon EKS optimized accelerated AMI).

NVIDIA device plugin installed in the cluster to expose GPUs to Kubernetes pods:

kubectl apply -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/v0.9.0/nvidia-device-plugin.yml

Amazon EMR on EKS virtual cluster registered to the EKS namespace.

To validate GPU availability on your nodes:

kubectl get nodes "-o=custom-columns=NAME:.metadata.name,GPU:.status.allocatable.nvidia\.com/gpu"

Note: Getting started with GPU-accelerated Spark on Amazon EMR is straightforward. To use the latest cuDF plugin, overlay the latest version (for example, 26.04.2 as of May 2026) onto the Amazon EMR RAPIDS image using an initContainer technique. This replaces the bundled cuDF JAR with a newer version while preserving all other Amazon EMR dependencies. We recommend using the latest Amazon EMR release to get the most up-to-date cuDF plugin for better performance. In our benchmarks, upgrading from cuDF plugin 25.08.0 to 26.04.2 reduced runtime by 36–38 percent. Download the latest cuDF plugin JAR from the NVIDIA repository. AWS Support covers Amazon EMR. For issues specific to a cuDF JAR, file a GitHub issue or contact NVIDIA at [email protected].

Performance benchmarks and cost efficiency

We ran the full TPC-DS benchmark suite (103 queries) at 3 TB scale on 8-node clusters in us-east-1. The following table summarizes the results:

.	GPU instances	CPU instances
Cost per run	$2.06	$2.93–$3.18
Total time (103 queries)	281s (4.7 min)	1,010–1,043s (16.8–17.4 min)
Speedup compared to CPU instances	3.7×	baseline

Cost per run is the total cluster cost for the benchmark’s duration: Cluster $/hr × (median runtime ÷ 3,600). The hourly rate combines the EC2 On-Demand cost for all 8 nodes and the Amazon EMR on EKS charge for the vCPU and memory the Spark pods consume. Both are billed per second (one-minute minimum), so you pay only for what a job uses while it runs. All runs used Amazon EMR on EKS 7.12.0 in us-east-1, with 8 × 4xlarge nodes (128 vCPU) on both the GPU and CPU sides. The g7.4xlarge cluster runs at $26.35/hr (8 × $3.042 EC2 = $24.34, plus $2.01 for Amazon EMR on EKS) and finishes in 281 seconds, at $2.06 per run. The CPU clusters run at a lower hourly rate ($10.43–$10.99) but take 1,010–1,043 seconds, landing at $2.93–$3.18 per run. All prices reflect On-Demand pricing in us-east-1 as of May 2026. G7 instances are also eligible for EC2 Spot and Compute Savings Plans, which can further reduce costs for recurring batch workloads.

Cost-per-run calculations include EC2 and Amazon EMR charges only. They exclude the EKS control-plane fee, EBS volumes, S3 request and storage costs, and the driver pod.

Bar chart of total TPC-DS runtime by instance type, showing g7.4xlarge finishing far faster than the CPU instances

Figure 1: Total runtime by instance type for all 103 TPC-DS queries at 3 TB scale. The g7.4xlarge with GPU acceleration completed the benchmark in 4.7 minutes, 3.7× faster than CPU instances (16.8-17.4 minutes)

Bar chart of total cost per benchmark run by instance type, showing the g7.4xlarge GPU instance costing less than the CPU instances

Figure 2: Total cost per benchmark run, including both Amazon EC2 instance and Amazon EMR on EKS cost across all 8 nodes. Despite a ~2.5× higher hourly rate, the g7.4xlarge GPU instance costs up to 31% less per run than Graviton because it finishes the workload 3.7× faster

Where GPU acceleration excels

GPU acceleration completed the 103-query power run in 281s compared to 1,032s on CPU, an overall 3.7× speedup that saves 750 seconds per run. GPU was faster on 102 of 103 query executions.

GPU acceleration delivers the largest gains on the long-running, compute- and shuffle-heavy queries where kernel throughput outweighs launch overhead. The biggest absolute time savings:

Query	CPU time	GPU time	Speedup	Time saved
q24 (part 1+2)	81.6s	15.9s	~5.1×	65.7s
q23 (part 1+2)	79.4s	16.4s	~4.9×	63.1s
q93	63.7s	5.6s	11.4×	58.1s
q76	30.4s	3.4s	9.0×	27.0s
q64	35.4s	8.5s	4.2×	26.9s
q50	27.6s	3.5s	7.9×	24.0s

Speedup distribution across all 103 executions:

Speedup band	Queries
≥5×	15
4–5×	13
3–4×	21
2–3×	26
1–2×	27
<1× (CPU faster)	1

Median per-query speedup 2.94× (geomean 2.84×). The heaviest wins (q50, q76, q93) are aggregation- and shuffle-join-intensive queries that convert cleanly to GpuHashAggregate and GpuBroadcastHashJoin.

Where CPU wins

With RAPIDS 26.04.2, the following query showcases a workload pattern where CPU was faster:

Query	CPU time	GPU time	Ratio	Root cause
q16	0.96s	1.44s	CPU 1.5x faster	Trivial/near-empty scan. Sub-second runtime where GPU kernel-launch overhead is not amortized

Choosing the right instance

Instance	Best for	Summary
g7.4xlarge (RTX PRO GPU)	Fastest and most cost-effective	Up to 3.7× faster than comparable CPU instances and up to 31% cheaper per run. Completes in 4.7 min compared to 17.2 min. Best choice for both speed and cost efficiency.
CPU instances (m8a / m8id / m9gd)	Flexibility, availability, and always-on workloads	Multiple architecture options deliver similar Spark SQL performance. Choose CPU when GPUs are unavailable, when clusters need to remain running continuously (for example, overnight jobs ready for next-day analysis), or when workloads cannot use GPU acceleration. CPU instances offer broad availability and predictable capacity without startup delays.

G7 instances require a G-instance vCPU service quota in your account (default is often 0 for GPU types). Request a quota increase through the Service Quotas console, or use On-Demand Capacity Reservations (ODCRs) to guarantee availability for recurring batch jobs.

Based on these benchmark results, consider evaluating GPU acceleration for your own Apache Spark workloads. Start by identifying compute-intensive operations in your current pipelines, particularly those involving large-scale aggregations, joins, or machine learning feature engineering that could benefit from the performance improvements demonstrated here.

Conclusion

Amazon EMR on EKS with NVIDIA RTX PRO 4500 together provide a meaningful step forward for teams running data-intensive Spark workloads at scale. Whether you’re building ML pipelines that demand rapid feature iteration, running complex ETL transformations across massive datasets, or powering real-time analytics that can’t afford to wait on slow batch jobs, GPU-accelerated Spark on G7 delivers the performance and speed to do more. As data and AI workloads continue to evolve, GPU-accelerated analytics on Amazon EMR is becoming the foundation for data teams. Get started with GPU-accelerated Spark on Amazon EMR on EKS today by visiting Amazon EMR documentation to launch your first G7-powered cluster and see the performance gains for yourself.

About the authors

Happy 20th Birthday, Amazon EC2

2026-08-25 Channy Yun (윤석찬)

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/happy-20th-birthday-amazon-ec2/

Twenty years ago today, Jeff Barr wrote a blog post that launched the Amazon EC2 Beta. That single post introduced resizable Linux virtual servers in the cloud, billed by the hour, with one instance type (m1.small) in one Region (US East). It was minimal yet useful, and it changed how the world thinks about computing infrastructure.

In 2021, Jeff covered the fifteen years of EC2 with the backstory and memorable EC2 launches. Over the last five years, AWS has continued to push the boundaries of what cloud computing can deliver, building custom silicon for general-purpose and AI workloads and expanding EC2 into new form factors and deployment models that our customers in 2006 could not have imagined.

The 20 years in brief
In his 15th anniversary post, Jeff chose important milestones of EC2 that established the foundational building blocks that customers still rely on today. Amazon Elastic Block Store (2008) provided persistent block storage. Elastic Load Balancing, Auto Scaling, and Amazon CloudWatch (2009) made applications scalable and highly available. Amazon Virtual Private Cloud (2009) gave customers logically isolated networks. AWS Nitro System (2017) enabled faster innovation and enhanced security. AWS Graviton processors (2018) were designed for cost-sensitive scale-out workloads.

Over 20 years, EC2 grew from one to over 1,200 instance types to meet customer needs across general-purpose, compute-, memory-, and storage-optimized, accelerated computing, and high-performance computing families. These instances expanded from one AWS Region to 39 Regions globally. AWS also extended EC2 beyond the Region boundary with AWS Outposts (2018) running EC2 instances locally, AWS Local Zones (2019) place globally, and AWS Wavelength (2019) inside global 5G telecommunications carrier networks.

While I hate to play favorites, I want to choose some of my favorite EC2 launches of the past five years:

AWS Inferentia for ML inference at scale (2019): We introduced purpose-built ML inference instances (inf1) with AWS Inferentia chips. Amazon EC2 Inf2 instances became generally available in April 2023 for large-scale generative AI inference workloads. Together with the Inferentia family, AWS Trainium instances now give customers a full stack of AWS-designed silicon optimized for every phase of the AI lifecycle both inference and training.
EC2 Mac instances (2020): The first Mac instances (mac1) were built on Apple Mac mini with Intel Core i7 (Coffee Lake) on the AWS Nitro System. Mac M1 (mac2) instances launched in July 2022 as the first Arm-based macOS instances on EC2. M2 Pro Mac instances followed in 2023, M4 and M4 Pro Mac instances in 2025, M3 Ultra Mac instances and M4 Max Mac instances in 2026, giving Apple developers a complete range of cloud-based build and test environments for macOS, iOS, iPadOS, tvOS, watchOS, and visionOS apps.
AWS Trainium for full-stack AI workloads at scale (2021): In November 2021, we previewed Trn1 instances with AWS Trainium accelerators optimized for high-performance deep learning training. In December 2024, Trn2 instances powered by AWS Trainium2 launched, with Trn2 UltraServers linking 64 Trainium2 accelerators via NeuronLink for training trillion-parameter foundation models. At AWS re:Invent 2025, Trn3 UltraServers powered by AWS Trainium3 deliver the best token economics for next-generation agentic, reasoning, and video generation applications. A single Trn3 UltraServer interconnects up to 144 Trainium3 chips to train and serve the largest frontier models. Now, AWS Trainium3 delivers the leading price-performance for high-performance AI training and inference at scale.
EC2 Capacity Blocks for ML (2023): This new EC2 usage model further democratizes ML democratizes ML by making it easy to access GPU instances to train and deploy ML and generative AI models. You reserve the GPU capacity you need (initially P5 instances) for a future date and only for the duration you require. In November 2024, EC2 Capacity Blocks for ML added supported for provisioning in a matter of minutes and extending up to six months. Now, EC2 Capacity Blocks for ML supports P6-B300, P6-B200, P5e, P5en, P4d, P4de, Trn1, Trn2, and Trn3 instances in addition to P5.
AWS Graviton5 (2025): Building on eight years of Graviton innovation since 2018, we previewed Graviton5 chips in AWS re:Invent 2025 and launched M9g and M9gd instances powered by Graviton5 and built on the sixth-generation AWS Nitro System. C9g and C9gd followed in June 2026. Now, Graviton5 features 192 cores, a 5x larger cache, and up to 33% lower inter-core latency, making it well suited for the growing demands of agentic AI workloads such as real-time reasoning, code generation, and multi-step task orchestration that require continuous, high-throughput CPU compute at scale.
AWS Nitro Isolation Engine (2026): Customers wanted to see, not just hear from us, proof of workload isolation in the Nitro Hypervisor. The Nitro Isolation Engine is a purpose-built component inside the Nitro Hypervisor, harnessing formal verification to provide mathematical assurance that customer workloads are isolated from each other and AWS operators, pioneering a new standard for mathematically proven cloud security. This feature is also based on the sixth-generation AWS Nitro System which has continued to evolve since our introduction in 2017.

The foundation underneath it all
Despite two decades of innovation, the fundamental value proposition of Amazon EC2 has not changed. Customers use it to get secure, resizable compute capacity in minutes, pay only for what they consume, and scale on demand without long-term commitments. That same flexibility now extends to AI workloads at a scale no one anticipated in 2006.

EC2 remains the foundational compute layer of AWS. Amazon ECS, Amazon EKS, AWS Lambda, AWS Fargate, AWS Batch, Amazon EMR, Amazon SageMaker AI, and Amazon Bedrock ultimately run on EC2 capacity. Every architectural pattern customers have built over the past twenty years, from simple web servers to trillion-parameter foundation model training clusters, starts with a decision to launch an instance.

We made strong foundational decisions in 2006, and we left room for the service to grow. Twenty years later, that strategy of creating services that are minimal-yet-useful, launching quickly, and iterating rapidly in response to your feedback continues to guide how we build. The next twenty years of cloud computing will demand capabilities we have not yet imagined. Amazon EC2 will continue to be the foundation where your workloads run.

To learn more about Amazon EC2, visit the Amazon EC2 product page or check out what’s new with EC2.

— Channy

AWS Weekly Roundup: Student Rewards on AWS Builder Center, Local Zone in Las Vegas, and more (August 24, 2026)

2026-08-24 Esra Kayabali

Post Syndicated from Esra Kayabali original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-student-rewards-on-aws-builder-center-local-zone-in-las-vegas-and-more-august-24-2026/

During my time at AWS, I have always looked for opportunities to work with students. I have delivered over 50 talks at universities across the region, and watching the potential in the room is always a strong motivator. It reminds me of why I do this work, and that the students I meet today may well become our customers and collaborators tomorrow. That is why I am happy to open this week with Student Rewards on AWS Builder Center.

Rick Suttles published Introducing Student Rewards on AWS Builder Center, a new benefit for verified higher education students. When you verify your enrollment through SheerID and complete your Builder Center profile, you unlock 12 months of premium AWS Skill Builder access (900+ courses, hands-on labs, certification exam prep, and game-based learning). From there, you earn badges through actions on Builder Center: publishing articles, commenting, and maintaining engagement. At 7 badges, you unlock $10 in AWS Credits. At 14 badges, another $20 in credits. At 21 badges, you earn an AWS Foundational Certification exam voucher ($100 value).

This represents a commitment of over $500 million in resources during this back-to-school season, providing students with the training, tools, and certification needed to start building their careers in cloud and AI. Student Rewards is available to students 18 years or older and enrolled at accredited higher education institutions worldwide, subject to verification and applicable terms.

Verify your student status and start learning, earning badges, and unlocking rewards!

Last week’s launches
Here’s what else happened this week.

A new AWS Local Zone in Las Vegas, Nevada – This new Local Zone supports Amazon EC2 C7i, M7i, R7i, and C8gn instances, Amazon EBS, Amazon ECS, Amazon EKS, Application Load Balancer, and AWS Direct Connect. AWS Local Zones are now available in more than 30 metropolitan areas worldwide. In addition, AWS added a fourth Availability Zone to the Europe (London) Region, delivering next-generation AI and ML capacity with Trn3 and P6 accelerated instances alongside general-purpose compute.
Amazon EC2 Auto Scaling now supports batch instance termination – You can now pass up to 100 instance IDs to the TerminateInstanceInAutoScalingGroup API to terminate them as a batch, reducing the number of API calls needed to scale down your Auto Scaling groups. Batch termination is designed for workloads that need to rapidly scale down, such as AI/ML training jobs, container orchestrators, or event-driven architectures that spin up large fleets temporarily.
AWS CloudShell now includes a built-in visual file editor – CloudShell now includes a visual file editor that you can launch directly from your shell session using a single edit command. The editor supports syntax highlighting, find-and-replace, multi-line selection, copy-paste, and undo-redo in a single browser session. Whether you are updating a deployment script, modifying an agent steering file, editing a CloudFormation template, or fixing a Lambda function, the editor provides a seamless edit-and-run experience without leaving CloudShell.
Amazon Bedrock now supports SpaceXAI Grok 4.6 with cross-Region inference – Grok 4.6, a frontier model built for coding, agentic tasks, and knowledge work, is now available on Amazon Bedrock. The model runs on the bedrock-runtime endpoint with support for the Responses, Chat Completions, and Converse APIs, and works with existing account-level controls including model invocation logging, Amazon CloudWatch metrics, and cost itemization in AWS Cost Explorer.
Amazon Bedrock expands API support and introduces cross-Region inference for OpenAI models – Amazon Bedrock now supports OpenAI GPT-5.6 models (Sol, Terra, and Luna) with the Responses, Converse, and Chat Completions APIs, and adds cross-Region inference. Geo cross-Region inference routes requests within a predefined geography (including new US Geo support with this launch), while Global cross-Region inference serves requests from any commercial AWS Region at a lower per-token cost.
AgentCore payments is now generally available in Amazon Bedrock AgentCore – At general availability, AgentCore payments includes Quick Create for Coinbase credential provisioning directly within the AgentCore console, a curated Coinbase Bazar MCP server of pay-per-use x402 endpoints via AgentCore gateway, support for the Machine Payment Protocol (MPP), and the “upto” scheme in the x402 protocol for pay-per-inference and dynamic pricing use cases. To learn more, visit the AI Blog post.
AWS Glue 6.0 delivers 30% price reduction and Iceberg v3 support – AWS Glue 6.0 is built on a fully modernized runtime, Apache Spark 4.1, Python 3.13, and Scala 2.13, delivering 30% lower pricing than previous AWS Glue versions. With Iceberg v3, Glue 6.0 adds the VARIANT data type with automatic shredding for faster reads on semi-structured data, deletion vectors for high-performance row-level updates, geometry and geography data types for spatial processing, and flexible schema evolution.

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news
Here are some additional posts you may find useful:

Updates to your AWS Sign-In experience – AWS is gradually introducing updates to the sign-in and sign-up experience. The redesigned sign-in page introduces a unified email entry point for root users and customers using the new email-based sign-in method, while IAM users continue signing in with their account ID, username, and password. The page also includes sign-in options for customers whose AWS account was created using a supported identity provider (Google, GitHub, Apple, or Amazon.com). A redesigned session selection page simplifies viewing and managing multiple active account and role sessions. If your organization relies on browser automation or scripted workflows that interact with the sign-in page, review the post to understand how these changes might affect your configuration.
In the works: AWS Builder Lofts in Berlin, Hyderabad, and São Paulo – My colleague Channy announced plans to open new Builder Lofts in three cities. Since the first Builder Loft opened in San Francisco in July 2025, it has welcomed more than 22,500 developers through its doors. Each new location will be a permanent community space offering free workshops, networking events, pitch nights, content creation spaces, and co-working areas. Berlin will focus on digital sovereignty and security-readiness, Hyderabad on AI and cloud-native architecture, and São Paulo on supporting Latin America’s developer ecosystem.
AWS and Amazon WorkSpaces recognized as a Leader in the 2026 Gartner Magic Quadrant for Desktop as a Service – AWS has been named a Leader in the 2026 Gartner Magic Quadrant for Desktop as a Service (DaaS) for the third consecutive year, evaluated on Completeness of Vision and Ability to Execute. Gartner noted strengths in operations, geographic strategy, and overall viability. This is also the first year the evaluation includes Amazon WorkSpaces for AI agents, a capability that runs AI agents within the same desktop environment, security perimeter, and audit trail as human users.

For a full list of AWS blog posts, be sure to keep an eye on the AWS Blogs page.

Upcoming AWS events
Check your calendar and sign up for upcoming AWS events:

AWS Summits – Free in-person events for builders and innovators to learn, connect, and explore the latest in cloud. Coming up: Zurich (September 2), São Paulo (September 3), Tel Aviv (September 10), and Dubai (September 30). You can also watch keynotes and sessions on the AWS Summits Global Livestream + On-Demand Hub. To access live and on-demand content for a particular Summit, you need to be registered for that event.
AWS Community Days – Community-led conferences planned and delivered by community leaders. Upcoming events include Bolivia, Santa Cruz (August 29), Toronto, Canada (August 29), JAWS SONIC, Tokyo (September 5), Warsaw, Poland (September 8).

Visit the AWS Builder Center to meet other builders, contribute solutions, and find resources that help you keep building.

Summer is slowly coming to an end, and I am already planning a few days off in the coming months to keep me motivated through the rainy autumn ahead. I hope you are doing the same. Come back next week for more!

— Esra

Amazon Linux default SSM parameter will now track the latest kernel

2026-08-20 Gokul Govindaraju

Post Syndicated from Gokul Govindaraju original https://aws.amazon.com/blogs/compute/amazon-linux-default-ssm-parameter-will-now-track-the-latest-kernel/

Today we are announcing that the Amazon Linux kernel-default AWS Systems Manager (SSM) parameter will now update to point to the latest Amazon Linux kernel version as new kernel versions get released. On August 17, 2026, for Amazon Linux 2023 (AL2023), the SSM parameter was updated from kernel 6.1 to kernel 6.18. As new kernel versions get released (expected annually), the parameter will continue to update to the latest kernel version after a validation period.

This post explains the default kernel behavior, what it means for your workloads, and how to manage the transition.

What’s changing?

Amazon Linux ships multiple kernel versions and has tracked a default kernel for each OS version. For example,
the AL2023 parameter:

ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-{minimal}-kernel-default-{x86_64 arm64}

has remained on kernel 6.1 since launch. Going forward, the kernel-default SSM parameter will update to the latest kernel as new versions are released. Each new kernel will go through a 3- to 6-month validation period after GA before we update the default. This window gives you time to test the new kernel before the change. We will announce the kernel-default upgrade date before it takes effect.

SSM Parameter	Resolved to (Before)	Resolves to (Now)
`al2023-ami-{minimal}-kernel-default-{x86_64, arm64}`	Kernel 6.1 AMI	Kernel 6.18 AMI (what’s changed)
`al2023-ami-{minimal}-kernel-6.18-{x86_64, arm64}`	Kernel 6.18 AMI	Kernel 6.18 AMI (unchanged)
`al2023-ami-{minimal}-kernel-6.1-{x86_64, arm64}`	Kernel 6.1 AMI	Kernel 6.1 AMI (unchanged)

Note: Already-running instances will keep the kernel they booted with and are not affected by this change. Only new instances launched from the kernel-default parameter will boot kernel 6.18. If you already use a version-specific SSM parameter, nothing changes for you.

Why are we making this change?

The Linux kernel is the foundation of workloads you run on Amazon Elastic Compute Cloud (Amazon EC2) and other services. Each new kernel brings meaningful improvements. For example, kernel 6.18 includes the Earliest Eligible Virtual Deadline First (EEVDF) CPU scheduler for fairer CPU time distribution and improved latency in mixed workloads. The kernel also increases Transmission Control Protocol (TCP) receive buffer for better network throughput on high-bandwidth instances.

Previously, customers who wanted to run the latest Amazon Linux kernel had to manually update their SSM parameter references and redeploy each time a new kernel became available. With this change, you can receive these improvements without needing to manually upgrade.

Evaluating the default kernel upgrade

Staying on the default kernel is the recommended approach as it allows your new instances to always run the latest validated kernel with no manual intervention. However, because the default will now advance annually, you should build processes to validate that the new kernel works for your workload before each upgrade takes effect. If your workload has specific requirements that mandate a fixed kernel version, evaluate whether the new default is compatible or revert to a kernel version that suits your use case.

If you haven’t validated kernel 6.18 yet, we recommend launching test instances on kernel 6.18 using the version-specific SSM parameter al2023-ami-{minimal}-kernel-6.18-{x86_64, arm64}. For instructions on referencing SSM parameters in your launch configuration, see the AL2023 User Guide.

Staying on or reverting to a specific kernel version

If you experience issues with the new default, or if your workload requires a specific kernel version for additional validation time or any other reason, revert to the version-specific SSM parameter. Change your references from al2023-ami-{minimal}-kernel-default-x86_64 to al2023-ami-{minimal}-kernel-{kernel_version}-x86_64 (for example, al2023-ami-kernel-6.1-x86_64). This applies anywhere you resolve an AL2023 AMI, including AWS CloudFormation templates, launch templates, Amazon EC2 Auto Scaling groups, CI/CD pipelines, or CLI scripts. For examples, refer to the AL2023 User Guide.

Each of the supported kernels (6.1, 6.12, and 6.18) continue to receive updates as defined in AL2023 kernel lifecycle. When staying on a specific version, we recommend tracking the kernel lifecycle and planning upgrades before the kernel reaches end of support.

Note: For Federal Information Processing Standards (FIPS) workloads, the default kernel may not always be the FIPS-validated kernel. If you require FIPS mode, see AL2023 FIPS FAQ.

Conclusion

In this post, we announced that the Amazon Linux default SSM parameter will now upgrade to the latest kernel as new kernel versions are released. The AL2023 kernel-default parameter was updated from kernel 6.1 to kernel 6.18 on August 17, 2026. We explained how the new cadence works, how already-running instances are unaffected, and how to stay on a specific kernel version if your workload requires it.

To learn more, see the AL2023 Kernel documentation and the AL2023 release notes. For questions or issues, contact AWS Support.

AWS Weekly Roundup: EC2 application status checks, IAM role manager, OpenAI Daybreak on Bedrock, and more (August 17, 2026)

2026-08-17 Channy Yun (윤석찬)

Post Syndicated from Channy Yun (윤석찬) original https://aws.amazon.com/blogs/aws/aws-weekly-roundup-ec2-application-status-checks-iam-role-manager-openai-daybreak-on-bedrock-and-more-august-17-2026/

Last week, the OpenSearch and Valkey teams visited Seoul to meet open source developers and contributors in the Open Source Summit Korea 2026 and MCP DevSummit Seoul 2026. At the four-day event, community leaders and users of open source projects and emerging agent AI gathered to share knowledge, collaborate on solutions, and push the projects forward.

Leaders of the Korean OpenSearch communities volunteered to participate in the booth, and also had time to network and interact in the user group meetup.

OpenSearch is an open source, enterprise-grade search and observability suite that brings order to unstructured data at scale. On June 9, 2026, OpenSearch 3.7 introduced new tools designed to query, alert, and track SLOs across logs, traces, and metrics through a single interface and retrieve vectors up to 5.5x faster for improved search performance. Since July 30, 2026, you can run OpenSearch version 3.7 on Amazon OpenSearch Service for improvements in vector search performance, search relevance, and Query Insights.

Valkey is an open source high-performance key/value datastore that supports a variety of workloads such as caching, message queues, and it can act as a primary database. On May 19, 2026, Valkey 9.1 introduced a redesigned I/O threading model that improves throughput by up to 17% and reduces memory usage for strings under 128 bytes by up to 20%. Since June 23, 2026, you can run Valkey 9.1 in Amazon ElastiCache for node-based clusters, delivering higher throughput, improved memory efficiency, and stronger access control for multi-tenant workloads.

You can meet our open source teams at upcoming OpenSearch and Valkey events.

Last week’s launches
Here are some launches that got my attention:

Amazon EC2 application status checks: Amazon EC2 introduces a new status check that helps you detect and respond to application-level issues on your EC2 instances. With application status checks, EC2 monitors applications to detect issues such as a web server that has stopped accepting requests, a Docker daemon that is not running, an incorrect networking configuration, or a network interface that is no longer passing traffic. To learn more, visit the Application status checks documentation.
AWS IAM role manager to set up IAM roles automatically: You can use a new role manager that automatically sets up the IAM roles your AWS services need. When you set up a supported service in the console, role manager creates a default role on your behalf, or reuses one that already exists in your account if it matches the required permissions. Role manager supports six AWS service consoles at launch. To learn more, read How AWS IAM role manager rethinks the starting point for IAM roles.
OpenAI Daybreak available to eligible customers on Amazon Bedrock: Daybreak is the cyber defense initiative from OpenAI that gives defenders governed access to frontier AI for cybersecurity work. For most security teams, Daybreak Blue, powered by GPT-5.6 Sol, serves as the starting point across defensive workflows including vulnerability discovery, detection engineering, and incident response. Daybreak Red, powered by a new GPT-5.6 Cyber, is designed for advanced, authorized tasks such as vulnerability research, exploit reproduction, and mitigation development. To enroll, contact OpenAI or reach out to your AWS account team for guidance on eligibility. To learn more, read the AI Blog post.
New foundation models in Amazon SageMaker JumpStart: We’re expanding the portfolio of foundation models available to AWS customers. These models address different enterprise AI challenges with specialized capabilities:

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news
Here are some additional projects and news items you may find interesting:

The deprecation of email validation in AWS Certificate Manager: ACM will discontinue support for email-validated public certificates by September 30, 2027. If you use email validation for your ACM public certificates, you need to migrate to DNS validation before that date. For Amazon CloudFront distributions, HTTP validation is also available.
The next-generation AWS VPN Client with CLI support and admin controls: You can use a new AWS VPN Client built on OpenVPN3. With the new client, you get full backward compatibility with existing AWS Client VPN endpoints while delivering the automation capabilities and security posture that enterprise networking teams have been asking for.
Oracle Exadata on Exascale for Oracle AI Database@AWS: ExaDB-XS brings Exadata-class performance and availability through a consumption-based model. With ExaDB-XS, you can scale compute and storage independently in small increments and pay only for what you consume.

For a full list of AWS blog posts, be sure to keep an eye on the AWS Blogs page.

Learn more about AWS, browse and join upcoming AWS-led in-person and virtual events, startup events, and developer-focused events including AWS Summits and AWS Community Days. Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development.

That is all for this week. Check back next Monday for another Weekly Roundup!

— Channy

Designing for failure: Building resilient systems on AWS

2026-08-13 Dhvani Vora

Post Syndicated from Dhvani Vora original https://aws.amazon.com/blogs/compute/designing-for-failure-building-resilient-systems-on-aws/

In cloud computing, failure in distributed systems isn’t a matter of if, but when. Modern applications span servers, Availability Zones, and Regions. Each component represents a potential point of failure. Resilient applications engineer fault tolerance into their architecture, building systems that self-recover and maintain availability. This post is written for engineers and architects who run distributed data systems such as Apache Cassandra, Apache Kafka, or HDFS on Amazon Elastic Compute Cloud (Amazon EC2) and want to build resilience against hardware failure.

We were working with a customer during one such incident and wanted to share the example. The customer runs a web application that uses Cassandra as its data store, handling both read-heavy and write-heavy workloads at a scale of millions of queries per day.

The 2 AM wake-up call nobody wants

Consider a platform that monitors millions of enterprise network devices across hospitals, universities, and airports worldwide. It detects problems before IT teams even notice them. For that platform, a 2 AM page is more than inconvenient. When your value proposition is catching failures before anyone else does, being caught off-guard by your own infrastructure failure is existential.

The engineering team was deep in quarterly planning when their monitoring dashboard lit up. Three Cassandra nodes had gone dark simultaneously. This was not a graceful shutdown or a rolling restart. It was a hard failure with no warning.

Their architecture is typical of high-scale telemetry platforms. Kafka-powered microservices ingest device telemetry, Apache Flink handles real-time anomaly detection, and Apache Airflow orchestrates batch analytics and firmware updates. All of these rely on Apache Cassandra as the distributed database backbone. The database stores billions of daily writes and handles millions of queries per day.

What actually happened

Three i4i.4xlarge instances running Cassandra nodes failed simultaneously in the SFO region. Investigation revealed that all three instances were colocated on the same physical host. That host suffered a hardware failure, taking all three instances offline at once.

Engineers spent ninety minutes digging through system logs trying to determine the root cause. The root cause was architectural. The deployment lacked Partition Placement Groups, creating a single point of failure where logical replication was undermined by physical collocation.

The good news: Cassandra maintained service availability with no data loss thanks to its replication factor. The bad news: for over an hour, the system ran on a thin safety margin. One more node failure in the same replication group would have caused data unavailability for a subset of queries. That is real customer impact for a platform that promises always-on monitoring.

This is the insidious nature of correlated failures. Individual node failures are expected and designed for. That is the whole point of replication. But when your replicas share physical infrastructure, replication becomes a paper guarantee. You have three copies of the data, but they all live on the same machine.

Making matters worse, their monitoring tools completely missed the initial failure. System status checks correctly flagged the host-level problem. But without Amazon CloudWatch alarms configured to act on those checks, detection was entirely reactive. The team found out because other things started behaving oddly, not because an alarm told them three nodes were down.

Hardware fails. You can’t fix it with a patch or configuration change. The real questions are how fast you detect it, how well your system handles it, and whether failures are correlated.

How the team responded and what they changed

The operations team manually replaced two failed instances with new ones on healthy hardware and restarted the third for log collection. Once replacement instances came online, new Cassandra nodes automatically rejoined their clusters and streamed data from surviving replicas. This process took several hours depending on data volume. Only after full synchronization did the clusters return to full redundancy.

The team recognized that this ninety-minute manual scramble wouldn’t scale. Similar problems had happened before, and each time they followed the same reactive pattern: page, investigate, manually replace, wait for streaming, breathe. Here’s what they implemented to break that cycle, and what you should implement too.

Two-track incident timeline. The top track, labeled Before automation: about 90 plus minutes of manual response, shows five milestones: at 0 minutes three nodes fail simultaneously. At about 5 minutes cascading errors are noticed with no alarm. At 90 minutes the root cause is found in system logs. At 90-plus minutes instances are manually replaced. And after several hours data streaming completes and full redundancy is restored. The bottom track, labeled After automation: under 5 minutes to recovery, shows four milestones: at 0 seconds the system status check fails. At about 60 seconds a composite alarm fires. At about 2 minutes Auto Scaling replaces the node. And in under 5 minutes a lifecycle hook rejoins the node to the cluster.

Figure 1: The same failure handled two ways. Manual response took over 90 minutes plus hours of streaming. The automated path completes recovery in under 5 minutes.

1. Use Partition Placement Groups to isolate failure domains

The three crashed servers shared a physical machine because no one told AWS otherwise. Without placement group constraints, instances are placed based on available capacity. That can mean multiple instances land on the same host. For stateless web servers, this rarely matters. For distributed databases whose entire resilience model depends on replicas being independent, it’s a silent architecture bug waiting to become a 2 AM incident.

Partition Placement Groups fix this by distributing instances across separate hardware racks. Each partition maps to a distinct set of physical infrastructure, with separate power and separate network switches. When one rack fails, it affects only the instances in that partition.

Diagram comparing two Cassandra deployments. On the left, labeled Before, all three Cassandra nodes run on a single physical host, so a host failure takes down all three replicas. On the right, labeled After, the three nodes are distributed across three Partition Placement Group partitions on separate racks (Rack A, Rack B, Rack C). When Rack B fails, only Node 2 is lost and the cluster survives.

Figure 2: Distributing Cassandra replicas across Partition Placement Group partitions so a single rack failure affects only one node.

You can create up to seven partitions per Availability Zone, with as many instances as needed in each. By mapping Cassandra replicas to separate partitions, a single hardware failure takes down one node instead of three. This applies to any distributed system that maintains replicas, such as Kafka, HDFS, or Cassandra.

Key insight: Align your Partition Placement Group partitions with your application’s replication topology. If Cassandra uses a replication factor of 3, place each replica in a different partition. This means the physical isolation boundary matches the logical replication boundary.

CLI example:

aws ec2 create-placement-group \
  --group-name cassandra-partitioned \
  --strategy partition \
  --partition-count 3

aws ec2 run-instances \
  --placement "GroupName=cassandra-partitioned,PartitionNumber=1" \
  --instance-type i4i.4xlarge \
  --image-id ami-xxxxxxxx

Partition Placement Groups (up to 7 partitions per AZ, unlimited instances per partition) are designed for large distributed workloads. Spread Placement Groups (max 7 instances per AZ, each on a separate rack) suit small critical clusters. For a Cassandra deployment at scale, Partition is the right choice. Learn more in the Amazon EC2 placement groups documentation.

2. Monitor system status checks and use composite alarms

The Cassandra team’s monitoring blind spot came down to a distinction many teams overlook. AWS runs two health checks on every instance: instance status checks (your guest OS and software) and system status checks (the physical hardware underneath). When a system status check fails, the problem is below your control. This includes a host crash, a power failure, or network loss at the rack level. No amount of SSH-ing will help, because the box is unreachable.

The Cassandra team had no Amazon CloudWatch alarms configured on either check type. That meant the only signal was cascading application errors noticed by engineers who happened to be awake. Set these up on day one, before your first production deployment.

To avoid false alarms during normal reboots, where metrics may briefly go missing, combine system status checks with application-level health monitoring using composite alarms. When both fail together, you know there’s a real problem. See the CloudWatch composite alarms documentation for setup details.

3. Automate instance recovery and replacement

The Cassandra team’s ninety-minute recovery wasn’t slow because the engineers were incompetent. It was slow because humans were in the loop. Waking up, assessing, deciding, acting, and verifying: each step adds minutes that compound under pressure. Auto Scaling groups remove the human from the critical path.

Place your Cassandra nodes in an Auto Scaling group. Auto Scaling continuously runs health checks on every instance, and when it marks an instance unhealthy, it terminates it and launches a replacement on different physical hardware, automatically placed within your Partition Placement Group. Under normal conditions, an instance whose system status checks fail is replaced within a few minutes.

The gap to close is detection, not replacement. Rather than waiting for Auto Scaling to reach its own conclusion, have the composite alarm from the previous section explicitly tell Auto Scaling the instance is unhealthy by calling the SetInstanceHealth API. As soon as your combined signal (system status check plus application-level check) confirms a real failure, mark the instance unhealthy and let Auto Scaling replace it immediately. This sidesteps any ambiguity in detection and starts recovery in seconds rather than minutes.

For stateless services, this is enough. For stateful systems like Cassandra, you need an additional step. Lifecycle hooks pause new instances before they join the cluster. A raw Amazon EC2 instance isn’t a functioning Cassandra node. It needs to join the ring, stream data from peers, and verify consistency before serving traffic. Read more in the Amazon EC2 Auto Scaling lifecycle hooks documentation.

In this customer’s case, automating these steps cut recovery time from ninety minutes of manual intervention to under five minutes of automated recovery.

A note on stateful recovery: automated replacement only handles the infrastructure layer. For Cassandra specifically, the new node still needs to stream data from peers before it’s fully operational. The key improvement isn’t eliminating that streaming time. It’s eliminating the human response time before streaming even begins.

4. Build automated incident response with AWS Systems Manager

When servers fail, you face competing priorities. You need to replace them fast to restore capacity, and you need to preserve logs for root cause analysis. These goals conflict when done manually. The Cassandra team restarted one failed node solely to collect diagnostic data before replacing it, adding time to an already long recovery.

AWS Systems Manager runbooks automate this tradeoff away. Build a workflow that runs these steps in sequence:

Isolate the failed instance by detaching it from the load balancer target group.
Create an Amazon EBS snapshot and capture available logs to Amazon S3.
Terminate the instance so that Auto Scaling can replace it.
Notify the on-call channel with the instance ID, failure type, and Amazon S3 log location.

A subtle but important detail: when the instance’s lifecycle is managed by an Auto Scaling group, let the group replace it. Terminating the instance directly only delays recovery, because the group first has to notice the instance is gone before it launches a replacement. Instead, call the TerminateInstanceInAutoScalingGroup API. This tells EC2 Auto Scaling to terminate the unhealthy instance and immediately launch a replacement in one coordinated action. Trigger this runbook automatically with Amazon EventBridge rules that match Amazon EC2 state-change events. The result is that forensic data is preserved, replacement happens in parallel, and the on-call engineer gets a notification after the system has already healed, rather than a page asking them to start fixing it.

Five-step automated recovery workflow shown left to right. Step 1: the Amazon EC2 system status check fails on the host. Step 2: an Amazon CloudWatch composite alarm triggers. Step 3: Auto Scaling terminates the unhealthy node and launches a replacement. Step 4: an AWS Systems Manager runbook takes a snapshot and sends logs to Amazon S3. Step 5: a lifecycle hook streams data, verifies, and rejoins the node to the cluster. The whole flow is triggered by Amazon EventBridge and reduces recovery from about 90 minutes of manual work to under 5 minutes.

Figure 3: The automated recovery workflow, from hardware failure detection through node rejoin, orchestrated by Amazon EventBridge, Auto Scaling, and AWS Systems Manager.

5. Invest in observability before you need it

After resolving the Cassandra incident, the team asked a harder question: what else is silently failing? They ran a broader health assessment, and the answer was sobering. Unstable Redis connections were dropping under load. Amazon EBS volumes were running with elevated latency. Application Load Balancer health check intervals were misconfigured. Secondary databases were approaching connection pool exhaustion. Any of these could cause the next outage, and none of them had triggered a single alert.

This is the pattern. Teams invest in monitoring for the system that recently broke while the next failure quietly builds elsewhere. The better approach is treating observability as infrastructure. Deploy it everywhere from day one, not bolted on after the post-mortem.

Deploy the CloudWatch agent for system-level and application-level metrics. Use Amazon CloudWatch Synthetics canaries to continuously test critical user paths such as login, data ingestion, and dashboard rendering. Set up distributed tracing with AWS X-Ray to identify latency bottlenecks across your microservice mesh. The goal isn’t only knowing that services are running. It’s continuously confirming they’re working correctly from the customer’s perspective.

The Cassandra team built what they call their “resilience dashboard.” It’s a single view surfacing Partition Placement Group distribution, replica lag, system status check state, and Auto Scaling group health. When the next incident happens, they won’t be scrambling to figure out what’s broken. They’ll open one dashboard and know immediately whether their defenses are holding.

Placement groups: Quick reference

The team’s outage involved Partition Placement Groups, but Amazon EC2 offers three placement group types. Choosing the wrong one is a common mistake, so here’s how they compare:

Type	Max instances	Isolation level	Best for
Partition	Unlimited (up to 7 partitions per AZ)	Separate racks per partition	Large distributed databases (Cassandra, Kafka, HDFS)
Spread	7 per AZ	Each instance on a separate rack	Small critical clusters needing maximum isolation
Cluster	Unlimited	Same rack (co-located)	HPC, ML training, low-latency workloads

If the Cassandra team had used Spread Placement Groups instead, they would have hit the 7-instance-per-AZ ceiling almost immediately at their scale. Partition Placement Groups gave them isolation and room to grow. For the highest-criticality deployments, combine placement groups with multiple Availability Zones. You get separate racks and separate data centers, protecting against both rack-level failures and zone-wide events like power grid outages.

The bigger picture: Resilience is a practice

Building resilient systems isn’t a one-time project. It’s a practice that evolves with your architecture. Start by assessing your workloads with the AWS Well-Architected Tool to identify single points of failure you might not see day-to-day. Define Service Level Objectives, so your team agrees on what “good enough” looks like. Not every service needs 99.99% availability, but you need to know which ones do.

Then layer your defenses. Placement groups prevent correlated hardware failures, composite alarms detect problems within minutes, and automated recovery fixes common issues without waking anyone up.

Test regularly. Run disaster recovery drills quarterly. Don’t rely only on tabletop exercises. Run actual failovers in pre-production environments. Use AWS Fault Injection Service to simulate hardware failures and zone outages in a controlled way. Hold blameless post-mortems after every incident to understand what broke, why it wasn’t caught earlier, and what you’ll change.

After this incident, the team deployed Partition Placement Groups, configured composite alarms, and automated their response process. The next time hardware fails, and it will, it won’t cause the same damage.

Consider adopting Chaos Engineering as a discipline. The principles of Chaos Engineering encourage teams to proactively inject failures into production-like environments to uncover weaknesses before they cause real outages. AWS Fault Injection Service makes it straightforward to run these experiments safely, with guardrails that automatically stop experiments if impact exceeds defined thresholds.

For related guidance, see the AWS Well-Architected Framework Reliability Pillar and the Amazon EC2 Auto Scaling User Guide. A sample Systems Manager runbook and AWS CloudFormation template for the automated recovery workflow described in this post is available in the AWS Samples GitHub repository.

If you’ve implemented similar resilience patterns or have questions about placement groups and automated recovery, share your experience in the comments.

Key takeaways

Challenge	Solution
Multiple instances on same physical host	Partition Placement Groups
No health notification for sudden failures	Amazon CloudWatch alarms on system status checks
Missing metrics during host reboots	Composite alarms with application-level health checks
Manual, slow incident response	Automated recovery with Auto Scaling and lifecycle hooks
Delayed root cause identification	Systematic triage starting at the infrastructure layer
Reduced redundancy after failure	Auto Scaling groups for automatic replacement
Recurring confidence erosion	Proactive architectural reviews and observability investment

Amazon EC2 provides tools like placement groups, managed services with built-in high availability, and automation frameworks like AWS Systems Manager. Select the right ones for your workload and test them relentlessly. Failure is inevitable. Your readiness determines the outcome.

Background: EC2 metal, Nitro, and Firecracker

Solution overview

Prerequisites

Density tunables on EC2 metal

From tunables to measurement: labsweep

Results on representative workloads

The 1.0x ratio rule

Core pinning reduces the tail latency for most real workloads

SMT depends on your workload

Configuration matters as much as hardware

When memory becomes the limiting resource

Reproduce these results on your own workloads

Clean up

Conclusion

Blog resources

Appendix

How it works

Creating a shared tag (CLI)

Listing all resources with shared tags (CLI)

Stopping tag sharing

Viewing shared tags as a recipient

A common use case: Golden AMIs

Considerations

Conclusion

Why use a CodeDeploy restart?

How RESTART works

Performance in feature testing

The safety controls remain familiar

Traffic-control behavior

Walk through a restart deployment

Prerequisites

Start a restart deployment with the AWS CLI

Start a restart deployment on the console

Monitor the restart

Handle alarms during incident recovery

Request constraints

Conclusion

About the authors

Prerequisites

Understanding the operational challenges

Challenge 1: Silent image aging

Challenge 2: The multi-region maintenance burden

Challenge 3: Barriers to onboarding

Challenge 4: Uncertainty about update success

What’s new: DescribeImages returns the associated SSM parameter

How it works: API response walkthrough

Practical examples

Example 1: Discover the SSM parameter for an AMI using the AWS CLI

Example 2: CloudFormation with dynamic SSM references

Example 3: Terraform with SSM parameter data source

Example 4: Auto Scaling launch templates with SSM parameters

Before-and-after workflow comparison

Best practices: Using SSM parameters for golden AMI pipelines

1. Discontinue hardcoding AMI IDs

2. Create custom SSM parameters for your golden AMIs

3. Use parameter versions and labels for controlled rollouts

4. Combine with Amazon EC2 Image Builder for end-to-end automation

5. Scope IAM permissions appropriately

Clean up

Conclusion

The challenge: Getting packages into an air-gapped approval-gated fleet

How the pattern works

Choosing the upstream source

Architecture and workflow

Package baseline and scheduled upgrade workflow

Evidence from the approval workflow

The frozen package repository on Amazon S3

Building, patching, and running the fleet

The package mirror

Security boundaries and limits

Cost and operations

Prerequisites

Reference implementation

Validate the internal package management workflow

Recommended practices

Clean up

Conclusion

How Spark Connect works on Amazon EMR on EC2

Share one cluster across your team

Getting started