Tag Archives: Foundational (100)

Part 2 – Cybersecurity journeys: What even is this job?

Post Syndicated from Shannon Brazil original https://aws.amazon.com/blogs/security/part-2-cybersecurity-journeys-what-even-is-this-job/

If you’ve ever looked at a list of cybersecurity jobs and felt like none of them fit you, you might just be looking at a short list. Most career guides cover the same few roles, like the analyst watching alerts or the pen tester (penetration tester, someone hired to legally break into systems to find weaknesses) looking for a way in. Those jobs matter, but they’re a small slice of a field with room for a lot of different skills. The three people in this post do work you probably won’t find in those guides, and none of them started out qualified for it. What they had in common was a willingness to aim at something before they were ready and to build the skills on the way there. If you’ve ever talked yourself out of a role because you didn’t tick every box, their stories are worth a few minutes, because the move each of them made is one you can make too.

In Part 1: I didn’t plan this, I shared how three people found their way into security from a warehouse floor, a marketing desk, and a teenager’s bedroom. This week is about where people end up after they’re inside. I’ve been in security for over a decade, and while putting this series together I kept running into work I hadn’t come across before: measuring the risk of a million applications at once, building detection tools from the customer service side of the house, and thinking seriously about how to secure infrastructure that’s being launched into orbit.

The skill tree

Jason Lane, Security Engineer

Jason Lane, Security Engineer

When Jason Lane told me he plans his career like a Dungeons & Dragons character build, I assumed he was being metaphorical, but he meant it literally. “Think of it like a video game skill tree,” he said, and then walked me through the whole framework. You start by picturing your level 20 character, the version of yourself with the skills and the role you actually want, and then you work backward. At level 19, what choices get you to 20? At level 18, what needs to be true so you even have those options? You keep stepping back until you’re standing at level 1, looking at a path you built from the top down.

For Jason, level 20 was vulnerability research (finding unknown security flaws in software before attackers do). At the time he was a systems administrator, keeping servers and networks running, which gave him a solid grasp of how systems fit together but little exposure to picking apart the applications on top of them. That gap, application security, was the next node on his tree, so he called an IT recruiter, laid out his plan, and was honest about where he stood. The recruiter found a credit card processing company that needed an application security engineer and was open to someone who wasn’t one yet. For Jason it was a paid apprenticeship in the exact skill he was after; for the company it was a motivated engineer who would grow into the role. He describes it as a deal where both sides got what they needed, and that framing matters: he wasn’t settling for less, he was trading a steeper learning curve for a faster path to where he wanted to be.

He spent 4 years there taking applications apart to understand what made them breakable, and when he had learned what that role could teach him, he applied the same framework again. A friend offered to get him an interview at Google, and Jason’s honest reaction, that he wasn’t smart enough to work there, is the kind of self-doubt most of us recognize (also known as imposter syndrome). His friend didn’t buy it, the interviewers didn’t either, and they hired him. The lesson he took from it is one worth borrowing: the voice telling you that you don’t belong in the room is often wrong, and the way to quiet it is to put in the work until the skills speak for themselves. He went on to spend nearly 7 years doing Android vulnerability research, the kind of deep technical work most companies don’t have the budget for.

That track record opened the door to Amazon. When the team that measures application risk interviewed him, the hiring manager’s pitch was simple: we could really use your vulnerability research experience. Years of tearing systems apart to find what made them breakable turned out to be the perfect background for judging which systems were most at risk. Today, Jason measures risk across roughly a million applications at Amazon, evaluating which ones would cause the most damage if compromised and which are most likely to be targeted, so the organization can focus on the most dangerous ones first. When his manager recently presented the team’s work to a new senior leader, the example he used was a project Jason had built from scratch. Jason’s name wasn’t on the slide, and he told me he didn’t need it to be, but you could hear the pride in his voice.

When I asked what he’d tell someone with a big vision and no idea where to start, he said to get obsessively clear about where you’re going, because the gaps will show up on their own. He compared it to planning a 100-mile trip and realizing you aren’t wearing shoes; once you see it, you can’t unsee it. And if it turns out the dream is really a hobby, he’s at peace with that too.

“If you can wait for Saturday, maybe you should just wait for Saturday. There’s nothing wrong with that.” —Jason Lane

If you’ve been waiting for someone to hand you a career ladder, Jason’s approach offers another option: decide where you want to end up, and let that tell you what to learn next.

Full circle

Jason Goodman, Security Engineer

Jason Goodman, Security Engineer

Where Jason Lane planned his career from the finish line, Jason Goodman’s career has looked more like a pinball machine, bouncing through three security disciplines in about a year and picking up new skills along the way.

The foundation was laid long before he knew it. Eight years ago, before Amazon acquired Ring, he was working in customer service, troubleshooting doorbells and helping customers with their accounts, and before that he sold computers at Best Buy. “That was the most tech I had,” he told me, grinning. What pointed him toward security was partly a nudge from his father-in-law, who worked in IT, and partly what he was seeing on the phones every day: the occasional account access that didn’t look right, the quiet signals that something might be off. Those patterns made him want to understand the systems behind them, so while he was still in customer service he went back to school for a cybersecurity degree.

He didn’t wait until graduation to start building toward the move. He reached out to the manager of the internal threat team (the people who look for misuse of access from inside an organization) and asked a question anyone can ask: what could I do to keep progressing while I’m in school? That turned into stretch projects, and here is the part worth noticing, he earned the bigger ones by delivering the smaller ones first. His standout project was a fraud prevention dashboard that helped the team spot potential misuse of customer account access, a blind spot nobody had flagged. He had the credibility to be handed it because he’d already shown up for the low-stakes work, and it opened the door to his first security engineering role on that same team.

From there he moved to an adversarial threat detection team (the people who watch for outside attackers trying to get in), which meant flipping his mindset overnight: instead of asking what someone on the inside could do with their existing access, he was now asking how someone on the outside would try to break in. A later move took him into application security, where the work shifted from responding after the fact to building security into processes from the ground up.

“I’m in my third discipline in a year since transitioning out of customer service into security engineering.” —Jason Goodman

What I find fascinating is how much carried over from one role to the next. The Splunk queries (Splunk is a tool for searching and analyzing large volumes of log data) he learned in customer service carried into internal threat work, the detection logic from insider investigations carried into adversarial detection, and now in application security he has come full circle, working alongside the same customer service teams he started with, except this time he’s helping build the security around them.

Through all of it, he has leaned on four different mentors, and the way he found them is worth stealing. Finding one mentor who truly invests in you is hard; finding four is rare. Jason didn’t wait to be assigned anyone. He reached out to people whose work he admired, starting with that internal threat manager during school, asked specific questions, and stayed in touch over the years as he grew. Each mentor served a different purpose: one for navigating school, one for stretch projects, one for learning to think like an adversary, and one for getting grounded in application security. “I’ve always felt like I could take at least a little nugget from each one of them,” he said. Next on his list is offensive security, meaning pen testing and red teaming (simulating real attackers to test how well an organization defends itself). He figures starting on defense was the right call: learn what attackers do and how to stop them, then learn to think like one.

If your current role feels unrelated to security, Jason’s story is a good reminder that the skills you’re building now may be exactly the ones your first security job needs. The analytical habit of noticing when something is off, the patience to explain a problem clearly, even the instincts you pick up from a completely different field all transfer. The trick is learning to name what you already have and point it at the work you want.

The edge of the map

Harshvardhan Chunawala, Solutions Architect, AWS Startups

Harshvardhan Chunawala, Solutions Architect, AWS Startups

Harshvardhan Chunawala was still in high school when he started finding security flaws in real products and reporting them to the companies that built them. He had started out building websites and web applications, and after he understood how software was put together, he couldn’t stop wondering how it could be pulled apart. Rather than wait until he had a degree or a job to explore that, he began poking at real products on his own, and when he found a flaw he contacted the company and worked with them to fix it.

Today he advises startup founders at AWS on security and volunteers his time to teach cloud and security courses to students, who often ask him where to start when entry-level work is being automated. His answer has changed over the years. Now that generative AI can answer the surface-level questions, he believes curiosity matters more than credentials, because the people who stand out are the ones who keep asking questions after the tool runs out of answers.

“AI tools put everyone on the same starting line. Interest and passion has become the driving force.” —Harshvardhan Chunawala

What surprised me most, though, was where his own curiosity took him. While studying at Carnegie Mellon University (CMU), he ended up working on space missions, including time as a mission operator for the Iris lunar rover (Iris and CMU’s next rover, MoonRanger, appear with him in the photo above). If you had asked me a few years ago whether security belonged anywhere near a rover on the Moon, I wouldn’t have known what to say, but Harshvardhan would have answered right away: anything that can be commanded remotely can be commanded by the wrong person.

When I asked about the future of security, he took the conversation completely off the map. He talked about robots entering people’s homes and about data centers moving from land to underwater to space. A satellite has to be protected in orbit, where nobody can physically reach a compromised system, and on the ground, where the stations that command it are targets too. “So are cyber security specialists going to go under astronaut training?” he asked, grinning, and he was only half joking. He has already written a two-part guide to incident response for satellite operations on AWS, so for him, this corner of the map is already part of the job.

If you’re drawn to an area that doesn’t look like a security job yet, Harshvardhan’s story is worth taking seriously: the move he keeps making is to follow a question past the point where the easy answers run out, into territory nobody has fully mapped. You don’t need a satellite to do that. Pick the corner of technology that genuinely fascinates you, start asking how it could be attacked and defended, and you might find you’re standing at the edge of a role that doesn’t have a name yet.

The thread

These three stories share one thread: the job you end up loving might be one you haven’t heard of yet. Jason Lane mapped his moves in advance, Jason Goodman carried his skills through three disciplines in a single year, and Harshvardhan is already helping AWS customers secure satellites and the ground systems that run them. None of them could answer “what do you do?” in a single sentence, and I suspect that’s true for a lot of people doing interesting work in this field.

If you’re curious about the roles that exist beyond the ones you already know, here are a few places to start exploring:

In Part 3: The work behind the work, we pull back the curtain on what the day-to-day of security actually looks like compared to what people imagine.

Have you found a corner of security most people don’t know exists? Share your story on LinkedIn!


Shannon Brazil

Shannon is a senior security engineer, managing a team on the AWS Customer Incident Response Team (CIRT), specializing in digital forensics and cloud security investigations. Known in the community as AWSlady, she is passionate about security education and mentoring the next generation of defenders.

Resolving concurrent write conflicts for Apache Iceberg tables in Amazon Redshift

Post Syndicated from Sanjay Devireddy original https://aws.amazon.com/blogs/big-data/resolving-concurrent-write-conflicts-for-apache-iceberg-tables-in-amazon-redshift/

In Getting started with Apache Iceberg write support in Amazon Redshift – Part 1, you learned how to create Apache Iceberg tables and write to them with INSERT. In Part 2, you modified data at the row level with DELETE, UPDATE, and MERGE. Both posts assumed your write had the table to itself. In production, that is rarely true: streaming jobs, scheduled pipelines, and one-time transformations frequently write to the same table at the same time. Amazon Redshift now resolves many of these concurrent writes automatically, committing changes that previously failed and were forced to retry. In this post, we explain why concurrent writes to Iceberg tables conflict, how Amazon Redshift resolves the conflicts that don’t actually overlap, how the engine escalates its checks from operations to partitions to individual files, and the throughput improvement it provides.

The challenge of concurrent writes

Apache Iceberg uses optimistic concurrency control (OCC) to keep a table consistent when more than one writer commits at the same time. Each writer prepares its changes against the table’s current version and commits by atomically advancing the table version in the catalog, on the condition that no other writer advanced it first. If another writer commits in the meantime, the condition fails and the entire write is discarded. This strictness maintains snapshot isolation semantics: if the later commit were applied anyway, it would replace the table version without the earlier writer’s changes, silently losing a committed write.

Readers don’t encounter this problem. An Iceberg scan always reads a committed snapshot, so a read is never invalidated by a concurrent write. That leaves write-write conflicts as the only case the engine must resolve, and resolving them correctly is what preserves snapshot isolation semantics for the table.

The problem is the bluntness of this check. The condition is evaluated against the table version, not against the rows or partitions each writer touched. At that grain, its answer is correct in that the version really did change. However, it says nothing about whether the two writes overlapped in the data underneath. Two writers that modified entirely separate partitions, or that independently inserted new data, conflict exactly as hard as two that rewrote the same rows. This happens whenever their query-start-to-commit windows overlap.

Consider a retail analytics platform with a customer_orders Iceberg table. A streaming pipeline might land new orders continuously with hundreds of inserts a minute. A nightly compliance job deletes records older than seven years, and takes about an hour to run. The two writes never touch the same data: the stream only adds recent orders, and the compliance job only removes old ones. Under basic optimistic concurrency, that doesn’t matter. If a single streaming insert commits at any point during that hour, the compliance job loses the race, its hour of work is discarded, and the next attempt re-scans terabytes of data only to re-enter the same race. The table version conflict was real even though the data conflict was not.

The common ways to avoid this are all manual, and each carries a cost. You can serialize your writers so that only one commits at a time, which gives up concurrency entirely. You can split a single logical table into several physical tables so that no two writers ever share one. This trades a clean data model and the ability to query across them for the ability to write in parallel. Or you can wrap every write in an application-side retry loop and absorb the cost of re-running statements that never actually conflicted. In each case, the effort goes toward working around the commit protocol rather than toward your data.

How Amazon Redshift resolves write conflicts

When a commit loses the race, Amazon Redshift now asks a more precise question before giving up: Did the writers that committed ahead of it actually change the same partitions or files or rows it changed? When the answer is no, there’s no reason to discard the work. Instead of aborting, Amazon Redshift rebases the change (similar to git rebase), replaying the already-prepared write onto the latest version of the table and committing that result. Both writes land, with no re-scan, no re-execution, and no retry loop in your application.

In the retail example, the compliance delete previously failed the moment a streaming insert committed inside its execution window. Now Amazon Redshift examines both writes, sees that the inserts only added data files while the delete only removed others, and confirms they never touched the same data. Because the changes are provably independent, Amazon Redshift rebases the delete onto the updated table version and commits it. Both writes succeed, the hour of work is preserved, and downstream consumers see both changes.

This behavior is automatic for eligible writes, with no clustering keys to choose, no repartitioning, and no setting to enable. When two writes genuinely conflict, the behavior is unchanged from today: the conflict surfaces and the later writer fails and must retry.

Two principles govern how the resolution works. The first is that a conflict is a matter of granularity. A failed race isn’t automatically a conflict. It’s a question that can be asked at different resolutions. The engine can ask whether any writer removed data at all or whether they only added it, and if data was removed, whether it came from the same partition, the same file, or the same rows that the current write touched. The finer the granularity examined, the more apparent conflicts turn out to be independent changes that can safely coexist. But finer checks also cost more: taken to the limit, the engine could intersect two change sets row by row, and that is expensive and usually unnecessary. So Amazon Redshift evaluates the least expensive level first and examines a finer level only when it must.

The second principle is that correctness takes precedence over concurrency. Conflict resolution works entirely from table metadata and never re-scans data or re-plans a query. When the engine can’t prove that two writes are independent, it doesn’t guess. It declines to resolve the conflict and the write retries, exactly as it would today. An unnecessary retry costs a small amount of time, whereas an incorrect resolution risks losing data, so the system is deliberately biased toward the safe outcome. Resolution operates under snapshot isolation, the same model Amazon Redshift already uses for Iceberg reads. Engines such as Apache Spark, Amazon EMR, and Amazon Athena honor it through the table’s isolation-level property, so a table shared across engines retains consistent semantics. For tables whose write isolation property is set to serializable, Amazon Redshift preserves the stricter contract and surfaces every conflict rather than resolving it. This is the Iceberg table property write.*.isolation-level. It’s unrelated to the Amazon Redshift database isolation level set with ALTER DATABASE … ISOLATION LEVEL, which governs Amazon Redshift transactions and has no effect on Iceberg commit conflict resolution.

Resolving at the right granularity

When a commit loses the race, the engine escalates through a sequence of checks, from the cheapest to the most precise. Each level is a complete resolution on its own and if a check proves the writes independent, the rebase commits and no finer check runs. The following diagram shows this escalation sequence: an operation-level check first, then partition-level, then file-level. Each level either proves the writes independent or escalates to the next, and a genuine conflict surfaces only after the last.

Flowchart of the conflict-resolution escalation: an operation check, then a partition check, then a file check, ending in a conflict

Figure 1: The conflict-resolution escalation from operation-level to partition-level to file-level checks

The first check is by operation. A write that only added data, for example an INSERT consisting of new data landing from a streaming job, can’t have removed anything another writer relied on, so any write can be rebased against it. This single rule resolves every combination in which at least one of the two writes only appended. A DELETE, UPDATE, or MERGE commits cleanly against a stream of concurrent inserts, and an INSERT commits cleanly over any other concurrent change, including a compaction.

Next, when both writes removed data, the engine compares the partitions they each touched. Writes that modified disjoint partitions, for example batch jobs updating different dates or per-Region pipelines maintaining their own slices of one table, are independent by construction and both can commit cleanly.

Finally, when both writes touch the same partition, the engine compares the individual files that each one changed. Two jobs deleting different rows of the same date partition usually land in different files, and when they do, both can commit cleanly. Only when two writes modified the same file does the conflict surface and the statement re-run. The protocol additionally has headroom for one further level: distinguishing changes to different rows within a single file.

The following table summarizes the outcomes by operation pair. Read down for the concurrent write that committed first, and across for your write.

 

Your write ↓ / Concurrent write → INSERT DELETE / UPDATE / MERGE Compaction ALTER
1 INSERT Resolves Resolves Resolves Re-run required
2 DELETE / UPDATE / MERGE Resolves Resolves based on partitions or files touched. A same-file change surfaces as a conflict Resolves based on partitions or files touched. A same-file change surfaces as a conflict Re-run required
3 ALTER Re-run required Re-run required Re-run required Re-run required

Performance

To measure the effect, we ran an increasing number of concurrent writers committing to a single Iceberg table over a fixed window. We compared basic optimistic concurrency mode, where a losing commit is aborted and re-runs, against automatic conflict resolution mode being enabled. Every writer issued the same single-statement INSERT, so the only variable across the various runs was contention rather than workload. We scaled 1–16 concurrent writers and recorded, for each configuration, how many commits succeeded and how many were rejected as conflicts.

Setup: one unpartitioned Iceberg table registered in the AWS Glue Data Catalog in the US East (N. Virginia) AWS Region (us-east-1). Each writer is an independent session issuing autocommit INSERT statements in a loop over a 60-second period. We report the commits completed and the attempts rejected as conflicts for each configuration.

The first figure shows write throughput as writers are added, normalized to the single-writer case so the comparison doesn’t depend on instance type or table size.

Write throughput versus number of concurrent writers for basic optimistic concurrency compared to automatic conflict resolution

Figure 2: Write throughput as concurrent writers increase, with and without automatic conflict resolution

Under basic optimistic concurrency, adding writers makes writing to the table slower. At 16 writers, throughput falls to 0.17× of a single writer, because nearly every writer’s work is discarded and redone rather than committed. With automatic conflict resolution the same workload scales to roughly 2×, and no commit is rejected in any measured configuration. At 16 concurrent writers, 12× more commits complete in the same window. The second figure shows why.

Commit failure rate versus concurrent writers, near (N−1)/N for the baseline and zero with automatic conflict resolution

Figure 3: Commit failure rate as concurrent writers increase, with and without automatic conflict resolution

The baseline’s failure rate is almost exactly (N−1)/N, for example 50 percent at two writers, 75 percent at four writers, and 94 percent at 16 writers. This illustrates the table-level check that occurred previously: only one writer wins each commit round and every other writer in flight is rejected. Additional writers translate almost entirely into repeated work rather than progress. With conflict resolution enabled, the same runs record no conflicts at all, because every write in this workload is an append, and an append never removes data another writer relied on.

This benchmark isolates the most basic append case deliberately: a single unpartitioned table and insert-only writers, which is the pattern behind continuous ingestion. Workloads that remove data exercise the partition and file checks described earlier, where the outcome depends on whether the two writes touched the same partition and the same files.

Fleet Results

Before conflict resolution, concurrent-write conflicts were the single largest cause of failed Iceberg writes in Amazon Redshift, accounting for roughly one in five failed Iceberg write queries each week. We rolled out this feature in stages, beginning with the operation-level check that covers concurrent appends. As that stage reached full fleet deployment in early September 2026, conflict errors from concurrent writes dropped to nearly zero, even as Iceberg write volume on the fleet more than doubled. One residual spike shown below came from a workload issuing concurrent UPDATE statements, a conflict class that subsequent partition and file level checks will resolve as they deploy across the fleet.

Figure 3: Conflict errors as a share of failed Iceberg writes

Figure 3: Conflict errors as a share of failed Iceberg writes

Observing conflict resolution

Every Iceberg commit attempt is recorded in the stl_iceberg_commit system table, including the outcome, the number of attempts, the concurrent operations the write rebased over, and the time spent rebasing. To see how often resolution is engaging on an example table:

SELECT outcome, count(*), avg(attempt_number)
FROM stl_iceberg_commit
WHERE trim(table_name) = 'customer_orders'
GROUP BY outcome;

Conclusion

Concurrent writes are a fact of production data lakes, and until now, they forced you to serialize writers, partition around the problem, or retry. With automatic conflict resolution, Amazon Redshift commits concurrent writes that don’t actually overlap. These include any mix of writes alongside concurrent appends, and row-level writes that changed different partitions or different files of the same table. To get started, write to your Iceberg tables from multiple jobs on patch 203 or later. Non-conflicting commits start to resolve on their own. To learn more about Iceberg write support in Amazon Redshift, see Part 1 and Part 2.


About the authors

Sanjay Devireddy

Sanjay Devireddy

Sanjay is a senior engineer in Amazon Redshift working in various areas such as Iceberg transactions, concurrency scaling, and data sharing mesh architectures.

Xiening Dai

Xiening Dai

Xiening is a principal software engineer leading various efforts in Amazon Redshift’s query execution and data lake processing.

Madhukiran Vaddi

Madhukiran Vaddi

Madhukiran is an engineering lead with the Amazon Redshift team. Leads Iceberg transaction support, including conflict resolution and multi-table transactions, as well as Redshift’s concurrency scaling, backups, and durability. Passionate about building reliable data infrastructure at cloud scale.

Srishti Mittal

Srishti Mittal

Srishti is a Product Manager at AWS, with a focus on making open data lakes performant and interoperable across analytics engines. She leads product strategy for open table formats such as Apache Iceberg, partnering with customers and field teams to turn real-world data lake challenges into product capabilities.

Reducing cross-AZ latency with the AWS Lambda metadata endpoint

Post Syndicated from Daniel Abib original https://aws.amazon.com/blogs/compute/reducing-cross-az-latency-with-the-aws-lambda-metadata-endpoint/

Customers use AWS Lambda to build applications that connect to stateful, latency-sensitive backends such as Amazon ElastiCache, Amazon Relational Database Service (Amazon RDS), and other services that expose per-Availability Zone (AZ) endpoints. For high availability, Lambda automatically provisions your execution environments across multiple Availability Zones in a Region. That resilience is transparent to your code. It also means that, until now, a function had no way of knowing which Availability Zone it was running in. When a function in one AZ connects to a resource in another AZ, the request crosses the AZ boundary, adding network latency and, in some cases, cross-AZ data transfer costs.

Lambda now exposes Availability Zone metadata through a metadata endpoint in the execution environment. Your function can now discover its AZ ID (for example, use1-az1) with an HTTP request. You can use that information to prefer same-AZ endpoints for downstream services, reduce cross-AZ latency, and implement AZ-aware resilience patterns such as AZ-specific fault injection testing. The feature is available at no additional cost in all commercial AWS Regions where Lambda is available. It works across all runtimes, including custom runtimes and container images, and is compatible with SnapStart, provisioned concurrency, and functions in a virtual private cloud (VPC). For more information about these compatibility details, see Using the Lambda metadata endpoint.

In this post, you will learn how the Lambda metadata endpoint works, how to retrieve the AZ ID using Powertools for AWS Lambda or by calling the endpoint directly, and how to apply the AZ ID to route to same-AZ resources for lower latency.

Why Availability Zone awareness matters

Consider a latency-sensitive read path: an API-backed Lambda function that reads from an ElastiCache cluster with a node in each of three Availability Zones. Lambda places your execution environments across those same AZs for resilience, but the placement is independent of where each cache node lives. Roughly two out of three invocations end up talking to a cache node in a different AZ than the one the function is running in.

A same-AZ round trip typically completes in single-digit milliseconds. A cross-AZ round trip adds latency on top of that, and on a hot read path executed thousands of times per second, those milliseconds compound into higher p99 latency and additional cross-AZ data transfer charges. Before this feature, avoiding cross-AZ traffic from Lambda required custom workarounds. These included embedding AZ hints in environment variables per function version, or inferring locality from IP ranges, none of which were reliable.

The same pattern appears across many of the VPC-based, AZ-scoped services your functions connect to. With Amazon RDS and Amazon Aurora, the writer instance lives in a single AZ, and each read replica lives in an AZ of its own. A read-heavy function that spreads its queries across replicas will, more often than not, reach a replica in a different AZ. Once your function knows its own AZ ID, you can direct reads to the replica that shares it. This keeps the hot query path in-AZ while still failing over to another replica when no same-AZ one is available. For more information, see Configuring Aurora read replica load balancing.

Amazon MemoryDB extends the same idea to a durable primary/replica topology: writes go to the shard’s primary in one AZ, while reads can be served from replicas spread across the others. Consider a high-volume, read-mostly path that tolerates eventual consistency, such as looking up reference data or serving cached lookups. Steering reads to the replica in the function’s AZ keeps the hottest part of the workload in-AZ, while writes continue to route to the primary wherever it lives. Because replicas lag the primary by a small, asynchronous delay, send any read that must reflect its own write directly to the primary rather than to a same-AZ replica.

Container platforms exhibit the same behavior when Lambda calls into them. If your function invokes a service running on Amazon Elastic Kubernetes Service (Amazon EKS) or Amazon Elastic Container Service (Amazon ECS), the pods or tasks behind that service are distributed across AZs, and by default a request can land on any of them. You can combine the function’s AZ ID with topology-aware routing to keep east-west traffic within the AZ. Examples include Kubernetes Topology Aware Routing or an internal load balancer with cross-zone balancing tuned. The same reasoning applies to a function calling a private service fronted by an internal Application Load Balancer or discovered through AWS Cloud Map: knowing the caller’s AZ turns “any healthy target” into “the nearest healthy target.”

The metadata endpoint removes that guesswork. Your function asks the execution environment where it is running, gets back a stable AZ ID, and routes accordingly.

Diagram showing a Lambda function reading from same-AZ resources and routing writes to a centralized primary using the AZ ID from the metadata endpoint

Figure 1: AZ-aware reads and centralized writes with the AWS Lambda metadata endpoint

How the metadata endpoint works

The metadata endpoint is a loopback HTTP API available inside the Lambda execution environment, accessible to both runtimes and extensions. Lambda automatically sets two environment variables in every execution environment:

  • AWS_LAMBDA_METADATA_API – The metadata server address in the format {ipv4_address}:{port} (for example, 169.254.100.1:9001). For more information about how this address and port are assigned, see Using the Lambda metadata endpoint.
  • AWS_LAMBDA_METADATA_TOKEN – A unique authentication token for the current execution environment, generated automatically at initialization. You include it in every metadata API request.

You retrieve metadata with an authenticated GET request:

GET http://${AWS_LAMBDA_METADATA_API}/2026-01-15/metadata/execution-environment

The response is a small JSON document:

{
  "AvailabilityZoneID": "use1-az1"
}

The Authorization header must contain a Bearer token (Authorization: Bearer <token>). This token-based authentication provides defense in depth against Server-Side Request Forgery (SSRF) vulnerabilities. Each execution environment receives its own randomly generated token, so a leaked or guessed URL alone is not enough to read metadata.

The response is immutable within an execution environment and returns a Cache-Control: private, max-age=43200, immutable header. We recommend that you cache the response and respect the TTL rather than calling the endpoint on every invocation. For SnapStart functions, the TTL is reduced during initialization so that clients refresh the metadata after restore, since a restored execution environment might come up in a different AZ. For more information, see Lambda SnapStart metadata behavior.

Retrieving the AZ ID with Powertools for AWS Lambda

The most direct way to consume the endpoint is with the Powertools for AWS Lambda metadata utility, available for Python, TypeScript, Java, and .NET. The utility caches the response after the first call and handles SnapStart cache invalidation automatically, so you don’t need to manage tokens, HTTP requests, or TTLs yourself. The following examples show how to use Powertools in each language.

Python

Install the Powertools package:

pip install "aws-lambda-powertools"

Use the metadata utility in your handler:

from aws_lambda_powertools.utilities.metadata import LambdaMetadata, get_lambda_metadata

def handler(event, context):
    metadata = get_lambda_metadata()
    az_id = metadata.availability_zone_id  # e.g., "use1-az1"

    return {"az_id": az_id}

TypeScript

Install the Powertools package:

npm install @aws-lambda-powertools/commons

Use the metadata utility in your handler:

import { getMetadata } from '@aws-lambda-powertools/commons/utils/metadata';

const metadata = await getMetadata();

export const handler = async () => {
  const { AvailabilityZoneID: azId } = metadata;
  return azId;
};

Java

Add the Powertools dependency to your pom.xml:

<dependencies>
    <dependency>
        <groupId>software.amazon.lambda</groupId>
        <artifactId>powertools-lambda-metadata</artifactId>
        <version>2.10.0</version>
    </dependency>
</dependencies>

Use the metadata client in your handler:

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import software.amazon.lambda.powertools.metadata.LambdaMetadata;
import software.amazon.lambda.powertools.metadata.LambdaMetadataClient;

public class App implements RequestHandler<Object, String> {

    @Override
    public String handleRequest(Object input, Context context) {
        LambdaMetadata metadata = LambdaMetadataClient.get();
        String azId = metadata.getAvailabilityZoneId(); // e.g., "use1-az1"

        return "{\"azId\": \"" + azId + "\"}";
    }
}

.NET

Install the Powertools package:

dotnet add package AWS.Lambda.Powertools.Metadata

Use the metadata class in your handler:

using Amazon.Lambda.Core;
using AWS.Lambda.Powertools.Metadata;

public class Function
{
    public string Handler(object input, ILambdaContext context)
    {
        var azId = LambdaMetadata.AvailabilityZoneId;
        return $"Running in AZ: {azId}";
    }
}

Calling the metadata endpoint directly

If you use a custom runtime, a container image, or prefer not to add a dependency, you can call the endpoint directly using the environment variables Lambda sets for you. The following example uses curl and jq to read the AZ ID:

# Variables are automatically set by Lambda
METADATA_ENDPOINT="http://${AWS_LAMBDA_METADATA_API}/2026-01-15/metadata/execution-environment"

# Make the request
RESPONSE=$(curl -s -w "\n%{http_code}" -H "Authorization: Bearer ${AWS_LAMBDA_METADATA_TOKEN}" "$METADATA_ENDPOINT")

HTTP_CODE=$(echo "$RESPONSE" | tail -1)

if [ "$HTTP_CODE" != "200" ]; then
  echo "Metadata request failed with status $HTTP_CODE" >&2
  exit 1
fi

# Parse the AZ ID
AZ_ID=$(echo "$RESPONSE" | sed '$d' | jq -r '.AvailabilityZoneID')

echo "Function is running in AZ ID: $AZ_ID"

Notice the configuration:

  • AWS_LAMBDA_METADATA_API: Lambda sets this to the metadata server address, so you never hardcode the host or port.
  • Authorization: Bearer ${AWS_LAMBDA_METADATA_TOKEN}: The token is required on every request. A missing or invalid token returns 401 Unauthorized.
  • GET only: The endpoint accepts GET requests. Any other method returns 405 Method Not Allowed.

When you call the endpoint directly, retrieve the AZ ID once during initialization (outside the handler) and cache it for the life of the execution environment, rather than calling on every invocation. If your function uses SnapStart, refresh the value after restore, because the restored environment might run in a different AZ.

Understanding Availability Zone IDs

The endpoint returns an AZ ID (for example, use1-az1), not an AZ name (for example, us-east-1a). Before you use it, it helps to read the ID itself. An AZ ID follows the pattern <region-code>-az<number>:

  • use1 is the shortened Region code, in this case US East 1 (us-east-1). Other Regions follow the same convention, such as usw2 for US West (Oregon) and euw1 for Europe (Ireland).
  • az1 identifies the individual Availability Zone within that Region. So use1-az1 reads as “Availability Zone az1 in us-east-1.”

This distinction matters. AZ IDs always refer to the same physical location across all AWS accounts, while AZ names might map to different physical infrastructure in each account in certain Regions. In other words, the us-east-1a name in your account and the us-east-1a name in another account can point to two different physical Availability Zones, but use1-az1 is the same physical zone in every account. There is no fixed mapping between a name like us-east-1a and an ID like use1-az1. AWS assigns names independently per account to balance resources across zones. That’s exactly why the endpoint returns the ID: comparing AZ IDs gives you a consistent, account-independent result about whether your function and a downstream resource share a physical Availability Zone. For more information, see AZ IDs for cross-account consistency.

If you need the AZ name (for example, for logging or for an API that expects a name), convert the AZ ID using the Amazon Elastic Compute Cloud (Amazon EC2) DescribeAvailabilityZones API. To call it, add the ec2:DescribeAvailabilityZones permission to your function’s execution role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "ec2:DescribeAvailabilityZones",
            "Resource": "*"
        }
    ]
}

Routing to same-AZ resources

Retrieving the AZ ID is only useful if you act on it. The following Python example shows the common pattern illustrated in Figure 1: look up the AZ ID once at initialization, select the connection endpoint that lives in the same AZ, and fall back gracefully when there is no same-AZ match.

from aws_lambda_powertools.utilities.metadata import LambdaMetadata, get_lambda_metadata

# Map each cache node to the AZ ID it lives in.
# Populate this from configuration (for example, environment variables or Parameter Store).
CACHE_NODES_BY_AZ = {
    "use1-az1": "cache-node-1.example.use1-az1.cache.amazonaws.com",
    "use1-az2": "cache-node-2.example.use1-az2.cache.amazonaws.com",
    "use1-az3": "cache-node-3.example.use1-az3.cache.amazonaws.com",
}

# Read the AZ ID once during initialization (cached across warm invocations).
AZ_ID = get_lambda_metadata().availability_zone_id

# Prefer the same-AZ node; fall back to any node if there isn't one.
CACHE_ENDPOINT = CACHE_NODES_BY_AZ.get(AZ_ID) or next(iter(CACHE_NODES_BY_AZ.values()))


def handler(event, context):
    # Use CACHE_ENDPOINT for your read path. When AZ_ID matches the node's AZ,
    # the request stays within the Availability Zone.
    return {"az_id": AZ_ID, "cache_endpoint": CACHE_ENDPOINT}

Notice the design choices:

  • Resolve at initialization: AZ_ID and CACHE_ENDPOINT are computed once per execution environment, not per invocation, so the metadata lookup and endpoint selection don’t add latency to the request path.
  • Always provide a fallback: If there is no node in your function’s AZ (for example, the service isn’t deployed in every AZ the function runs in), fall back to any available endpoint. Correctness must not depend on a same-AZ match being present.
  • Compare AZ IDs, not names: Because the endpoint returns AZ IDs, and services like ElastiCache and RDS expose the AZ of their nodes, you can compare the two directly for a reliable same-AZ decision.

Beyond latency-aware routing, the same AZ ID unlocks resilience testing. You can, for example, inject faults only in functions running in a specific AZ to validate that your application degrades gracefully during a simulated AZ impairment, without affecting traffic served from other AZs.

Best practices

Based on the endpoint’s behavior, keep these recommendations in mind:

  • Cache the metadata: The response is immutable for the life of an execution environment. Read it once at initialization and reuse it. If you call the endpoint directly, respect the Cache-Control TTL rather than requesting on every invocation.
  • Handle SnapStart correctly: A restored SnapStart environment might resume in a different AZ than the one it was snapshotted in. Refresh the AZ ID after restore. Powertools handles this invalidation for you automatically.
  • Ignore unknown fields: Additional fields might be added to the response in the future. Parse defensively and don’t fail if new fields appear.
  • Degrade gracefully: Treat same-AZ routing as an optimization, not a requirement. Always keep a working fallback path so your function stays correct when no same-AZ resource is available.
  • Keep the token server-side: The AWS_LAMBDA_METADATA_TOKEN is scoped to a single execution environment. Never log it or forward it outside the function.

Conclusion

In this post, you learned how the Lambda metadata endpoint helps your functions discover the Availability Zone they run in. With a single authenticated HTTP request, or one call through Powertools for AWS Lambda, your function can retrieve a stable AZ ID and use it to route to same-AZ resources, cutting cross-AZ latency and data transfer on hot paths, and to build AZ-aware resilience patterns like targeted fault injection. The feature works across all runtimes, integrates with SnapStart, provisioned concurrency, and VPC-enabled functions, and is available at no additional cost in all commercial AWS Regions where Lambda is available.

The key takeaway: if your Lambda function reads from or writes to AZ-scoped resources such as ElastiCache or RDS on a latency-sensitive path, resolve the AZ ID at initialization, prefer the same-AZ endpoint, and always keep a fallback. You get lower, more predictable latency with no change to how Lambda manages high availability for you.

To get started, see Using the Lambda metadata endpoint in the AWS Lambda Developer Guide.

Additional resources

Part 1 – Cybersecurity journeys: I didn’t plan this

Post Syndicated from Shannon Brazil original https://aws.amazon.com/blogs/security/part-1-cybersecurity-journeys-i-didnt-plan-this/

If you’ve ever wondered whether your background qualifies you for a career in cybersecurity, you’re not alone — and you’re probably more prepared than you think.

My name is Shannon Brazil, and I work in security communications at Amazon Web Services (AWS). I have been in the security space for over a decade, and my own journey into it wasn’t exactly a straight line. I started in Statistics Canada helping people fill out export declaration forms over the phone, then got a college placement in IT, moved into Digital Forensics and Incident Response, and eventually found my way into the side of security that most people don’t think about: how we communicate about it, how we tell the stories, and how we help people understand what this work looks like from the inside.

Nearly every time I’m asked that “how do I get in?” question, my answer is different. Not because I’m making it up as I go, but because I try to make it relatable to the person asking. Every person I’ve met in this field started from a different and took a completely different road to get here. There is no checklist, no degree that unlocks the door, and no straight line.

I decided to create this series for Cybersecurity Awareness Month, however I didn’t want to write another “Top 10 Tips to Break Into Security” post. Instead, I interviewed with 11 security professionals across AWS, each from a different team and a different background, and asked them one basic question: how did you actually get here?

What came out of those conversations surprised me. The stories were funny, honest, sometimes hard to hear. Over the next four weeks, I will be sharing them in a four-part series, grouped by theme:

  1. I didn’t plan this (this post) explores the non-linear paths that brought people into security from places you wouldn’t expect.
  2. What even is this job?” dives into the lesser-known corners of security that most people don’t realize exist.
  3. The work behind the work pulls back the curtain on what the day-to-day looks like compared to what people imagine.
  4. What I’d tell myself wraps the series with advice, growth moments, and the kind of wisdom that only comes from having lived it.

Nearly every person I spoke to had a different story, but one thing kept coming up: nobody planned this. And that’s kind of the whole point. So let’s start there.

Mr. Robot and a box cutter

Justin Knight, Security Engineer

Justin Knight, Security Engineer

Justin Knight was packing boxes in an Amazon warehouse when his career in cybersecurity started. He just didn’t know it yet.

It was around 2015 or 2016, and he’d just started at Amazon, working fulfillment, scanning, packing, shipping, with no tech background and no degree in computer science. What he did have was a TV show.

“I saw Mr. Robot,” he told me, grinning. “And I was like, man, I want to do that.” I’ve heard a lot of origin stories in this field, but something about watching a fictional hacker-vigilante while surrounded by conveyor belts and cardboard, and deciding that’s the career for me, just felt different. Justin started teaching himself by turning YouTube into his classroom (Daniel Messler for the fundamentals, Stök for the hacker mindset, NetworkChuck for the energy), and spinning up a Kali Virtual Machine (a free operating system built for security testing) and diving into Hack the Box (an online platform where you practice cybersecurity kills by solving challenges) and TryHackMe labs (a beginner-friendly platform for learning cybersecurity through guided exercises), all while still packing boxes.

When an IT role opened up inside the warehouse, Justin jumped on it and started doing the tasks the team disliked the most: fixing broken laptop screens. He would proactively walk the floor looking for cracked displays before anyone even filed a ticket, replace printers (the silent soul-killer of every IT professional), and eventually became the “computer guy” while already looking for what was next.

Justin found a bug bounty role posted on LinkedIn, and even though he had no idea what bug bounty even was, he reached out anyway. The hiring manager had a one-on-one with him, saw the passion, and did something I’ve seen happen before but never gets old: he lowered the role level requirements to a lower seniority level so Justin could make the jump.

Four years later, Justin is still on that team. He just got back from DEFCON, his second time, where he met the very YouTubers who taught him everything he knows.

“I didn’t even know what bug bounty was until I joined the team.” —Justin Knight

What gets me about Justin’s story is that none of the traditional ingredients were there: no degree, no bootcamp, no network of security professionals opening doors. All Justin had was a show that lit a fire, a YouTube playlist, and the kind of curiosity that made him walk the warehouse floor looking for broken screens nobody asked him to fix.

The accidental investigator

Zlata Pavlova, Sr Intel Analyst

Zlata Pavlova, Sr Intel Analyst

If Justin’s story resonated with you, Zlata’s might hit even closer to home; she started with a marketing gig at a pen testing firm.

Zlata has a degree in political science. After school, she worked the kinds of jobs that have nothing to do with either politics or science: hospitality, restaurants, retail. At some point, she got interested in social media marketing, and that led to a contract with a small penetration testing (pen testing) firm. Not to do anything security-related, just to run their social media, maintain their website, and represent them at conferences. But something happened when she started spending time around people who think for a living. The pen testers, the red teamers, the people who look at a locked door and think “how would I get through that without the key?” Their energy was contagious.

“I had no idea that there’s actually a title or a role based on finding information online,” she said. “I didn’t know that could be transformed into a career.”

Zlata had always been good at sleuthing, finding things and connecting dots that other people missed. She just didn’t know there was a word for it: OSINT, or open source intelligence. Before long, she was supporting the red team’s operations, preparing phishing engagements and doing reconnaissance for physical security assessments, and eventually conducting the assessments herself. And then she found Trace Labs.

For those unfamiliar, Trace Labs runs capture-the-flag competitions focused on finding missing persons. Zlata started as a competitor, moved into judging, and then took it even further by volunteering with the National Child Protection Task Force, doing OSINT investigations on cases involving real victims.

One of those cases led to people being rescued and suspects being arrested. A political science major who took a marketing contract at a pen testing firm helped rescue real people from real danger, because she followed her curiosity into a field she didn’t even know existed.

“That was one of the biggest cases I personally worked on. Hearing that our efforts actually helped people being rescued and the bad guys arrested… that was really rewarding.” —Zlata Pavlova

Today, Zlata works on a technical risk team, bridging the gap between cybersecurity and physical security for executive protection. She came from a world where none of this was on the radar. And her advice for anyone feeling like they don’t belong?

“Impostor syndrome is real, but you belong here. Share what you learn. Don’t assume everyone already knows it. They might not.” —Zlata Pavlova

Access denied

Arman Sadri, Security Engineer

Arman Sadri, Security Engineer

Arman Sadri’s story starts in a place most cybersecurity career guides don’t typically cover: getting in trouble as a teenager.

As a teenager, Arman was into video games. Really into them. So into them that he ended up hacking into a major tech company and stealing hardware prototypes. He was 16, maybe 17. “I’m dumb. I’m a kid,” he told me, laughing about it now, but you could hear the weight of it in his voice. That experience taught him a lot, but it also left him terrified that no legitimate employer would ever give him a chance, especially not a Fortune 500 company.

He enrolled in Year Up, a program that pairs 6 months of college with 6 months of an internship, and did so well they had him teaching the classes. Amazon offered him a job before the internship even started, letting him skip ahead and start running. He landed in IT support (the help desk, the person who remotes into your computer when something breaks) and for a while, that was the job, but Arman had his sights set on security.

He applied to a mentorship program for aspiring security engineers and found himself on a 2-year waitlist, but he waited it out. When he finally got in, he met with a senior leader every week, showed what he could do, and coached the other mentees in the room. At the end of the program, there was a hiring challenge. No other candidate could complete it, and even after they extended it externally. Arman was the only one who finished.

Four months of silence went by before he got a response: “Hey, there’s this role opening up. I think you’d be a great fit.” The role didn’t exist before that conversation; they created it for him.

“If you knew by the 80th no it was a yes, you’d be so happy every time you got told no.” —Arman Sadri

What stays with me about Arman’s story is the refusal to let his past define his ceiling. He didn’t hide from it, he just outworked it. A 2-year waitlist, and he waited. A challenge nobody could finish, and he finished it. A role that didn’t exist, and now it does. And when I asked him what makes the biggest difference for someone trying to break in, he didn’t say certifications. He said: “What can I Google about your name and see? Show me what you’ve built. Show me the impact.”

The thread

These three stories share one thread: curiosity, persistence, and a willingness to start before feeling ready. If you’re looking for a place to begin:

In Part 2: What Even Is This Job?, we explore the roles most people don’t know exist in cybersecurity — and why that matters for your career.

Have your own unconventional path into security? Share your story on LinkedIn!


Shannon Brazil

Shannon is a senior security engineer, managing a team on the AWS Customer Incident Response Team (CIRT), specializing in digital forensics and cloud security investigations. Known in the community as AWSlady, she is passionate about security education and mentoring the next generation of defenders.

ICYMI: August 2026 @AWS Security

Post Syndicated from Rodolfo Brenes original https://aws.amazon.com/blogs/security/icymi-august-2026-aws-security/

Read all about the latest AWS security features, compliance updates, and hands-on resources in our monthly digest posts. You’ll find expert blog posts, new service capabilities, code samples, and workshops.

AWS Security Blog posts

August brought 20 AWS Security Blog posts organized across seven categories. Identity and access management led the month with five posts covering self-service rate limits for Amazon Cognito, a decade of AWS Managed Microsoft AD, a redesigned sign-in experience, console Private Access for isolated VPCs, and automated IAM Identity Center governance. Data protection followed with four posts on AWS KMS data key caching, ACME protocol support in AWS Certificate Manager, Amazon S3 over-permissioned access remediation, and the upcoming deprecation of email-based domain validation. AI security continued to grow with four posts on custom authentication in Amazon Bedrock AgentCore Gateway, user authorization propagation in AI agents, and extending Bedrock Guardrails to tool interactions. Threat detection, governance and networking.

Identity

From 2 weeks to 2 minutes: Amazon Cognito launches provisioned limits for self-service rate limit management

Authors: Kiran Dongara, Howie Li | Published: August 5, 2026

Learn to use Amazon Cognito provisioned limits for on-demand authentication rate limit adjustments, replacing the previous 10–14 day support ticket process with self-service capacity scaling in minutes.

A decade of enterprise identity in the cloud with AWS Managed Microsoft AD

Authors: Vladimir Provorov, Tekena Orugbani, Rodney Underkoffler | Published: August 7, 2026

AWS Managed Microsoft AD celebrates 10 years of fully managed Active Directory in the cloud, now offering Standard, Enterprise, and Hybrid editions with multi-Region replication and 20+ AWS service integrations.

Updates to your AWS sign-in experience

Authors: Vaibhav Chowla, Ella Segura | Published: August 17, 2026

AWS is gradually rolling out a redesigned sign-in page with a unified email entry point, social identity provider options, and an updated session selection experience for managing multiple active sessions.

Extend your data perimeter to the AWS Management Console with Private Access

Authors: Madhur Kulkarni, Abhijit Barde, Sujay Ghosh, Mateusz Jaworski | Published: August 28, 2026

AWS Management Console Private Access now supports VPCs without internet connectivity, routing all console traffic – authentication, static assets, and service API calls – through AWS PrivateLink endpoints to strengthen your data perimeter.

Automate IAM Identity Center governance with continuous discovery and reporting

Author: Jonathan Nguyen | Published: August 31, 2026

Learn to deploy automated discovery and reporting for AWS IAM Identity Center applications and assignments across your organization, with event-driven monitoring that validates naming conventions and enables near real-time enforcement of governance policies.

Data Protection

Caching KMS data keys in multi-thread environments: per-tenant encryption for event-driven systems at scale

Authors: Maria Gutovsky, Hemmy Yona | Published: August 6, 2026

Learn to solve the cache stampede problem in multi-tenant envelope encryption using the AWS-recommended hierarchical keyring pattern or a custom Caffeine-based caching approach to reduce AWS KMS costs.

Automate certificates with ACME support in AWS Certificate Manager

Authors: Anthony Harvey, Chandan Kundapur | Published: August 6, 2026

Learn to use ACME protocol support in AWS Certificate Manager to automate public certificate issuance and renewal using standard clients like Certbot and cert-manager, with enterprise controls for domain scoping and centralized visibility.

Securing your Amazon S3 buckets: identifying and remediating over-permissioned access

Authors: Hetal Kolekar, Fernando Chiera di Vasco Freitas, Manonmayi Vedam | Published: August 7, 2026

Learn to detect and fix over-permissioned Amazon S3 buckets across multi-account environments using AWS Lambda, AWS Config, and AWS Security Hub, with automation for continuous monitoring.

AWS Certificate Manager will discontinue email validation to prove domain validation for certificates

Authors: Adam Aboudi, Poojil Tripathi | Published: August 13, 2026

ACM will discontinue email-validated public certificates by September 30, 2027, aligning with CA/B Forum standards – learn the timeline and how to migrate to DNS validation in place.

AI Security

Implement custom authentication for tools integration using request Lambda interceptor in AgentCore Gateway

Authors: Nishant Mainro, Ram Ramani | Published: August 18, 2026

Learn to use a request Lambda interceptor in Amazon Bedrock AgentCore Gateway to bridge legacy authentication mechanisms like Basic Auth, isolating credentials from AI agents using AWS Secrets Manager.

Propagate user authorization context in AI agents with Amazon Bedrock AgentCore

Authors: Anshu Bathla, Prafful Gupta, Rohit Verma | Published: August 19, 2026

Learn to enforce least-privilege access in AI agents by propagating user identity through Amazon Bedrock AgentCore to Amazon DynamoDB, Knowledge Bases, and Salesforce, without embedding authorization logic in agent code.

Extend Amazon Bedrock Guardrails to tool interactions using the Strands Agents SDK

Authors: Stephan Traub | Published: August 27, 2026

Learn to extend Amazon Bedrock Guardrails beyond the model boundary to tool calls, external data, and MCP server interactions using three validation checkpoints built with Strands Agents SDK lifecycle hooks.

Threat detection and incident response

Security Hub Extended adds supply chain security as its tenth category

Author: Michael Fuller | Published: August 18, 2026

AWS Security Hub Extended now includes supply chain security with Chainguard and Socket as curated partners, helping you verify open source dependencies and block malicious packages through a single AWS billing relationship.

Detecting multi-stage attacks on AWS: a guide to cross-service signal correlation

Authors: Nisha Kashyap | Published: August 26, 2026

Learn to correlate signals across AWS CloudTrail, VPC Flow Logs, and Route 53 Resolver logs to detect multi-stage attacks by layering your business context, data classification, access norms, and change windows – on top of Amazon GuardDuty Extended Threat Detection.

AWS partners with Anthropic and OpenAI to bring AWS Continuum into developer workflows

Author: Chet Kapoor | Published: August 5, 2026

AWS Continuum for code vulnerabilities now integrates with Anthropic Claude Code, OpenAI Codex, and Kiro, enabling developers to discover, prioritize, validate, and remediate vulnerabilities within their coding environments.

We invited a direct competitor into Security Hub Extended. Here’s why.

Authors: Michael Fuller | Published: August 31, 2026

AWS Security Hub Extended now includes Upwind, a runtime-first cloud security company, offering eBPF-based workload protection with pay-as-you-go pricing through a single AWS bill, reinforcing customer choice even where capabilities overlap with AWS offerings.

Infrastructure security

AWS Network Firewall now supports rule hit count

Authors: Preetkumar Shah, Amit Gaur, Cheriyan Mundapuzha, Santosh Shanbhag, Srivalsan Mannoor Sudhagar | Published: August 20, 2026

AWS Network Firewall now tracks how often stateful rules match traffic, helping you identify unused rules, validate security controls for compliance, and accelerate incident response at no additional cost.

Governance and compliance

Landing Zone Accelerator independent assessment report for C5:2020 now available on AWS Artifact

Authors: Kevin Donohue, Michael Wahlers | Published: August 11, 2026

An independent assessment by Schellman evaluates how Landing Zone Accelerator on AWS aligns to C5:2020 requirements, implementing 325 security controls to accelerate your compliance journey.

Fast track ISM-ready cloud environments and IRAP assessments with Landing Zone Accelerator on AWS

Authors: Kevin Donohue, Dave Connell, Dan Friebe | Published: August 25, 2026

A new independent assessment by gwi.digital evaluates Landing Zone Accelerator against 1,081 ISM controls, achieving 91% coverage of addressable scope to help Australian customers accelerate IRAP assessment readiness.

How Moeve scales AWS governance with automated AWS Organization Service Control Policies

Authors: Gonzalo Guerrero, Jonatan De Martín, Rayco Martinez | Published: August 26, 2026

Learn how Moeve manages 150 SCPs across 300+ accounts in three AWS Organizations using a governance-as-code model; policies live in Git, deploy through GitHub Actions pipelines, and attach dynamically based on account metadata during onboarding, with Amazon EventBridge and AWS Lambda providing real-time observability of every organizational change.

August Security Bulletins

In August 2026, AWS published 23 security bulletins addressing vulnerabilities across open-source SDKs, MCP servers, developer tools, and the OpenSearch ecosystem. A dominant theme was the AI agent tool surface: prompt-injection consent bypasses in Strands Agents Tools enabled command and code execution, an insecure direct object reference exposed cross-tenant agent memory, and credential disclosure and authorization flaws affected the Amazon MQ, DocumentDB, and AWS Transform MCP servers and the Bedrock AgentCore harness. Remote code execution recurred throughout, from prototype pollution and Java deserialization in OpenSearch to path-traversal-to-root in amazon-ssm-agent, Zip Slip in awsdac, and an uncontrolled search path in the Kiro IDE and CLI on Windows.

Other notable issues include disabled SSH host key verification in the AWS CLI, memory-safety flaws in the AWS SDK for C++, privilege escalation in the FreeRTOS-Kernel, and memory-amplification denial of service in Amazon ion-java. OpenSearch accounted for a large share of the month, spanning authorization, input validation, SSRF, stored XSS, and denial of service, while Athena Federated Query connectors exposed Secrets Manager secrets. A common thread: insufficient authorization and input validation at the boundary between AI agents and the systems they reach. All patches are available, upgrade promptly. For more information, see AWS Security Bulletins.

AWS Samples

In August 2026, we published 17 new code samples organized into five categories: governance and compliance (7), AI security (3), data protection and privacy (3), identity and access management (2), and infrastructure security (2). This month’s collection reflects the rapid growth of agentic AI workloads: most samples focus on governing, auditing, and securing AI agents built on Amazon Bedrock AgentCore, from platform-level governance and telemetry to fraud investigation and biosecurity screening.

Governance and compliance

Agentic Governance Platform

Learn to deploy an AWS-native control plane for governing AI agents across your enterprise with centralized registry, Microsoft Entra ID single sign-on, Cedar tool policies, multi-vendor agent inventory, and Langfuse observability, all self-hosted on Amazon Bedrock AgentCore.

Enterprise Agentic AI Platform Accelerator

Learn to deploy a secure, governed foundation for production AI agents on Amazon Bedrock AgentCore with modular CDK stacks covering identity, gateway, memory, runtime, and observability; supporting Strands Agents, LangGraph, and Claude Agent SDK with opt-in security controls.

Video Compliance Agent

Learn to deploy an end-to-end pipeline that automatically verifies video content against broadcast compliance guidelines such as Ofcom; extracting frames, audio transcripts, and OCR text shot by shot, then using Amazon Bedrock to flag potential violations and produce a structured per-shot compliance report.

Intelligent Security for Healthcare APIs

Learn to add behavioral anomaly detection, automated data sensitivity classification, and HIPAA compliance reporting to your FHIR API using Amazon Bedrock; running asynchronously so clinical workflows are never blocked, with Amazon Comprehend Medical and Bedrock Guardrails anonymizing PHI throughout the monitoring path.

Governed Agentic Companion

Learn to deploy a governed, orchestrator-driven multi-agent builder companion on Amazon Bedrock AgentCore, reachable from Kiro, Claude Code, or any MCP client; the kit enforces 13 codified tenets through an always-on governance gate with no off switch, routing each request to a specialist while blocking deploys, secret leaks, and ungrounded answers by construction.

Audit the Agent

Learn to deploy a serverless daily executive audit pipeline for AWS AI agents (AWS DevOps Agent, AWS Security Agent) using AWS Step Functions and AWS Lambda; the report answers five questions: what the agent accessed, who authorized it, what it cost, its risk posture across five trust dimensions, and whether you should be concerned, all sourced deterministically from AWS CloudTrail, CUR, and IAM with AI-generated summaries bounded by layered guardrails.

AI Security

Telemetry Enablement for AgentCore CloudFormation

Learn to deploy a single AWS CloudFormation stack that enables Amazon CloudWatch logs and X-Ray traces for every Amazon Bedrock AgentCore resource type – Runtime, Gateway, Memory, Browser, CodeInterpreter, and WorkloadIdentity – using native and custom telemetry rules.

Bedrock Guardrails to OCSF on CloudWatch

Learn to transform Amazon Bedrock Guardrails intervention events into OCSF Detection Finding records and land them in the Amazon CloudWatch unified data store; enabling you to query guardrail violations alongside AWS CloudTrail, Amazon VPC Flow Logs, and other sources with Amazon Athena or CloudWatch Logs Insights.

Bedrock Readiness Agent

Learn to deploy a read-only assessment agent built with the Strands Agents SDK that evaluates your Amazon Bedrock environment across six dimensions: IAM governance, data retention, quota headroom, model selection fitness, cost projection, and operational observability; generating severity-rated findings with AWS CloudFormation and Terraform remediation templates you can apply directly.

Infrastructure security

DDoS Guardian

Learn to install an agent skill that reviews an AWS WAF web ACL as a system: evaluation order, rule interactions, and L7 DDoS posture; then delivers a severity-ranked HTML report with ready-to-apply remediation. Offline, read-only, no AWS resources modified.

Biosecurity Screening Policy on Amazon Bedrock AgentCore Gateway

Learn to use Policy in Amazon Bedrock AgentCore to deterministically screen AI agent tool requests for biosecurity risks, combining Cedar policies with three independent screening layers: MMseqs2 sequence alignment, ESMC-600M embedding similarity, and Foldseek structural homology; enabling defense-in-depth controls that block high-risk protein sequences before they reach downstream tools.

MCP Fraud Investigation Agent

Learn to deploy an end-to-end AI-powered e-commerce fraud investigation agent built with the Strands SDK on Amazon Bedrock AgentCore, connecting through an AgentCore Gateway over MCP to query transaction history, customer profiles, login activity, support cases, and fraud playbooks; a React dashboard on AWS Amplifystreams the agent’s reasoning token by token as it works each case.

Identity

IAM Account Access Manager with ABAC

Learn to implement workforce access using IAM account access manager and attribute-based access control, where one IAM role per project shares a single policy document and access decisions are made by comparing role tags against resource tags at request time; onboarding a new project requires only tagging and entitlement configuration with no policy authoring.

Operationalizing Least Privilege: Automate IAM Remediation through Your CI/CD Pipeline

Learn to automate remediation of unused IAM permissions using AWS IAM Access Analyzer, AWS CloudTrail, and Amazon Bedrock for AI-generated AWS CDK code; the solution attributes each role to its origin (IaC or manual), then creates pull requests for IaC-managed roles or issues for manually created roles in GitLab or GitHub, with configurable exclusion rules and policy diffs for human review.

Data Protection

Data Residency Chatbot with Amazon Bedrock AgentCore

Learn to deploy a data-residency-compliant natural-language chatbot on Amazon Bedrock AgentCore where all data and AI inference stay within a single AWS Region; the solution uses a Strands agent that answers plain-English questions from Aurora PostgreSQL through governed, whitelist-validated read-only tools exposed via an AgentCore Gateway, with a residency guard that rejects any cross-region inference profile, demonstrated with a rooftop-solar subsidy program and adaptable to any sector or geography.

LLM-based PII Detection

Learn to detect personally identifiable information in conversational text using Amazon Bedrock; the solution prompts any Converse-compatible model to return PII spans with category, value, and character offsets, handles long inputs via word-boundary chunking, recovers near-miss labels, and supports custom categories, few-shot examples, and pluggable backends beyond Bedrock.

Agentic Data Classification and Redaction

Learn to build a conversational AI research assistant that automatically classifies documents for MNPI, PII, and security sensitivity, then enforces per-user redaction at query time using Amazon Bedrock AgentCore, Guardrails, and Amazon OpenSearch Serverless vector search.

Conclusion

August 2026 provides comprehensive guidance and runnable code for governing agentic AI workloads at enterprise scale, from orchestrator-driven governance gates and biosecurity screening policies to daily executive audit pipelines and OCSF-normalized guardrail telemetry. The posts and samples provide patterns for console Private Access in isolated VPCs, attribute-based access control with IAM account access manager, automated least-privilege remediation through CI/CD, and cross-service signal correlation for multi-stage threat detection. Each resource includes deployment steps or runnable code so you can validate in your own environment before adopting. Subscribe to the AWS Security Blog RSS feed to receive updates as they publish, and revisit this digest monthly for a consolidated view of what changed and what to act on.

If you have feedback about this post, submit comments in the Comments section below.


Rodolfo Brenes

Rodolfo Brenes

Rodolfo is a Principal Solutions Architect focused on Cloud Governance and Compliance. With over 18 years of experience, he currently leads a technical field community in AWS helping customers scale and improve their security and governance frameworks. Besides work, Rodolfo enjoys video games, playing with his four cats, and won’t say no to a good outdoor adventure.

Anna Brinkmann

Anna has 18 years of experience in the technical content space and has spent the last 6 years managing the AWS Security Blog. Outside of work, she enjoys spending time with her family.

Supporting ASD’s multi-factor authentication campaign: Why MFA matters more than ever

Post Syndicated from Grace Zhang original https://aws.amazon.com/blogs/security/supporting-asds-multi-factor-authentication-campaign-why-mfa-matters-more-than-ever/

The Australian Signals Directorate (ASD) has this month issued a clear call to action through its Multi-factor authentication: Switch it on campaign, urging businesses, organisations, and individuals to enable multi-factor authentication (MFA) across their online accounts. At AWS, we strongly support this message.

As threat actors continue to target credentials through phishing, credential stuffing, and social engineering, passwords alone are no longer enough. MFA is one of the most effective security controls available. It’s a cornerstone of ASD’s Essential Eight maturity model and a recognized component of major cybersecurity frameworks worldwide. ASD’s campaign reinforces what the security community has long advocated: switching on MFA is one of the simplest and most impactful steps any organization or individual can take to protect themselves online, and we encourage all to heed ASD’s call.

How AWS enforces MFA across every account type

At AWS, we’ve put this principle into practice at scale. In June 2025, AWS Identity and Access Management (IAM) achieved comprehensive MFA enforcement for root users across all account types, a significant milestone and the first of its kind among major cloud providers. This was the culmination of a deliberate, phased security journey: beginning with requiring MFA for AWS Organizations management account root users in May 2024, expanding to standalone account root users in June 2024, introducing centralized root access management in November 2024, and completing enforcement across all account types including member accounts. MFA prevents over 99 percent of password-related attacks and is available to all AWS customers at no additional cost, with support for FIDO2 passkeys and FIDO-certified security keys for phishing-resistant authentication. This milestone reflects our ongoing commitment to secure-by-design principles, setting a high bar for our customers’ default security posture and demonstrating that organizations of any scale can, and should, make MFA the standard rather than the exception.

ASD’s Multi-factor authentication: Switch it on campaign banner

Extending MFA beyond your AWS environment

A compromised email account can be used to reset AWS passwords. A breached source control system can expose infrastructure-as-code secrets. Enable MFA on your email, collaboration tools, source control, and other services that support it. Visit the ASD Multi-factor authentication campaign page for broader guidance.

Getting started with MFA on AWS

AWS enforces MFA automatically for root users. To extend that same protection to your IAM users—the identities your team members and applications use daily—you can configure MFA individually through the AWS Management Console for IAM. To learn more, see Security best practices in IAM. For phishing-resistant authentication with FIDO2 passkeys, see Passkeys and security keys in IAM.

If you have questions or feedback about MFA on AWS, leave a comment below or reach out on AWS re:Post. If you haven’t already, heed ASD’s call and switch on MFA across every account you own.

This post was written in support of ASD’s Multi-factor authentication: Switch it on campaign. For more AWS security content, visit the AWS Security Blog.

If you have feedback about this post, submit comments in the Comments section below.


Grace Zhang

Grace Zhang

Grace is the Regulatory and Security Compliance Lead for Australia and New Zealand (ANZ), based in Sydney. She supports security assurance and compliance initiatives across the ANZ region, helping customers navigate regulatory requirements and build confidence in the security of the AWS Cloud.

AWS Security Reference Architecture: A deep dive into PCI DSS compliance

Post Syndicated from Avik Mukherjee original https://aws.amazon.com/blogs/security/aws-security-reference-architecture-a-deep-dive-into-pci-dss-compliance/

Amazon Web Services (AWS) is excited to announce the publication of the AWS Security Reference Architecture (AWS SRA) Payment Card Industry (PCI) Data Security Standard (DSS) Deep Dive. This new guide extends the core AWS SRA to provide prescriptive, architecture-level guidance for organizations that store, process, or transmit cardholder data on AWS.

Organizations subject to PCI DSS have long asked for a comprehensive reference that bridges the gap between general AWS security best practices and the specific technical and organizational controls required to achieve and maintain PCI DSS compliance. This guide answers that need by showing how AWS SRA patterns address PCI DSS intent from account scoping and network segmentation to encryption, logging, and access control.

What is the AWS SRA PCI DSS Deep Dive?

The AWS SRA is a holistic, prescriptive security architecture guide that describes how AWS security services fit together across a multi-account AWS environment. It’s built around a modular, three-tier web architecture and is intentionally designed to be adapted as needed. Not every workload needs every service, but the AWS SRA provides the full range of options and their architectural relationships.

This PCI DSS deep dive doesn’t replace the AWS SRA, it extends it:

  • Mapping AWS SRA account types to PCI DSS scoping boundaries showing which accounts are in scope, connected-to or security-impacting, or out-of-scope in a typical payment architecture.
  • Layering PCI-specific controls onto existing AWS SRA service configurations. For example, additional logging granularity, encryption requirements, or network restrictions that go beyond the AWS SRA baseline.

The architectural patterns and controls described in this guide apply equally to merchants and service providers.

Who should use the guide

The guide is intended for:

  • Security architects designing or extending an AWS multi-account landing zone for workloads under PCI DSS scope.
  • Compliance engineers mapping AWS controls to PCI DSS requirements during assessment.
  • Cloud platform teams building shared security services that must accommodate a cardholder data environment (CDE).
  • Qualified Security Assessors (QSAs) and Internal Security Assessors (ISAs) who want to understand how AWS SRA patterns address PCI DSS intent.

Key AWS SRA design principles for PCI DSS

The guide applies six foundational AWS SRA design principles that are particularly relevant to PCI DSS compliance:

  • Implement a strong identity foundation: Enforce least privilege, separation of duties, and centralized identity management. Eliminate reliance on long-term static credentials.
  • Enable traceability: Monitor, alert, and audit actions in real time. Integrate log and metric collection with automated investigation and response systems.
  • Apply security at all layers: Defense-in-depth with preventive and detective controls at edge, virtual private cloud (VPC), load balancing, compute, OS, application, and code layers.
  • Protect data in transit and at rest: Classify data by sensitivity and apply encryption, tokenization, and access control mechanisms.
  • Keep people away from data: Reduce or eliminate direct access to cardholder data through automation and tooling.
  • Prepare for security events: Establish incident management processes, run simulations, and implement automated detection and recovery.

How to use the guide

The AWS SRA PCI DSS deep dive can be consumed in two ways:

  • As a narrative: Read the guide from beginning to end, starting with the PCI DSS primer, through the architecture and account scoping model, to the detailed requirement mappings. This approach gives you a complete understanding of how AWS SRA and PCI DSS intersect.
  • As a reference: Navigate directly to specific PCI DSS requirements or AWS SRA account types relevant to your current project. The guide includes architecture diagrams, requirement mapping tables, and service-specific configurations that you can use independently.

The guide includes downloadable architecture diagrams and detailed control mapping tables that complement the narrative content, making it straightforward to reference during security reviews and PCI DSS assessments.

Next steps

Security is a journey, not a destination. Review the AWS SRA PCI DSS Deep Dive guide and begin mapping its patterns to your own cardholder data environment and then validate existing environments against SRA best practices using SRA verify.

If you need assistance, contact AWS Professional Services, your AWS account team, or the AWS Partner Network, who can work with you to translate the reference architecture into a customized AWS environment that you can then operate.

If you have feedback about this post, submit comments in the Comments section below. If you need assistance architecting or implementing a PCI DSS-compliant AWS environment, contact the AWS Security Assurance Services team.


Author

Avik Mukherjee

Avik is a Senior Security Solutions Architect with more than a decade of experience in IT governance, security, risk, and compliance across retail, financial, and technology industries. He’s one of the original authors of the AWS Security Reference Architecture and leads the effort to extend the AWS SRA to compliance frameworks.

Akanksha Chaturvedi

Akanksha is a Senior Security Assurance Consultant with over 10 years of specialized experience in risk-based security assessments and regulatory compliance across highly regulated industries. She is an expert practitioner in HIPAA, PCI-DSS, GDPR, FedRAMP, and IRAP frameworks, with demonstrated success in architecting and deploying enterprise security programs from conception through full implementation.

Nimesh Ravas

Nimesh Ravasa

Nimesh is a Senior Assurance Consultant at AWS who focuses on security assurance and compliance for cloud-centered architectures. He brings extensive experience in PCI DSS assessments and security architecture reviews, helping organizations build and maintain compliant environments on AWS. He is passionate about translating complex compliance requirements into actionable technical guidance.

Omner Barajas

Omner Barajas

Omner is a Senior Security Solutions Architect at AWS with deep expertise in designing secure, scalable architectures for regulated industries. He specializes in network security, identity management, and security automation, and works with customers across financial services and payments to implement defense-in-depth strategies aligned with industry standards.

Viktor Mu

Viktor Mu

Viktor is a Senior Assurance Consultant at AWS with a strong background in information security governance, risk, and compliance. He specializes in helping organizations achieve and maintain PCI DSS compliance in complex cloud environments, with particular expertise in scoping, segmentation, and continuous compliance monitoring.

Announcing 90-minute function timeout on AWS Lambda Managed Instances

Post Syndicated from Tarun Rai Madan original https://aws.amazon.com/blogs/compute/announcing-90-minute-function-timeout-on-aws-lambda-managed-instances/

AWS Lambda now supports a 90-minute function timeout for asynchronous and event source mapping (ESM) invocations on AWS Lambda Managed Instances (LMI), a capability of AWS Lambda. This is a 6x increase from the previous 15-minute limit. Customers running data processing, media transcoding, financial calculations, AI inference, and batch workloads can now use Lambda functions for jobs that require longer continuous execution, without re-architecting their applications. This also applies to invocations within Lambda durable functions, which use checkpoints to track progress and automatically recover from failures through replay, skipping completed work. When invoked asynchronously, a multi-step durable execution can run for up to 1 year.

Evolution of function timeout on Lambda

Lambda’s function timeout has increased over time, from 5 minutes at launch in 2014 to 15 minutes in 2018. As customers sought to use the simplicity of Lambda for data-intensive workloads, the 15-minute timeout limit forced architectural tradeoffs for applications where customers needed longer continuous execution time. Several patterns emerged:

  • Media processing: Speech-to-text transcription and video transcoding that routinely need longer than 15 minutes of continuous execution.

  • Financial calculations: Monte Carlo simulations, bond pricing, and portfolio risk analysis that are memory-intensive and often require longer than 15 minutes.

  • Data processing and ETL pipelines: Batch jobs processing multi-gigabyte datasets or aggregating data from external sources that exceed 15 minutes during peak volumes.

  • AI inference: Model testing and inference jobs (for example, reasoning tasks) that fit Lambda’s memory and CPU profile but exceed its timeout.

  • Web scraping and file transfer: Crawling external sites or pulling large file sets from vendors that exceed 15 minutes when sources respond slowly.

In each case, customers preferred Lambda’s simplicity but had to re-architect when jobs hit the 15-minute limit.

Fast forward to 2026, Lambda supports two form factors: functions (event-driven, 15-minute timeout), and MicroVMs for user or AI-generated just-in-time code (HTTP-driven, 8-hour duration). To allow customers to benefit from the simplicity of serverless compute with the flexibility and pricing model of EC2 for steady-state workloads, we extended the on-demand capacity mode of Lambda to add Lambda Managed Instances (LMI). With Lambda Managed Instances, you can process multiple concurrent requests per instance, access specialized compute configurations, and drive cost efficiency through EC2 pricing advantages, without managing infrastructure.

We also support Lambda durable functions (powered by the durable execution SDK) on both on-demand and LMI capacity modes. Durable functions provide application-level checkpointing and workflow-as-code: your code saves execution state using step() and wait() operations, and gracefully recovers from infrastructure failures by resuming from the last checkpoint rather than restarting from scratch. While a durable execution (the complete lifecycle of a durable function) can run for up to a year, each invocation was still limited to 15 minutes.

As customers onboard more workloads to serverless compute to benefit from its simplicity, they need longer continuous execution for data-intensive use cases like AI inference, media transcoding, scientific modeling, and financial calculations that do not fit Lambda’s 15-minute duration constraints. Today, we are extending the function timeout on Lambda Managed Instances to 90 minutes for asynchronous and ESM invocations. This includes invocations within a durable function, where a multi-step application can continue to run for up to 1 year when invoked asynchronously.

Activating 90-minute function timeout

You can now configure any Lambda function running on a Managed Instance with a timeout of up to 90 minutes (5,400 seconds) for async and ESM invocations. Synchronous invocations retain the existing 15-minute maximum. The function executes exactly as before: same runtime, same handler, same IAM execution role, same virtual private cloud (VPC) configuration. The only difference is that your function now supports longer continuous execution. Your initialization code (Init phase) is still limited to 15 minutes on Lambda Managed Instances.

To set the timeout, update your function configuration using the AWS CLI:

aws lambda update-function-configuration \
    --function-name my-data-processor \
    --timeout 5400 \
    --region us-east-1

Or in AWS CloudFormation / AWS Serverless Application Model (AWS SAM):

MyFunction:
  Type: AWS::Serverless::Function
  Properties:
    FunctionName: my-data-processor
    Runtime: python3.12
    Handler: app.handler
    Timeout: 5400
    MemorySize: 10240

You do not need to change any code. You can also update the timeout from the Lambda console under Configuration > General Configuration (Figure 1), or configure it through natural language prompts in your AI coding assistants (like Claude Code or Kiro) by installing the Agent Toolkit for AWS.

Lambda console General configuration page showing the function timeout field set to 90 minutes

Figure 1: Configuring Lambda function timeout

The change takes effect on subsequent invocations after the function timeout is updated. For event source mappings, allow a few minutes for the new configuration to propagate. Your existing observability setup continues to work as expected: Amazon CloudWatch metrics, AWS CloudTrail, and AWS X-Ray capture the full invocation lifecycle without any changes. For details, see monitoring Lambda functions and monitoring durable functions.

There is no additional charge for using the 90-minute timeout. Standard Lambda Managed Instances pricing applies.

90-minute timeout and durable functions

The 90-minute function timeout and durable functions are complementary. The function timeout (--timeout) controls how long each individual invocation can run, while the durable execution timeout (ExecutionTimeout in --durable-config) controls the total elapsed time from execution start to completion. Durable functions use checkpoints to track progress and automatically recover from failures through replay, re-executing from the beginning while skipping completed work. With today’s launch, each asynchronous invocation in a durable function running on a Managed Instance can now execute for up to 90 minutes continuously, while the corresponding durable execution can run for up to 1 year. For synchronous and event source mapping invocations, both the invocation and the corresponding durable execution are limited to 90 minutes.

For idempotent jobs (for example, an ETL pipeline step triggered by SQS), the extended timeout alone might be sufficient. If the host fails, the message returns to the queue and a fresh invocation starts. For jobs where re-execution is expensive (for example, a 40-minute inference run already 30 minutes in), combine both. Enable durable functions to checkpoint periodically, so a failure at minute 35 resumes from the last checkpoint rather than restarting from zero.

Invocation behavior: asynchronous, event source mappings, and synchronous

Asynchronous invocations (up to 90 minutes): If the function fails or times out, Lambda applies your configured retry policy (up to two retries by default) and routes failed events to your dead-letter queue or on-failure destination.

Event source mappings (up to 90 minutes): For SQS, configure your queue’s visibility timeout to be at least six times the function timeout. This gives Lambda enough time to retry if a function is throttled while processing a previous batch. Lambda validates this at event source mapping creation time, but does not prevent subsequent changes to queue or function settings that might create a mismatch.

For Amazon Kinesis and Amazon DynamoDB Streams, configure the maximum batching window and parallelization factor to account for longer processing times per batch.

If your batch contains multiple records and you want to avoid re-processing the entire batch when one record fails, enable partial batch failure reporting. This is available for SQS, Kinesis, DynamoDB Streams, Amazon Managed Streaming for Apache Kafka (Amazon MSK), and self-managed Apache Kafka event source mappings. With partial batch failures enabled, only the failed records are retried, not the entire batch.

Note that invocations for Amazon MQ ESM and Amazon DocumentDB (with MongoDB compatibility) ESM remain limited to 15 minutes.

Synchronous invocations (15 minutes maximum): Synchronous invocations retain the existing 15-minute maximum timeout. If you set your function timeout to greater than 15 minutes and invoke it synchronously, Lambda continues to apply the 15-minute timeout. The GetFunctionConfiguration API reports the configured timeout value.

To see which event sources invoke Lambda functions synchronously or asynchronously, refer to Lambda documentation.

Considerations and best practices

Because your functions now support longer continuous execution, consider these best practices for components that might be ephemeral in nature, such as network connections and credentials.

Networking: Make sure idle connection timeouts on downstream services (RDS, Amazon ElastiCache, external APIs) accommodate the full function duration. If your function routes traffic through a NAT Gateway, send keep-alive packets to prevent idle connections from being dropped (350-second idle timeout). Respect DNS TTL values for external hostname resolution. The AWS SDK handles this automatically, but custom HTTP clients might cache DNS records beyond their TTL.

Credentials: If your function acquires temporary credentials or tokens, verify they remain valid for the full execution duration or refresh them in the background.

Idempotency: Lambda does not guarantee exactly-once processing. With longer-running functions, the window for retries and duplicate deliveries increases. You can use Powertools for AWS Lambda to implement idempotency in your function code so that operations like payments or database writes produce the same result even if executed more than once. If you use Lambda durable functions, steps have at-least-once execution semantics by default. The SDK skips completed steps during replay, but steps that fail before checkpointing may re-execute. You can use execution names as idempotency keys for durable functions.

Conclusion

The 90-minute function timeout on Lambda Managed Instances addresses one of the most common customer needs for building data-intensive applications on AWS Lambda. Data processing, media transcoding, AI inference, and financial computation workloads that exceed 15 minutes can now run on Lambda without code changes or architectural workarounds. We look forward to hearing from you if you need a longer timeout for synchronous invocations, or for the on-demand capacity mode, on our AWS Lambda Roadmap GitHub page.

To get started, update your function’s timeout configuration and deploy. For a step-by-step walkthrough, see Getting started with Lambda Managed Instances. For sample code demonstrating long-running functions with durable checkpointing, see durable functions examples. To learn more about AWS Lambda, visit aws.amazon.com/lambda.

OSPAR 2026 report now available with 167 services in scope

Post Syndicated from James Chang original https://aws.amazon.com/blogs/security/ospar-2026-report-now-available-with-167-services-in-scope/

We’re pleased to confirm the successful completion of our annual Amazon Web Services (AWS) Outsourced Service Provider’s Audit Report (OSPAR) assessment on July 29, 2026, in line with the OSPAR version 2.0 framework.

The Association of Banks in Singapore (ABS) established the Guidelines on Control Objectives and Procedures for Outsourced Service Providers (ABS Guidelines) to set out baseline control criteria for outsourced service providers (OSPs) operating in Singapore. These guidelines cover key areas such as cyber hygiene, technology risk management, business continuity, data security, cryptography, and software application development and management, drawing on regulatory direction from the Monetary Authority of Singapore (MAS).

This year’s certification cycle broadens the scope with five additional services, covering the 167 AWS services within the AWS Asia Pacific (Singapore) Region. The newly added services are:

This latest certification reinforces our commitment to the security standards expected of cloud providers within Singapore’s financial services industry. For customers, OSPAR offers a way to ease due diligence efforts typically associated with compliance reviews.

You can download the latest OSPAR report from AWS Artifact, a self-service portal for on-demand access to AWS compliance reports. Sign in to AWS Artifact in the AWS Management Console, or learn more at Getting Started with AWS Artifact. The list of services in scope for OSPAR is available in the report and is also available at AWS Services in Scope by Compliance Program.

We remain committed to expanding the OSPAR program’s scope over time, guided by customer architectural and regulatory needs. For any questions regarding the OSPAR report, reach out to your AWS account team.

If you have feedback about this post, submit comments in the Comments section below.

James Chang

James Chang

James is part of the Global Security Assurance team and has taken on major audit programs across the Asia Pacific Japan (APJ) region, including Japan’s ISMAP certification. He has also delivered APJ regulatory assessments and customer assurance engagements.

Ignatius Lee

Ignatius Lee

Ignatius is a Security Assurance professional based in Singapore, covering audits across the Asia Pacific Japan (APJ) region. Since joining Security Assurance in early 2025, he has contributed to key audit programs across the region, including Hong Kong, Singapore, Australia, Japan, Indonesia, and Korea.

Joseph Goh

Joseph Goh

Joseph is the APJ ASEAN Lead at AWS, based in Singapore. He leads security audits, certifications, and compliance programs across the Asia Pacific region. Joseph is passionate about delivering programs that build trust with customers and providing them assurance on cloud security.

Agentic security: Detection and response at machine speed

Post Syndicated from Gee Rittenhouse original https://aws.amazon.com/blogs/security/agentic-security-detection-and-response-at-machine-speed/

After talking with enterprise security leaders over the past year, one thing has become clear: the rise of autonomous AI agents is the most significant shift in security posture since the move to cloud. Organizations across every industry are adopting AI agents that authenticate on behalf of users, execute multistep workflows, and make decisions across infrastructure, often without waiting for human approval. Security operations need to keep pace.

At Amazon Web Services (AWS), we believe security should evolve ahead of AI adoption, not behind it. That belief drove our team to collaborate with the SANS Institute on a new chapter in the 2026 Cloud Security Exchange eBook, where we lay out a practical framework for securing agentic workloads at enterprise scale.

The challenge: Threats now move at machine speed

Traditional security was built for deterministic systems with predictable inputs and outputs. Agentic workloads break those assumptions. The same prompt can produce a compliant response on one request and a policy-violating response on the next. Agents adapt their behavior over time as they interact with users, data, and tools and operate with genuine autonomy: connecting to APIs, chaining actions together, and making independent decisions.

These properties mean that security controls designed for one-time assessments no longer suffice. Detection and response need to operate continuously and at machine speed.

What makes this urgent is the gap between adoption velocity and security maturity. Although 80% of organizations have adopted AI, only 10% govern it. Agents are being built by an expanding population of developers—including those using low-code tools—creating governance challenges that existing security programs must be extended to address.

Extending what already works

The good news, agentic security isn’t a blank slate. It builds on the same principles security teams already apply: identity governance, least privilege, defense in depth, and backup and recovery. What changes is how those principles are implemented when workloads are autonomous and probabilistic. In our eBook chapter, we cover four foundational areas:

  • Agent identity and governance: Every agent needs its own identity with temporary, scoped credentials rather than persistent, broad access. This extends zero trust principles to AI agents, where every request is authenticated and authorized independently, and every action has a traceable authorization chain. When a single agent combines access to sensitive data, the ability to communicate externally, and exposure to untrusted content, the risk profile changes significantly. Design patterns that prevent any single component from combining all three reduce that risk substantially.
  • Evolving detection for agentic workloads: Static, rule-based detection designed for human activity patterns can’t keep up with agent behavior. Organizations need continuous behavioral monitoring, living baselines that adapt as agents evolve, and instrumented observation that surfaces anomalies in real time. Amazon GuardDuty delivers this today, analyzing security signals continuously to detect threats as they emerge.
  • Response that balances speed with precision: When threats move at machine speed, response must be automated and tiered: some agent behaviors should be contained immediately, others require human judgment. The response framework we outline distinguishes between actions that can be automated safely and those that need escalation.
  • From single agents to multiagent ecosystems: Agents are already composing into teams, delegating subtasks, negotiating access, and coordinating across organizational boundaries. Each stage of this evolution inherits every security requirement that came before it, meaning organizations securing today’s basic chat agents are already laying the foundation for tomorrow’s multiagent ecosystems.

Security as an enabler of agentic AI adoption

The security leaders I speak with aren’t asking whether to adopt AI agents. They’re asking how to adopt them responsibly, at speed, and without slowing down the business.

AWS approaches this challenge by building security into the platform at every layer. Agentic AI built on AWS inherits nearly two decades of experience securing mission-critical workloads. Amazon GuardDuty, Amazon Inspector, and AWS Security Hub work together to provide continuous threat detection, vulnerability management, and unified security operations, all adapting to the unique characteristics of agentic workloads.

This isn’t about building new security from scratch. It’s about extending the security foundations your teams already trust into an environment where AI operates with increasing autonomy.

Read the full framework

Our chapter in the 2026 Cloud Security Exchange eBook goes deeper on each of these areas, with specific architectural patterns, implementation guidance, and frameworks for security teams at every stage of agentic AI maturity, whether you’re evaluating, piloting, or operating at scale.

Read the 2026 Cloud Security Exchange eBook: Agentic Security: Detection and Response at Machine Speed

You can learn more about AWS security services at AWS Cloud Security, or explore our AI Security Framework for a comprehensive view of how AWS secures AI workloads with the right controls, at the right layers, at the right phases.

If you have feedback about this post, submit comments in the Comments section below.


Gee Rittenhouse

Gee Rittenhouse

Gee is the Vice President of Agentic Security at AWS. He holds a PhD from MIT and brings extensive leadership experience across enterprise security and cloud. He previously served as CEO of Skyhigh Security and Senior Vice President and General Manager of Cisco’s Security Business Group, where he was responsible for Cisco’s worldwide cybersecurity business.

Measuring and improving search quality with Amazon OpenSearch Service

Post Syndicated from Aruna Govindaraju original https://aws.amazon.com/blogs/big-data/measuring-and-improving-search-quality-with-amazon-opensearch-service/

Search is the front door of many applications, yet most teams struggle to answer a deceptively simple question: “Is my search actually returning relevant results?” Query logs tell you what users typed, not what they saw, what they selected, or why they left. When search feels broken, the culprit is rarely the engine. It’s the lack of deliberate signal collection, measurement, and a feedback loop to act on it.

You can close this gap on Amazon OpenSearch Service using User Behavior Insights (UBI), an open schema standard for capturing search behavior, and Search Relevance Workbench (SRW), a toolkit for measuring and evaluating search quality. Your application generates the UBI-formatted records. Together, UBI and SRW give you a repeatable framework: collect signals, turn them into relevance judgments, and validate every change before it ships.

In this post, we show you how to capture UBI data on an Amazon OpenSearch Service domain and use those signals to evaluate search quality. This is the first post in a two-part series. We build the foundation here, and Part 2 covers automating the workflow end to end.

The challenge: You can’t improve what you can’t measure

Consider a shopper searching for “handbag” on an ecommerce site. The catalog has 16 products (tote bags, duffel bags, laptop bags), but every title only says “bag.” The search returns zero results. Most shoppers leave. A patient one retries with “bag” and finds what they were looking for.

Your server log recorded that first query as a clean sub-second response: no error, no alert, no signal. What it missed entirely was a customer with purchase intent. That customer hit a vocabulary gap between how they search and how you write your catalog. Zoom out and apply this lens to misspelled queries, poor handling of long-tail searches, and abandoned sessions. The blind spot is larger than you think.

There’s a second problem: click signals are position biased. Users select the first result far more than the fifth, regardless of relevance, so raw click counts reflect where results appeared, not whether they deserved to be there. Any judgment derived from clicks must correct for this bias. We return to it when generating judgments.

Capturing behavioral data with UBI

UBI defines two indices. The ubi_queries index holds one record per executed query: the text the user typed, the full query that ran (filters and facets included), and the IDs of the documents returned. The ubi_events index holds every subsequent user action: impressions, hovers, clicks, add-to-carts, each stamped with the result position and the product’s business identifier (object_id). A shared query_id links every event back to the query that triggered it. Two additional identifiers complete the picture: client_id tracks the browser across visits, and session_id scopes events to a single visit.

A query record captures what the user asked and which document IDs the engine returned, including zero-result cases like the handbag search, which appears as a record with an empty result list. Here’s the shopper’s follow-up search for “bag”:

{
  "query_id": "1bf736d4-d673-4763-9193-4bc8a2282115",
  "client_id": "9a9968ac-664b-42d7-9a9e-96f412b5ab49",
  "user_query": "bag",
  "query": "{"multi_match": {"query": "bag", "fields": ["title", "description", "category", "brand"]}}",
  "query_response_hit_ids": [
    "3760170840499",
    "8400000000042"
  ],
  "timestamp": "2026-07-23T07:53:35.264Z",
  "application": "retail-shop"
}

The UBI queries schema reference documents the complete query schema, including the mandatory attributes.

The event record captures what the user did next. For each result rendered, emit an impression event. When the user selects a result, emit a click event. Here is the impression event for the first result of the bag search:

{
  "action_name": "impression",
  "query_id": "1bf736d4-d673-4763-9193-4bc8a2282115",
  "client_id": "9a9968ac-664b-42d7-9a9e-96f412b5ab49",
  "session_id": "0f2e6f2a-8f4e-4f60-9f6e-2a1b3c4d5e6f",
  "user_query": "bag",
  "timestamp": "2026-07-23T07:53:41.112Z",
  "event_attributes": {
    "position": {
      "ordinal": 1
    },
    "object": {
      "object_id": "3760170840499",
      "object_id_field": "object_id"
    }
  }
}

event_attributes also accepts custom fields of your own alongside the standard position and object structures. The action_name attribute is critical: The judgment model you use later consumes only impression and click events. Treat a paginated results page as the same logical query: reuse the query_id and record absolute positions. The UBI events schema reference documents the complete event schema.

Collecting UBI data on Amazon OpenSearch Service

Behavioral data (what results ranked, what users saw, what they selected) exists only in the application layer. Your application owns the records, and Amazon OpenSearch Ingestion (OSI), a fully managed, serverless data collector powered by Data Prepper, provides the managed delivery path. Your application sends the records as SigV4-signed HTTP POST requests to the OSI pipeline endpoints. Route browser events through your backend for signing. One thing to understand before you write any code: Your application generates and owns the query_id attribute. The application creates the ID when it runs a search and stamps it on every subsequent event the user produces, until the user issues a new search or the session ends.

Prerequisites

To follow along, you need an Amazon OpenSearch Service domain running OpenSearch 3.5 or later with the OpenSearch UI application, permissions to create OpenSearch Ingestion pipelines with an AWS Identity and Access Management (IAM) pipeline role, and a search application you can instrument to emit behavioral records.

Create the UBI indices

Before you start collecting user metrics, you need the two indices in place with the right mappings. Field types matter here: query_id as keyword supports exact joins between queries and events, timestamp as date supports time-range queries, and event_attributes as dynamic means you can extend events with custom fields without schema changes.

Create ubi_queries first in Dev Tools. It holds the query-side records. We abbreviated the mappings here. Refer to the published queries-mapping.json file for the complete version:

PUT ubi_queries
{
  "mappings": {
    "properties": {
      "query_id": { "type": "keyword" },
      "client_id": { "type": "keyword" },
      "user_query": { "type": "keyword" },
      "query_response_hit_ids": { "type": "keyword" },
      "timestamp": {
        "type": "date",
        "format": "strict_date_time"
      },
      "application": { "type": "keyword" }
    }
  }
}

Then create ubi_events. It holds every user action that follows (refer to the full events-mapping.json file):

PUT ubi_events
{
  "mappings": {
    "properties": {
      "query_id": { "type": "keyword", "ignore_above": 100 },
      "action_name": { "type": "keyword", "ignore_above": 100 },
      "client_id": { "type": "keyword", "ignore_above": 100 },
      "session_id": { "type": "keyword", "ignore_above": 100 },
      "user_query": { "type": "keyword" },
      "timestamp": {
        "type": "date",
        "format": "strict_date_time"
      },
      "event_attributes": {
        "dynamic": true,
        "properties": {
          "position": {
            "properties": {
              "ordinal": { "type": "integer" }
            }
          },
          "object": {
            "properties": {
              "object_id": { "type": "keyword" },
              "object_id_field": { "type": "keyword" }
            }
          }
        }
      }
    }
  }
}

With both indices created, the next step is routing data into them. You can deliver UBI data to your domain in several ways. This post uses OSI pipelines, shown end to end in the diagram that follows the setup.

Set up the OSI pipelines

Create two OSI pipelines: one for queries and another for events. Each pipeline exposes an HTTP source endpoint that your application writes to (shown on each pipeline’s console page) and sinks data to the corresponding index. The following configuration defines the events pipeline:

version: '2'
ubi-events:
  source:
    http:
      path: /ubi/events
      max_request_length: 10mb
  processor:
    - date:
        from_time_received: true
  sink:
    - opensearch:
        hosts: ["https://<domain-endpoint>"]
        aws:
          serverless: false
          region: <region>
          sts_role_arn: <pipeline-role-arn>
        index_type: custom
        index: ubi_events
    - s3:
        aws:
          region: <region>
          sts_role_arn: <pipeline-role-arn>
        object_key:
          path_prefix: 'ubi_events/%{yyyy}/%{MM}/%{dd}'
        bucket: <bucket-name>
        threshold:
          maximum_size: 50mb
          event_collect_timeout: 60s
        codec:
          ndjson:

Note: the queries pipeline follows the same pattern, with /ubi/queries as the path and ubi_queries as the sink index and S3 prefix. Create the pipeline role yourself or let OpenSearch Ingestion create it. If your domain uses fine-grained access control, also map the pipeline role to a backend role so the domain accepts the pipeline’s writes. Refer to the tutorial Collecting UBI-formatted data in Amazon OpenSearch Service for detailed steps.

With the pipelines running, your application can start sending data. The following diagram illustrates the end-to-end flow:

UBI collection flow from the search application through OpenSearch Ingestion into the ubi_queries and ubi_events indices

Figure 1: The UBI collection pattern on Amazon OpenSearch Service

The workflow consists of the following steps:

  1. Users interact with your search application.
  2. The application sends signed query records to the OSI HTTP endpoint.
  3. OSI writes queries to the ubi_queries index.
  4. Users interact with the results, viewing and selecting documents.
  5. The application sends signed event records, carrying the same query_id, to the OSI HTTP endpoint.
  6. OSI writes events to the ubi_events index.
  7. Optionally, both pipelines archive records to Amazon Simple Storage Service (Amazon S3).
  8. Search Relevance Workbench (OpenSearch UI) works with the collected data in the ubi_queries and ubi_events indices.

Note: if you’re already collecting site analytics through an existing third-party tool, you don’t need to replace it. Map your search-related events (queries, clicks, and conversions) into the UBI schema and store them in OpenSearch. That’s enough to unlock the out-of-the-box evaluation framework, implicit judgment generation, and the full SRW metrics pipeline, without defining a single custom metric from scratch.

Visualize the data collected

After the UBI behavior metrics start to trickle in, you can review the data in the Discover tab on the OpenSearch UI dashboard. Filtering ubi_queries for empty result lists ranks your vocabulary gaps. You can also visualize the data collected through the sample User Behavior Insights (UBI) dashboards in OpenSearch.

OpenSearch Discover view of UBI records for the zero-result handbag query and the follow-up bag query

Figure 2: UBI records in Discover, showing the zero-result handbag query and the follow-up bag query with its impressions and pagination events

With data flowing into your indices, keep these things in mind as you scale to production:

  • Keep telemetry off the search critical path – Queue records and forward them asynchronously. Losing a fraction of behavioral data is statistically harmless. Blocking users isn’t.
  • Manage volume deliberately – Batch impression events, and if you sample, sample whole queries rather than individual events to preserve the click-through ratios that drive judgments.
  • Isolate analytical load for larger deployments – Route pipelines to a separate analysis domain with the same engine version, mappings, and analyzers as production. This keeps behavioral writes from touching live search latency.
  • Plan for retention and integrity – Register the UBI mappings as an index template and apply an Index State Management (ISM) retention policy as your indices grow. You should validate and rate-limit the event write path, and cover query text and client identifiers with your data retention policy.

Evaluating search quality with Search Relevance Workbench

With ubi_queries and ubi_events collecting data, you now have the signals needed to evaluate search quality. Search Relevance Workbench, generally available in the OpenSearch UI from Amazon OpenSearch Service 3.5, turns those signals into structured experiments: comparing query configurations, scoring results against relevance judgments, and surfacing metrics that guide iterative tuning.

The Search Relevance Workbench home screen in the OpenSearch UI

Figure 3: Search Relevance Workbench in the OpenSearch UI

SRW experiments rely on three components. You set them up once, then reuse them across every experiment you run: a query set (the fixed queries you evaluate against), search configurations (the query structures you want to compare), and a judgment list (the relevance ground truth). The following sections walk through each one.

Step 1: Create a query set

A query set is the fixed collection of queries you evaluate against. Keeping it fixed makes results comparable across experiments. Effective query sets reflect real traffic, not intuition. You can seed one from your top queries, a random sample, or a hand-picked mix that includes long-tail and low-performing queries. Alternatively, SRW can sample directly from ubi_queries using Probability-Proportional-to-Size (PPS) sampling, which selects queries in proportion to how often users issue them. This approach represents frequent queries like “bag”, so your metrics reflect search quality as users experience it.

Query set creation screen sampling queries from real traffic in the ubi_queries index

Figure 4: Creating a query set sampled from real traffic in ubi_queries

Step 2: Define search configurations

A search configuration defines how a search executes: the index, the query structure, and a %SearchText% placeholder that SRW replaces with each query in your set. Creating two configurations and running them against the same query set and judgment list is how you validate a change before any user sees it.

As an example, here we define two configurations: a baseline multi_match query (retail_query) and a variant that boosts title matches (retail_boosted_query), so we can measure whether the boost actually helps ranking.

retail_query retail_boosted_query
{
  "query": {
    "multi_match": {
      "query": "%SearchText%",
      "fields": [
        "title",
        "description",
        "category",
        "brand"
      ]
    }
  }
}
{
  "query": {
    "multi_match": {
      "query": "%SearchText%",
      "fields": [
        "title^2",
        "description",
        "category",
        "brand"
      ]
    }
  }
}

Configurations go beyond query variants: a candidate can be an entirely different retrieval strategy, like hybrid search combining keyword and neural retrieval. You can use judgments to rate query-document pairs independently of your retrieval approach. You can test a semantic or hybrid approach offline against your existing traffic before shipping it.

Step 3: Create the judgment list

A judgment is a relevance rating for a query-document pair: the ground truth that quality metrics measure against. You can create judgments that are explicit (from stakeholders or a large language model acting as judge), imported, or implicit (derived from behavior). Here we use implicit judgments derived from UBI selection behavior, scored using the Clicks Over Expected Clicks (COEC) model. The COEC model helps correct position bias by comparing each document’s actual click rate against the expected rate for its rank position. Documents that outperform their position score as relevant. Those that users select because they ranked first score near average.

Judgment list creation screen with the Implicit click-based type and COEC click model selected

Figure 5: Creating an implicit judgment list with the Implicit (Click based) type and the COEC click model

Three things to get right before you run experiments:

  1. object_id in your events must match the document _id from your product catalog. The search configurations you define return this _id, which lets SRW join judgments to results.
  2. Implicit judgments are statistical. They need volume and query coverage. As a working rule of thumb, aim for hundreds to thousands of real sessions per query to separate signal from noise.
  3. Max Rank controls how deep in the result list events count. If users paginate, set it beyond a single page. We use 20 here.

Step 4: Run experiments

This post uses three SRW capabilities: Query Analysis, Query Set Comparison, and Search Evaluation. Query Analysis is a quick eyeball check: compare two configurations side by side for a specific query to see exactly what changed and why the metrics moved. The other two answer harder questions with numbers: how good a configuration is, and how two configurations compare against real relevance signals.

Query Set Comparison (also called pairwise comparison) takes two configurations and computes ranking similarity. Jaccard overlap measures how much the two result lists share, while Rank-Biased Overlap (RBO) weights agreement at the top of the list more heavily. Near-identical scores mean the change will barely register with users. Low overlap means a real ranking shift worth reviewing carefully before shipping. In this run, the two configurations score 0.93 Jaccard and 0.92 RBO, a modest but real shift. SRW cannot score zero-result queries like “handbag”: They show zero similarity in a comparison and Failed in an evaluation, a signal they need a different fix than ranking adjustments.

Query Set Comparison results showing Jaccard and Rank-Biased Overlap scores for the two configurations

Figure 6: Query Set Comparison showing Jaccard and Rank-Biased Overlap between the two configurations

Search Evaluation (also called pointwise evaluation) scores one configuration against your query set and judgment list across four metrics, each computed over the top k results (k=10 by default):

Metric What it measures What it tells you
Coverage@k Proportion of returned documents that have judgments How much to trust the other three metrics. Low Coverage means many results were never judged
Precision@k Fraction of the top k results that are relevant How many irrelevant results appear on the first page
MAP@k (Mean Average Precision) Precision averaged across ranks, rewarding relevant documents placed early Whether relevant results appear early, even when Precision ties
NDCG@k (Normalized Discounted Cumulative Gain) Graded judgment values, discounted by position (rank 1 counts more than rank 9) Whether the best results appear first. The primary comparison metric

Each pointwise experiment evaluates one configuration. To compare candidates, run one experiment per configuration and compare the results. In this run, the baseline (retail_query) scores Coverage@10 of 1.0, Precision@10 of 1.0, MAP@10 of 0.95, and NDCG@10 of 0.93, with the zero-result “handbag” query showing as Failed in the per-query detail.

Search evaluation results showing Coverage, Precision, MAP, and NDCG at 10 with per-query detail

Figure 7: Search evaluation results for one configuration: Coverage, Precision, MAP, and NDCG at 10, with per-query detail

From measurement to improvement

The preceding experiments are the harness. The following are common levers to test with it. Express each as a new search configuration, evaluate it against the same query set and judgment list, and adopt it only if the metrics move:

  • Synonyms – One option for addressing known vocabulary gaps is to build synonyms. A search-time synonym token filter treats “handbag” and “bag” as equivalent, and with Amazon OpenSearch Service, you can hot deploy custom synonym packages without reindexing.
  • Field weights – Adjust the fields and boosts in a multi_match query, like the title^2 variant tested earlier.
  • Semantic retrieval – A hybrid query combines keyword and neural scores, addressing vocabulary mismatch as a class rather than term by term. Judgments evaluate it offline exactly like a lexical candidate.
  • Reranking – A rerank processor in a search pipeline reorders the top results using a cross-encoder model.

Clean up

To avoid future charges, delete the resources you created for this walkthrough:

  • Delete the two OpenSearch Ingestion pipelines. To reuse them later, stop them instead. A stopped pipeline keeps its configuration and incurs no OpenSearch Compute Unit (OCU) hour charges.
  • If you configured the optional Amazon S3 archive, delete the archived objects (or the bucket).
  • If you keep the domain, optionally delete the ubi_queries and ubi_events indices and the query sets, judgment lists, and experiments you created. These live on the domain and incur no separate charges.
  • If you created the domain specifically for this post, delete it to remove everything, including the resources in the previous step. Deleting a domain is irreversible. Don’t delete a domain that serves other workloads.

Conclusion

UBI collects the evidence, COEC turns it into judgments, and SRW experiments deliver the verdict: Coverage, Precision, MAP, and NDCG in place of guesswork. Ship the winning configuration, keep collecting, and the next round of judgments shows whether the improvement holds with real behavior. Where there used to be an opinion, there is now a number.

Everything here follows a repeatable pattern, and repeatable patterns lend themselves to automation. Part 2 walks through the Search Relevance Agent, available through the AI Assistant chat (the Ask AI button) in the OpenSearch UI. The agent analyzes your UBI signals, generates tuning hypotheses, and validates them offline before recommending changes. The pipeline you built in this post is the foundation. Stay tuned for Part 2.

To go deeper on the evaluation features, refer to the Search Relevance Workbench documentation.


About the authors

Aruna Govindaraju

Aruna Govindaraju

Aruna is an Amazon OpenSearch Specialist Solutions Architect and has worked with many commercial and open source search engines. She is passionate about search, relevancy, and user experience. Her expertise with correlating end-user signals with search engine behavior has helped many customers improve their search experience.

Sean Bjurstrom

Sean Bjurstrom

Sean is an Enterprise Support Lead in ISV accounts at Amazon Web Services, where he specializes in Analytics technologies and draws on his background in consulting to support customers on their analytics and cloud journeys. Sean is passionate about helping businesses harness the power of data to drive innovation and growth. Outside of work, he enjoys running and has participated in several marathons.

Utkarsh Agarwal

Utkarsh Agarwal

Utkarsh is a Cloud Support Engineer in the Support Engineering team at AWS. He provides guidance and technical assistance to customers, helping them build scalable, highly available, and secure solutions in the AWS Cloud. In his free time, he enjoys watching movies, TV series, and, of course, cricket! Lately, he has also been attempting to master foosball.

We invited a direct competitor into Security Hub Extended. Here’s why.

Post Syndicated from Michael Fuller original https://aws.amazon.com/blogs/security/we-invited-a-direct-competitor-into-security-hub-extended-heres-why/

When customers keep pointing you to a solution that overlaps with parts of your own offering, you have a choice to make. This post is about the choice we made with Upwind, and why we’d make it again.

AWS Security Hub Extended exists because customers told us what was working for them in enterprise security and asked us to simplify adoption and integration. Upwind was one of the solutions customers kept naming, so we brought them in. Upwind didn’t only agree to participate, they committed fully to integration. They brought their full solution portfolio into Extended with aggressive pay-as-you-go pricing from day one. They got their field organization fully aligned on joint deal flow and have driven more customer activity and closed deals through Security Hub Extended than any other partner in the program.

Giving customers choice, even when it overlaps

Multiple best-of-breed options in cloud security—including one that overlaps with our own capabilities—are straightforward when you start with what customers need. Some will choose Security Hub Essentials for cloud security posture management and vulnerability scanning. Some will choose Upwind for runtime-first protection. Some will run both and get stronger outcomes from the combination. The customer decides, not us. That principle applies to every partner in Security Hub Extended. We listen to what’s working, and we simplify adoption through the same AWS relationship customers already have.

“Our customers run on AWS, and Security Hub is where their security operations live,” said Amiram Shachar, Co-Founder and CEO of Upwind. “Being inside Security Hub means customers get Upwind’s cloud workload protection with the same billing, the same support path, and the same operational model they already know. We’re here because it’s a better outcome for the customers we share.”

Who is Upwind?

Upwind is a cloud security company trusted by Siemens, Peloton, Roku, Wix, Nextdoor, and Nubank. Fast Company named them one of the Most Innovative Companies of 2026.

What makes them different is runtime. Most cloud security solutions scan configurations periodically and report what could be a risk based on static posture. Upwind deploys an eBPF-based sensor directly in the Linux kernel that sees what workloads are doing in real time, including process behavior, network connections, API calls, and container interactions. All observed continuously. That means Upwind can tell you not only what could theoretically be exploited, but what is actively at risk right now. That distinction cuts alert noise dramatically and lets security teams focus on what genuinely matters.

Better together. Not only with AWS, but with each other

Now extend that to the rest of your security stack. If you’re already running other Security Hub Extended solutions, they work together without you building the integrations.

A customer running Chainguard for supply chain security, Upwind for runtime protection, and Splunk for security operations gets a connected experience. Chainguard helps ensure clean, malware-resistant dependencies at build time. Upwind validates workload behavior at runtime and enriches those findings with real-time context. Everything flows into Splunk through Security Hub for unified triage. One experience, one bill, no custom integration work. The security team sees the full lifecycle from build to production without stitching tools together.

That same pattern applies with 7AI, where AI-driven automation can triage and investigate Upwind’s runtime events alongside endpoint, identity, and network signals, all without manual pipeline work.

This is the multi-way partnership that Security Hub Extended was designed to enable. These solutions aren’t only easier to buy together, they’re building toward each other. The findings flow into Security Hub in OCSF (Open Cybersecurity Schema Framework), get correlated and prioritized together, and route to the downstream tools your team already uses. Your security stack gets stronger as a whole, not only solution by solution.

How it works commercially

This isn’t a paper partnership. We’re closing multi-million dollar deals together through Security Hub Extended. Upwind has engaged faster than any other partner in the program, bringing their own customer opportunities and joining AWS-originated deals to close them jointly. One enterprise customer recently replaced their incumbent CNAPP with Upwind through a Security Hub Extended Private Offer. The deciding factors were runtime visibility that their previous solution couldn’t deliver and a single predictable commercial model that replaced complex per-module pricing across multiple vendors. The commercial model has momentum, and it’s because Upwind invested not only in signing an agreement but in the engineering and go-to-market work that makes joint success real.

Upwind is available through Security Hub Extended with pay-as-you-go pricing, one AWS bill, and no required long-term commitment. For enterprises that prefer committed-pricing agreements, Security Hub Extended Private Offers are also available with deeper discounts and the ability to aggregate spend across partners. You choose the path that fits how you buy. If you’re already running Security Hub for posture management and vulnerability scanning, adding Upwind gives you runtime visibility alongside what you already see. No new tooling to stand up, no new workflow to learn. It shows up in your existing prioritized view of risk.

What Upwind is building next

Upwind continues to expand. AI workload protection that monitors model behavior and agent tool calls at runtime. Windows Server VM coverage across AWS, Azure, and GCP. Deeper integration with the Security Hub correlation engine so runtime context enriches attack-path intelligence automatically. The partnership deepens as both sides invest.

“We believe runtime context and AWS-native signals together produce stronger outcomes than either alone,” said Amiram Shachar, Co-Founder and CEO of Upwind. “As Security Hub deepens its correlation and Upwind extends its runtime fabric, customers who use both will have a view of risk that no single solution can replicate. That’s the future we’re building toward together.”

What this means for you

Security Hub Extended exists to give you access to the solutions your peers are already succeeding with through the AWS relationship you already have. Upwind is what that philosophy looks like when applied to a category where AWS has an existing offering. We listened to customers, saw what was working for them, and made it available with the same commercial model as everything else.

Enable Upwind through the AWS Security Hub console. Pay-as-you-go. No commitment required. If you want to understand what consolidation looks like with Security Hub Extended, talk to your AWS account team.

We’re just getting started, but the momentum is real.

If you have feedback about this post, submit comments in the Comments section below.


Michael Fuller

Michael has been with AWS for 16 years and led product for AWS Security Services for 11 years. Michael has 29 years in the industry and held several roles in product management, business development, and software development for IBM, Cisco, and Amazon. Michael has a Bachelor’s of Science in Computer Engineering from the University of Arizona and an MBA from the University of Washington.

ICYMI: July 2026 @AWS Security

Post Syndicated from Rodolfo Brenes original https://aws.amazon.com/blogs/security/icymi-july-2026-aws-security/

If you found time for a bit of vacation this summer, you might be in catch-up mode. Here’s a list to help: all the expert blog posts, new service capabilities, code samples, and workshops, in case you missed it, from July 2026.

AWS Security Blog post

This month’s AWS Security Blog posts covered AI agent security, supply chain protection, network firewall automation, DDoS mitigation, and compliance readiness. Read on for guidance on securing AI coding agents, implementing dependency cooldowns, choosing the right key management solution, and preparing for HIPAA Technical Safeguard requirements.

AI Security

Enforce least-privilege authorization in multi-agent AI chains using Cedar
Authors: Dhananjay Karanjkar | Published: July 6, 2026
Learn to implement a three-layer Cedar policy model with OAuth 2.0 authentication to prevent authorization scope expansion across multi-agent delegation chains using Amazon Verified Permissions.

Enforce zero data retention on Amazon Bedrock with Bedrock Projects and service control policies
Author: Rob Higareda | Published: July 7, 2026
Learn to use Amazon Bedrock Projects and SCPs to centrally enforce zero data retention policies, preventing accounts from enabling data sharing with third-party model providers across your organization.

Designing for the inevitable: System prompt leakage and mitigations in generative AI applications
Author: Manideep Konakandla | Published: July 8, 2026
Learn to implement defense-in-depth mitigations for system prompt leakage using Amazon Bedrock Guardrails prompt attack filters, canary tokens, semantic similarity detection, and sandwich instruction patterns.

Balancing speed and safety: A control framework for AI coding agents
Authors: Daniel Begimher, Danny Cortegaca | Published: July 30, 2026
Learn to implement an application security control framework for AI coding agents, with author-time controls that shape what agents produce and build-time controls that verify what reaches production.

Data Protection

How to use the AWS Workload Credentials Provider for cross-account secret retrieval and prefetching secrets
Authors: Derik Wang, Paras Dhawan | Published: July 1, 2026
Learn to configure the AWS Workload Credentials Provider for cross-account secret retrieval using IAM role chaining and prefetching secrets at startup to reduce cold-start latency.

The CISO’s guide to post-quantum mandates and migrations
Author: Rushir Patel | Published: July 8, 2026
A strategic playbook for CISOs navigating post-quantum cryptography migration, covering regulatory timelines, dependency classification, cryptographic telemetry, and building crypto-agile organizations.

AWS KMS or AWS CloudHSM: Choose the right key management solution
Author: Derek Tumulak | Published: July 28, 2026
Learn how to choose between AWS KMS and AWS CloudHSM based on integration needs, cost, and whether you require traditional HSM interfaces or legacy algorithms.

Secure your npm and pip package updates in Amazon Linux
Author: Norbert Manthey | Published: July 29, 2026
Learn to implement a one-line dependency cooldown for npm and pip that skips packages published in the last 24 hours, protecting against supply chain events while still allowing urgent security patches.

Infrastructure security

Secure Amazon container workloads using container attribute-based rules in AWS Network Firewall
Authors: Amit Gaur, Amish Shah, Preetkumar Shah, Akash Kumar Sinha | Published: July 1, 2026
Learn to define AWS Network Firewall rules for Amazon EKS and Amazon ECS workloads using native container attributes like namespaces, pod names, and labels instead of ephemeral IP addresses.

Authenticate legitimate AI agent traffic with AWS WAF Bot Control
Authors: Harith Gaddamanugu, Kaustubh Phatak | Published: July 14, 2026
Learn to use Web Bot Authentication (WBA) in AWS WAF Bot Control to cryptographically verify legitimate AI agent traffic using HTTP message signatures and ed25519 keys.

Accelerating AWS Network Firewall troubleshooting with AWS DevOps Agent
Author: Salman Ahmed | Published: July 24, 2026
Learn to use AWS DevOps Agent to automate root cause analysis for AWS Network Firewall connectivity issues, including domain deny lists, stateless rule priority misconfigurations, and asymmetric cross-AZ routing drops.

AWS Shield Advanced is embracing the AWS WAF Anti-DDoS managed rule group: What changes and how to prepare
Authors: Eitav Arditti, Andrew Chen, Justin Kurpius | Published: July 27, 2026
AWS Shield Advanced is adopting the AWS WAF Anti-DDoS managed rule group as its default application-layer DDoS protection, with a phased migration from July 2026 through January 2027.

Threat detection and incident response

Introducing the Amazon GuardDuty investigation agent: on-demand AI-powered threat assessment
Author: Allan Holmes | Published: July 20, 2026
Learn to use the new Amazon GuardDuty investigation agent (public preview) to automate threat correlation and receive structured assessments with risk levels, confidence scores, MITRE ATT&CK mappings, and actionable recommendations.

Amazon identifies North Korean hacker group behind open-source supply chain attacks
Author: CJ Moses | Published: July 29, 2026
Learn how Amazon Threat Intelligence linked the compromises of axios, debug, chalk, and typo-crypto NPM packages to a single DPRK-linked threat actor, and how attacker tradecraft is evolving with generative AI.

Extend Amazon Inspector SBOM Generator with plugins
Authors: Michael Long, Anthony Verleysen, Charlie Bacon | Published: July 30, 2026
Learn to write custom Lua plugins for the Amazon Inspector SBOM Generator to inventory package ecosystems that aren’t supported out of the box, without modifying source code or waiting for an official release.

Security Hub adds AI workload protection and multicloud support for Microsoft Azure
Author: Michael Fuller | Published: July 14, 2026
AWS Security Hub now monitors Microsoft Azure resources for misconfigurations and vulnerabilities, adds GuardDuty AI Protection for Amazon Bedrock and Amazon SageMaker AI, and introduces an AI inventory for organization-wide visibility.

Governance and compliance

AWS designated as a critical third party to the UK financial sector
Author: Michael Jefferson | Published: July 10, 2026
AWS has been designated as a critical third party to the UK financial sector by HM Treasury, establishing direct regulatory oversight by the Bank of England, PRA, and FCA.

New compliance guidance available: HITRUST i1 on AWS
Authors: Abdul Javid, Shreya Singh | Published: July 13, 2026
AWS published new implementation guidance for HITRUST i1 certification, covering 11 technical control domains with AWS-specific controls for healthcare organizations seeking i1 assessment readiness.

HIPAA Security Rule on AWS – Technical Safeguards Implementation and Readiness Guidance
Authors: Abdul Javid, Hector Rodriguez, Kapil Temghare, Shreya Singh | Published: July 31, 2026
New guidance helping covered entities and business associates implement and evidence compliance with HIPAA Security Rule Technical Safeguards (§164.312) on AWS, including 2025 NPRM proposed changes.

Identity

Introducing OAuth support for AWS MCP Server
Authors: Vaibhav Chowla, Jaimin Bhatt, Ankur Joshi | Published: July 9, 2026
AWS MCP Server now supports OAuth 2.1 authorization through AWS Sign-In, enabling agents like Claude Code,Kiro, and Gemini CLI to connect using existing IAM credentials with browser-based authentication.

July Security Bulletins

In July 2026, AWS published 21 security bulletins (2026-049 through 2026-069) addressing vulnerabilities across open-source SDKs, MCP servers, and developer tools. Key themes include credential disclosure and SSRF, affecting HealthLake, HealthOmics, and API MCP servers, plus Strands Agents tools that could inadvertently expose secrets to unauthorized endpoints. Command and code injection impacted aws-cdk-lib, jsii-diff, Bedrock AgentCore SDK, and Amplify Codegen UI. The smithy-rs framework received three patches for denial-of-service via uncontrolled recursion and Slowloris issues.

Other notable issues include insecure file permissions in the AWS CLI, deserialization remote code execution in the Advanced JDBC Wrapper, SQL injection in mcp-gateway-registry, TLS 1.3 flaws in s2n-tls, and stored XSS in AWS Ops Wheel. A common thread: insufficient input validation in tools interacting with AI agents, reflecting the expanded surface area of LLM-integrated workflows. All patches are available, upgrade promptly. For more information, see AWS Security Bulletins.

AWS Samples

This month brings 14 new AWS samples spanning AI security, identity, data protection, governance, threat detection, and security posture management. From deploying governed AI agent platforms on Amazon Bedrock AgentCore to building data-residency-compliant chatbots and DevSecOps baselines for Kiro, these repositories help you implement security and governance best practices across your AWS environment.

AI Security

Lark MCP on AgentCore
Learn to deploy a hosted remote MCP service on Amazon Bedrock AgentCore that lets AI agents operate Feishu/Lark through 450+ tools, with per-user identity isolation and smart multi-step orchestration via 20+ domain Skills.

Lark CLI MCP Wrapper on AgentCore Runtime and Identity
Learn to securely wrap a CLI tool as an MCP server on AgentCore Runtime using a sidecar credential-isolation pattern, where the CLI process never holds real tokens and all secrets are resolved through AgentCore Identity’s Token Vault.

LiteLLM Bedrock Gateway on EKS
Learn to deploy a production-grade LiteLLM proxy on Amazon EKS as a unified OpenAI/Anthropic-compatible gateway to Amazon Bedrock, with four progressive layers covering network isolation, cross-region inference profiles, and cross-account delegation.

Enterprise Agentic AI Platform Accelerator on AgentCore
Learn to deploy a secure, governed foundation for production AI agents on Amazon Bedrock AgentCore with CDK stacks covering identity, gateway, memory, runtime, and observability; supporting multiple agent frameworks (Strands, LangGraph, Claude SDK) and opt-in security controls including VPC isolation, KMS encryption, Cedar policies, and Bedrock Guardrails.

FlowAMP: AI Agent Governance on AWS
Learn to deploy a single-pane-of-glass agent management platform on Amazon Bedrock AgentCore that discovers, monitors, scores, controls, and cost-accounts AI agents across an AWS Organization with agentic discovery, compliance scanning (NIST AI RMF, ISO 27001, SOC 2), Responsible-AI scoring, FinOps via Cost Explorer, and Cedar-based policy enforcement.

Kiro SecOps Baseline
Learn to deploy a DevSecOps security baseline for Kiro as a single Go CLI that installs global guardrails (permissions.yaml, steering, skills, a security-review agent) and per-project workspace hooks (fail-closed guard, PR/pipeline review gates, scanner configs for gitleaks, trivy, and checkov) with enterprise fleet distribution via MDM and Administration scope.

Identity

OAuth 2.0 Token Exchange with Amazon Cognito
Learn to implement RFC 8693 OAuth 2.0 Token Exchange using Amazon Cognitowith a true delegation pattern, enabling services to act on behalf of users while maintaining distinct service identities and least-privilege access in microservices architectures.

Lark Identity on AgentCore — Gateway Interceptor
Learn to implement enterprise identity pass-through on Amazon Bedrock AgentCore using a Gateway Request Interceptor that forwards the user’s identity and injects per-user credentials to downstream MCP tools, so the agent never holds a token and tools act only as the authenticated user against Lark.

Data Protection

Automated PII Detection Pipeline with Amazon Macie
Learn to build an event-driven pipeline that automatically detects PII in Amazon S3 objects using Amazon Macie, AWS Step Functions, and custom data identifiers, with CSV/JSON reporting and SNSalerting for high-severity findings.

Data-Residency Chatbot with Amazon Bedrock AgentCore
Learn to build a data-residency-compliant natural-language chatbot on Amazon Bedrock AgentCore that keeps all data and AI inference within a single AWS Region, using governed text-to-SQL with whitelist-validated queries, Aurora PostgreSQL in private subnets, and AgentCore Gateway for secure tool access.

Governance and compliance

Video Compliance Agent
Learn to build an end-to-end automated video compliance verification pipeline using Amazon Bedrock, ECS Fargate, and AWS Step Functions that processes videos shot-by-shot, extracting frames, audio transcripts, and OCR text, then flags potential broadcast guideline violations with structured per-shot reports.

Contract Compliance Search with Amazon OpenSearch
Learn to build a contract compliance search system that combines semantic search with semantic highlighting using Amazon OpenSearchService, Amazon Titan V2 embeddings, and a SageMaker-hosted highlighting model to surface relevant clauses across contract documents.

Threat detection and incident response

Multicloud Security Posture Assessment
Learn to deploy a centralized security assessment solution that scans AWS, Azure, Google Cloud Platform, and Oracle Cloud Infrastructure environments from a single AWS deployment using Prowler, with AWS CloudFormation templates for each provider and unified reporting in HTML, CSV, and JSON-OCSF formats.

Centralize AWS Security Agent Findings
Learn to deploy a AWS CloudFormation stack that automatically exports AWS Security Agentpenetration test findings to Amazon S3 and queries them centrally with Amazon Athena, using Amazon EventBridge, Step Functions, and a AWS Glue catalog for tracking findings over time.

Sentinel Harness — Production SecOps Agents as Configuration
Learn to build production security-operations agents as pure configuration on Amazon Bedrock AgentCore Harness, declaring model, prompt, tools, skills, memory, and limits in YAML while AWS runs the agent loop with human-in-the-loop gates, detection-engineering tools, adversary emulation, and a self-improvement closed loop.

AWS Labs

This month brings 1 new AWS Labs repository focused on data protection, helping organizations build automated PII detection and redaction pipelines with AI-powered processing across documents and audio files.

Data Protection

PII Anonymizer
Learn to build an automated PII detection and redaction pipeline using AWS Step Functions, Amazon Bedrock, Amazon Textract, and Amazon Transcribe; supporting PDFs, Word, Excel, images, and audio files with synthetic replacement or blackout modes, concurrency control, and customer-managed KMS encryption.

Conclusion

July 2026 provides guidance and examples for securing AI agent architectures at scale, from governed text-to-SQL with data residency controls and agent management platforms to DevSecOps baselines for AI coding tools. The posts and samples provide patterns for least-privilege authorization in multi-agent chains using Cedar, post-quantum migration planning, container-aware network firewall rules, and multicloud security posture management. Each resource includes deployment steps or runnable code so you can validate in your own environment before adopting. Subscribe to the AWS Security Blog RSS feed to receive updates as they publish, and revisit this digest monthly for a consolidated view of what changed and what to act on.

If you have feedback about this post, submit comments in the Comments section below.


Rodolfo Brenes

Rodolfo Brenes

Rodolfo is a Principal Solutions Architect focused on Cloud Governance and Compliance. With over 18 years of experience, he currently leads a technical field community in AWS helping customers scale and improve their security and governance frameworks. Besides work, Rodolfo enjoys video games, playing with his four cats, and won’t say no to a good outdoor adventure.

Anna Brinkmann

Anna has 18 years of experience in the technical content space and has spent the last 6 years managing the AWS Security Blog. Outside of work, she enjoys spending time with her family.

Announcing in-place ZooKeeper-to-KRaft cluster upgrades for Amazon MSK

Post Syndicated from Austin Groeneveld original https://aws.amazon.com/blogs/big-data/announcing-in-place-zookeeper-to-kraft-cluster-upgrades-for-amazon-msk/

Apache Kafka 4.0 officially removes ZooKeeper. If your Amazon Managed Streaming for Apache Kafka (Amazon MSK) Provisioned clusters still run in ZooKeeper metadata mode, now is the time to plan your migration. Amazon MSK now supports in-place upgrades from ZooKeeper to KRaft metadata mode, so you can modernize your existing cluster’s metadata management through the familiar version upgrade workflow.

For more than a decade, Apache ZooKeeper provided dependable metadata management for Kafka, including controller election, partition state, broker registration, and topic configuration. With Apache Kafka 4.0, ZooKeeper is officially removed in favor of KRaft, an embedded Raft-based consensus protocol that handles metadata management internally. It brings those responsibilities into Apache Kafka itself, creating a more streamlined foundation for the continued evolution of Kafka. Amazon MSK has supported KRaft-mode clusters since May 2024, and all Kafka 4.x versions on Amazon MSK use KRaft.

With the in-place upgrade, you can retain your cluster data and metadata while Amazon MSK manages the control-plane transition. Your cluster remains available for produce and consume traffic throughout the process, with no expected downtime if you’re following best practices. By using the existing version upgrade workflow, the move to KRaft becomes a natural step in your cluster’s lifecycle. This prepares your cluster for Kafka 4.x and future Kafka releases.

Prerequisites

Before initiating the upgrade, review the following requirements to confirm your cluster is ready for the transition.

Supported source versions

Clusters must be running Kafka 3.9.x in ZooKeeper mode to use the in-place upgrade. If your cluster is running an earlier version, such as 3.6.0, 3.7.x, or 3.8.x, first complete a standard in-place version upgrade to 3.9.x. You can then initiate the upgrade to 3.9.x.kraft.

Kafka 3.9 is the bridge release for this transition because it supports both ZooKeeper and KRaft modes. To support customers through this migration process, Amazon MSK provides extended support for 3.9.x for a minimum of 2 years from its April 2025 release.

Client compatibility

Requirement Detail
Minimum client library Apache Kafka client v3.0+
Recommended client version v3.9 or above
Connection strings Must use bootstrap.servers only. Any ZooKeeper connection strings (the --zookeeper flag) must be removed before upgrade.

The --zookeeper admin flag was deprecated in Kafka 2.5 and removed in 3.0. Before upgrading, update any remaining applications or tools that connect directly to ZooKeeper.

Pre-upgrade checklist

Before beginning the upgrade, confirm the following:

  • For Standard brokers, the cluster must be deployed across three Availability Zones. Express brokers provide this by default.
  • The cluster is running Kafka 3.9.x in ZooKeeper mode.
  • Standard brokers expose direct ZooKeeper access on ports 2181 (plaintext) and 2182 (TLS). Before upgrading, validate that you’ve disabled ZooKeeper access on the cluster and none of your applications rely on these connections.
  • Solutions using dynamic Kafka configurations that relied on ZooKeeper have been removed before attempting the upgrade operation.
    • If you previously configured custom domain names on a ZooKeeper-based deployment using the dynamic override (kafka-configs.sh --alter on advertised.listeners), be aware that KRaft does not support this dynamic configuration. If you attempt to upgrade your MSK cluster to KRaft with altered advertised.listeners, the upgrade operation fails.
    • If you’re implementing your custom domain name solution on MSK moving forward with KRaft, we recommend our coinciding MSK release for custom domain name support by statically configuring the custom.advertised.listeners property through the UpdateClusterConfiguration API.
  • The cluster has no under-replicated partitions.
  • The cluster is running within per-broker partition limits for standard or express broker clusters.
  • For clusters running above the KRaft brokers-per-cluster limit, you might need an additional quota increase. If you previously raised a quota increase for your ZooKeeper brokers-per-cluster, submit another quota increase for the KRaft limit before attempting the upgrade.
  • The cluster has enough reserve capacity to support rolling broker restarts while serving client traffic.
  • As a best practice, verify that monitoring is ready for the transition from ZooKeeper-specific metrics to KRaft controller metrics.
    • After the migration, ZooKeeper-specific Amazon CloudWatch metrics such as ZookeeperRequestLatencyMsMean and ZookeeperSessionState are no longer available.
    • If you use Open Monitoring, Kafka also stops publishing ZooKeeper metrics. Plan to update or retire related alerts and dashboards as part of your migration preparation.

How the upgrade works

When you initiate the upgrade, Amazon MSK performs a managed, multi-phase migration:

  1. Controller quorum bootstrap: Amazon MSK provisions KRaft controller nodes alongside the existing ZooKeeper infrastructure. Both systems operate in parallel during this phase.
  2. Metadata migration: The KRaft controller reads the cluster state from ZooKeeper and writes it to the internal KRaft metadata log.
  3. Broker transition: Amazon MSK performs a rolling update and registers with the KRaft controller quorum. Data plane operations remain available during the transition.
  4. Validation and bake period: Amazon MSK verifies cluster health under KRaft, including partition leadership, replication state, and controller responsiveness.
  5. ZooKeeper decommissioning: After validation succeeds, Amazon MSK removes the ZooKeeper infrastructure and the cluster operates entirely in KRaft mode.

During the upgrade, the cluster enters UPDATING state. You can continue producing and consuming data, while Amazon MSK administrative API operations are temporarily unavailable until the cluster returns to ACTIVE.

Amazon MSK maintains a high bar for durability during the transition. It uses rigorous safety checks at each phase of the migration to protect customer metadata in both roll-forward and rollback scenarios.

Built-in recovery

Amazon MSK monitors cluster health throughout the upgrade. If it detects a condition that prevents the migration from completing, it automatically returns the cluster to its pre-migration state. No customer action is required during recovery.

The operation status changes to Reverting to pre-migration state while Amazon MSK restores the original Kafka version and reconnects ZooKeeper. After the cluster returns to ACTIVE, the describe-cluster-operation API provides error codes, failure reasons, and recommended remediation steps. You can use these to address the issue before starting the upgrade again.

How to perform the upgrade

The following steps walk you through the upgrade process using the Amazon MSK console. You can also perform these steps programmatically using the AWS Command Line Interface (AWS CLI) or SDK.

Step 1: Disable ZooKeeper access (standard brokers only)

Note: This step applies only to Standard broker clusters. Express broker clusters don’t expose direct ZooKeeper access and can skip directly to Step 2.

Standard brokers expose direct ZooKeeper access on ports 2181 (plaintext) and 2182 (TLS). Before upgrading, validate that none of your applications rely on these connections.

Navigate to your cluster’s Properties tab, choose Network settings, and then choose Edit ZooKeeper access.

Figure 1: Editing ZooKeeper access from the cluster network settings

Figure 1: Editing ZooKeeper access from the cluster network settings

In the pop-up window, verify that ZooKeeper access is set to Disabled, and then choose Save.

Edit ZooKeeper access dialog with access set to Disabled and the Save button

Figure 2: Confirming ZooKeeper access is disabled

Confirm that producers, consumers, and admin tooling continue operating normally without ZooKeeper connectivity. This step is fully reversible. Re-enable ZooKeeper access immediately if anything breaks.

Figure 3: Verifying client traffic continues without ZooKeeper access

Step 2: Initiate the version upgrade

In the Amazon MSK console, under Properties, choose Upgrade in the Apache Kafka version section.

Figure 4: Starting a version upgrade from the Apache Kafka version section

Select your cluster and start a version upgrade to 3.9.x with Target metadata mode set to KRaft. Choose Upgrade.

Figure 5: Selecting KRaft as the target metadata mode

You can monitor your upgrade progress on the cluster properties page.

Figure 6: Monitoring upgrade progress on the cluster properties page

Step 3: Monitor upgrade progress

Track progress on the Cluster operations tab in the Amazon MSK console or with the describe-cluster-operation API.

Figure 7: Tracking the upgrade on the Cluster operations tab

Step 4: Validate the KRaft cluster

After the cluster returns to ACTIVE state in KRaft mode:

  • Verify that topics, partitions, and consumer groups are present.
  • Confirm producer and consumer throughput aligns with pre-migration baselines.
  • Update or disable any ZooKeeper-specific monitoring alerts.
  • Update operational documentation and runbooks to reflect KRaft mode.

Figure 8: Cluster running in KRaft mode after the upgrade

After the upgrade completes, your cluster appears in an Active state with KRaft enabled as the metadata mode.

Get ready for the next generation of Kafka on Amazon MSK

The in-place ZooKeeper-to-KRaft mode upgrade makes it straightforward to prepare existing Amazon MSK clusters for the future of Apache Kafka. Beyond removing external metadata dependencies, KRaft delivers faster failover times and higher partition limits per cluster. Amazon MSK handles the entire metadata transition, rolling broker updates, validation, and recovery workflow for you. With the new in-place experience, you have a clear, streamlined path to upgrade on your schedule and unlock enhanced scalability and resilience.

For more details, see the Amazon MSK Developer Guide and the supported Kafka versions documentation.


About the authors

Austin Groeneveld

Austin Groeneveld

Austin is a Streaming Specialist Solutions Architect at Amazon Web Services (AWS), based in the San Francisco Bay Area. In this role, Austin is passionate about helping customers accelerate insights from their data using the AWS platform. He is particularly fascinated by the growing role that data streaming plays in driving innovation in the data analytics space. Outside of his work at AWS, Austin enjoys watching and playing soccer, traveling, and spending quality time with his family.

Ashley Millette

Ashley Millette

Ashley is a Specialist Solutions Architect for Streaming and Analytics at AWS. She partners with customers to design and implement real-time data streaming architectures using services like Amazon MSK, helping them build scalable, cost-effective pipelines that turn data in motion into actionable insights. She is passionate about simplifying complex streaming workloads and enabling customers to modernize their data infrastructure with confidence.

Amazon MSK Service 101: How many partitions does an Amazon MSK topic need?

Post Syndicated from Yashika Jain original https://aws.amazon.com/blogs/big-data/amazon-msk-service-101-how-many-partitions-does-an-amazon-msk-topic-need/

Customers new to Amazon Managed Streaming for Apache Kafka (Amazon MSK) often ask how many partitions their topics need. Choosing the right partition count is one of the most impactful architectural decisions you make, because it directly affects throughput, scalability, and operational complexity.

In Apache Kafka, a topic is the fundamental unit for categorizing data streams, but to achieve high scalability and performance, Kafka divides topics into smaller, independent units called partitions.

In this post, we provide practical guidance for determining the ideal partition count for your use case.

Understanding Kafka partitions

In
Apache Kafka, a partition is the unit of storage and parallelism. Each partition is an ordered, immutable log that can store records as they are produced to a topic. When you create a topic, Kafka distributes its partitions across the brokers in the cluster. Partitions allow Kafka to scale in three key ways:
  • Parallelism – Within a consumer group, each partition can be read by only one consumer at a time. Each partition maps to a dedicated log file in storage on the broker, and Kafka manages these logs through separate processing threads. This architecture allows more partitions to support more consumers processing data in parallel, with each partition’s log being independently managed for read and write operations.
The following diagram shows how Kafka distributes partition replicas across a three-broker cluster, with each broker serving as a leader for some partitions and a follower for others.
Partitions 0, 1, and 2 replicated across three brokers, each a leader for some partitions and a follower for others

Figure 1: Partition replicas distributed across a three-broker cluster

The following diagram illustrates how producers append new records to the end of a partition log, while consumers read sequentially from their current offset position.

Producers append records to the tail of partition logs while consumers read sequentially from their offset position

Figure 2: Producer writes and consumer offset positions in two partition logs

  • Throughput – Producers and consumers can read and write data in parallel across partitions, increasing overall throughput.
  • Scalability – Partitions allow Kafka to spread data and load across multiple brokers instead of concentrating it on a single node.

However, increasing partitions comes with trade-offs. Each partition adds metadata overhead, consumes memory, and requires file handles on the broker. While more partitions improve throughput and parallelism, they also increase the operational burden on the cluster. Too many partitions can lead to longer leader election times during broker failures, increased end-to-end latency, and higher memory consumption for both producers and consumers managing connections to multiple partitions.

Trade-offs when choosing partition count

Choosing a partition count is a balancing act between parallelism and resource utilization.

Benefits of more partitions

Using more partitions can significantly improve throughput by allowing Kafka to distribute read and write traffic across more brokers. This is particularly useful for high-volume ingestion pipelines and real-time analytics workloads. More partitions also allow consumer groups to scale horizontally, because the maximum number of active consumers in a group is limited by the number of partitions. In addition, choosing a partition count that is evenly divisible by the number of brokers helps provide balanced leadership and replica distribution, reducing the risk of uneven load.

Operational costs of more partitions

However, higher partition counts also come with costs. When a broker fails or undergoes maintenance, Kafka must perform recovery operations for each affected partition. During recovery, Kafka elects new leaders for partitions that were hosted on the unavailable broker and replicates data from the remaining in-sync replicas to newly assigned brokers. This process involves copying partition data across the network to restore the replication factor, which can be resource intensive. As the number of partitions increases, these recovery operations take longer because each partition requires its own leader election and data replication cycle.

You might encounter clusters with very high partition counts that experience extended recovery times during rolling upgrades, even when overall traffic volumes are modest. Amazon MSK Express brokers address this challenge by recovering 90x faster and providing 180x faster elasticity when scaling out clusters. This significantly reduces the operational impact of high partition counts during maintenance windows and failure scenarios.

Infrastructure cost implications

Beyond operational complexity, more partitions can directly increase infrastructure costs. Amazon MSK publishes partition-per-broker limits that vary by instance type. When the total partition count (including replicas) exceeds what the current broker fleet can support, you must add brokers to stay within recommended limits, even if throughput alone does not warrant the additional capacity.

Amazon MSK partition-per-broker guidelines

Amazon MSK publishes recommended partition-per-broker guidelines to help you operate clusters reliably. These values are strict limits. Exceeding them can lead to operational challenges, particularly during broker replacement or rolling upgrades, and can block cluster operations such as configuration updates or scaling down.

Express brokers support up to 5x more partitions per broker compared to Standard brokers. For example, the largest Standard broker (kafka.m7g.16xlarge) supports a recommended maximum of 4,000 partitions per broker. The equivalent Express broker (express.m7g.16xlarge) supports up to 20,000 recommended partitions per broker. This higher partition density means partition-bound workloads can be hosted on fewer brokers, improving price-performance by up to 50% for such workloads.

We recommend setting Amazon CloudWatch alarms on PartitionCount per-broker metrics to proactively monitor your partition distribution. When an alarm triggers, evaluate your partition strategy and consider rebalancing partitions across brokers, consolidating topics, or scaling out your cluster to stay within recommended limits. For detailed guidance, see Right-size your cluster: Number of partitions per Standard broker and Express broker partition quota.

Practical guidance for choosing a partition count

There is no single formula that works for every Kafka workload. In practice, you typically combine several considerations when sizing partitions.

  • Start with throughput requirements – The first step is to determine your per-partition throughput capacity, which then informs how many partitions you need.

For Express brokers, use the per-broker throughput capacity as the primary means for sizing your cluster. Express brokers feature a fully managed storage layer, so you do not need to separately account for storage I/O constraints. The published per-broker limits represent the effective capacity available to your workload.

For Standard brokers, the achievable throughput depends on additional factors beyond the broker instance size. These factors include provisioned EBS storage throughput, the number of consumer groups reading from the broker, and how much data is served from memory versus disk. Storage I/O is consumed when producers write, when data replicates between brokers, and when consumers read data that is not in memory. For this reason, validate the effective per-partition throughput for Standard brokers through load testing in your environment.

Once you know your per-partition throughput, calculate the required number of partitions: Number of partitions = Peak throughput of the topic ÷ Throughput per partition

For example, if a topic must handle 40 MB/sec at peak and your testing shows each partition can sustain 5 MB/sec, you would need: 40 ÷ 5 = 8 partitions. Always validate these assumptions with load testing, as actual throughput varies based on your workload characteristics. For initial sizing estimates, refer to the Amazon MSK Sizing and Pricing worksheet and the Amazon MSK Best Practices documentation.

  • Consider your consumer parallelism needs – If you know the number of consumers required during peak processing times, use that as your partition count. We don’t recommend having more active consumers in a consumer group than partitions. For example, if you have 5 partitions, only 5 consumers can actively process data. Additional consumers remain idle. These idle consumers still maintain active TCP connections to the brokers, sending frequent heartbeats and group coordination requests. This might result in unnecessary overhead on broker resources and contribute to high CPU usage despite low egress traffic.
Consumer group with more consumers than partitions, leaving the extra consumers idle

Figure 3: Idle consumers when a consumer group has more consumers than partitions

  • Producer throughput and partition keys – When sizing partitions, consider producer-side throughput in addition to consumer parallelism. If producers generate data faster than a single partition can handle, additional partitions can help distribute write traffic across brokers. Partition keys also play a critical role. Poorly distributed or low-cardinality keys can create hot partitions and limit throughput. In such cases, increasing the number of partitions alone does not improve throughput unless records are evenly distributed.
  • Plan for even distribution and future growth – Kafka works best when partitions can be spread evenly across brokers. Instead of focusing on specific numbers, aim for partition counts that divide reasonably well across your expected broker count. This reduces reassignment churn when brokers are added or replaced. But avoid excessive over-partitioning. It’s reasonable to leave some headroom for future growth. However, creating thousands of partitions “just in case” often causes more harm than good. Increasing partitions later is supported, but it can affect ordering guarantees and may require consumer changes. Start with a conservative number, monitor real traffic patterns, and scale gradually.

From an operational perspective, Amazon MSK provides recommended partition-per-broker guidelines based on broker instance type. Exceeding these guidelines increases operational risk and can block cluster operations such as version upgrades, scaling, or configuration changes. Large partition counts can also increase consumer group rebalance duration, temporarily pausing message processing and increasing end-to-end latency.

Keep in mind that partitioning improves scalability, but it does not address application-level bottlenecks such as slow consumers, inefficient processing logic, or downstream system constraints.

Conclusion

Determining the right number of partitions for an Amazon MSK topic is a foundational design decision. It affects throughput, scalability, failure recovery, and day-to-day operability of your Kafka cluster. Start by understanding your throughput and consumer parallelism needs, respect Amazon MSK partition-per-broker guidelines, avoid excessive over-partitioning, and validate assumptions through load testing. Most importantly, there is no universal “correct” number, only a number that fits your workload, operational goals, and cost.

For more information, see the Amazon MSK Developer Guide and Recommended best practices for Amazon MSK.


About the authors

Yashika Jain

Yashika Jain

Yashika is a Senior Cloud Analytics Engineer at AWS, specializing in real-time analytics and event-driven architectures. She is committed to helping customers by providing deep technical guidance, driving best practices across real-time data platforms and solving complex issues related to their streaming data architectures.

Ali Alemi

Ali Alemi

Ali is a Principal Streaming Solutions Architect at AWS. Ali advises AWS customers with architectural best practices and helps them design real-time analytics data systems which are reliable, secure, efficient, and cost-effective. Prior to joining AWS, Ali supported several public sector customers and AWS consulting partners in their application modernization journey and migration to the Cloud.

Fast Track ISM-ready cloud environments and IRAP Assessments with Landing Zone Accelerator on AWS

Post Syndicated from Kevin Donohue original https://aws.amazon.com/blogs/security/fast-track-ism-ready-cloud-environments-and-irap-assessments-with-landing-zone-accelerator-on-aws/

This post announces the availability of a new independent assessment report available on AWS Artifact analyzing how Landing Zone Accelerator on AWS (LZA) can automatically deploy multi-account environments in Amazon Web Services (AWS) with Australian Government Information Security Manual (ISM) security controls coverage at scale. The report includes findings from an independent third-party analysis conducted by AWS Partner gwi.digital. In addition to the report, we talk about ISM compliance applicability to LZA and a new testing mechanism for measuring configuration drift, which together can provide Australian customers with a documented and validated foundation to accelerate IRAP assessment readiness.

Background

Australian organizations in public sector, defense, and critical infrastructure agencies must build cloud environments that meet (ISM requirements. The ISM defines 1,081 security control requirements across 22 guideline chapters. Demonstrating compliance is central to achieving an IRAP assessment outcome, but assessments typically require months of preparation, evidence gathering, and testing.

In October 2025, we introduced the LZA Universal Configuration and LZA Compliance Workbook. LZA provisions a multi-account security architecture that automates the deployment of nearly 200 security controls based on AWS Well-Architected pillars and AWS security best practices. The LZA Compliance Workbook, available on AWS Artifact, documents how the Universal Configuration (UC) maps to technical security requirements from 17 global compliance frameworks, with more being added. LZA is an ideal solution for customers with security and compliance obligations—both existing and anticipated—because the guardrails it deploys are applied automatically to new accounts as environments grow.

What’s in the report

To see how LZA can help customers in Australia we teamed up with AWS Partner gwi.digital to run LZA as a customer would. gwi.digital is a consultancy partner specializing in cybersecurity and governance, risk, and compliance (GRC) and has deep experience in IRAP assessments and the ISM framework. The team conducted an independent analysis and evaluation of LZA UC against 1,081 ISM controls. The assessment was conducted in a greenfield AWS environment hosted in the ap-southeast-2 (Sydney) AWS Region, positioned within Phase 2 of the ASD Cloud Security Assessment and Authorization Framework and focused on cloud consumers building on already-authorized AWS services. It builds on existing AWS IRAP foundations: Underlying AWS services were most recently independently assessed at the PROTECTED level by CyberCX last year. While this report does not constitute an official IRAP authorization, certification, or accreditation, it provides a professional evaluation of evidence based on what LZA delivers out of the box (and what it does not), so organizations can make informed decisions.

What the assessment found

Of the 1,081 ISM controls, 256 are within the addressable scope of LZA and include the technical infrastructure controls that a solution like LZA can meaningfully address. Of those 256, LZA achieves Full or Partial coverage for 234 (91%). The remaining 825 controls are outside the scope of LZA: physical security, personnel, organizational governance, and classification-level exclusions. A key contribution for the assessment was analysis of the shared responsibility model that goes beyond the traditional AWS/customer binary. It considers a three-tier view—AWS (provided), LZA (enabled), and Customer (responsibility)—and further categorizes the 825 out-of-scope controls into subcategories so customers can quickly determine which controls require their attention compared to which are already addressed at the infrastructure level. By combining automated deployment, ISM-specific compliance mappings, independent validation, and continuous evidence generation, Australian customers can reduce IRAP assessment timelines while achieving more reliable compliance outcomes.

Note: Results are based on the configuration as-provisioned based on the LZA version and during the time of this assessment. Results may vary depending on customer implementation and configuration choices.

Continuous validation with CATS

Confirming controls are implemented and operating effectively across your environment is complex and often lacks complete coverage. To simplify testing your LZA deployment, we developed the Controls Acceptance Testing Suite (CATS). CATS is an automated compliance validation engine that runs purpose-built tests against the security configuration baseline deployed by LZA UC. For the ISM assessment, CATS executed over 3,600 individual tests across six AWS accounts, evaluating account structure, identity and access, network configuration, logging, encryption, and backup. For Australian customers, CATS enables:

  • Automated evidence generation: Machine-readable results may replace weeks of manual evidence collection
  • ISM-enriched reporting: gwi-digital developed a conversion script that maps CATS output to ISM control references, enabling auditors to interpret results from an ISM perspective
  • OSCAL export: Results in Open Security Controls Assessment Language (OSCAL) format for interoperability with assessment tools
  • Continuous drift detection: Repeatable execution identifies when configurations deviate from the validated baseline between assessment cycles

Note: CATS availability is limited and currently accessible only through AWS Professional Services as a private beta solution and may be subject to change. Availability, features, and pricing are subject to change without notice. Contact your AWS account representative or reach out to a member of the LZA team for questions about CATS, or to provide feedback on the LZA ISM report.

ISM-optimized configuration

Through the assessment, gwi-digital identified configuration adjustments that elevate specific control ratings from Partial to Full, with minimal effort; for example, increasing the default password length from 14 to 15 characters or adjusting log retention to align with Australian Federal Disposal Authority standards. These recommendations have been communicated to the LZA UC team for inclusion in a future ISM-specific guidance section in LZA GitHub documentation.

Getting started

In addition to the LZA ISM report, you can also find the LZA Compliance Workbook available on AWS Artifact. It maps related ISM requirement identifiers to security implementation statements, giving you a starting point from which you can customize and enhance your compliance documentation for your unique use cases after deploying LZA.

  1. Sign in to your AWS account and then download the LZA Australia ISM Third-Party Analysis and Evaluation Report and LZA Compliance Workbook from AWS Artifact.
    Figure 1: LZA report and workbook in AWS Artifact

    Figure 1: LZA report and workbook in AWS Artifact

  2. Visit the LZA Universal Configuration GitHub repository to review and download the latest configuration baseline.
  3. Use the LZA Implementation Guide to see use cases, review pre-deployment considerations and then follow deployment steps.
  4. Set-up a security compliance chat agent. Consider creating a knowledge base with Amazon Bedrock using the LZA Compliance Workbook, LZA Implementation Guide, web crawlers to the LZA GitHub, and your custom resources to set up your own chat agent.

If you have questions, contact a gwi.digitalexpert, AWS LZA team member, or your AWS account representative.

Conclusion

The combination of ISM mappings in the LZA Compliance Workbook, the gwi-digital analysis report, and CATS evidence gives Australian customers a head start on IRAP assessments.

  • Weeks saved on scoping: Pre-determined control applicability significantly reduces upfront scoping effort
  • Documentation ready: Implementation statements and control requirement alignment reduce the documentation burden
  • Independent assurance: A third-party report that customers and their assessors can reference directly
  • Continuous evidence: CATS generates repeatable security evidence between assessment cycles, replacing periodic manual audits with ongoing assurance

Acknowledgements

The Landing Zone Accelerator team would like to thank Baden Hughes, Henrik Melkonyan, Iain Lindsay-German, and Ian Roderick from gwi.digital for their professionalism and expertise in performing an incredibly thorough assessment, helping us to make LZA better for customers, and for their contributions to this blog post.

If you have feedback about this post, submit comments in the Comments section below.


Kevin Donohue

Kevin Donohue

Kevin is a Senior Security Compliance Engineer at AWS, where he builds solutions and resources to help AWS customers achieve their security and compliance goals. Prior to joining the Landing Zone Accelerator team in AWS Professional Services in 2024, Kevin began his tenure with AWS Security in 2019 specializing in FedRAMP compliance and the shared responsibility model.

Dan Friebe

Dan Friebe

Dan is a Senior Solutions Architect at AWS, helping customers design and build secure cloud solutions. With over a decade of AWS experience and 16+ years supporting public sector missions, he has spent 8 years delivering solutions for customers and enabling partners to deliver at scale. He’s passionate about removing friction between compliance and innovation, helping customers move faster without compromising security.

Dave Connell

Dave Connell

Dave Connell is a Senior Systems Development Engineer within AWS Professional Services. He has spent the last 7 years building software to secure and scale AWS usage, enabling builders to delight customers and deliver business value through excellent software in the cloud. Dave is passionate about enabling delivery of predictable, safe and valuable customer outcomes by empowering the builder.

Security Hub Extended adds Supply Chain Security as its tenth category

Post Syndicated from Michael Fuller original https://aws.amazon.com/blogs/security/security-hub-extended-adds-supply-chain-security-as-its-tenth-category/

Since February, we’ve grown AWS Security Hub Extended from 14 curated partners across 9 categories to 23 partners across 10. At Black Hat this month, 14 of those partners were at the Amazon Web Services (AWS) booth demoing live. Four of those partners delivered theater talks and ten were featured on SecurityLive streaming. We hosted a partner reception that brought our leadership together with partner executives to plan what comes next. These are companies investing real engineering and real go-to-market (GTM) alongside us, and increasingly with each other, because the model resonates with the customers they’re talking to every day. The most common question we heard at the booth was when Supply Chain Security was coming.

It’s here. And that’s the thing I want to spend the most time on today, because it’s the category customers keep asking us about.

Supply Chain Security: The category customers have been asking for

Software supply chain risk has moved from a security-team concern to a board-level conversation. SolarWinds showed what happens when a build system is compromised. Log4j showed what a single transitive dependency vulnerability can do at global scale. The xz utils backdoor showed the patience of a maintainer-compromise attack executed over years. Each demonstrated a different dimension of the same problem, and the pace is accelerating. Attackers know that a fast way into an enterprise is through the open source packages that enterprise unknowingly trust.

Every customer I talked to at Black Hat had this on their risk register. Most still hadn’t operationalized a solution, because doing so meant a standalone deployment, a new contract, a new console, and integration work their security team couldn’t prioritize. That’s the friction we aim to remove.

Security Hub Extended now offers Supply Chain Security with Chainguard and Socket as the curated partners. Supply Chain Security uses the same model as everything else in Extended. Every offering has pay-as-you-go pricing, one bill, no required long-term commitment. For enterprises that prefer to continue using the procurement process they always have, Security Hub Extended Private Offers are also available. These are committed term agreements with deeper discounts, the ability to aggregate spend across partners on a single AWS bill, and both monthly and annual payment options throughout the term. You pick the path that fits how you buy.

What Chainguard does

Chainguard gives you open source dependencies rebuilt from source in a hardened, verified build process, so what enters your environment is malware-resistant and provenance-backed. Their research shows that rebuilding from source would have stopped 98% of known malicious packages from ever reaching production. If you can’t verify the source, it never appears in the Chainguard repository. That’s the filter between the public registry and your developers.

What Socket does

Socket analyzes the actual behavior of open source packages to block malicious dependencies at the time of install. Not after a Common Vulnerability and Exposures (CVE) is published days or weeks later. At the moment the package tries to land in your environment, Socket flags it based on what it does, not what a database says about it. Its reachability analysis then tells you which vulnerabilities are exploitable from your code instead of drowning your team in noise. You pay for the distinct packages you check, not for how often your builds run.

Why they work together

Together, Chainguard and Socket cover the two questions that matter:

  • Can I trust what I’m pulling in?
  • Can I stop malicious components before they get built into my applications?

Chainguard helps secure the foundation your code is built on. Socket secures the packages you pull into it. Both help protect your software supply chain regardless of where you deploy—across clouds or on-premises. Activate both through Security Hub Extended and their findings flow into Security Hub in OCSF (Open Cybersecurity Schema Framework) alongside everything else, so a supply chain risk is correlated and prioritized next to your endpoint, identity, and cloud signals. From there, it routes out to the downstream tools you’ve already integrated, so it fits the pipeline your builders run today.

23 partners, 10 categories. Built on what customers asked for

Every partner in Security Hub Extended is here because customers told us they needed that capability and that specific solution was already working for them. We add categories because the threat landscape evolves, and we add partners because customers point us to who’s solving those problems well. The goal is straightforward: Simplify adopting the security solutions your peers are already succeeding with, through the AWS relationship you already have.

The full set today spans endpoint, identity, email, network, data, browser, cloud, AI, security operations, and now supply chain. The 23 curated partners are 7AI, Britive, Chainguard, CrowdStrike, Cyera, Island, LayerX, Native Security, Noma, Okta, Oligo, Opti, Palo Alto Networks, Proofpoint, SailPoint, SentinelOne, Socket, Splunk, Sublime, Upwind, Varonis, Zenity, and Zscaler.

Our focus now is deepening integrations and reducing activation friction so these solutions work together, not in isolation. That’s where the real value compounds.

What we’re building next

Everything I’ve described so far is the commercial model working: Customers buying best-of-breed security through one AWS relationship with the flexibility they expect. But the bigger vision is the integration layer that makes these tools genuinely better together, not just easier to buy together.

The integration we’re most focused on is cross-partner correlation, turning signals from an endpoint solution, an identity solution, and a cloud solution into one exposure and one attack path instead of three disconnected alerts. Right alongside that, we’re dramatically reducing the activation, deployment, and integration friction so customers go from subscribing to seeing value in hours rather than weeks. Both efforts enable the curated solutions you already trust to deliver stronger outcomes together than they do apart.

That’s the build we’re accelerating with our partners now, and you’ll hear more leading into re:Invent.

Explore what’s available

If you’re running open source in production and don’t yet have supply chain visibility, start there. Activate Chainguard and Socket through the Security Hub console today. If you’re managing multiple security vendor relationships and want to understand what consolidation looks like with Security Hub Extended, talk to your AWS account team. Pricing for every partner is published on our pricing page, no sales call required. And if you’re already using Security Hub for posture management and threat detection, the Extended plan is available in the same console you already use.

We’re just getting started.

If you have feedback about this post, submit comments in the Comments section below.


Michael Fuller

Michael has been with AWS for 16 years and led product for AWS Security Services for 11 years. Michael has 29 years in the industry and held several roles in product management, business development, and software development for IBM, Cisco, and Amazon. Michael has a Bachelor’s of Science in Computer Engineering from the University of Arizona and an MBA from the University of Washington.

Introducing public preview runtimes on AWS Lambda, starting with Node.js 26 and Python 3.15

Post Syndicated from Jonathan Tuliani original https://aws.amazon.com/blogs/compute/introducing-public-preview-runtimes-on-aws-lambda-starting-with-node-js-26-and-python-3-15/

Today, AWS Lambda introduces public preview runtimes, a new way to try upcoming language versions on Lambda before their general availability (GA) release. Starting today, you can create and update Lambda functions using Node.js 26 and Python 3.15, the first runtimes available as public previews.

Previously, Lambda has always launched new runtimes as Generally Available (GA), giving you a production-ready experience from day one. But this means you couldn’t run your functions on Lambda using a pre-release language version, and we couldn’t hear your feedback while breaking changes were still possible. Public preview runtimes change that. By putting pre-GA runtimes in your hands months earlier, we can listen to your feedback and address it before GA, while we still have the opportunity to make breaking changes to improve the runtime.

Preview runtimes are available in all AWS commercial Regions, AWS GovCloud (US) Regions, and China Regions. They use the same runtime identifier as the eventual GA runtime, so your functions graduate automatically when the runtime reaches GA, with no action required.

Why public preview runtimes

When Lambda launches a new runtime as GA, that means it is ready for use in production workloads from day one. Historically, the Lambda team has validated new runtimes through internal testing and pre-release benchmarking. However, without real customer workloads running on the runtime, some issues only surface after the GA launch. And once the runtime is GA, the scope to address those issues is much reduced since we cannot risk breaking existing production workloads.

Public preview runtimes address this by opening up a pre-GA feedback window. During this period, you can deploy functions using the upcoming runtime, and the Lambda team can act on what you find, including making potentially breaking changes if necessary. In addition, because the upstream language is still in its pre-release phase, there’s also the opportunity that issues discovered during preview can be fixed in the runtime itself, not just worked around.

This benefits everyone involved. You get a runtime that’s been tested against a broader range of real workloads before it reaches GA. Third-party partners, including observability providers, infrastructure-as-code tools, and deployment frameworks, get time to validate compatibility. And upstream language communities get a signal from a major cloud platform while they can still act on it.

This is the first time we’re launching runtimes as public previews. As such, it’s an experiment. We hope to make public previews the default for all future runtime launches, depending on the success of this experiment and the feedback we receive.

What’s included in the preview runtimes

The Node.js 26 and Python 3.15 preview runtimes are built on the latest upstream pre-release of each language version. At launch, they are a straightforward version bump. There are no additional Lambda-specific enhancements beyond what the new language version itself provides. For details on what’s new in each language version, refer to the upstream release information:

Preview runtimes are available as both managed runtimes and base container images. The base images are published to the Lambda base image ECR repository with image tags starting with 3.15-preview (for Python) and 26-preview (for Node.js).

During the preview period, we may introduce additional features or enhancements to these runtimes. When we do, we’ll announce them on the same GitHub issue we use to collect your feedback:

Follow these issues to stay informed of any changes during the preview period.

What to expect during preview

Preview runtimes follow the same patching cadence as GA runtimes. When an update is released upstream, Lambda applies it to the preview runtime on the same schedule as any other supported runtime. All Lambda features supported by the current GA runtimes are available on the preview runtimes, including Lambda Managed Instances and durable functions.

The key difference is that the underlying language version has not yet reached its stable release. In addition, the Lambda team is still working on the runtimes to add features and optimize performance. This means breaking changes may occur during the preview period. A function that works today may require a fix after Lambda rolls out the next runtime update. This is by design: the preview period exists so that these issues can be found and resolved before GA, not after.

Because of this potential for breaking changes, preview runtimes are not covered by the AWS Lambda SLA or AWS technical support plans. We strongly recommend against using them for production workloads. Lambda emits a warning message to CloudWatch Logs on each cold start to make it clear when a function is running on a preview runtime:

WARNING: This is a preview runtime version and should not be used for production workloads. For further information and to provide feedback, see https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html.

You may notice that, at launch, preview runtimes have slower performance than GA runtimes, in particular for cold starts. This is because of a combination of lack of optimization and less caching in internal Lambda sub-systems. We will benchmark and optimize performance during the preview period, prior to GA.

Functions that use preview runtimes are billed at standard Lambda rates. There is no additional cost or separate pricing.

Share your feedback

We want to hear from you during the preview period. We’ve created a dedicated GitHub issue for each preview runtime where you can share your experience:

Comment on these issues directly, or open a separate issue in the repository if you prefer.

We’re interested in all feedback, not just bug reports. If you see an opportunity to take advantage of a new language feature in the runtime, or a way to improve the Lambda programming model for that language, we want to hear about it. The preview period is when we can still make meaningful changes, so this is the best time to share your ideas.

Note that feedback should be scoped to the runtime itself: the execution environment, language integration, and programming model. For broader Lambda feature requests, refer to the AWS Lambda public roadmap.

Transition to GA

Both Node.js 26 and Python 3.15 are expected to reach their stable upstream releases in October 2026. Lambda GA for each runtime is targeted within two months following those releases. For the latest estimated GA dates, see Lambda documentation.

For Node.js, the GA timeline is tied to the Node.js “Active LTS” release, which is scheduled for October 2026. Only at that point is the release considered suitable for production workloads by the Node.js project, and only then is it sufficiently stable for Lambda’s automatic runtime patching in which patches are applied to your functions without action on your part. Lambda will not GA the Node.js 26 runtime until it reaches Active LTS.

When a preview runtime reaches GA, your functions graduate automatically. The runtime identifier does not change: nodejs26.x in preview is the same nodejs26.x at GA. You do not need to update your function configuration, templates, or code. The preview label is removed from the console and documentation, the runtime becomes covered by the Lambda SLA and AWS Support, and the GA performance and quality bar applies from that point forward.

If you have pinned your function to a specific runtime version using Runtime Management Controls during the preview period, it remains pinned. You can unpin at any time to move to the GA runtime. Functions pinned to a pre-GA runtime version are not covered by the Lambda SLA and AWS Support.

Getting started

You can start using the Node.js 26 and Python 3.15 preview runtimes today using the Lambda console, AWS Command Line Interface (AWS CLI), AWS CloudFormation, AWS Serverless Application Model (AWS SAM), or AWS Cloud Development Kit (AWS CDK).

Console

In the Lambda console, choose “Node.js 26 (Preview)” or “Python 3.15 (Preview)” from the runtime list when creating or updating a function.

Screenshot of the Lambda console runtime list showing “Node.js 26 (Preview)” and “Python 3.15 (Preview)” options.

AWS CLI

Create a function using the preview runtime with the standard runtime identifier:

aws lambda create-function \
  --function-name my-function \
  --runtime nodejs26.x \
  --handler index.handler \
  --role arn:aws:iam::123456789012:role/my-role \
  --zip-file fileb://function.zip

For Python 3.15, use --runtime python3.15. These are the same identifiers the GA runtimes will use, there is no separate preview-specific value.

AWS CloudFormation

Specify the preview runtime in your CloudFormation template using the same runtime identifier:

Resources:
  MyFunction:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: my-function
      Runtime: nodejs26.x
      Handler: index.handler
      Role: arn:aws:iam::123456789012:role/my-role
      Code:
        S3Bucket: amzn-s3-demo-function-code
        S3Key: function.zip

AWS SAM

AWS SAM supports preview runtimes using the standard runtime identifier in your template:

Resources:
  MyFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: python3.15
      Handler: app.lambda_handler
      CodeUri: src/

When you run sam init, preview runtimes appear in the template list with a “(Preview)” label, so you can scaffold a new project directly.

AWS CDK

The AWS CDK does not yet include built-in enum members (such as Runtime.NODEJS_26_X). During the preview phase, you can use the public Runtime constructor to specify the runtime directly, for example:

import { Stack, StackProps } from "aws-cdk-lib";
import { Construct } from "constructs";
import { Function, Runtime, RuntimeFamily, Code } from "aws-cdk-lib/aws-lambda";

export class LambdaStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    new Function(this, "MyFunction", {
      runtime: new Runtime("nodejs26.x", RuntimeFamily.NODEJS),
      handler: "index.handler",
      code: Code.fromAsset("lambda"),
    });
  }
}

Or, for Python 3.15, replace new Runtime("nodejs26.x", RuntimeFamily.NODEJS) with new Runtime("python3.15", RuntimeFamily.PYTHON).

This synthesizes identical CloudFormation to what a built-in enum produces. When the runtime reaches GA, a corresponding enum member will be added. There is no functional difference in the meantime.

Conclusion

Public preview runtimes give you a seat at the table while Lambda’s next runtimes are still taking shape. Try using Node.js 26 or Python 3.15 today to deploy a function, run your test suite, and let us know what you find.

Share feedback and follow along:

These GitHub issues are where we’ll post any enhancements or breaking changes during the preview period, so they’re worth watching even if you don’t have immediate feedback. The preview runtimes are available today in all AWS Regions, including AWS GovCloud (US), and the AWS China Regions. To learn more, see the Lambda runtimes documentation.

AWS Certificate Manager will discontinue email validation to prove domain validation for certificates

Post Syndicated from Adam Aboudi original https://aws.amazon.com/blogs/security/aws-certificate-manager-will-discontinue-email-validation-to-prove-domain-validation-for-certificates/

Today, we’re announcing that AWS Certificate Manager (ACM) will discontinue support for email-validated public certificates by September 30, 2027. If you use email validation for your ACM public certificates, you need to migrate to DNS validation before that date. This change aligns with the Certification Authority/Browser (CA/B) Forum’s industry-wide deprecation of email-based domain validation and gives you a full year to migrate ahead of the Forum’s March 2028 deadline.

In this blog post, we share the rationale for this change, the timeline, and the steps you can take to migrate your certificates to DNS validation.

Background

The CA/B Forum sets the standards that browsers and certificate authorities must follow for publicly trusted certificates. In November 2025, they voted to end support for email-based domain validation effective March 15, 2028. After that date, certificates validated through email won’t be trusted by browsers, regardless of which certificate authority issued them.

ACM will be deprecating its email validation in-line with the CA/B Forum’s requirements, by September 30, 2027. The ACM timeline gives customers one year to migrate before the CA/B Forum’s hard deadline.

Timelines for these changes

If you currently use email validation for certificates requested from ACM, there are a few important dates that you should be aware of:

  • January 1, 2027: ACM will no longer offer email validation in new AWS Regions.
  • March 31, 2027: ACM will no longer offer email validation for new certificate requests in any Region.
  • September 30, 2027: ACM will no longer renew existing certificates that use email validation in any Region.
  • March 15, 2028: Per the CA/B Forum, public certificate authorities can no longer use email-based domain validation to issue or renew publicly trusted certificates. Certificates issued before this date remain valid until they expire.

Check for existing email validated certificates

If you have any ACM issued public certificates, you can check whether any of them are email validated by using the AWS Management Console for ACM or the AWS Command Line Interface (AWS CLI).

Identify email validated certificates using the ACM console

Use the following steps in the console to find email validated certificates.

  1. Open the ACM console.
  2. Select the filters Validation method = Email and Type = Amazon Issued for a list of email validated certificates.
  3. Any certificates listed are public email-validated certificates and should be migrated before September 30, 2027.
  4. Figure 1: List of all public email validated certificates

    Figure 1: List of all public email validated certificates

Identify email-validated certificates using the AWS CLI

Use the following commands to find email validated certificates.

# Discover email validated public certificates 
region="${1:-us-east-1}" 

aws acm list-certificates --region "$region" \ 
--query "CertificateSummaryList[?Type=='AMAZON_ISSUED'].CertificateArn" --output text | tr '\t' '\n' | while read -r arn; do aws acm describe-certificate --region "$region" --certificate-arn "$arn" \
--query 'Certificate.[DomainName,Type,DomainValidationOptions[0].ValidationMethod]' \ 
--output text done | awk -F'\t' '$3 == "EMAIL"' | column -t 

Usage: 
chmod +x list-email-validated-certs.sh 
./list-email-validated-certs.sh # default region, us-east-1 
./list-email-validated-certs.sh us-west-2 # another region

Migrate existing email validated certificates

To assist you in this migration, ACM is updating the UpdateCertificateOptions API so you can switch a certificate’s validation method from email to DNS in place. This means the certificate Amazon Resource Name(ARN) will remain the same and no changes will be needed to your AWS resources that reference the certificate.

When you update a certificate to DNS validation, ACM provides a CNAME record to add to your DNS configuration, and you have 72 hours to add that record. During this window, the certificate continues to function normally on email validation. If the 72 hours elapse without a DNS update, the certificate stays active on email validation and you can retry when ready. After DNS validation is complete, ACM is designed to automatically renew your certificate before it expires without further manual intervention required. We recommend completing migration before September 30, 2027, so that ACM can keep your certificates up to date without interruption.

To migrate using the console

  1. After you’ve identified a certificate that needs updating, open it and select Update validation method at the top of the page.
    Figure 2: DNS Validation prompt when viewing an email validated public certificate.

    Figure 2: DNS Validation prompt when viewing an email validated public certificate.

  2. After the update is triggered, you will see a View DNS records flashbar at the top of the certificate page.
    Figure 3: View DNS validation records after updating validation method.

    Figure 3: View DNS validation records after updating validation method.

  3. Select View DNS Records in the flashbar to open a dialog box from which you can download the CSV file for the CNAME records to export to other DNS providers.
    Figure 4: Get DNS validation information from the dialog box

    Figure 4: Get DNS validation information from the dialog box

  4. For Route 53 users, there is a Create records in Route 53 that makes the validation available as a one-click option.
    Figure 5: Create DNS validation records into Route 53

    Figure 5: Create DNS validation records into Route 53

To migrate using the AWS CLI:

For instructions on how to update certificates using the AWS CLI, see the email to DNS migration user guide.

Alternatives after email validation is no longer available

ACM supports two validation methods for new certificates going forward:

  • DNS validation – Add a CNAME record to your DNS configuration. ACM automatically renews DNS-validated certificates as long as the record remains in place. We recommend this method for most use cases.
  • HTTP validation for CloudFront – ACM provides a unique token that you host at a well-known URL path on your domain. This method is only available for certificates used with Amazon CloudFront.

Both methods remove the manual approval step required by email validation and let ACM renew your certificates automatically.

Conclusion

The deprecation of email validation and use of the new UpdateCertificateOptions API helps keep your certificates trusted and your applications running as industry standards evolve. The updated UpdateCertificateOptions API is designed to make this migration straightforward: switch your validation method in place, add the DNS record, and ACM is designed to handle renewals automatically from that point forward.

If you have questions or need assistance migrating, contact AWS Support or start a new thread on the AWS re:Post ACM Forum.

If you have feedback about this post, submit comments in the Comments section below.


Adam Aboudi

Adam Aboudi

Adam is a Senior Technical Product Manager on the AWS Certificate Manager team. He focuses on helping customers simplify certificate lifecycle management and maintain strong security posture across their AWS environments.

Poojil Tripathi

Poojil Tripathi

Poojil is a Solutions Architect at AWS based out of Austin, TX. They work with AWS customers to help them design secure architectures for all types of workloads. They would like to remind you to dance like no one is watching, but encrypt like everyone is.

Landing Zone Accelerator Independent Assessment Report for C5:2020 now available on AWS Artifact

Post Syndicated from Kevin Donohue original https://aws.amazon.com/blogs/security/landing-zone-accelerator-independent-assessment-report-for-c52020-now-available-on-aws-artifact/

Organizations operating in Germany and across Europe increasingly need to demonstrate cloud security compliance under the Cloud Computing Compliance Criteria Catalogue (C5:2020), published by Germany’s Federal Office for Information Security (BSI). Last year, we introduced Landing Zone Accelerator on AWS support for digital sovereignty and today we’re announcing the availability of a new independent assessment report available on AWS Artifact which evaluates how the Landing Zone Accelerator (LZA) on AWS solution provides enhanced coverage for C5:2020 requirements by implementing nearly 200 native security controls. LZA is available using a standard multi-account configuration or as a container-based deployment option in the AWS European Sovereign Cloud, enabling customers with data residency requirements to use the same security configuration baseline.

How this accelerates your compliance journey

Security and compliance are a shared responsibility. LZA takes on part of this responsibility by defining a security architecture baseline and automatically provisioning your AWS environment that scales as your organization grows. Where AWS already provides C5 Type 2 attestation reports for “security of the cloud”, the LZA assessment report offers an independent opinion of how the security baseline LZA provisions aligns with C5:2020 criteria for “security in the cloud”. Instead of starting from scratch, you can deploy with LZA, evaluate the scope of coverage from the report, and use the LZA Compliance Workbook to build on and customize for your organization’s unique use case. These resources can help you reduce time in architecture design, evidence collection, and preparation for C5:2020 assessments. The free LZA Compliance Workbook available on AWS Artifact and open source Universal Configuration GitHub repository are also excellent sources to add to a knowledge base, enabling you to create a security compliance chat agent with Bedrock to assist your governance or assurance teams.

What’s in the report

AWS Partner Schellman, an independent third-party assessor, evaluated the LZA Universal Configuration architecture and security control baseline, which maps to C5:2020 controls in the LZA Compliance Workbook, to determine how the LZA infrastructure aligns to C5:2020 technical requirements. The report concluded that LZA can help implement 325 security controls in aggregate, aligning to technical requirements from eight C5:2020 control areas. It also describes the LZA architecture design, security best practices, and scoping considerations for C5:2020 assessments. This is the first installation of the independent C5 report for LZA, which will be updated in 2027 to evaluate coverage for the pending C5:2026 revision.

In addition to the LZA C5:2020 report, you can also find the LZA Compliance Workbook available on AWS Artifact. It maps C5:2020 requirement identifiers to security implementation statements, giving you a starting point from which you can customize and enhance your compliance documentation for your unique workloads or operational practices after deploying LZA.

Getting started with LZA for C5

  1. Sign in to your AWS account first and then download the LZA C5:2020 Independent Assessment Report and LZA Compliance Workbook from AWS Artifact.
  2. Visit the LZA Universal Configuration GitHub repository to review and download the latest configuration baseline. Also, see guidance for European Sovereign Cloud LZA deployments.
  3. The LZA Implementation Guide walks you through deployment steps, use cases, and pre-deployment considerations.

To learn more, submit a question to a LZA team member or contact your AWS account representative.

If you have feedback about this post, submit comments in the Comments section below.


Kevin Donohue

Kevin Donohue

Kevin is a Senior Security Compliance Engineer at AWS, where he builds solutions and resources to help AWS customers achieve their security and compliance goals. Prior to joining the Landing Zone Accelerator team in AWS Professional Services in 2024, Kevin began his tenure with AWS Security in 2019 specializing in FedRAMP compliance and the shared responsibility model.

Michael Wahlers

Michael Wahlers

Michael is a Principal Solutions Architect and Public Sector Specialist working across Germany, Austria, and Switzerland. With passionate enthusiasm, he supports public institutions with innovative digital solutions. His expertise ensures seamless service delivery, making him a valuable asset in shaping the digital future of the public sector. He also enjoys exploring complex distributed systems and incorporating local contexts into his work.