Tag Archives: security

Build your own vulnerability harness

Post Syndicated from Dan Jones original https://blog.cloudflare.com/build-your-own-vulnerability-harness/

A few weeks ago, we published our initial findings from Project Glasswing, looking at what happens when you point frontier security models at an enterprise codebase. We also explored how our defensive structures adapt to protect our infrastructure and customers from threats posed by frontier AI. Since then, the AI ecosystem has continued to shift rapidly — developers who’ve built tightly around a single model have already experienced what happens when that model is no longer available or gets superseded by a more capable one. These market shifts only reinforce our core thesis: no matter which underlying model is leading the pack on any given day, the future of agentic workflows will not be found in standalone models, prompts, or single-agent sessions.

Moving from a localized security “skill” to a continuous, fleet-wide scanning pipeline requires an architecture where models are treated as interchangeable components. Relying on a single model inherently limits defensive coverage, as the same system will tend to look at code paths through the exact same lens. To counter this, models should be frequently interchanged and cross-tested. By varying the models across the pipeline — such as using one model for initial discovery and an entirely different one for validation — we can ensure that vulnerabilities are cross-checked by distinct sets of logic. Furthermore, a true enterprise-scale harness must look beyond isolated repositories to trace vulnerabilities across cross-repo dependencies, ultimately filtering thousands of raw candidates down to a trusted, triaged queue of actionable fixes.

This post serves as a practical look at how to build that model-agnostic layer, focusing on how we manage state controls, eliminate false positives, and coordinate end-to-end triage at scale.

Two objections, up front

The first post made the case for why generic coding agents can’t do this job. The main issue is that agents only hold one hypothesis at a time, fill their context window after covering a sliver of a real repo, and then lose information during context compaction. For more details, read that post.

Before we move forward, we would like to answer two likely questions.

“Why not use subagents instead of a harness?” Subagents are useful, and they are a good starting point. But security analysis needs hundreds of separate investigations that survive across runs, don’t share a context window, and can be re-scoped and cross-referenced later. It needs persistence, deduplication, resumability, and eventually fleet-wide dependency tracing. That’s an orchestration problem, and a prompt can’t get you there.

“Is this blog post just an ad for frontier models?” No. Our approach centers on the harness, not the model. When it comes to vulnerability discovery, we run it with whatever frontier model is currently best at what we need. When we point different models at the same target, they each turn up a different share of the bugs. The harness is the bit that lasts. If you build your own system, design it to be model-agnostic from day one. This will allow you the freedom to use any model of choice without constraints.

It all starts with a skill

We started with a ~450-line security-audit skill that we ran on a single repository, and adjusted the prompts until we surfaced real bugs. Later, we added the orchestration that became the plumbing of the entire system. The real value lives in the prompts themselves, and our prompts continue to carry the initial skill’s attacker scenarios, bug classes, and anti-pattern detections nearly unchanged.

The skill was written to run a 7-phase audit in one session:

  • Three parallel research agents do recon and write an architecture.md.

  • One Hunter agent runs per class attack, trying to break the code rather than review it.

  • Adversarial validators try to disprove each finding.

  • The survivors are written up as a human-readable vulnerability report.

  • They’re also emitted as findings.json against a schema, and a mechanical check validates that file.

  • Finally, a fresh agent independently re-verifies every finding against the source.

  • The surviving, re-verified findings are submitted to the ingest API.

That first skill maps almost directly onto the later harness:

Skill phase

Harness stage

Recon agents write architecture.md

Recon

Hunters run per attack class

Hunt

Validators disprove findings

Validate

Surviving findings become a report

Report

findings.json is checked mechanically for schema adherence, not correctness

Mechanical validation of line numbers and functions in findings

Fresh agent re-verifies findings

Independent validation

The skill worked, but it quickly revealed its limits. Looking at the coverage metrics, a single run finds only about half the bugs you’d catch across multiple runs. In our experience the ones it did find skewed toward the simpler and less subtle. Once your process is basically “run it ten times and diff by hand,” you probably need to start looking at a real harness.

While running and fine-tuning the skill, we ran into three walls: 

  • Context exhaustion: An hour in, the context window fills up and the model will cannibalize its own memory, instantly forgetting the bugs it spent all morning tracking down. We broke this bottleneck by externalizing the state entirely, treating the LLM as a stateless compute engine. 

  • Persistence: A crash mid-run means starting over. Losing hours of work to one AI rate-limit error or connection flakiness is an incredibly expensive way to realize you need a better architecture. 

  • Cross-repo reasoning: A single repo session is completely blind to the relationships between applications that consume it, and the number of bugs that surface when you inspect the interface between components is probably more than one might expect.

ADVICE: A real but minimal harness consists of just Recon, Hunt, and Validate stages kept in a database, alongside a separate Validator that can’t file its own findings. You should skip cross-repo tracing entirely until you have more than one repository that matters. Skip a dedicated Deduplication agent until you are actively drowning in noise. Start with a skill in your development environment, get your prompts working well, and only build the next architectural stage when not having it is the specific thing slowing you down.

Codifying the skill into a pipeline

Most AI security write-ups in this space are about a single repo or a curated benchmark; running a whole fleet this way, with cross-repo tracing, isn’t something we’ve seen written up elsewhere. Our codebase spans a massive mix of languages — Rust, Go, C, Lua, TypeScript and Python, alongside various configuration management systems, static configs, and all sorts of additional context. So we had to come up with something new that worked for us. Going from that first slash-command run to a fleet scanner that could cover 128 distinct repos, automatically finding and interrogating relevant dependencies, took about six weeks. Codification was mostly mechanical: we lifted each phase of the skill into its own agent, put a database behind it and an orchestrator in front. The mapping was almost one-to-one.

The entire fleet runs on one unified harness with no per-language tuning and traces the dependencies between repos. While offloading syntax to a model makes the system language-agnostic, the differentiator is its ability to trace dependencies between repos. The harness itself doesn’t care if it’s looking at C pointers or a TypeScript file; it focuses on the higher-level logic of security orchestration. This allows us to scale across hundreds of different codebases, without having to write custom language parsing. 

A two-stage vulnerability research workflow

Our entire vulnerability research workflow is built on a two-stage operational framework: the Vulnerability Discovery Harness (VDH) and the Vulnerability Validation System (VVS).

The VDH functions as our discovery engine, proactively scanning codebases to surface potential security issues. Once bugs enter the VVS, which allows multiple harnesses to feed into it, they go through stages of Deduplication, Judgment, and finally Fixing, as we’ll talk about later.

We use one model for VDH, but we use a completely different model for VVS, so the models are effectively double-checking each other. There is an obvious security benefit to this: by forcing Model B (VVS) to judge the output of Model A (VDH), you ensure that the finding is evaluated by an entirely different set of logical weights and training data — one that acts as an unbiased, adversarial third party whose sole job is to ruthlessly stress-test Model A’s assumptions.  And operationally, we benefit from treating model providers like interchangeable commodities. Model providers can change temperature, caching, and inference effort budgets over time, even within one model version. Instead of building a system that depends on a model behaving predictably over time, our harness is built to absorb downstream volatility without breaking.

Stage 1: Vulnerability Discovery Harness (VDH)

The first post covered what each agent/stage is for, so we’ll talk about the parts it didn’t: the glue between stages, and the handful of details that decide whether any of it works.

Agent/stage

Primary Role

Sub-agents / Tooling

Recon

Maps out the target architecture and maps potential threat vectors

3 parallel Recon sub-agents write architecture.md

Hunt

Runs per-class attacks, compiles fragments, probes binaries

It spawns siblings (these handle between 9% and 20% of fleet-wide tasks depending on the model). It reaches out to and writes to the Wishlist tool. 

Validate

Mechanically checks the finding, then adversarially disproves it

Runs in two passes: plain code handles the initial schema/path checks, then a single isolated agent tries to disprove the finding before it can be filed. 

Gapfill

Generates new hunt tasks for empty coverage cells

Enqueues fresh hunt tasks for any under-tested (area × attack-class) cells that still look thin

Dedup

Identifies and consolidates overlapping findings

Combines deterministic code and agents to cluster findings by root cause, folding them together in real time

Trace

Walks dependency graph; spawns consumer-repo tasks

Walks the graph to add hunt tasks inside every identified consumer repo to make sure cross-repo bugs are caught

Feedback

Learns from pre-existing reports and optimizes future runs

Takes validation failures, shallow runs, and repeated misses, and instantly rewrites queued prompts to make future tasks sharper.

Report

Renders human-readable report

Just a script, no model required

Table 1: Vulnerability Discovery Harness (VDH)

Stages four through eight run as a continuous producer-consumer loop. As the initial hunt progresses, the Gapfill, Feedback and Trace agents generate new tasks; Dedup folds overlapping findings back together and the rest of the loop keeps consuming the queue. This ensures a vulnerability discovered late in the cycle is still validated, reported and checked against other code to make sure it doesn’t contain the same bug, all within the same run.

Splitting the pipeline this way guarantees strict context controls. If you fill the context window, the model starts hallucinating. We keep each agent’s job hyper-focused, keeping context usage below 25% of the total window. A naive “read all files” approach will blow past this limit every single time.

One thing that caught us out was that persistence needs to be factored in before parallelism. You do not want to throw away a five-hour run because of an unforeseen error. Every stage writes to one SQLite database keyed by (run_id, repo, stage). Any stage can resume, retry, or get pulled into a later run without redoing work. Findings are streamed and saved as they happen, so a crash costs you the task in flight and nothing else.

ADVICE: Sometimes a transient API error comes back as text in the (200 OK) response stream instead of throwing a code exception. To the orchestrator, this looks exactly like a task that finished cleanly. You must explicitly classify the response text, not just trust the exception type, or you end up logging empty runs as successes.

Dynamic threat modeling

During the Recon stage, the agent writes the threat model instead of being handed one. Beyond about ten built-in attack classes (many forms of injection, memory corruption, protocol parsing, timing side channels, and others), the Recon agent can invent repo-specific classes on the spot, each with its own methodology. It writes a custom taxonomy tailored specifically to that codebase, which is used to more tightly scope the Hunter agents.

Reading source code isn’t enough to understand how it behaves under stress, especially for subtle undefined-behavior bugs in C and other lower-level languages. The Hunter agents move past code reading and transition into active execution. They compile fragments, build small versions, and attack them. The biggest jump in quality came from giving Hunters a sandbox (built on unshare) to crash binaries.

ADVICE: If the harness itself runs inside Docker, that sandbox needs seccomp=unconfined and apparmor=unconfined or it will silently fail to start. It’s a one-line fix that saves you a day of head-scratching if you aren’t an expert in nested containerization, like us.

Micro-forks and the wishlist

Beyond the core pipeline stages, we added two specialized mechanisms that grant the Hunters significant autonomy to adapt their focus and request external resources without derailing an ongoing analysis:

Sibling Forking: This helps ensure that if a Hunter agent trips over an interesting code path that is outside the current scope, it doesn’t wander off track. It uses a tool call to fork a sibling agent with a precise structural seed. Fleet-wide, this accounts for roughly 9% of tasks, though the rate is highly model-dependent — from near-zero to about a fifth, depending on which model is hunting.

The Wishlist: When an agent needs a tool it doesn’t have, often a Validator confirming a Proof of Concept (PoC) or a Hunter wanting to build something (like a specific build environment, a VM, or some prod config files), it writes to a central wishlist. It provides enough context for the system to automatically re-run that exact task once a human provides the dependency. Some of these can be partly self-healing: if the container needs to be rebuilt with some changes, this can autonomously happen after the run by having a generic coding harness monitor the logs.

The wishlist has been written to 25,472 times across 128 repos since the wishlist was added, and it’s the main way the agents talk back to us. One that landed while we were writing this: “I need a FreeBSD VM to confirm this PoC end-to-end.“

Fleet-wide cross-repo tracing

After the initial cleanup, a Tracer agent checks how different software components are connected. It looks for a specific path: can a potential attacker send harmful input from the outside to a vulnerable part of the system? If the answer is yes, the Tracer agent automatically spawns fresh hunt tasks inside the consumer repository. To make this work, you need a unified, cross-repo symbol index and an accurate dependency graph. This allows you to uncover deep, systemic flaws that a standard single-repo scan would miss.

Running our harness across an entire fleet of repos revealed two lessons that only surfaced when this was done at scale. 

First, deduplication is its own problem, big enough to need its own agents. When you are scanning a handful of repositories, you can manually eyeball overlapping bugs. Simple string matching or file-path checks won’t save you here. Determining whether two complex logic flaws are actually the exact same root bug sounds trivial, but it isn’t. It requires so much cognitive reasoning that we had to deploy dedicated Dedup agents just to clean up the noise, along with their own heuristics and ways of reducing the work.

The second is to not wire in static analysis early. We plumbed Semgrep all the way through, and the Hunters invoked it zero times in a month of runs. They would rather read and run the code. The wishlist, by contrast, was the single most-used tool in the system. It’s worth paying attention to what the agents actually reach for, rather than what you think they’ll want.

Making findings you can trust

The agent will edit the source code so its own exploit works, then triumphantly report the bug it just created. It will write a test that proves something entirely tautological like “exec() executes things, therefore critical vulnerability”. Or it builds an exploit that runs fine but proves nothing, because the threat model behind it is nonsense. If your harness doesn’t actively fight this, all you’ve built is a faster way to produce junk.

A Hunter has to state the threat model before it’s allowed to file anything. It has to define exactly who the attacker is, and what boundary the vulnerability crosses or what assumption it breaks. The output schema ordering enforces it. This requirement eliminates the vacuous findings, the “if a user has database write access, they can write to the database” kind.

Every confirmed finding ships with a PoC written as a test that runs against the original, untouched codebase. This prevents the agent from editing the source files to force an exploit to land. If there is no working PoC, we treat the finding as fake. In practice, that’s a Hunter compiling a thirty-line parsing loop, running it with memory protection enabled, and demonstrating that the incorrect read stride is originating from a stack address rather than the expected message body. You can re-run it yourself. Furthermore, every confirmed finding must also ship a proposed patch. What actually reaches our review queue is a verified bug, a working test, and a functional git diff, not just a vague text description of a problem.

Before an exploit path survives, deterministic code (written in plain code, not another model) mechanically verifies that the cited files and paths actually exist, and confirms that both the patch and the test parse correctly. This Validator cannot log findings of its own; its sole job is to aggressively disprove the Hunter‘s theory. If a Hunter is allowed to grade its own homework, it will confidently validate everything it outputs.

We don’t claim a false-negative rate for our system. There’s no labeled set of every real bug in a codebase, so any claimed recall number is entirely speculative. What we can watch is whether re-runs keep turning up new bugs (they do) and whether coverage is still growing across runs. It’s all a proxy, as you don’t know for sure how many bugs exist in a single codebase, but it’s a good-enough way of measuring effectiveness.

Stage 2: Vulnerability Validation System (VVS)

A finding coming out of the harness is just the start of the triage process, with all discoveries landing in a single, shared VVS that currently holds 13,841 findings across 145 repos in total. Triaging that volume is its own massive engineering problem, and it matters just as much as the hunting. That triage engine runs on a different model from the harness, broken down into three distinct jobs.

Agent/stage

Primary role

Spawns/ sub-agents/tooling

Dedup

Identifies if a vulnerability is already in the system, or raised as internal Jira ticket already

Deterministic: plain code builds inverted indexes over files, functions, trust boundaries, and rare tokens, then hands each finding a short candidate list

Probabilistic: Dedup agent reasons over that short list, Stable cross-run key reopens existing records

Judgment

Production reachability and validation

Single agent — builds context about the bug from MCP servers, to get the shape of what the service looks like in production. Searches the wiki, Jira, git, config, and all available other sources to try and understand whether a bug is truly applicable to our production environment, and then score the vulnerability against this. It also validates the bug against source code to understand if the bug still exists on the latest main branch.

Fixing

Generates patches, runs regression tests

Runs the regression test before and after (filtered to the affected test; full suite only when per-test filtering isn’t available). It requires a clean fail→pass flip on the target test to clear the gate. If the post-patch test fails, or if a global run detects downstream regressions, the commit is automatically blocked and flagged for human intervention.

Table 2: Vulnerability Validation System (VVS)

Deduping

Comparing every single finding against every other finding using an LLM scales at O(N^2), which falls apart completely at scale. To keep the model off the critical path, deterministic code builds inverted indexes over the structured data (touched files/functions, trust boundary, rare tokens) to generate a short list of real candidates. Only then does an agent look at that short list to see if a single fix would close several of them. Stable cross-run keys ensure re-found bugs reopen existing records rather than spawning new ones.

Contextual judgment

Judgment is a second, independent pass over what survived. The agent rechecks the latest information, pulling from deployment, environment, and config context to determine if the code path is reachable in prod, and identify the repo owner. This process filters “exploitable now” from “real but latent” and from “real but filed against the wrong component.” It’s moving a pile of chaotic findings into a risk-driven orchestration workflow.

Automated fixing

The Fixer takes the proposed patch and unit tests, rewrites them to match the repo’s style, applies the diff, and runs targeted tests. A clean fail→pass flip is the ideal and the only auto-cleanup case; a failing post-patch test blocks the commit. The Fixer never merges code on its own; a human must review the branch. This gate is the non-negotiable, human-in-the-loop safeguard that enables a clean, unbreakable cryptographic trail for change management compliance. Left to patch freely, a model will happily fix a security bug while quietly breaking an unrelated feature or adding dozens of new bugs.

Across all three triage jobs, each agent is confined to one narrow task wrapped in deterministic bookkeeping code, and nothing writes to production without a human signing off on a dry run. While this pipeline moves the engineering bottleneck from finding bugs to reviewing and landing fixes, the Fixer remains the youngest and slowest part of the system. 

What it costs

Running hundreds of agents over a fleet of repos is not cheap, but at least the shape of the spend is predictable. Almost all of the compute budget goes directly into the hunt stage. This makes Gapfill our cost-to-coverage lever, as each additional pass costs roughly half as much as the initial hunt.

Because the cost per repository varies wildly, we budget per repo rather than per run. We enforce a strict task cap per repository and spin up a worker pool of anywhere from 50 to 200 workers. That way you can spend money on the repos that are actually finding things, and not waste it on the ones that aren’t.

It’s also why, for us, the big scans are a periodic backlog sweep and not a per-PR check. A full scan of a complex repo can take hours; the worst run took just over 14 hours. Cheaper, smaller harnesses are the right tool for that job.

How we tell it’s working

We measure our system’s effectiveness by tracking how efficiently our automated pipeline filters deliberate engineering noise into high-quality, actionable findings. Because we intentionally tune our Hunters to over-report subtle primitives that could be chained into larger attacks, our true indicator of success is how sharply we can refine that initial mountain of raw data, before it ever reaches a human.

To gauge this, we track exactly how many raw findings survive each validation stage over time. Thanks to better context injection from our Recon phase, our initial validation rejection rate dropped from 40% down to 11%, while the share of high-integrity findings climbed from 35% to 58% (representing ~12,057 lifetime findings).

Here’s the lifetime breakdown from raw candidates to actionable findings, at the point in time this blog post was written.


Vulnerability Discovery Harness (VDH)

  • Raw candidates: Everything the discovery harness emitted before independent validation.
  • Needs repro: Findings that appeared plausible but required manual reproduction before being trusted.
  • Rejected at validation: The validator disproved the threat model, exploit path, affected code, or evidence.
  • Duplicates: Candidates collapsed onto another finding from the same harness.
  • Survived validation: Findings that passed the independent validation gate and moved into the VVS.
  • Bugs that went elsewhere: Findings deliberately routed outside this flow.

Vulnerability Validation System (VVS)

  • Another vulnerability harness: Other automated sources feeding the same validation system.
  • Total bugs in system: The combined pool after ingest.
  • Duplicates: Findings the dedup pass identified as already covered by another canonical finding or ticket.
  • Wrong repo / other / not a risk: The noise bucket: misattributed findings, defense-in-depth, or latent risks.
  • Bugs sent to teams: Finalized, clean findings ready for remediation.
  • Judged Internet-exploitable: High-urgency findings a realistic attacker could trigger in production.
  • Not judged Internet-exploitable: Lower-urgency, actionable bugs (production issues, dependency risks, or config errors).
  • Final severity split: The categorization used to assign priority for the engineering teams.

The core metric of the harness isn’t a speculative recall score — it’s keeping the number of unconfirmed findings in front of real humans as close to zero as possible. The architecture needs to be a relentless filtering funnel.

  • Out of 20,799 raw candidates generated by VDH, only about 12,057 survived validation.

  • When these were pushed into the VVS, joining findings from another harness, the central pool was brought to 13,841. 

  • The Dedup agent folded away 5,442 findings as duplicates. 

  • 1,154 were routed to the queue as ‘wrong-repo’ or ‘low-risk’ and were recycled back into the system where appropriate. 

  • Ultimately this left 7,245 actionable findings for engineering teams to act on.

Traditional compliance rules dictate arbitrary remediation windows based entirely on a static CVSS score (e.g., “Fix all Highs in 30 days”). Our contextual judgment layer turns this compliance checkbox into actual risk management. 

The architecture is capable of tracking findings back to their origin, meaning that fixing a single root cause resolves an entire cluster of findings rather than just patching individual issues. VDH system performance is also measured by dividing repos into (area x attack-class) cells and running the Gapfill agent iteratively until it stops producing findings. Whenever we update an underlying prompt, we test it against a held-out repository to see if that total coverage cell number actually moves.

The harness wires automated health signals to catch system failures early in the pipeline. If a hunt finished suspiciously fast and fails to spawn sub-hunts or gap tasks, it usually indicates a crashed dependency rather than a clean codebase. To remedy this, the system flags any Hunter agent that finishes with zero findings as “shallow” and immediately requeues it for a new run. 

Finally, our system’s robustness is reinforced by the independent triage pass described earlier. By re-judging all submissions with a different model and separate logical weights, we ensure an unbiased, adversarial verification that is decoupled from the specific model used for discovery, providing a trust layer that persists regardless of which model is in use.

None of this is finished. We change our system constantly, and it is nowhere near a perfect science. But raw candidate findings are cheap now, and the only work worth doing is turning them into sound, verifiable code fixes.

Building your own harness means accepting that AI models are volatile, but your orchestration layer doesn’t have to be. By decoupling your security logic from any single provider, forcing adversarial verification, and automating your triage pipeline, you can turn a mountain of LLM noise into a reliable, fleet-wide defense engine.

Our “North Star” metrics: measuring real-world velocity

Every codebase is a little different, so to show you how this actually works in the real world, we mapped out a realistic benchmark based on a standard repo run. Keep in mind that this represents a single pass on one repo; over time, as the continuous fleet-wide loop deduplicates, filters, and recycles findings, it reduces the volume of lifetime candidates by roughly 65%.

Engineering hours saved via automated patching: Rather than focusing on static baselines, we measure the health of our pipeline by its technical throughput, processing velocity, and its ability to eliminate the manual triage bottleneck:

  • Initial Validation Cut: For a standard repository (~30k lines of code), this yields 100 initial findings, with a full run taking 3-4 hours, maintaining a hyperfocused context window throughout. 

  • Compression: The Deduplication and Contextual Judgment Layers process these candidates in parallel. Within 3 hours, the system compresses and refines the batch of findings from ~100 raw candidates to 80 distinct, high-fidelity bugs.

  • Remediation: The automated Fixer processes these 80 distinct bugs at an average rate of 5 minutes per bug. In total, the system can discover, validate, deduplicate, and open functional pull requests in approximately 14 hours.

Shrinking mean-time-to-resolve for critical flaws: Of course, you can’t dump 80 patches into production all at once without breaking things. To keep deployments safe, our system uses a tiered rollout:

  • Critical Exposure Containment: The system isolates the critical, high, and exploitable bugs (avg. 10 out of 80). We fast-track these for a human review and introduce them into release cycles, getting them fully patched in production in 5 days.

  • Incremental Hardening: The remaining latent risks, minor config anomalies, and lower-urgency bugs are incrementally rolled into prod over a 15-20 day window to guarantee platform stability.

How we’re handling all of this patching

These findings are the result of an isolated, ring-fenced research experiment designed to stress-test our code. They do not represent active, unpatched vulnerabilities in our live production environment.

Because the harness runs constantly in our test environments, these specific numbers are completely out of date by the time you’re reading this. Every single bug surfaced by the pipeline came attached to a working test case to demonstrate the bug and a draft patch. Our security teams are systematically processing the reports and applying the necessary fixes, meaning the Cloudflare products you use every day are already actively hardened against these vectors.

Along with this blog post, we’re releasing the initial skill we used to develop the harness, it’s been slightly cleaned up before release so it’s easier to understand and integrate, but the skill itself remains substantially the same. Hopefully the harness itself will follow shortly. This could be a starting point for your own vulnerability harness, your own skill, or whatever suits your needs best:
github.com/cloudflare/security-audit-skill

If your team is working on the same problems and would like to compare notes, reach out to us at [email protected].

Reducing SMS OTP fraud with Vonage network-powered solutions and Amazon Cognito

Post Syndicated from Tito Milla original https://aws.amazon.com/blogs/architecture/reducing-sms-otp-fraud-with-vonage-network-powered-solutions-and-amazon-cognito/

User authentication remains one of the most targeted touchpoints in application security. With the industrialization of fraud threats by generative AI, cybercrime costs are expected to reach $23 trillion in 2027, an increase of 175 percent from 2022. 20 percent of fraud is attributed to synthetic identity and authentication exploits, with account takeover (ATO) surging 141 percent since 2021.

But the damage goes beyond security. SMS One-time passcodes (OTPs) achieve only approximately 80 percent conversion on authentication flows, meaning 1 in 5 legitimate users is lost at the point of verification. Enterprises absorb hundreds of thousands of password recovery helpdesk tickets annually, representing significant support costs tied to OTP-based verification. Every abandoned authentication attempt today represents an opportunity to maximize your conversion rates across checkout, account recovery, and onboarding flows. The industry has long assumed that stronger security requires more user friction. That isn’t a law of physics. It’s a limitation of the tools available. Mobile operator network data removes that constraint and provides stronger identity assurance and a smoother experience, not one at the expense of the other.

In this post, we show how Vonage network-powered solutions work with Amazon Cognito to enhance many mobile-first use cases with network-level identity verification. Vonage network-powered solutions are a composable stack of real-time mobile operator intelligence, silent authentication, and integrated fraud protection, which uses the CUSTOM_AUTH flow to complete identity verification in under 5 seconds, with zero user interaction.

About Vonage

Vonage, part of Ericsson, is an AWS Partner with multiple AWS Marketplace listings. The company provides enterprise and CIAM deployments with cloud-based access to mobile operator network APIs, including real-time mobile identity and authentication across key regions. These complement Vonage’s global communications, voice, and video APIs backed by Ericsson’s global telecommunications infrastructure.

What network-powered means and why it matters

Before diving into architecture, it’s worth being precise about what separates Vonage’s network-powered solutions from the identity and fraud tools enterprises already have in their stack.

Most identity verification signals today are derived from aggregated, cached, or behavioral data. Traditional phone number lookup services query static databases that may be days or weeks out of date. Device fingerprinting analyzes browser characteristics that might be spoofed. Behavioral biometrics builds models from historical sessions. This is useful, but a lagging indicator by definition.

Enterprise customers who implement Vonage’s network-powered solutions operate from a fundamentally different layer: real-time data sourced directly from mobile network operators (MNOs). When you query whether a SIM was recently swapped, you’re querying the network that performed the swap. When Silent Authentication verifies a user, the proof of possession is the cellular data session itself. This session can’t be phished, intercepted, or socially engineered.

In fraud scenarios where SIM swaps are weaponized for account takeover (ATO), “recently” means minutes or hours, not days. Static databases refreshed weekly are not detecting these events. They’re logging them after the fact. Real-time operator queries close that window entirely.

The three pillars: Identity Insights, Verify, and Fraud Defender

Vonage network-powered solutions combine three API service components into a composable security stack that integrates with Amazon Cognito through the CUSTOM_AUTH flow:

1. Identity Insights: Pre-verification intelligence

Identity Insights runs before verification channels are initiated, surfacing real-time operator signals that are directly actionable in authentication policy decisions. The following list shows a representative set of JSON elements that might be returned by a request. Customers have the option to select which data is most valuable given a specific authentication use case and industry combination.

  • format and network_type: Filters invalid numbers, VoIP, landline, and premium-rate numbers used in synthetic account creation and bot-driven fraud.
  • sim_swap: Detects SIM swaps within a configurable look-back window, a leading indicator of ATO events in progress.
  • subscriber_match: Compares subscriber identity (name, address) against operator Know Your Customer (KYC) records.
  • device_swap: A recent change in the mobile device associated with a phone number signals that a bad actor might have taken control of the SIM card. (coming soon)
  • recycled_number: Numbers previously deactivated and reassigned to a new subscriber can trigger false identity matches in onboarding flows, creating risk in account creation. (coming soon)

These pre-checks trigger your defined risk policy: step-up challenge, hard block, or silent logging. Critically, fraudulent attempts are identified and blocked before a single OTP is sent, before verification costs are incurred, and before fraud processing overhead is generated.

2. Verify with Silent Authentication: Alleviating the friction tax

Every additional step a user must finish during authentication carries a measurable cost: abandoned sign-ups, failed conversions, and support tickets from users who don’t receive or mistyped a code. We call this cumulative loss the friction tax. For SMS OTP flows with approximately 80 percent completion rates, the friction tax means roughly 20 percent of legitimate users drop off before they ever reach your application.

After a number passes the risk pre-checks, the Verify API delivers the authentication challenge. The primary authentication method is Silent Authentication.

When a user initiates sign-in from a mobile device, Vonage routes an HTTP request through the user’s cellular data connection. The mobile operator confirms that the SIM registered to the phone number matches the session making the request. The exchange happens in the background, in seconds. The user doesn’t see, type, copy, or enter any code.

If Silent Authentication can’t finish or is unavailable, Verify automatically falls back to traditional SMS, RCS, Voice, WhatsApp, or email, remaining transparent to the user.

Key benefit: Silent Authentication alleviates the three primary exploit vectors against SMS OTP: SIM swap (bad actor receives the code), SS7 interception (message diverted in transit), and social engineering (user tricked into sharing the code). All without additional input from the end user.

3. Fraud Defender: Protecting the verification channel

Fraud Defender addresses a threat familiar to enterprise finance teams: artificially inflated traffic (AIT) and SMS pumping. Automated systems trigger high volumes of OTPs sent to premium-rate numbers that bad actors control. At enterprise verification volumes, these events can run undetected for extended periods.

Fraud Defender provides real-time traffic monitoring and intelligent blocking at the point of outbound delivery, intercepting these malicious events before costs accumulate. The financial impact is immediate and measurable. Fraud Defender typically absorbs its own cost in toll fraud prevention within the first billing cycle. For most enterprises, it quickly becomes a net revenue-positive investment. Vonage customers have collectively saved over $3M in SMS-related fraud costs since deployment. The savings continue to compound as the blocking algorithm evolves to counter new exploit patterns. For Verify customers, the value is even more compelling: Fraud Defender activates automatically with the Vonage Verify API at no additional cost. This makes it one of the highest-ROI fraud protections available.

Prerequisites

To implement this solution, you need:

  • An AWS account with permissions to create and manage Amazon Cognito, AWS Lambda, AWS Secrets Manager, Amazon CloudWatch, and AWS WAF resources.
  • An Amazon Cognito user pool (existing or new).
  • A Vonage API account with access to Identity Insights and Verify APIs.
  • AWS Command Line Interface (AWS CLI) or AWS Serverless Application Model (AWS SAM) CLI installed and configured.
  • For client integration: the Vonage Silent Authentication SDK for your mobile platform (iOS/Android).

Solution architecture with Amazon Cognito

Enterprise customers that integrate the Vonage solution use the Amazon Cognito CUSTOM_AUTH flow, which uses three AWS Lambda functions that orchestrate the solution stack without changing your existing user pool configuration or downstream service integrations.

Architecture diagram showing the Risk-Adaptive Customer Sign-In flow with layers including user devices, edge protection with Amazon CloudFront and AWS WAF, Amazon API Gateway, identity layer with Amazon Cognito, verification layer with Vonage Identity Insights, Verify API, and Fraud Defender, and the carrier network with Mobile Network Operators.

Architecture components

The solution connects five layers, each handling a distinct step in the authentication flow:

  • Client app (mobile/web) – Initiates the CUSTOM_AUTH flow with the Vonage Silent Authentication SDK, follows check_url redirects over the cellular network, and submits the verification code back to Amazon Cognito.
  • Amazon Cognito user pool – Orchestrates the CUSTOM_AUTH challenge flow and issues JWT tokens upon successful verification.
  • AWS Lambda triggers – Define Auth Challenge (orchestrator), Create Auth Challenge (calls Vonage APIs), and Verify Auth Challenge (validates response).
  • Vonage Network APIs – Identity Insights pre-check, Verify with Silent Auth and OTP (built-in failover), and Fraud Defender (automatic).
  • Mobile network operators – SIM-level identity verification through CAMARA/Open Gateway APIs.

Authentication flow

The following steps represent an authentication workflow sequence between Amazon Cognito and Vonage network-powered solutions:

  1. The client calls InitiateAuth with CUSTOM_AUTH, passing the user’s phone number.
  2. The Define Auth Challenge Lambda function instructs Amazon Cognito to issue a CUSTOM_CHALLENGE.
  3. The Create Auth Challenge Lambda function calls Identity Insights for pre-verification risk assessment. If the number passes pre-checks, Lambda calls Vonage Verify to initiate Silent Authentication and returns the check_url to the client.
  4. Upon receiving the check_url, the client opens an HTTPS connection to it, triggering HTTP redirects to the mobile carrier’s network for direct mobile-device-to-mobile-network-operator verification. Upon completion, the client receives a verification code from the operator.
  5. The client calls RespondToAuthChallenge with the code.
  6. The Verify Auth Challenge Lambda function submits the code to Vonage’s check endpoint. On success, it returns answerCorrect = true and Amazon Cognito issues the appropriate session token.

Sequence diagram showing the User Login flow with SIM-swap pre-check using Vonage Identity Insights and Silent Authentication via Vonage Verify, orchestrated through the Amazon Cognito CUSTOM_AUTH flow with Lambda triggers.

Coexistence and phased rollout

A critical design principle: zero disruption to existing infrastructure. The Vonage Network API plugs into the Amazon Cognito CUSTOM_AUTH flow without changes to your existing user pool, app client configurations, or downstream service integrations. Deployment requires a single sam deploy command.

This design approach allows for a phased rollout. Start with the highest-risk journeys (password recovery, high-value transactions) where security ROI is clearest, then expand to daily login and onboarding as you measure impact. Traditional SMS, RCS, and Voice OTP remain options for lower-risk flows during the transition.

Risk-aware workflows by journey type

The strategic value of combining Vonage’s network-powered solutions with the Amazon Cognito policy-driven CUSTOM_AUTH flow is context-aware authentication calibrated to actual risk. CRITICAL journeys are recommended for the first phase of implementation as they aim to meaningfully mitigate synthetic identity and account takeover. The following table describes risk-aware workflows by journey type.

Journey Risk Vonage Workflow
New account signup CRITICAL Identity Insights filters invalid/non-mobile numbers + Subscriber Match validates KYC → Silent Auth for zero-tap onboarding
Daily login MEDIUM SIM swap recency + device consistency check → Silent Auth passively, step-up only on elevated signals
Password recovery, profile change (contacts), 2FA settings change HIGH Mandatory SIM swap hard-check (tight lookback window) + Subscriber Match → Silent Auth required, no passive bypass
High-value transaction CRITICAL Full signal stack (line type, SIM swap, subscriber match) → Silent Auth + secondary challenge if risk elevated

Low-risk actions (for example, viewing account details, browsing content, or checking order history) generate no friction and no unnecessary verification cost. High-risk actions trigger the full assurance stack. The calibration is policy-driven and configurable per journey.

Implementation considerations

Configuring Amazon Cognito starts with setting up the user pool to allow the CUSTOM_AUTH authentication flow and accept phone numbers as the primary sign-in attribute. After the user pool is in place, associate the three required Lambda functions with their corresponding Amazon Cognito trigger hooks and store your Vonage API credentials in AWS Secrets Manager.

Layer in security from the start, following the AWS Well-Architected Security Pillar. Scope each Lambda function’s AWS Identity and Access Management (IAM) role to only what it needs: Amazon Cognito trigger invocations and AWS Secrets Manager access. Enforce TLS 1.2+ on all communication for encryption in transit. For observability, turn on Amazon CloudWatch logging on each Lambda function and turn on AWS CloudTrail to capture Amazon Cognito API audit trails. Finally, deploy AWS WAF with rate-limiting rules in front of the authentication endpoint to protect against brute-force attempts.

To configure the solution, follow these steps:

  1. Set up the Amazon Cognito user pool to allow the CUSTOM_AUTH authentication flow.
  2. Configure the user pool to accept phone numbers as the primary sign-in attribute.
  3. Associate the three required Lambda functions with their corresponding Amazon Cognito trigger hooks.
  4. Store your Vonage API credentials in AWS Secrets Manager.

Important: This solution creates AWS resources that incur charges. These include Amazon Cognito (per monthly active user), AWS Lambda (per invocation), AWS Secrets Manager (per secret per month), Amazon CloudWatch Logs, AWS CloudTrail, and AWS WAF (per rule and request). See the pricing page for each service and delete resources when no longer needed.

Privacy and compliance

The architecture is designed so that PII doesn’t leave the mobile operator. Subscriber Match performs a comparison within the operator’s environment and returns only a match score. The underlying subscriber data isn’t transmitted. Silent Authentication operates without PII exchange. The cellular session is the credential.

  • GDPR: Only match scores are returned. No subscriber PII is stored or transmitted, supporting GDPR data minimization.
  • PSD2 / Open Banking: Silent Authentication qualifies as a possession-factor for Strong Customer Authentication (SCA).
  • HIPAA: Subscriber Match supports identity assurance for healthcare applications.
  • DORA: Multi-channel fallback achieves > 99.9 percent verification availability.
  • CCPA: Same data-minimization architecture as GDPR.

Production results: Lydia Solutions

Lydia Solutions, one of Europe’s fastest-growing mobile financial services applications, deployed Vonage Verify with Silent Authentication in October 2024. The results demonstrate the real-world impact at scale, including up to 50 percent reduction in latency when compared to Lydia Solutions’s previous authentication services.

“Vonage Verify with Silent Authentication has been a real innovation for us. The solution has elevated our ability to deliver a simpler, seamless and more secure user experience while protecting against increasingly sophisticated threats and fraud patterns.”

— William Brulin, Senior VP, Lydia Solutions

Lydia’s results sit at the high end of outcomes observed. Across deployments in ecommerce, digital banking, and consumer services, conversion improvements of 2–8.5 percent compared to SMS-only are the norm, with authentication journey latency reductions of 50–75 percent.

Conclusion

This is where mobile operator data shifts the approach. Rather than applying identical verification friction to every session, enterprises can use real-time network signals to make adaptive authentication decisions. Verify silently when conditions are right, step up when risk indicators appear, and block when fraud is detected.

Enterprise implementation of the offering makes those risk signals and authentication methods accessible through a composable API layer. The combination of Identity Insights for pre-verification intelligence, Verify for network-layer authentication, and Fraud Defender for channel protection delivers risk-proportionate authentication that’s in production at scale today.

The solution deploys with minimal changes to your existing Amazon Cognito user pool. Start with high-risk journeys, measure impact, and expand. Vonage Verify API is available across over 700 MNOs in over 200 countries and territories, and the integration requires only three Lambda functions.

Next steps

Vonage is an AWS Partner. To learn more, visit the Vonage partner page.

The content and opinions in this post are those of the third-party author and AWS is not responsible for the content or accuracy of this post.

About the authors

Scaling out Distroless adoption with AI

Post Syndicated from Grab Tech original https://engineering.grab.com/scaling-out-distroless-adoption-with-ai

Introduction

Grab is migrating from heavy base images like Ubuntu to Distroless images to reduce security risks. By stripping containers down to the bare application and its runtime, we eliminate unnecessary binaries and Common Vulnerabilities and Exposures (CVEs).

This migration is more than a compliance mandate; it is a strategic security decision to build a more resilient and defensible production environment. By moving to Distroless, we are fundamentally shrinking our attack surface; eliminating the binaries and shells that attackers use for lateral movement. With over 900 services already transitioned, we are on track for 80% adoption by mid-2026.

Why Distroless requires rigorous testing

Distroless adoption risk: runtime failure

However, shifting to Distroless images introduces a critical technical risk: runtime failure. A service might build perfectly in Continuous Integration (CI), but fail at the deployment stage due to:

  • Missing shared objects: Binaries might require specific libraries (.so files) present in Ubuntu but absent in Distroless.
  • Implicit links: Third-party tools may expect specific system utilities or directory structures.

Testing is required to ensure two things:

  • The service spins up with the correct config.
  • All runtime dependencies remain intact.

Scaling this verification across thousands of services manually? That would take years unless we found a way to automate the trust.

The testing methodology

As we perform changes to the Dockerfile definition of our services, it is crucial for us to include the corresponding test strategy so that changes we make do not introduce regressions in our running services. Assessing the change introduced to our services, the lowest possible testing boundary would be that of what we define as Medium Tests in Grab.

Medium tests in Grab

At Grab, we categorize our test suites into three main sizes: small, medium and large. Small tests refer to functional tests whereby mocks are introduced via dependency injection. Large tests refer to end-to-end tests that run on actual services in our staging environment where nothing is mocked.

Architecture diagram of a medium test environment.
Figure 1. Architecture diagram of a medium test environment.

Medium tests belong in the middle ground, whereby external dependencies (such as service to service dependencies) are mocked with a network proxy layer in a similar concept as WireMock, but internal dependencies like MySQL are not mocked and instead spun up using Testcontainers. In this setup, systems under test are actually built into Docker containers and run in Docker before their endpoints are being hit by test inputs, with the corresponding responses being asserted on. As such, we could now effectively test if any changes of the Dockerfile definition broke the service. An added bonus is that all of these could occur within the CI environment, without reaching the Continuous Deployment (CD) stage.

Happy path for Distroless changes.
Figure 2. Happy path for Distroless changes.

This makes Medium Test effective and efficient for testing changes to the services associated with distroless adoption. We could now largely scale up our adoption process by:

  1. Raising batch Merge Requests to dockerfile definitions for Distroless adoption.
  2. Running medium tests in CI.
  3. Upon passing the medium tests, automatically merge the changes and trigger CD.

Introduction of toil

The approach above works nicely for services that already have Medium Tests defined. However, we quickly hit a blocker running this rollout methodology for services without a Medium Test setup. Inherently, scaffolding Medium Tests for a service is a tedious task. Most of the toil comes from first figuring out the internal dependencies, then spinning up their corresponding test containers in test time before wiring the internal dependencies up with the service under test by updating the test environment configurations.

Current gap in Medium Test coverage.
Figure 3. Services without Medium Test setup blocked the rollout.

These tasks are not challenging but are generally tedious to set up. At the same time, they cannot be automated completely given the different internal dependency combinations that each service uses, as well as the difference in how the configurations are being defined and used in each service. With ~400 services in scope without Medium Test setup, this became a huge blocker for our distroless migration campaign.

The need for flexibility in how each task is executed, together with each task’s fairly low complexity, made artificial intelligence (AI) a natural tool to accelerate distroless adoption work.

AI: The toil buster

Solution leveraging AI.
Figure 4. Solution overview: AI-driven workflow for Medium Test scaffolding and migration.

AI was a good fit because the work we needed to automate had clearly defined output, and we could tell, deterministically, whether it worked. Success was straightforward: the CI pipeline would turn green, running basic Medium test health checks. With a measurable end goal and a reliable success signal, we pursued an agentic workflow rather than a one-off generation attempt.

The starting point

We started by adopting skills to guide the agent on how to proceed with Medium test work and how to unblock itself when it hit repo-specific friction. These skills gave context for scaffolding basic Medium tests, setting up internal dependencies, and debugging issues in the code. Once those foundations were in place, we rolled the approach out to a batch of 20 services, completed by the AI in about two working days. That batch validated the core hypothesis: the AI could scaffold Medium tests first, then use those tests to verify that our Dockerfile change (building distroless images) introduced no regressions.

Teaching an agent to test

At that point, the real shift was turning “can do the task” into “can repeat the behavior.” We captured the Medium-test knowledge as a list of skills grounded in Grab’s internal Medium test SDK.

Then DevSecOps wrapped those skills into an Entrypoint Skill, an orchestrator that runs a multi-phase workflow across services. The result is a single agent loop that moves from candidate detection, to scaffolding, to fixing failures, and onward to CI verification without treating each service as a brand-new, one-off problem.

Workflow overview for Medium Test generation.
Figure 5. Workflow overview for Medium Test generation.

Leveraging the skills we’ve acquired, we utilized Claude Code, Anthropic’s agentic coding tool. This tool operates by accepting a list of services and then processing them in a batch.

  • Detect: Is this a deployable service or is it a library? Is it still maintained? The agent skips anything that doesn’t qualify, so human time is only spent on real candidates.
  • Scaffold: Using Grab’s scaffolding tool, the agent generates the medium test boilerplate.
  • Fix: The scaffold rarely works on the first try due to the unique setup of each repository like missing environment variables, database dependencies at startup, port mismatches, and similar issues. The agent reviews its knowledge base and pattern-matches errors against known fixes.
  • Raise MR: Once the medium test passes locally, the agent creates a draft merge request on GitLab with a description explaining what changes were done for that specific service and why.
  • Monitor CI: The agent polls the pipeline, reads job logs on failure, and attempts CI-specific fixes. If the same error persists after two attempts, it flags the issue for human review.
  • Repeat: Push the fix and move to the next service while the pipeline runs. The agent doesn’t sit idle waiting for CI! It starts scaffolding the next service asynchronously, checking back on previous pipelines as results come in.

What made it work

Getting the workflow to function was the easy part. Getting it to function reliably across hundreds of services required deliberate design choices.

Model Context Protocol (MCP): The agent never leaves Claude Code. GitLab interactions like creating branches, raising MRs, reading pipeline error logs, all happen through a MCP server. When the agent needs Grab-specific context like what a service does, or who owns it, it queries Glean, an enterprise search tool used by Grab through its MCP integration rather than guessing. For code-level context, finding how a service is structured or how dependencies are wired across repositories, it queries Sourcegraph through its own MCP integration.

Guardrails over autonomy: The agent can only touch test files and CI configs. Application code is off-limits, enforced before every commit. It can’t gut tests to make them pass. If it can’t fix the problem, it escalates.

Knowledge that compounds: We maintain a feedback loop for scaffolding, mocking, and known failure patterns. After each batch, we review what the agent hit and promote recurring fixes into the skill. The agent improves not because the model gets better, but because its instructions do.

Integrating scripts with skills: For deterministic tasks like boilerplate generation, scripts are far more reliable than raw AI logic. By integrating these scripts as “skills,” we also optimize the agent’s performance in context window management. During test execution, standard output often produces hundreds of lines of repetitive logs that could exhaust token limits or distract the model. Using a script as an intermediary allows us to programmatically filter logs, extracting only the specific error messages or stack traces required for debugging. This ensures the AI receives a clean, actionable summary rather than being overwhelmed by noisy data.

Token efficiency: Batch runs across dozens of services burn through tokens fast. We configured a compressed communication style that cuts output by ~75%, keeping technical substance while stripping filler. Proper communication is reserved for MR descriptions and messages to service owners.

Isolated execution: Each batch run spawns the agent in its own context window. Long sessions processing dozens of services don’t bloat the main conversation, keeping the agent focused and responsive.

Human-in-the-loop: Every MR is raised as a draft; a human reviews before anything merges. A human also decides which learnings become permanent knowledge. The agent proposes; people approve.

From tests to migration at scale

With medium tests in place across our service fleet, we had the safety net we needed. The next step was automating the distroless migration itself.

The patch-test-compare loop

Patch–test–compare loop for Distroless migration.
Figure 6. Patch–test–compare loop: baseline Medium Tests, apply Distroless Dockerfile changes, re-run tests, and triage results.

Before touching a single Dockerfile, the system runs the service’s existing medium tests to establish a baseline. Pre-existing test failures are baselined, allowing for a clear distinction between legacy issues and new regressions introduced by the distroless patch.

Then comes the distroless patching. The system inspects each service’s Dockerfile for OS-level package dependencies by scanning for apt-get install lines and filtering out packages already included in the distroless base image. Two scenarios to consider here:

  • If no extra packages are needed, it’s a straightforward base image swap.
  • If packages are detected, the system generates a multi-stage build: a builder stage installs the required packages, then copies only the necessary shared libraries into the distroless runtime stage. The result is a minimal image that still has everything the service needs to run.

After patching, the same medium tests run again. Results fall into clear categories: pass (tests still green – safe to migrate), regression (tests broke – the patch caused a problem), or already failing (was broken before we touched it). Regressions trigger an automated remediation step. A separate AI agent inspects the container for missing shared libraries and attempts to fix the Dockerfile. If it can’t resolve the issue, the service is flagged for human review.

Scaling with batch changes

The previous section explains the patch-test-compare loop, but how can we scale to handle more than one service at a time? To migrate at scale, we use batch change tooling that applies the Dockerfile transformation across dozens of repositories simultaneously, creating merge requests automatically. The system handles both standalone GitLab repositories and Grab’s shared Go monorepo, adapting the patching and MR strategy to each.

Impact on our services

Medium test generation at scale

With medium tests in place, services with possible regressions have higher chances of being caught before reaching staging, providing the safety guarantee we needed. Each generated test also became a permanent safety net for the service, not just for the distroless migration but for all future changes. Over 1.5 months, the agent raised 100+ medium test MRs across repositories, bringing more services into compliance with Grab’s “shift-left” testing initiative.

Distroless adoption

The campaign moved the needle significantly across our service fleet. Overall distroless adoption for our scope grew from 52.7% in December 2025 to 70.8% by April 2026, covering 997 out of 1,408 services.

Autonomous with oversight

The agent autonomously handles the majority of medium test generation and Dockerfile migration work with little human intervention for standard cases. Engineers remain in the loop, reviewing every draft MR and making the final call on what merges.

Engineering bandwidth reclaimed

Manually generating a basic medium test requires familiarity with Grab’s internal SDK, typically 1–3 days per repository for developers new to the framework. Across ~400 services without medium tests, that adds up to 400–1,200 engineer-days. By leveraging AI we brought this down to roughly 0.1 days per service, compressing what would have taken well over a year into a fraction of the calendar time. This freed the team to focus on higher-leverage work like improving migration tooling, handling edge cases, and advancing the roadmap beyond distroless.

Conclusion

With distroless images and stronger medium test coverage, we made Grab’s services more secure and easier to verify. We demonstrated that AI can shoulder much of the scale-up effort.

Join us

Grab is Southeast Asia’s leading superapp, serving over 900 cities across eight countries (Cambodia, Indonesia, Malaysia, Myanmar, the Philippines, Singapore, Thailand, and Vietnam). Through a single platform, millions of users access mobility, delivery, and digital financial services, including ride-hailing, food delivery, payments, lending, and digital banking via GXS Bank and GXBank. Founded in 2012, Grab’s mission is to drive Southeast Asia forward by creating economic empowerment for everyone while delivering sustainable financial performance and positive social impact.

Powered by technology and driven by heart, our mission is to drive Southeast Asia forward by creating economic empowerment for everyone. If this mission speaks to you, join our team today!

Dependencies should be fetched directly from VCS

Post Syndicated from arp242.net original https://www.arp242.net/deps-vcs.html

I’ve been writing Ruby at my new $dayjob in the last month. After spending most
of the last decade writing Go it’s been a fun change of scenery. I did Ruby
before (years ago) and I can’t really tell you which is “better” – Ruby is very
different from Go in almost every respect and I find both quite effective in
getting stuff done in their own way.

One aspect where I do feel Go is clearly better is dependency management;
specifically the security aspect thereof. Go is not magically immune to
malicious dependencies, but it is a lot more resistant to them chiefly because
there is no “publish a package” step.

In Go dependencies are identified by URL, e.g. github.com/user/pkg. Go
identifies which VCS is being used (git in this case) and fetches the tag or
commit you specified in your go.mod file. The go.mod file serves as both a
dependency specification and “lock file”. It lists exact versions; there is no
~>1.1. It includes both direct and indirect dependencies and lists your full
dependency tree (the go command writes to go.mod).

There are more aspects to Go Modules, including security features, but I will
skip over them for the purpose of this article. The relevant bit is
“dependencies are identified by URL, the code is fetched directly from the VCS,
and it does this for both direct and indirect dependencies”.

Auditing and updating dependencies is easy: I do git log -p old..new (usually
via a forge web UI), read all the commits, and update the go.mod file. I don’t
have many dependencies and those I do have don’t change much. It’s usually
pretty fast. I don’t need to do careful in-depth reviews here; just look for
suspicious stuff. Something like exec.Command(..) or http.Post(..) in a
globbing library would stand out. It’s hard to really hide stuff.

I’ve been doing this for years for every dependency. As a solo developer. It’s
easy. Some projects have much larger dependency trees and this becomes more
time-consuming, but not hard or confusing. It’s still easy, just takes a bit of
time.

For Ruby things are different as it has a “publish a package” step: you create a
.gem archive and upload that to rubygems.org. You can put anything in there – no
guarantee the .gem contents correspond to the source repo. To audit it I need to
do something like:

curl -s https://rubygems.org/downloads/example-2.7.5.gem >old.gem
curl -s https://rubygems.org/downloads/example-2.8.2.gem >new.gem

mkdir old new
tar xf old.gem -C old
tar xf new.gem -C new

(cd old && tar xf data.tar.gz)
(cd new && tar xf data.tar.gz)

diff -urN old new

It works, I guess, but is far from easy. The individual commits are lost and is
generally harder to audit. In some cases the diff is small enough that it’s
okay. In other cases it’s huge and not having access to the commits is a pain. I
can also totally see myself get confused about what I did and didn’t audit. I
guess the number of people doing this sort of auditing is very low because it’s
just such a pain.

This is not unique to Ruby: this is how many (most?) package systems work.

Almost all of the “side-channel attacks” I’ve seen are perhaps more accurately
described as “package publishing attacks”. They rely on injecting something in
the “publish a package” step. Whether that’s RubyGems, npm, PyPI, .tar.gz FTP
downloads, or something else is a relatively minor detail. It’s rare that the
actual source repo gets compromised as it’s just too visible. You need to at
least slightly hide your exploit for it to be effective.

The recent npm compromises all relied on gaining access to the npm account and
injecting something in the published package. xz had some exploit code in the
source repo but was inert, hidden in a binary test file and only activated in
the modified .tar.gz release. Back in 2018 event-stream added a dependency on
flatmap-stream, which had nefarious code in index.min.js only on the published
npm package.

Which points to a second problem: packages containing “compiled” resources. The
JavaScript that TypeScript generates is not completely unreadable, it’s
certainly much less readable and auditable than the original TypeScript. To
say nothing of minified files or binary blobs. This is less of a problem in
Ruby, but a far bigger problem in npm.

Last week RubyGems added a cooldown option and “AI-assisted vulnerability
scanning against the most critical gems”. Not a bad thing to do as a short-term
move, but I feel a more appropriate solution would be to reconsider the entire
“publish a package” model. It just lacks the required transparency. AI tools are
not going to magically fix that.

I probably would have created something similar to RubyGems myself twenty years
ago. Distributing .tar.gz files on SourceForge was by and large how things
worked and many projects did not have a publicly accessible VCS repo, or did not
have one at all. Generally this worked fairly well at the time. I’m not blaming
any one here – I would have done the same. RubyGems is just designed in a
different world.

I’m not saying RubyGems and npm need to copy what Go does exactly in every
respect and I’m not saying all of this is completely perfect either. But as far
as I know, it’s the best anyone has come up with thus far. Some other aspects of
Go modules (such as Minimal Version Selection) are less important.

I appreciating that completely changing how this works is hard and potentially
disruptive. But dealing with this endless stream of hijacked packages is also
hard and disruptive. So…

In the specific case of Bundler, there is already some support for this. You
can do:

gem 'rails', git: 'https://github.com/rails/rails.git', tag: 'v8.1.2'

And it will fetch the rails gem from git, but will still fetch dependencies from
rubygems.org. Maybe there is some way to cajole Bundler in to using git for
everything, but it’s an uphill battle and easy to accidentally use rubygems.org.

Just thinking out loud here, but something like this would probably go a long
way, and won’t break existing Gemfiles:

# Do not allow fetching anything from rubygems.org;
# changes gem() behaviour to use git.
must 'use-git'

gem 'github.com/rails/rails', 'v8.1.2'

# Indirect (automatically written and updated by "bundle install" and similar)
indirect do
    gem 'github.com/rails/actionview', 'v8.1.2'
    gem 'github.com/fxn/zeitwerk',     'v2.8.2'
    # ... etc...
end

Gemfile.lock can still be used for hashes (similar to go.sum), but is otherwise
not all that useful.

There’s a bunch of details to be sorted out here and I’m not pretending those
are straight-forward to sort out well, but there does seem to be a reasonable
path there, I think?

Or maybe something else entirely. I don’t know. The main point is: I want to
reliably audit my dependencies like a responsible developer and RubyGems makes
it too hard, as do other package managers (but I don’t care about them).

Defend against frontier cyber models: Cloudflare’s architecture as customer zero

Post Syndicated from Rohit Chenna Reddy original https://blog.cloudflare.com/frontier-model-defense/

A few weeks ago, we wrote about Project Glasswing and what we observed when we pointed cyber frontier models at our own code. Since then, we’ve seen that the part of the post that has resonated most deeply is the argument that the architecture around the vulnerability matters more than the speed of the patch.

In the conversations we’ve had with CISOs and security teams since, the questions have been consistent: what does our architecture actually look like, what should we monitor for, where do we start, and how can Cloudflare help?

Before getting into the details: the architecture below is built almost entirely from Cloudflare’s own products, because Cloudflare security is customer zero for the security products we build. The Cloudflare stack already exists in front of our code, employees, and customer-facing applications. If you’re a Cloudflare customer, every layer below is available to you today. If you’re not, the principles still apply to whatever stack you’ve built.

What a cyber frontier model actually changes

In the previous post, we showed how a cyber frontier model like Mythos changes the attacker’s timeline. It can find vulnerabilities, reason through exploit chains, and generate working proofs faster than earlier models. While models like Mythos do not change the shape of an intrusion — reconnaissance, initial access, lateral movement, persistence, and exfiltration still have to happen — the difference is in the speed and scale. When pointed at the open web, a model can find and hit low-hanging fruit quickly. Against a hardened target, it still has to probe, and adapt, and it often produces more noise than a careful human operator would.

Discovery, exploit chain construction, and proof-of-concept generation used to be the gating constraints on producing a working attack. A frontier model handles all three in a fraction of the time. Work that used to be slow and methodical is now fast and indiscriminate.

While AI is accelerating how fast developer teams at Cloudflare and many other companies can ship code, the security team’s work has not compressed the same way. An attacker only needs one opening to get in, while security teams need to find and close them all. Writing a fix, regressing it, and shipping it without breaking the code around it has constraints that AI doesn’t remove. We learned this the hard way when we let an AI coding assistant write its own patches against our own bugs, as we described at the end of the previous post. Some of those patches fixed the original bug while quietly breaking something else the code depended on.

As these models become more competent and capable, our main focus from a threat standpoint comes down to three things. Each one shapes the architecture we walk through in the rest of this post.

  • The first is the speed of discovery. Frontier models make it easier to search large bodies of public code, including the open-source libraries that many companies depend on. That does not mean every bug in a library is exploitable, or that library bugs are where most vulnerabilities live. Exploitability still depends on how the code is used, whether attacker-controlled input can reach the vulnerable path, and the protections that sit around it. But widely used open-source libraries and frameworks give attackers a shared surface to study at scale. When a real, reachable vulnerability exists there, a model can help find it, reason about possible exploit paths, and generate proof-of-concept variants faster than maintainers and defenders can review every downstream use. The gap between when an attacker discovers a vulnerability and when defenders learn it exists is what worries us most. If you are not running these models against your own code, it is safe to assume someone else is.

  • The second is exploit volume and adaptation. A model can produce thousands of variations of a single exploit and run reconnaissance at the same scale. All that volume gives an attacker an advantage, but it won’t necessarily get them past signature-based detections. Many of those iterations will have the same underlying signature, so a rule that catches the first one will catch the rest. Adaptation is how they will get past signature-based detections. Ask a model to show you a SQL injection, and it will return a textbook example. Tell it there is a WAF in the way, and it will start probing, learning what gets blocked, and rewriting the payload until it can slip past the rule blocking it.

  • The third is the impact when a vulnerability is inevitably exploited. No architecture catches everything. After the vulnerability is exploited, the question we ask ourselves is: where can the attacker get to with one identity, one path, or one credential, before something else stops them? If the answer is “anywhere they want,” the vulnerability was never the problem. The architecture around the vulnerability was.

Cloudflare’s superpower: visibility

We see roughly a fifth of the world’s web traffic and that traffic tells us, in real time, which payloads are mutating, which patterns are picking up, and where attacker tooling is moving next. Two teams turn that visibility into defense.

First is Cloudforce One, our threat intelligence, research, and operations team, which sits within the Cloudflare security organization. They turn what we see across the network into insights the rest of the stack can act on: tracked adversaries, emerging campaigns, and indicators of compromise (IOCs). The hard part of this work was never knowing what is malicious — it was the delay in mitigation. Knowledge of a new threat normally has to travel from a threat report, into a feed, and then into a company’s defense before it can be used to block anything. Attackers have learned to move faster than that. Our network closes that gap: Cloudflare customers can now use Cloudforce One threat intelligence directly within the WAF to block high-risk traffic.

Second is the team that owns the WAF engine that does the actual detecting: the managed rulesets that run in front of our own properties and are available to every Cloudflare customer, the machine learning behind WAF Attack Score, and the relationships that sometimes let us ship a rule before a CVE is publicly disclosed. The team is globally distributed and moves fast, releasing rules within hours of a proof-of-concept of an attack becoming known. Once a detection is deployed, it reaches our entire network, along with every Cloudflare customer, in under 30 seconds. React2Shell is a recent example: a managed WAF rule was protecting our own properties, and everyone else’s on Cloudflare, hours before the official advisory was published.

The scoring layer, the defenses we put in front of the application, and the containment around the vulnerability all build on what these two teams see. 

Scores over signatures

Signature-based defenses were built for a world where novel exploits were scarce and variations took weeks. Cloudflare’s traditional SLA from a fresh proof-of-concept to a live, deployed rule has been 12 hours. With the advent of frontier models, this is not good enough anymore. Detections need to be in place before a CVE is discovered. This is why we layer ML-based detection in front of the traditional signature-based WAF.

The model is trained on a large body of past attack traffic, and it catches new variants of vulnerabilities before they’re publicly known. A novel SQL injection or remote code execution chain is almost always a rearrangement of attack shapes the model has seen before, even when the specific exploit is brand new. We run the model on every request and assign a WAF Attack Score between 1 and 99, based on how closely the request resembles those underlying shapes, not against a list of known-bad signatures. The lower the score, the more aggressively we treat the request. That score determines whether we let the request through. We apply a similar scoring methodology to AI prompts with AI Security for Apps: rather than check each prompt against a list of known malicious prompts, we score how closely a prompt resembles an actual attack. 

The architecture around the vulnerability

Those capabilities only matter once they’re stacked in front of an application, and the first layer in our defense-in-depth approach is the WAF. Anything that matches a known-bad pattern gets dropped before it reaches the application, which clears the bulk of the obvious traffic and lets the more specialized layers below focus on what’s left.

On the API surface, we run a positive security model through API Shield. Instead of trying to anticipate every bad request, we describe what a valid request to each API looks like, either from the API’s own definition or learned from our real traffic, and anything that doesn’t fit doesn’t get through. This neutralizes the advantage of frontier AI models: because we only permit validated traffic, generating thousands of new attack variations fails to bypass the system.


Cloudflare’s layered architecture

Bot Management catches probing traffic on our network before frontier models can build a map. It scores every request on how likely it is to be automated, using the same signals across our whole network: how the client behaves, whether it looks like a real browser, and whether the connection matches a known-bad pattern. An attack only lands if it can find a soft spot. 

Zero Trust Network Access is used for every internal application. The implicit trust of being inside the network is replaced with explicit per-request identity and policy for every employee accessing every tool. The value of this was clear when one of our engineers shipped a misconfigured tool. A flat network would have exposed everything on the same segment, but in our deployment, the exposure stopped at the tool itself. We built Require Access Protection afterwards so newly deployed or misconfigured applications can’t be reachable before an access policy is in place.

IdP Federation makes that secure by default posture easier to keep consistent across every Cloudflare account — which becomes even more necessary when more people are shipping internal tools quickly. Instead of asking each team to wire up SSO separately, we configure our identity provider (IdP) once and share it across the organization. New accounts get SSO automatically, recipient-side IdP connections are read-only, and Access policies in each account still evaluate the resulting identity as part of the normal request flow. 

MCP Server Portal gives teams a controlled way to connect AI agents to enterprise systems. Agents access MCP servers that are centrally managed through a single portal, with every action logged. That way when an agent acts on someone’s behalf, we know what it did, what it touched, and whether it should have been allowed to. The full picture of how we built it is in our post on enterprise MCP.

AI Gateway runs in front of our internal AI tools the same way AI Security for Apps runs in front of customer-facing AI features, with the same scoring and the same visibility. Inside the company, the visibility piece is more useful than the blocking, because we needed to see what engineers were actually building before we could write meaningful policy on it.

Where your teams can start 

Frontier models can help attackers find vulnerabilities, adapt payloads, and move faster, but they still have to pass through the layered defense you deploy in front of your application. That is where teams should start:

  • Put inspection in front of public applications.

  • Define what valid API traffic looks like.

  • Use bot detection to limit automated probing.

  • Require identity and access policy before any internal tool is reachable.

For AI and agentic systems:

  • Route model traffic through a gateway.

  • Keep agents connected through approved MCP servers.

  • Log what they do. 

The goal is to make sure that when one layer misses, the next layer limits what the attacker can see, reach, or change.

That is the point of the architecture around the vulnerability: to limit the scope of an attack. The vulnerability may be what starts the attack, but the architecture determines how far it can go.

How do we know this approach works?

Plenty of security stacks look impenetrable on a whiteboard but fall over in practice. That is why we test ours continuously, both at the perimeter and inside our environment, with our red team involved across both.

At the perimeter, frontier models are one tool we use to test our application security stack as an adaptive attacker. These models sit alongside the rest of our red team and detection workflows including: manual testing, threat intelligence, observed traffic patterns, proof-of-concept analysis, and signals from our own network. Together, those inputs help us decide where to aim testing: newly launched products, recently changed surfaces, and the paths an attacker is most likely to probe first. The most important part is the process that follows. When something gets through, we identify the gap, use the right mix of tools to understand it, write the rule or mitigation, ship the update, and test again to make sure the gap is closed.

Inside the environment, our red team starts from the assumption that the perimeter has already failed. They look at what has changed, where sensitive systems carry risk, and whether one compromised identity, path, or credential can reach farther than it should. When we change the architecture based on what they find, they run the scenario again against the new version to confirm the gap is actually closed.

We confirm that this architecture is working by continuously testing its behavior during failures, rather than relying on the perfection of individual layers.

If your team is working on the same problems and would like to compare notes, reach out to us at [email protected].

Operationalizing AWS security: A maturity roadmap

Post Syndicated from Joseph Sadler original https://aws.amazon.com/blogs/security/operationalizing-aws-security-a-maturity-roadmap/

Enabling security tooling is the starting point. Making it operational—where findings drive decisions, response times are measurable, and your security posture improves week over week—is where most organizations struggle.

This blog post provides a phased maturity roadmap for organizations that have already enabled AWS Security Hub and Amazon GuardDuty. These two services form the foundation of a cloud-centered security operations capability on AWS. Security Hub provides centralized security posture management and aggregates findings from multiple AWS security services, while GuardDuty provides intelligent threat detection by continuously monitoring for malicious activity and unauthorized behavior. For any production or enterprise AWS environment, having both services enabled across all accounts and AWS Regions is a baseline expectation; not because they’re optional add-ons, but because effective security operations require both the ability to detect threats and the ability to understand your overall security posture. If you haven’t yet enabled them, the Security Hub documentation and GuardDuty documentation provide setup guidance, including multi-account deployment with AWS Organizations.

Customers consistently tell us that while individual AWS security service documentation is thorough, what’s missing is a consolidated operational playbook—one resource that ties the services together into a working security operations practice with clear phases, progression criteria, and an operational cadence. That’s the gap this post fills. Rather than covering how each feature works (the documentation does that well), this post focuses on when and why to use each capability, and how to build the organizational habits that make them effective.

What follows is a six-phase roadmap for moving from these services are active to these services are driving our security operations. Each phase builds on the previous one, and each is designed to deliver tangible, measurable improvement.

Phase 0: Assess your current state

Goal: Understand what’s working before changing anything.

Estimated timeline: 1–2 weeks

Move to Phase 1 when: You have a documented current-state assessment covering all the following items.

Before introducing new processes or automation, establish a clear picture of the current environment. This assessment informs every decision that follows.

Actions:

  • Findings inventory: Review existing active GuardDuty findings to determine how many there are, the severity distribution, and how old the oldest findings are. A large backlog of untouched HIGH or CRITICAL findings that have been sitting for weeks is a strong signal about where to focus first.
  • Security Hub score baseline: Determine your current compliance score against AWS Foundational Security Best Practices (FSBP) and The CIS AWS Foundations Benchmark. Check to see which standards are enabled; if multiple standards are enabled, review for overlapping standards (creating noise) or unused standards.
  • Multi-account and multi-Region check: Look to see if GuardDuty is enabled in every account and every Region, or only in Regions with active workloads. Threat actors frequently operate in Regions that organizations don’t actively monitor. Also check to see if Security Hub aggregation is configured with a delegated administrator account or if each account is being managed independently.
  • Integration check: Determine if GuardDuty findings are flowing into Security Hub and if Amazon Inspector and Amazon Macie are enabled and feeding findings in. Without integration, Security Hub might be only surfacing its own compliance checks.
  • Notification check: See if there’s an Amazon EventBridge rule configured for notifications and if so, how findings are being routed and to whom. Know if notifications are being sent using an Amazon Simple Notification Service (Amazon SNS) topic or a chat channel integration. Without a clear notification and response workflow, findings can accumulate silently in the console with no one looking at them.

Deliverable: A one-page current state assessment that identifies what’s enabled, what’s flowing where, who’s looking at it, and what’s in the existing backlog.

Phase 1: Reduce the noise

Goal: Make the signal meaningful before asking anyone to act on it.

Estimated timeline: 2–3 weeks

Move to Phase 2 when: Remaining findings represent items requiring real decisions, compliance scores reflect actual posture, and you can articulate why every suppression rule and disabled control exists.

This is the single most important phase. If this step is skipped in favor of jumping straight to automation, the result is automated chaos. Alert fatigue is the primary reason security tooling is ignored, and addressing it first is what makes everything that follows sustainable.

GuardDuty tuning:

  • Create suppression rules for known-benign findings. The goal is to suppress activity you’ve already evaluated and accepted—such as expected traffic from corporate egress IPs (based on trusted IP lists), internal tools that trigger DNS-based findings, or internet-facing resources that naturally receive port scanning. The principle: if you’ve investigated a finding and it’s expected, suppress it so your team can focus on what matters.
  • Triage every active HIGH and CRITICAL finding into three categories: needs immediate investigation (real threat, not yet reviewed), true positive, already addressed (archive using workflow status), or false positive or expected behavior (create a suppression rule). Every finding must be categorized into one of these three states.
  • Review GuardDuty protection plans and enable any that are relevant but not yet active. Organizations that enabled GuardDuty years ago might not have activated protection plans released since then (such as Runtime Monitoring, Malware Protection, RDS Protection, and Lambda Protection). Evaluate each against your workload profile and enable what applies.

Security Hub tuning:

  • Disable controls that aren’t relevant to the environment. This is the highest-value quick win. If a service isn’t in use, disable its controls. If a control is addressed by an alternative solution, disable it. A 47% compliance score where half the failures are irrelevant trains teams to ignore the dashboard entirely. See the Security Hub controls reference for the full list.
  • Choose a primary standard. AWS Foundational Security Best Practices is a strong default. The CIS AWS Foundations Benchmark adds value when there’s a specific compliance mandate. Avoid enabling PCI DSS or NIST 800-53 standards unless there’s a reporting requirement—they add significant volume without proportional signal for most organizations.
  • Configure cross-Region aggregation to the delegated administrator account if not already in place. A single aggregated view eliminates the need to check findings across multiple Regional consoles.
  • Use the workflow status field operationally. Findings should progress from NEW to NOTIFIED to RESOLVED or SUPPRESSED. If everything remains in NEW indefinitely, the system carries no operational meaning.

Deliverable: A tuned environment where remaining findings represent items that require real decisions. Compliance scores should now reflect your organization’s actual security posture rather than noise.

Phase 2: Build the notification and routing layer

Goal: Get the right findings to the right people at the right time.

Estimated timeline: 2–3 weeks

Move to Phase 3 when: CRITICAL and HIGH findings reach the security team within minutes, MEDIUM findings create tracked tickets, and notifications include enriched context. No action is taken until a person or an automation is informed that something needs attention.

Architecture: Security Hub to EventBridge rule to routing logic to destination

Tiered notification strategy:

CRITICAL Page on-call immediately PagerDuty or Opsgenie 15 minutes
HIGH Alert security team channel Slack or Teams channel and ticket creation 4 hours
MEDIUM Create ticket for review Jira or ServiceNow 48 hours
LOW or INFORMATIONAL Batch digest Weekly email summary or dashboard review Next review cycle

Key design decisions:

  • Route from Security Hub, not individual services. Because findings from GuardDuty, Inspector, Macie, and Security Hub compliance checks all aggregate in Security Hub, build your EventBridge rules there for centralized management.
  • Create a fast path for the most dangerous finding types. Certain GuardDuty findings, particularly those involving credential exfiltration, cryptocurrency activity, trojans, and active compromises, warrant a separate, faster routing path that bypasses normal triage. Identify these based on your threat model and the GuardDuty finding types reference.
  • Enrich notifications before delivery. A raw JSON finding in a chat channel provides little actionable context. Use an AWS Lambda function to format notifications with the information responders need: account alias, Region, Amazon Resource Name (ARN), finding type, severity, a console deep link, and a plain-language description. The Security Hub CloudWatch Events integration guide describes the event format.

Deliverable: A working notification pipeline where CRITICAL and HIGH findings reach the security team within minutes, MEDIUM findings create tracked work items, and LOW and INFORMATIONAL findings are batched for periodic review.

Phase 3: Build automated remediation for high-confidence findings

Goal: For findings where the correct response is deterministic, remove the human from the loop.

Estimated timeline: 3–4 weeks

Move to Phase 4 when: At least 3–5 high-confidence finding types have automated responses deployed with audit trails, and the team has established a process for evaluating new auto-remediation candidates.

The guiding principle: Only auto-remediate when all three conditions are met: the finding is high-confidence, the response is deterministic, and the blast radius of the automated action is limited. Automated remediation must not create the risk of a production outage.

Decision framework:

Confidence level High – no false positive risk Medium – context-dependent Low – requires investigation
Response complexity Single, well-defined action Multiple steps or judgment calls Requires forensic analysis
Blast radius Limited to one resource Could affect dependent services Production-wide impact
Rollback difficulty Straightforward to reverse Moderate effort to reverse Difficult or impossible to reverse

Common auto-remediation categories:

  • Instance isolation for confirmed compromise findings (cryptocurrency mining, malware, and trojans): Replace the security group, snapshot volumes for forensics, and notify.
  • Credential revocation for confirmed credential compromise: Attach deny-all policies, revoke sessions, and deactivate access keys as appropriate to the credential type.
  • Compliance drift correction for deterministic misconfigurations: Re-enable Amazon Simple Storage Service (Amazon S3) Block Public Access, revoke overly permissive security group rules, and re-enable AWS CloudTrail logging.
  • Notification-only escalation for findings that require human judgment before action: Amazon Elastic Block Store (Amazon EBS) encryption gaps (require migration) and access key rotation (requires coordination with the key owner).

For implementation, AWS provides Security Hub Automated Response and Remediation (SHARR), a solution that includes pre-built remediation playbooks deployed as AWS Step Functions workflows triggered by EventBridge. This is a strong starting point—evaluate the provided playbooks, enable the ones that fit, and extend with custom remediations as needed.

Note: For findings that recur because the environment lacks preventive guardrails, the best long-term response is often a service control policy (SCP) that prevents the misconfiguration from occurring in the first place. Phase 5 covers this preventive controls layer.

Deliverable: A library of automated and semi-automated remediation runbooks with full audit trails, and a documented decision framework the team uses to evaluate new auto-remediation candidates.

Phase 4: Build the operational rhythm

Goal: Turn security findings management into a sustained organizational practice, not a one-time cleanup.

Estimated timeline: 4–6 weeks to establish, then ongoing

Move to Phase 5 when: The weekly cadence has been running consistently for at least 8 weeks, monthly metrics show positive trends, and the first quarterly review has been completed.

This is where many organizations stall, and it’s the most important phase in the entire roadmap. The technology is working, the notifications are flowing, automated remediations are firing, but there’s no organizational habit built around it. Without this phase, everything you’ve built in Phases 0–3 will gradually decay. Suppression rules will go stale, new team members won’t know the system exists, and findings will start accumulating again. The operational rhythm is what converts a security tooling deployment into a security operations practice.

Weekly security review (30 minutes)

Attendees: Security team lead, cloud platform team representative, rotating engineering lead from an application team

Why the rotating engineering lead matters: Security findings don’t exist in a vacuum; they’re generated by workloads that engineering teams own. Rotating an engineering representative through this meeting accomplishes three things: it builds security awareness across the organization, ensures findings are routed to people with the context to resolve them, and creates organizational accountability beyond the security team.

Agenda template:

5 minutes Compliance score trend – Review Security Hub scores by account and standard. Is the trend improving, declining, or flat? If declining, why? Security lead Identified regression areas
5 minutes Critical and high findings review – Walk through new HIGH and CRITICAL GuardDuty findings from the past week. Are there any that need immediate escalation? Security lead Escalation actions assigned
10 minutes Top five failing controls – Identify the five Security Hub controls with the most failures. Assign an owner and a target date for each. Platform lead Owners and dates documented
5 minutes Automation review – Did any auto-remediations fire this week? Did they work correctly? Were there any false triggers? Security lead Automation adjustments queued
5 minutes Tuning decisions – Are new suppression rules needed based on this week’s findings? Are any new finding types candidates for auto-remediation? All Tuning backlog updated

Running the meeting effectively:

  • Keep a running document (such as a wiki page or shared document) that captures decisions and action items week over week. This becomes your institutional memory.
  • If the compliance score hasn’t moved in over 3 weeks, that’s a signal. Either the assigned work isn’t happening, or the remaining findings are genuinely difficult to address. Both need to be discussed.
  • Track action items from previous weeks. A review that generates action items but never follows up on them will lose credibility and attendance quickly.

Escalation procedures

Define clear escalation paths before they’re needed:

CRITICAL finding not acknowledged within the SLA Auto-escalate to security team manager 15 minutes after SLA breach
HIGH finding not resolved within the SLA Escalate to finding owner’s manager 4 hours after SLA breach
Compliance score drops more than 5 points in a week Escalate to cloud platform team lead for investigation Next business day
Auto-remediation failure Page security on-call Immediate
New finding type not covered by existing runbooks Add to weekly review agenda for triage and runbook development Next weekly review

Monthly metrics report

Compile these metrics monthly and review them with security and engineering leadership. The goal is to tell a story about whether the organization’s security posture is improving, stable, or degrading, and why.

Mean time to acknowledge (MTTA) for CRITICAL findings Are findings being seen promptly? Decreasing month over month
Mean time to resolve (MTTR) for CRITICAL and HIGH findings Are findings being acted on? Decreasing month over month
Security Hub compliance score by standard, by account What is the posture trend over time? Increasing month over month
Number of active GuardDuty findings by severity Is the backlog growing or shrinking? Decreasing for HIGH and CRITICAL
Findings auto-remediated compared to manually resolved Is automation delivering value? Auto-remediation ratio increasing
Number of suppressed findings (with quarterly justification review) Is noise being managed, or are problems being hidden? Stable or decreasing
New findings introduced compared to resolved this month Is the organization getting ahead or falling behind? More finding resolved than introduced
SLA adherence rate by severity Are response commitments being met? More than 95% for CRITICAL, and more than 90% for HIGH

Building the dashboard: Use Amazon CloudWatch dashboards for real-time operational visibility or Amazon QuickSight connected to Security Hub findings through Amazon Security Lake for historical trend analysis and executive reporting. The dashboard should be visible to—and regularly viewed by—everyone in the weekly review, not locked in a security team tool.

Quarterly reviews

The quarterly review is a deeper inspection of the system itself; not just the findings, but the machinery processing them.

Quarterly review checklist:

  • Suppression rules audit: Review every active suppression rule to determine if the underlying condition is still present and the suppression is still justified. Document the review outcome for each rule.
  • Disabled controls audit: Review every disabled Security Hub control. Check that the justification is still valid and if the environment changed (for example, a service that wasn’t in use is now in use).
  • Automation audit: Review AWS Identity and Access Management (IAM) roles used by remediation functions and verify least privilege. Review execution logs for any anomalies or failures that weren’t caught.
  • New capabilities review: Evaluate newly released GuardDuty protection plans and Security Hub controls from that quarter. AWS releases new detection and compliance capabilities regularly. If you’re not reviewing them quarterly, you’re accumulating blind spots.
  • Process effectiveness review: Determine if the weekly meeting is well-attended and if action items are being completed. Make sure SLAs are being met. If attendance, action item completion, and SLA compliance aren’t where they should be, explore structural changes to address the gaps.

Operational maturity scoring

Use this rubric to assess the maturity of your operational rhythm itself. Score each dimension 1–3 and use the total to track progress over time.

Review cadence One time reviews when someone remembers Weekly review happens, but attendance is inconsistent Weekly review is consistently attended with documented outcomes
Metrics tracking No metrics captured Metrics are collected monthly but not acted on Metrics drive decisions and declining trends trigger specific actions
Finding ownership Findings sit in queue with no owner Findings are assigned to teams but SLAs aren’t tracked Every finding has an owner, SLAs are tracked, and breaches are escalated
Automation management Set-and-forget automations Automation logs are reviewed periodically Automation is reviewed weekly, and new candidates are evaluated continuously
Tuning lifecycle Suppression rules created but never reviewed Annual review of suppressions and disabled controls Quarterly reviews with documented justification for every rule
Cross-team engagement Security team works in isolation Platform team participates Engineering teams actively participate and own remediation

Scoring (revisit quarterly):

  • Beginning: 6–9
  • Established: 10–14
  • Optimized: 15–18

Deliverable: A documented operational cadence with clear ownership (consider a RACI matrix), metrics dashboards, escalation procedures, and a continuous improvement loop. The cadence should survive team member turnover—if it depends on one person remembering to run it, it’s not yet operational.

Phase 5: Mature the architecture

Goal: Fill remaining gaps and build toward a comprehensive security operations capability. Estimated timeline: Ongoing. Prioritize based on organizational risk profile and compliance requirements.

  • Amazon Inspector integration: Enable Amazon Inspector for Amazon Elastic Compute Cloud (Amazon EC2) instances, Lambda functions, and Amazon Elastic Container Registry (Amazon ECR) container images. Findings flow into Security Hub automatically, adding vulnerability management alongside threat detection and posture management. Prioritize this if you have Amazon EC2 or container workloads without an existing vulnerability scanning solution.
  • Amazon Macie: Enable Amazon Macie for S3 buckets containing potentially sensitive data. Particularly important for organizations with compliance requirements around personally identifiable information (PII), protected health information (PHI), or Payment Card Industry (PCI) data. Configure automated sensitive data discovery and route findings to Security Hub.
  • Amazon Security Lake: Amazon Security Lake centralizes security-relevant logs in OCSF format for long-term retention, forensic investigation, and threat hunting. This is valuable when you need historical analysis beyond the Security Hub retention window, or when feeding a third-party Security Information and Event Management (SIEM) tool.
  • Preventive controls layer: Convert recurring detective findings into preventive policies. Use SCPs to prevent disabling GuardDuty, Security Hub, and CloudTrail, IAM permission boundaries on developer roles, AWS WAF on public endpoints, and AWS Network Firewall for VPC traffic inspection. The pattern is to make recurring misconfigurations impossible to introduce.
  • Detective controls expansion: Use AWS IAM Access Analyzer for external access and unused access findings, AWS CloudTrail Lake for long-term queryable audit logs, and AWS Config custom rules for organization-specific compliance checks.
  • Incident response readiness: Have incident response playbooks referencing specific GuardDuty finding types, pre-built forensics infrastructure (isolated VPC, forensic AMIs, and pre-configured IAM roles), regular tabletop exercises, and AWS CloudFormation templates to deploy isolation infrastructure on demand. See the AWS Security Incident Response Guide for a comprehensive framework.

Conclusion

In this post, I provided a six-phase roadmap for operationalizing Security Hub and GuardDuty and showed that it isn’t a single project, but a progression. Phase 0 and Phase 1 can typically be completed in 3–5 weeks and deliver immediate clarity. Phases 2 and 3 build the response infrastructure that turns findings into action over the following 5–7 weeks. Phase 4 is what makes everything sustainable and is where you should invest the most attention. And Phase 5 expands the aperture from Security Hub and GuardDuty into a comprehensive security operations capability.

If you walked away from this post and did one thing, run the Phase 0 assessment this week. That single deliverable tells you exactly where to focus next. Use the following self-assessment checklist to identify your current phase, then focus on the next one. A tuned environment with working notifications and a weekly review cadence is dramatically more effective than a fully featured but neglected deployment. Start where you are, reduce the noise, build the habits, and iterate. To learn more, explore the AWS Security Hub User Guide, the Amazon GuardDuty User Guide, and the AWS Security Incident Response Guide. If you’ve implemented a similar operational cadence, or have questions about any phase, share your experience in the comments.

Self-assessment checklist

Phase 0 We know how many active GuardDuty findings exist across all accounts ☐
We know our current Security Hub compliance score ☐
We know whether GuardDuty is enabled in every account and region ☐
We know who (if anyone) is reviewing findings today ☐
Phase 1 GuardDuty suppression rules exist for known-benign activity ☐
Irrelevant Security Hub controls have been disabled with documented justification ☐
All active HIGH and CRITICAL findings have been triaged ☐
Security Hub compliance scores reflect actual posture, not noise ☐
Phase 2 HIGH and CRITICAL findings generate real-time notifications to the security team ☐
MEDIUM findings automatically create tracked work items ☐
Notifications include enriched context (account alias, resource ARN, and console link) ☐
Phase 3 At least three high-confidence finding types trigger automated remediation ☐
Auto-remediation actions have full audit trails ☐
Remediation runbooks are documented and version-controlled ☐
Phase 4 A weekly security review meeting occurs with defined attendees and agenda ☐
MTTA and MTTR are tracked monthly for CRITICAL and HIGH findings ☐
Suppression rules and disabled controls are reviewed quarterly ☐
Security metrics trend positively over the past 3 months ☐
Phase 5 Amazon Inspector, Macie, or Security Lake are integrated ☐
Preventive controls (SCPs, permission boundaries) address recurring findings ☐
Incident response playbooks exist and are tested through tabletop exercises ☐

If you have feedback about this post, submit comments in the Comments section below.

Joseph Sadler

Joseph Sadler

Joseph is a Senior Solutions Architect on the Worldwide Public Sector team at AWS, specializing in cybersecurity and machine learning. With public and private sector experience, he has expertise in cloud security, artificial intelligence, threat detection, and incident response. His diverse background helps him architect robust, secure solutions that use cutting-edge technologies to safeguard mission-critical systems

Turning Cloudflare’s threat indicators into real-time WAF rules

Post Syndicated from Alexandra Moraru original https://blog.cloudflare.com/realtime-threat-intel-waf-rules/

Cloudflare’s Threat Events provides security analysts with a window into the global threat landscape. The platform offers a peek into the immense traffic that Cloudflare processes every day, so you can see in real time which IPs are attacking specific industries or which threat actors are trending globally. However, translating that visibility into active mitigation has often been a manual, reactive process.

Security teams have faced a recurring frustration: knowing that certain IP addresses were associated with specific threat actors (like Tycoon 2FA or RaccoonO365) or had been seen targeting their specific industry in other regions, but they couldn’t easily automate the blocking of these high-risk IPs within their own WAF unless they manually configured the rules. 

We are excited to announce a new integration that brings Cloudflare’s vast threat intelligence directly into your WAF engine: you can now write proactive rules using live intelligence data. This means you can add more intelligence context to protect your application against known bad actors — before they even attempt to touch your infrastructure.

By populating specialized fields during the early stages of a request, the WAF can now screen traffic based on:

  • Who is attacking by matching specific threat actor names

  • Who they are targeting via the industry or country filters to see who the IP has targeted in the past

  • What type of attack using enriched threat context, filtering by attack type (DDoS, WAF, cybercrime, etc.) and the timeframe it was last seen

Always-on detection

This new capability is built on the same always-on detection framework we recently introduced for Attack Signature Detection, a system that identifies common attack patterns in real time without requiring pre-configured rules. By separating detection from mitigation, we ensure that threat intelligence is constantly running in the background, enriching your HTTP request analytics with insightful threat metadata before you even decide to take an action.

The primary advantage of an “always-on” model is the elimination of the traditional “log vs. block” trade-off: visibility in log mode, or protection in block mode. That’s because when a rule blocks a request, you lose visibility into how other signatures would have assessed it — insight that could have helped you strengthen your defenses.

If you have a Cloudforce One subscription, these insights appear in your analytics automatically. You can see which threat actors are hitting your site and which industries those IPs usually target, allowing you to verify traffic patterns before “flipping the switch” to block.

These detections execute with negligible latency, ensuring your performance remains lightning-fast while providing the high-confidence data needed to build robust security policies. While this initial release focuses on IP-based matching, we are already looking toward extending these capabilities to JA3 fingerprints and domain-based matching. This will allow you to block malicious traffic even when attackers rotate IPs, by identifying the unique software signatures or malicious destination links they use in their payloads.

New WAF fields

To make this possible, we’ve exposed the following specific signals directly to the WAF engine:

Field

Description

cf.intel.ip.attacker_names

Names of known threat groups (e.g., CRAVENFLEA).

cf.intel.ip.target_industries

Industries targeted by this IP (e.g., Cryptocurrency, Automotive).

cf.intel.ip.attacker_countries

The source country of the threat event.

cf.intel.ip.target_countries

The countries targeted by the threat event.

cf.intel.ip.datasets

The source feed providing the data (e.g., ddos, waf).

Example rule expressions

Because a single IP address could be associated with multiple threat actors or targeted industries simultaneously, these fields are represented as arrays. We use the any() function and [*] wildcard to check whether any value within that threat profile matches your criteria:

  • Block known DDoS participants targeting your region: any(cf.intel.ip.target_countries[*] == "FR") and any(cf.intel.ip.datasets[*] == "ddos")

  • Protect against specific threat actors targeting the Finance sector: any(cf.intel.ip.target_industries[*] == "Banking & Financial Services") and any(cf.intel.ip.attacker_names[*] == "BLACKBASTA")

  • Broad protection against specific high-risk origin countries: any(cf.intel.ip.attacker_countries[*] == "IR")

How to use Threat Events data in your workflows

Whether you prefer a UI-driven approach or Infrastructure as Code, these fields are integrated into your existing workflows.

The WAF rule builder (API & Terraform)

For teams that prefer Infrastructure as Code, the new cf.intel fields are fully integrated into the WAF rule builder for WAF custom rules and rate limiting. You can write complex expressions using the same syntax you use today. Because these are standard WAF fields, they are fully supported via the Cloudflare API and Terraform, allowing you to automate threat blocking across your selected domains or even on your whole account.


New fields added to the WAF rule builder to allow users to choose the relevant configuration based on the Threat Events indicators. 

Visibility in Security Analytics

Deployment is only half the battle. All matches triggered by these threat intelligence fields are logged in Security Analytics. You can drill down into your traffic to see exactly which rule was triggered and which specific indicator matched. These enriched logs allow for faster auditing and postmortem analysis when a rule triggers.


Threat event matches surface in Security Analytics, with full context and a one-click option to create a custom security rule.

One-click rule from the Threat Events dashboard

If you are already using the Threat Intelligence Dashboard to investigate trends, you don’t have to copy and paste IP lists. You can create Saved Views based on your specific filters, such as “IPs seen attacking the Financial sector in the last seven days.” With a single click, you can export these filters directly into a WAF rule.


Saved Views now allow users to easily create WAF rules to match the saved view configuration. 

Global intelligence across our network

Visibility and ease of use are only possible if the underlying engine is fast. How do we handle millions of threat indicators without slowing down your traffic?

These threat intelligence datasets are compressed into a high-performance format and distributed to every single Cloudflare data center globally. When a request hits our network, the Cloudflare WAF performs an O(1) constant-time lookup against these local datasets. This ensures that whether we are checking against ten indicators or ten million, the latency overhead remains effectively zero (measured in microseconds).

Because an IP can be associated with multiple threat vectors, our engine doesn’t stop at the first match. It evaluates the set of all signals associated with that IP simultaneously. This ensures that a rule looking for “Attacker = RU” AND “Target Industry = Banking” will trigger correctly by evaluating the intersection of these attributes in a single pass, providing maximum coverage against multi-vector actors without increasing computational complexity.

Ready to get started?

This feature is available today for customers with any active Cloudforce One subscription:

  • Cloudforce One Essentials allows customers to access the default datasets in Threat Events, search for indicators, and conduct threat-hunting investigations

  • Cloudforce One Advantage allows customers to access our Threat Intelligence Analyst custom insights via requests for information

  • Cloudforce One Elite — our most complete package — includes brand protection, a high number of requests for information, and access to all Threat Events datasets

Ready to turn global insights into local defense? Head over to Threat Events or the WAF section of your Cloudflare Dashboard to start building your first Threat Intel rule, or contact your account team to learn more about subscribing to Cloudforce One.

Why Policy in Amazon Bedrock AgentCore chose Cedar for securing agentic workflows

Post Syndicated from Liana Hadarean original https://aws.amazon.com/blogs/security/why-policy-in-amazon-bedrock-agentcore-chose-cedar-for-securing-agentic-workflows/

Agents have agency: they adapt and find multiple ways to solve problems. This autonomy creates a fundamental security challenge: the large language model (LLM) at the heart of the agent is non-deterministic, and its decisions can’t be predicted or guaranteed in advance. It can hallucinate harmful actions with complete confidence. It’s vulnerable to prompt injection attacks, where adversaries inject malicious commands through tool responses or user inputs. LLMs don’t robustly differentiate between commands and data, everything is only tokens. For these reasons, if you want defense in depth, you must treat the LLM as an untrusted actor from a security point of view.

The insight is that the LLM can’t affect the external world directly: it has to go through an orchestrator that invokes tools based on the LLM’s output. This is precisely where the controls must be applied. What you need at this boundary is authorization: a decision about whether each tool invocation should be allowed and under what conditions. Consider a customer service agent for an online retailer. Without proper controls, it could process refunds that exceed authorized limits, apply discounts to product categories that should be excluded, or look up one customer’s data while handling another customer’s session.

If you control agents’ access to tools, you can establish a safety envelope within which the agent can operate freely. This differs from two common but unsatisfactory approaches:

  • Creating hard-coded workflows eliminates uncertainty, but by itself defeats the purpose of using an LLM as the brain of the agent, because you’ve built a traditional application with an LLM interface. And even with this restriction, using LLM outputs at any step can open up the same risks. While it’s a useful technique for well-understood workflows, it’s not sufficient for agents that need to adapt.
  • Human-in-the-loop provides a safety net for critical operations, and it will always have a role. But relying on it as the main control mechanism sacrifices autonomy and can lead to approval fatigue.

You need agents that are safe and autonomous. This requires an auditable, deterministic enforcement layer that sits outside the agent and tools. Why outside? Because the LLM’s plan is the thing you can’t trust—it can’t be responsible for enforcing its own constraints. Controls at the LLM layer—such as system prompts and training-time alignment—can be bypassed by prompt injection or hallucination. Hard-coded checks in agent or tool code are more robust, but become difficult to audit and manage at scale, especially when security logic is scattered across many tools and services. Centralizing authorization outside both gives you a single checkpoint the LLM can’t circumvent; one that’s auditable and can be verified independently of the application code.

This is where AgentCore Policies come in. Amazon Bedrock AgentCore Gateway sits between the agent and the remote tools it calls. When you associate a Policy with a Gateway, it blocks everything by default. Policies selectively open this boundary by specifying which tool invocations are allowed and under what conditions. This enforcement applies to all tool traffic routed through the Gateway. For this approach to scale, it must be more straightforward to reason about the policies than about the agent’s behavior.

AgentCore policies are expressed in Cedar. Cedar is an open source authorization policy language developed by AWS that has recently joined the Cloud Native Computing Foundation (CNCF). Cedar was designed with exactly these properties: it’s purpose-built for authorization, readable by humans, and analyzable by machines using automated reasoning. This gives enterprises the ability to scale policy definition and enforcement to their AI agents.

How Cedar is used by Amazon Bedrock AgentCore

Amazon Bedrock AgentCore provides the infrastructure to deploy and manage agents at scale. It includes AgentCore Runtime for hosting agents, AgentCore Gateway for managing how agents connect to tools using Model Context Protocol (MCP), and Policy in AgentCore. Policy intercepts all agent traffic through AgentCore gateways and evaluates each request against defined policies in the policy engine before allowing tool access. Cedar powers the policy layer.

AgentCore Policy uses Cedar and its mathematical analysis capabilities at several points in the AgentCore Gateway workflow: the Cedar authorization engine is used at policy evaluation and Cedar Analysis is used during policy authoring, and in the control plane.

Policy authoring: Developers can write Cedar policies directly or use natural language that gets translated to Cedar through a neuro-symbolic AI feedback loop. Neuro-symbolic AI combines machine learning’s flexibility with automated reasoning’s provable correctness. An LLM generates policies from natural language, while Cedar Analysis validates them using symbolic, mathematical reasoning. The following diagram illustrates this workflow:

Figure 1: Cedar policy generation workflow

Figure 1: Cedar policy generation workflow

An administrator specifies—in natural language—which MCP tools the agent can call and under what conditions. The neuro-symbolic feedback loop then formalizes this description into Cedar policies. Here’s how it works: first, the LLM translates the natural language into Cedar policies. These policies are then run through two stages of verification. In the first stage, AgentCore Policy uses a Cedar schema generator that takes the MCP tool descriptions and produces a Cedar schema. Cedar validates the policies against this schema, helping to ensure that they reference valid tools and parameters and ruling out whole classes of runtime errors. If validation passes, the second stage runs Cedar Analysis, which encodes each policy as a mathematical formula and detects issues like policies that grant or deny everything, or that contain impossible conditions. These mathematical proofs identify errors in the process of translating from the natural language description to Cedar policies, and guide corrections.

The neuro-symbolic feedback loop significantly improves the accuracy of the generated policies. This demonstrates the power of combining neural and symbolic approaches—the LLM provides creative translation from natural language, while automated reasoning provides rigorous validation.

Control plane: When attaching policies to an AgentCore Gateway, Cedar Analysis performs holistic analysis of the entire policy set. Instead of analyzing policies in isolation, it examines how they interact and their combined effect. This analysis identifies potential logical errors—such as conflicting or redundant policies—and detects whether the policy set produces unintended authorization outcomes. When Cedar Analysis detects these errors, the operation fails and returns a description of the issue, so the policy author can fix and retry. See the Formal analysis for policy verification section for examples of the checks.

MCP tool invocation enforcement: Each agent tool request made to the AgentCore gateway is evaluated against Cedar policies which determine whether the MCP tool invocation with the given arguments should be allowed. This creates the safety envelope while allowing the necessary bridges to enable the agent to perform its job.

MCP tool filtering: Cedar enables an additional layer of protection that operates before any tool invocation occurs. When an agent issues a list tools command, AgentCore Gateway uses Cedar’s partial evaluation capability to determine which actions would always be denied under the current policy set. Those actions are omitted from the list tool response. The agent and the underlying LLM never see those tool actions, eliminating an entire class of risk: the agent and LLM can’t attempt to invoke a tool it doesn’t know exists. This is a direct benefit of Cedar’s partial evaluation: the system can determine that certain tool actions are unreachable without needing to wait for an actual tool invocation attempt.

Why Cedar: Analyzability enables safety at scale

Natural language is too ambiguous for security-critical infrastructure, and general-purpose programming languages, like Python, are very expressive but too difficult to analyze. They can have unintended side effects, termination issues, and can be difficult to understand.

Cedar avoids these issues by excluding loops and stateful operations, so policy evaluation terminates in O(n) time in common cases. This bounded execution time means agents can make authorization decisions without disrupting user experience or workflow efficiency.

Cedar is straightforward to read. Regulatory compliance and security audits require policies that humans can understand and verify. Cedar policies read like structured natural language, making them accessible to security teams, compliance officers, and business stakeholders:

// Only allow bulk discounts for premium customers with sufficient quantity
permit (
  principal is AgentCore::OAuthUser,
  action == AgentCore::Action::"ApplyBulkDiscount",
  resource
)
when
{
  principal.hasTag("customer_tier") &&
  principal.getTag("customer_tier") == "Platinum" &&
  context.input.orderQuantity >= 50
}
unless
{
  context.input
    .productTypes
    .containsAny
    (
      ["limited_edition", "seasonal_specials"]
    )
};

Auditors without a technical background can understand this policy: “Allow bulk discounts for platinum customers who order at least 50 items, except for limited edition or seasonal special products.” The unless clause makes the exception clear, which is how business rules are typically expressed in natural language. Notice that this single policy constrains two different sources of data. The customer tier comes from a JSON Web Token (JWT) claim—it can’t be hallucinated or manipulated by the LLM. The tool inputs like order quantity and product types, however, originate from the LLM’s tool call. Cedar policies constrain these inputs to only allowed values, ensuring that even if the LLM produces unexpected arguments, the policy enforcement layer rejects them deterministically.

Cedar is the right choice because it’s fast, straightforward to read, and analyzable through automated reasoning. This analyzability is why you can reason about the safety envelope around agents that’s expressed as Cedar policies. As agentic systems grow the number of tools grows. Without proper tooling, policy management becomes intractable; policies can conflict, create security gaps, or produce unintended authorization outcomes.

In the rest of this section, we examine how Cedar’s analyzability directly addresses this challenge through its deterministic, mathematically sound analysis. Because Cedar analysis can reliably detect conflicts and logical errors across large policy sets it enables scalable policy management through neuro-symbolic AI.

Formal analysis for policy verification

Cedar policies can be encoded as mathematical formulas and analyzed using automated reasoning techniques through a symbolic encoder. This enables AgentCore Policy to provide sophisticated policy verification capabilities during policy authoring and beyond. AgentCore Policy uses this analysis when authoring or attaching policies to detect possible logical errors, such as conflicting or redundant policies. Policy analysis, including policy comparison is available as an open source CLI tool. Next, we will take a look at some concrete examples of these checks.

Detecting logical errors in policies: Cedar Analysis can detect when policies contain logical errors. For example, the following policy has contradictory constraints that mean it can’t allow any request: the customer tier can’t be both gold and platinum at the same time. The intention was to use an || instead of &&, a mistake that can be made by both humans and AI systems that author policies.

// This policy cannot allow any requests due to logical errors
permit (
  principal is AgentCore::OAuthUser,
  action == AgentCore::Action::"ProcessRefund",
  resource
)
when
{
  principal.hasTag("customer_tier") &&
  principal.getTag("customer_tier") == "Gold" &&
  principal.getTag("customer_tier") == "Platinum"
}
unless { context.input.refundAmount > 1000 };

Similarly, Cedar Analysis can detect policies that always allow a given action, usually an indication of an overly permissive policy. For example, the following policy will allow all ApplyBulkDiscount requests because any order quantity will either be greater than or equal to 100 or less than 100.

// This policy allows all ApplyBulkDiscount requests
permit (
  principal is AgentCore::OAuthUser,
  action == AgentCore::Action::"ApplyBulkDiscount",
  resource
)
when
{
  context.input.orderQuantity >= 100 ||
  context.input.orderQuantity < 100 ||
  (principal.hasTag("customer_tier") &&
   principal.getTag("customer_tier") == "Platinum")
};

Detecting such logical errors isn’t easy for humans, and can’t be done by pattern matching: you need the formal rigor of mathematical analysis, which is exactly what Cedar Analysis does.

Detecting policy conflicts: Cedar Analysis can also analyze the entire policy set to detect inconsistencies between different individual policies:

// These policies conflict - Analysis will detect the subtle issue
permit (
  principal is AgentCore::OAuthUser,
  action == AgentCore::Action::"ProcessRefund",
  resource
)
when
{
  principal.hasTag("customer_tier") &&
  principal.getTag("customer_tier") == "Gold" &&
  context.input.refundAmount < 100
};

forbid (
  principal is AgentCore::OAuthUser,
  action == AgentCore::Action::"ProcessRefund",
  resource
)
when
{
  principal.hasTag("customer_tier") &&
  ["Gold", "Platinum"].contains(principal.getTag("customer_tier")) &&
  context.input.refundAmount < 500
};

The permit policy allows gold customers to process refunds less than $100, while the forbid policy blocks gold customers (and platinum customers) from processing refunds less than $500. Because forbid overrides permit in Cedar, the forbid policy would block all gold customer refunds despite the permit policy.

Comparing policy changes: When updating policies, Cedar Analysis can also determine the exact impact of a change. Consider the following update to the unless clause (the policy lines with + have been added and those with - have been removed): we now block ApplyBulkDiscount only when the product type is limited_edition and the quantity exceeds 200.

 permit (
   principal is AgentCore::OAuthUser,
   action == AgentCore::Action::"ProcessRefund",
   resource
 )
 when
 {
   context.input.refundAmount < 500
 };
 
 permit (
   principal is AgentCore::OAuthUser,
   action == AgentCore::Action::"ApplyBulkDiscount",
   resource
 )
 when
 {
   context.input.orderQuantity >= 50
 }
 unless
 {
-  context.input.productTypes.containsAny(["limited_edition"])
+  context.input.productTypes.containsAny(["limited_edition"]) &&
+  context.input.orderQuantity > 200
 };

At first glance, adding a condition to the unless clause might seem more restrictive. In fact, it’s the opposite: narrowing when the unless applies means the permit now covers more requests. For example, an order of 73 units of a limited_edition product would have been blocked before but is now allowed. Cedar Analysis can automatically detect this and generates the following table showing the difference in permissiveness between the original policy set and the updated one:

Principal type

Action

Resource type

Status

OAuthUser

ProcessRefund

Gateway

Equivalent

OAuthUser

ApplyBulkDiscount

Gateway

More permissive

In the preceding example, the analysis tells us that the updated policy allows allows exactly the same ProcessRefund requests, but allows more ApplyBulkDiscount requests.

This formal verification capability is essential when agents operate autonomously and can affect the real world. Organizations need mathematical certainty that their policies will behave as intended.

Deterministic behavior for reliable governance

Unlike probabilistic AI models, enterprise security requires deterministic guarantees. Cedar policies always produce the same authorization decision for identical requests, regardless of evaluation order or system state. Cedar’s default deny, forbid wins, no ordering semantics help ensure predictable behavior.

// Policy evaluation order does not affect the authorization decision
permit(
    principal,
    action == AgentCore::Action::"ProcessRefund",
    resource
) when {
    context.input.refundAmount < 500
};

forbid(
    principal,
    action == AgentCore::Action::"ProcessRefund", 
    resource
) when {
    context.input.orderDate.offset(duration("90d")) < context.system.now
};

Whether the permit or forbid policy is evaluated first, a refund request over $500 will always be denied, and any refund issued more than 90 days after the order date will also be denied. This predictability gives enterprises confidence in their agent governance.

From policies to production

By choosing AgentCore Policy and Cedar, organizations can deploy autonomous agents with policies they can reason about mathematically, not only hope the agents work correctly. Cedar’s combination of expressiveness, readability, and formal verification means that you can design agents with the flexibility needed to function and the certainty security teams demand.

Automated reasoning has already proven its value across AWS, from AWS IAM Access Analyzer verifying access policies to provable security for network configurations. Applying these same techniques to agentic AI is a natural extension: as agents take on more responsibility, the need for mathematically grounded guarantees only grows. The neuro-symbolic approach we’ve described in this post—combining LLM flexibility with the rigor of automated reasoning—points toward a future where agents can be both more autonomous and more trustworthy, because the verification keeps pace with the autonomy.

Learn more

Policy is now available as part of Amazon Bedrock AgentCore Gateway. To learn more about Cedar and its capabilities, visit the Cedar website, try the Cedar playground, or join the Cedar community on Slack.

For more information about Policy in Amazon Bedrock AgentCore Gateway, visit the AWS documentation or explore the AgentCore Gateway console.

If you have feedback about this post, submit comments in the Comments section below.

Liana Hadarean

Liana Hadarean

Liana is a Principal Applied Scientist at AWS. She has worked on the code analysis tools that power Amazon Q Java security detectors, and is now a contributor to the Cedar policy language.

John Tristan

Jean-Baptiste Tristan

Jean-Baptiste is a Senior Principal Applied Scientist at AWS Agentic AI where he works on neurosymbolic AI and agentic safety.

Project Glasswing: what Mythos showed us

Post Syndicated from Grant Bourzikas original https://blog.cloudflare.com/cyber-frontier-models/

For the last few months, we’ve been testing a range of security-focused LLMs on our own infrastructure. These LLMs help identify potential vulnerabilities in our own systems, so we can fix them – and they also show us what attackers are going to be able to do with the latest models.

None of these LLMs has captured more attention than Mythos Preview, from Anthropic. A few weeks ago, we were invited to use Mythos Preview as part of Project Glasswing. We soon pointed it at more than fifty of our own repositories – to see what it would find, and to see how it works.

This post shares what we observed, what the models did well and what they didn’t, and how the architecture and process around them needs to change, so they can be used at scale.

What changed with Mythos Preview

Mythos Preview is a real step forward, and it’s worth saying that plainly before getting into anything else. We’ve been running models against our code for a while now, and the jump from what was possible with previous general-purpose frontier models to what Mythos Preview does today is not just a refinement of what came before.

It’s a different kind of tool doing a different kind of work, and that makes a clean apples-to-apples comparison to earlier models difficult. So rather than trying to benchmark Mythos Preview against general-purpose frontier models, it’s more useful to describe what it can actually do, and two features that stood out across the work we did with Mythos Preview:

  • Exploit chain construction – A real attack rarely uses one bug. It chains several small attack primitives together into a working exploit. For instance, it might turn a use-after-free bug into an arbitrary read and write primitive, hijack the control flow, and use return-oriented programming (ROP) chains to take full control over a system. Mythos Preview can take several of these primitives and reason about how to combine them into a working proof. The reasoning it shows along the way looks like the work of a senior researcher rather than the output of an automated scanner.

  • Proof generation – Finding a bug and proving it’s exploitable are two different things, and Mythos Preview can do both. It writes code that would trigger the suspected bug, compiles that code in a scratch environment, and runs it. If the program does what the model expected, that’s the proof. If it doesn’t, the model reads the failure, adjusts its hypothesis, and tries again. The loop matters as much as the bugs it finds, because a suspected flaw without a working proof is speculation, and Mythos Preview closes that gap on its own.

Some of what we describe above is not entirely unique to Mythos Preview. When we ran other frontier models through the same harness, they found a fair number of the same underlying bugs, and in some cases they got further than we expected on the reasoning side too. Where they fell short was at the point of stitching the pieces together. A model would identify an interesting bug, write a thoughtful description of why it mattered, and then stop, leaving the actual chain unfinished and the question of exploitability open. What changed with Mythos Preview is that a model can now take those low-severity bugs (which would traditionally sit invisible in a backlog) and chain them into a single, more severe exploit. 

Model refusals in legitimate vulnerability research

The Mythos Preview model provided by Anthropic, as part of Project Glasswing, did not have the additional safeguards that are present in generally available models (like Opus 4.7 or GPT-5.5).

Despite this, the model organically pushes back on certain requests – much like the cyber capabilities that made it useful for vulnerability hunting, the model has its own emergent guardrails that sometimes cause it to push back on legitimate security research requests. But as we found, these organic refusals aren’t consistent – the same task, framed differently or presented in a different context, could produce completely different outcomes as illustrated in the examples below.


Example of Mythos Preview pushing back on building a working proof of concept 

For example, the model initially refused to do vulnerability research on a project, then agreed to perform the same research on the same code after an unrelated change to the project’s environment. Nothing about the code being analyzed had changed.

In another case, the model found and confirmed several serious memory bugs in a codebase, and then refused to write a demonstration exploit. The same request, framed differently, got a different answer, and even the same request can produce different outcomes across runs due to the probabilistic nature of the model. Semantically equivalent tasks can produce opposite outcomes depending on how and when they’re presented to the model.

This matters because while the model’s organic refusals/guardrails are real, they aren’t consistent enough to serve as a complete safety boundary on their own. That’s precisely why any capable cyber frontier model made generally available in the future must include additional safeguards on top of this baseline behavior – making it appropriate for broader use outside of a controlled research context like Project Glasswing.

The signal-to-noise problem

One of the hardest parts of triaging security vulnerabilities is deciding which bugs are real, which are exploitable, and which need fixing now. This was a hard problem even in the pre-AI world. AI vulnerability scanners and AI-generated code have made it worse, and at Cloudflare we’ve built multiple post-validation stages to deal with it.

Two factors dominate the noise rate:

  • Programming language – C and C++ give you direct memory control and, with it, bug classes – buffer overflows, out-of-bounds reads and writes – that memory-safe languages like Rust eliminate at compile time. We saw consistently more false positives from projects written in memory-unsafe languages.

  • Model bias – A good human researcher tells you what they found and how confident they are. Models don’t. Ask a model to find bugs, and it will find them, whether the code has any or not. Findings come back hedged with “possibly,” “potentially,” “could in theory,” and the hedged findings vastly outnumber the solid ones. That’s a reasonable bias for an exploratory tool. It’s a ruinous one for a triage queue, where every speculative finding spends human attention and tokens to dismiss, and that cost compounds across thousands of findings.

Mythos Preview represents a clear improvement here, particularly in its ability to chain primitives – combining multiple vulnerabilities into a working proof of concept rather than reporting them in isolation. A finding that arrives with a PoC is a finding you can act on, and it means far less time spent asking “is this even real?”

Our harnesses are deliberately tuned to over-report, so we see more (and miss less), which comes with a lot more noise. But at triage time, Mythos Preview’s output has noticeably higher quality: fewer hedged findings, clearer reproduction steps, and less work to reach a fix-or-dismiss decision.

Why pointing a generic coding agent at a repo doesn’t work

When we first started AI-assisted vulnerability research last year, our instinct was the obvious one: point a generic coding agent at an arbitrary repository and ask it to discover vulnerabilities. This approach works, in the sense that the model will produce findings, but it doesn’t work in producing meaningful coverage of a real codebase and identifying findings of value. There are two main reasons for this:

  • Context – Coding agents are tuned for one focused stream of work: building a feature, fixing a bug, writing a refactor. They ingest a lot of source code, hold a single hypothesis at a time, and iterate against it. That’s exactly the wrong shape for vulnerability research, which is narrow and parallel by nature. A human researcher picks one specific thing to look at and investigates it thoroughly. That one thing might be a single complex feature, transitions across security boundaries, or a specific vulnerability class like command injections, where attacker input ends up being run as a shell command. Then they do it again, for a different feature, security boundary, or vulnerability class, several thousand times across the codebase. A single agent session (even with subagents) against a hundred-thousand-line repository can cover maybe a tenth of a percent of the surface in a useful way before the model’s context window fills up and compaction kicks in – potentially discarding earlier findings that would have mattered.

  • Throughput – A single-stream agent does one thing at a time, but real codebases need many hypotheses against many components at once, with the ability to fan out further when something interesting turns up. You can drive a single agent harder, but at some point you stop being limited by the model and start being limited by the shape of the interaction itself. Using the model directly in a coding agent turns out to be fine for manual investigation when a researcher already has a lead and wants a second pair of eyes. However, it’s the wrong tool for achieving high coverage. Once we accepted that, we stopped trying to make Mythos Preview do the wrong job and started building the harness around it instead.

What a harness actually fixes

Four lessons came out of running the work at scale, and each one pointed to the need for a harness that manages the overall execution:

  • Narrow scope produces better findings – Telling the model “Find vulnerabilities in this repository” makes it wander. Telling it “Look for command injection in this specific function, with this trust boundary above it, here’s the architecture document and here’s prior coverage of this area” makes it do something much closer to what a researcher would actually do.

  • Adversarial review reduces noise – Adding a second agent between the initial finding and the queue – one with a different prompt, a different model, and no ability to generate its own findings – catches a lot of the noise that the first agent would miss if it just checked its own work. It turns out that putting two agents in deliberate disagreement is way more effective than just telling one agent to be careful.

  • Splitting the chain across agents produces better reasoning – Asking “Is this code buggy?” and “Can an attacker actually reach this bug from outside the system?” are two different questions, and the model is better at each one when you ask them separately, because each question is narrower than the combined version.

  • Parallel narrow tasks beat one exhaustive agent – Coverage improves when many agents work on tightly scoped questions and we deduplicate the results afterward, rather than asking one agent to be exhaustive.

Each of those observations is about model behavior, and put together they describe something that isn’t a chat interface anymore. It’s a harness that helps you achieve the final outcomes. The first steps to building a harness are simple, as you can ask the model to help, which is what we did. We used Mythos Preview to build on, tailor, and improve our original harnesses to suit its strengths.

An example of what a harness looks like in practice is described below.

Our vulnerability discovery harness

Here’s what our vulnerability discovery harness looks like, stage by stage. It was used to scan live code across our runtime, edge data path, protocol stack, control plane, and the open-source projects we depend on.


Stage What it does Why it matters

Recon 		An agent reads the repository from the top down, fans out to subagents responsible for each subsystem, and produces an architecture document covering build commands, trust boundaries, entry points, and likely attack surface. It also generates the initial queue of tasks for the next stage.   Gives every downstream agent shared context. Cuts the wander problem.
 
Hunt 		Each task is one attack class paired with a scope hint. Hunters (the agents that actually look for bugs) run concurrently, typically around fifty at once, each fanning out to a handful of exploration subagents. Each hunter has access to tools that compile and run proof-of-concept code in a per-task scratch directory. This is where most of the work happens. Many narrow tasks in parallel, not one exhaustive agent.

Validate 		An independent agent re-reads the code and tries to disprove the original finding. It uses a different prompt and has no ability to emit new findings of its own. Catches a meaningful fraction of the noise the hunter wouldn’t catch when reviewing its own work.

Gapfill 		Hunters flag areas they touched but didn’t cover thoroughly. Those areas get re-queued for another pass. Counteracts the model’s tendency to drift toward attack classes it has already had success with.

Dedupe 		Findings that share the same root cause collapse into a single record. Variant analysis is a feature, not a way to inflate the queue with duplicates.

Trace 		For each confirmed finding in a shared library, a tracer agent fans out (one instance per consumer repository), uses a cross-repo symbol index, and decides whether attacker-controlled input actually reaches the bug from outside the system. Turns “there is a flaw” into “there is a reachable vulnerability.” This is the stage that matters most.

Feedback 		Reachable traces become new hunt tasks in the consumer repositories where the bug is actually exposed. Closes the loop. The pipeline gets better as it runs.

Report 		An agent writes a structured report against a predefined schema, fixes any validation errors against that schema itself, and submits the report to an ingest API. Output is queryable data, not free-form prose.

What this means for security teams

The loudest reaction to Mythos Preview from other security leaders has been about speed – scan faster, patch faster, compress the response cycle. More than one team we have spoken with is now operating under a two-hour SLA from CVE release to patch in production. The instinct is understandable: when the attacker timeline shortens, the defender timeline has to shorten with it. Faster is not going to be enough, and we think a lot of teams are about to spend a lot of time, effort, and money learning that the hard way.

Patching faster does not change the shape of the pipeline that produces the patch. If regression testing takes a day, you cannot get to a two-hour SLA without skipping it, and the bugs you ship when you skip regression testing tend to be worse than the bugs you were trying to patch. We learned a version of this when we tried letting the model write its own patches and watched a few go out that fixed the original bug while quietly breaking something else the code depended on.

The harder question is what the architecture around the vulnerability should look like. The principle is to make exploitation harder for an attacker even when a bug exists, so that the gap between when a vulnerability is disclosed and when it is patched matters less. That means defenses that sit in front of the application and block the bug from being reached. It means designing the application so that a flaw in one part of the code cannot give an attacker access to other parts. It means being able to roll out a fix to every place the code is running at the same moment, rather than waiting on individual teams to deploy it. 

We also recognize this topic cuts both ways. The same capabilities that helped us find bugs in our own code will, in the wrong hands, accelerate the attack side against every application on the Internet. Cloudflare sits in front of millions of those applications, and the architectural principles described above are exactly the ones our products are built to apply on behalf of customers. We will share more on what that means for customers in the weeks ahead.

If your team is doing similar work and would like to compare notes, reach out to us at [email protected].

Our research with Mythos Preview was conducted in a controlled environment against our own code; every vulnerability surfaced through this work was triaged, validated, and remediated where action was needed under Cloudflare’s formal vulnerability management process.

This work was a team effort. Thanks to Albert Pedersen, Craig Strubhart, Dan Jones, Irtefa Fairuz, Martin Schwarzl, and Rohit Chenna Reddy for their contributions to the research, engineering, and analysis behind this blog post.

Enabling AI sovereignty on AWS

Post Syndicated from Stéphane Israël original https://aws.amazon.com/blogs/security/enabling-ai-sovereignty-on-aws/

Cloud and AI are transforming industries and societies at unprecedented speed, from accelerating research and enhancing customer experiences to optimizing business processes and enriching public services. At Amazon Web Services (AWS), we believe that for the cloud and AI to reach their full potential, customers need control over their data and choices for how and where they run their workloads. In 2022, we formalized our commitment to control and choice—offering all AWS customers the most advanced set of sovereignty controls and features available in the cloud with the AWS Digital Sovereignty Pledge. As AI adoption accelerated, we’ve been working with customers to help them embrace AI innovation while meeting sovereignty requirements. We’re committed to ensuring customers can continue to harness AI’s transformative capabilities without compromising on the capabilities, performance, innovation, security, and scale of the AWS Cloud to meet their sovereignty needs, including AI sovereignty. Our approach to AI sovereignty is grounded in a deep understanding of these needs and the real-world implementation challenges that come with them.

Through discussions with customers, partners, analysts, and regulators, we’ve learned that digital sovereignty—and AI sovereignty—means different things to different stakeholders. Each country and region has unique, evolving sovereignty requirements, with no uniform guidance on which workloads or sectors must comply. Despite this variation, we’ve identified consistent themes: data sovereignty (including data residency and operator access restrictions) and operational sovereignty (including resilience, survivability, and independence). AI sovereignty builds on these foundations, adding emerging considerations such as preserving cultural norms, values, and local languages in AI outputs. Ultimately, meeting digital and AI sovereignty requirements comes down to providing customers with more control and choice.

Enabling customer control and choice across the AI stack

AI sovereignty requires control and choice across the AI stack—comprehensive cloud infrastructure that combines compute, networking, data management, security controls, specialized application services, and talent. This includes the ability to make deliberate choices across the stack such as location, dependencies, services, and partners that align with customers’ unique needs, regulatory requirements, and innovation objectives. With AWS, customers can develop AI on a trusted foundation where their data remains secure and under their control. Customers have the freedom to choose from a comprehensive range of AI optimized chips—including purpose-built AWS silicon and chips from NVIDIA, AMD, and Intel—so they can select the right chip for the right workload. AWS applies two decades of learned expertise to our comprehensive AI stack, enabling organizations to maintain complete control over their data and operations while accessing cutting-edge capabilities to solve local challenges.

AWS provides customers with the infrastructure and tools to embed AI across the full value chain—not just in isolated use cases, but as a foundational capability enabling them to train and deploy models and build sophisticated AI and generative AI applications with exceptional performance. This enables customers to focus on innovation instead of their infrastructure, bringing the cloud to where they need it most with a range of options including AWS AI Factories, AWS Outposts, AWS Local Zones, AWS Dedicated Local Zones, and AWS Regions including the AWS European Sovereign Cloud. For example, customers who require dedicated deployments to meet their sovereignty requirements for their mission-critical AI workloads can use AWS AI Factories. These physically isolated, dedicated deployments built exclusively for the customer combine the latest AI infrastructure, including AWS Trainium accelerators, NVIDIA GPUs, dedicated networking, and storage. AWS AI Factories address AI sovereignty needs by delivering on-premises AI capabilities to securely perform training, fine tuning and real-time inference.

The AWS AI portfolio offers a comprehensive range of services—from foundation models (FMs) through Amazon Bedrock, to machine learning offerings like Amazon SageMaker, application services like Amazon Q, and developer tools like Kiro—designed to give customers control over their data and choice in how they deploy AI. With Amazon Bedrock, customers can choose from hundreds of models from leading providers like AI21 Labs, Anthropic, Amazon, Cohere, Mistral AI, and OpenAI. Customers can evaluate and select the most suitable FMs for their specific needs and choose where they deploy them, and fine-tune models privately with their own data. Customers are always in control of their data. Critically, no customer inputs to or outputs from Amazon Bedrock are used to train Amazon Nova or any third-party models.

Supporting national AI strategies

Successful AI strategies require building a holistic environment nurturing local talent, supporting startups, developing industry-specific applications, and fostering public-private partnerships. The cloud has transformed AI from an exclusive technology requiring massive investment into an accessible tool for innovation across all sectors and organization sizes. While technical infrastructure gets much of the attention when considering AI sovereignty, the cultural and strategic dimensions of national FMs are equally critical. These FMs aren’t merely computational tools, they can encode elements of cultural knowledge, linguistic nuance, and societal context, making local relevance a design consideration rather than an afterthought. These FMs serve purposes that extend beyond technical capabilities. Locally trained FMs can reflect national educational curricula and cultural values while understanding local legal systems, business practices, and regulatory frameworks. Models trained on local languages, dialects, and cultural contexts support linguistic diversity and help underrepresented languages gain representation in AI products and services.

AWS supports vital national priorities and customers’ missions, such as the preservation of culture norms, values, and local languages development of regional and local language model capabilities. To customize models, customers can use Amazon SageMaker AI for voice, domain specialization, and to evaluate models for accuracy. For example, the first Greek LLM made available in March 2024 was Meltemi—built on top of Mistral-7B, running on AWS infrastructure, and continually pretrained to extend its proficiency in the Greek language using a dataset of 28.5 billion Greek tokens. Meltemi is available on HuggingFace. SEA-LION—a family of open source, multilingual LLMs for Southeast Asia—was trained entirely on AWS with managed GPU clusters. Their team completed a 3B-parameter model in only 3 months—a 60% faster timeline than comparable on-premises projects.

Verifiable control over data access

Sovereignty isn’t only about where data resides—it’s about who can access it and under what conditions. In the AI context, access restriction extends beyond infrastructure to cover model inputs, outputs, training processes, and the operational environments in which AI runs. Unlike traditional infrastructure, AI workloads introduce new access surfaces: the model itself, the data used to train it, and the inference pipeline through which sensitive inputs flow. This furthers the need for verifiable governance and identity propagation in IT systems.

To help ensure the confidentiality and integrity of customer data, all modern Amazon Elastic Compute Cloud (Amazon EC2) instances including those that offer AI accelerators, such as AWS Inferentia and AWS Trainium, are backed by the industry-leading security capabilities of the AWS Nitro System. By design, there is no mechanism for anyone at AWS to access customer data on Nitro EC2 instances that customers use to run their workloads. AWS services—including those with AI capabilities built on Amazon EC2—inherit these same protections. These protections apply to AI data running in the AWS Nitro System so that they’re protected at every stage—from model training to inference. The NCC Group, an independent cybersecurity firm, has validated the design of the Nitro System. We believe providing this level of transparency is critical in building and sustaining trust.

As AI agents increasingly take actions across systems on behalf of users, controlling who and what can access resources—and ensuring appropriate human oversight—becomes critical. AWS Identity and Access Management (IAM) helps ensure that only authorized users and applications can access AI resources through fine-grained permissions and comprehensive audit trails. For AI agents and automated workloads, Amazon Bedrock AgentCore Identity provides identity and credential management, so agents operate with the right permissions and nothing more.

Transparency and assurance

Transparency is at the core of our digital sovereignty commitment. We provide comprehensive industry-leading technical measures, operational controls, and contract protections that give customers control over where they locate their data, who can access it, and how it’s used. To give greater assurance on how AWS services are designed and operated, we continue to seek out and secure third-party attestations, accreditations, and certifications that help our customers meet their compliance needs.

We continue to deepen our assurances and transparency to customers—such as updating our AWS Service Terms to reflect our technical protections commitments (e.g. AWS Nitro System), providing detailed commitments as to our handling of thirid-party requests for customer data in our agreements, and providing supplemental explanations and resources (e.g. CLOUD Act blog) to empower customers to make informed choices on sovereignty matters. These efforts extend into our commitment to responsible AI, providing customers the confidence to build and operate AI applications responsibly using AWS Services. ISO/IEC 42001 is an international management system standard that outlines requirements and controls for organizations to promote the responsible development and use of AI systems. AWS is the first major cloud service provider to achieve ISO/IEC 42001 accredited certification for AI services, covering Amazon Bedrock, Amazon Q Business, Amazon Textract, and Amazon Transcribe. In November 2025, AWS successfully completed its first surveillance audit for ISO 42001:2023 with no findings, reiterating the continual commitment of AWS to responsible AI practices.

Innovative technology requires a secure and trustworthy foundation. AWS supports more than 140 security standards and compliance certifications that our customers and partners can inherit to help comply with local laws and regulations. For two decades, we’ve deeply engaged with regulators and cybersecurity authorities to align our offerings with national priorities and ensure our solutions support both innovation and control. We actively contribute to frameworks that respond to new developments without stifling progress.

Sustained commitment to helping customers achieve their sovereignty goals

AWS is committed to giving customers the same control and choice over their AI systems as they have over their data. We help customers harness AI’s transformative power while maintaining the capabilities, performance, innovation, security, and scale of AWS Cloud. As cloud and AI evolve, AWS will continue offering the most advanced sovereignty controls and features available.

If you have feedback about this post, submit comments in the Comments section below.

Stephane Israel

Stéphane Israël

Stéphane is the leader and Managing Director of the AWS European Sovereign Cloud. He is responsible for the management and operations of the AWS European Sovereign Cloud, including infrastructure, technology, and services, in addition to broader digital sovereignty efforts at AWS. Prior to AWS, he was the CEO of Arianespace, where he oversaw numerous successful space missions, including the launch of the James Webb Space Telescope.

How Cloudflare responded to the “Copy Fail” Linux vulnerability

Post Syndicated from Chris J Arges original https://blog.cloudflare.com/copy-fail-linux-vulnerability-mitigation/

On April 29, 2026, a Linux kernel local privilege escalation vulnerability was publicly disclosed under the name “Copy Fail” (CVE-2026-31431). Cloudflare’s Security and Engineering teams began assessing the vulnerability as soon as it was disclosed. We reviewed the exploit technique, evaluated exposure across our infrastructure, and validated that our existing behavioral detections could identify the exploit pattern within minutes. 

There was no impact to the Cloudflare environment, no customer data was at risk, and no services were disrupted at any point. Read on to learn how our preparedness paid off. 

Background

Our Linux kernel release process

Cloudflare operates a global Linux server infrastructure at an immense scale, with datacenters located across 330 cities. We maintain a custom Linux kernel build based on the community’s Long-Term Support (LTS) versions to manage updates effectively at this volume. At any given time, we may utilize multiple LTS versions from various series, such as 6.12 or 6.18, which benefit from extended update periods.

The community regularly merges and releases security and stability updates which trigger an automated job to generate a new internal kernel build approximately every week. These builds undergo testing in our staging data centers to ensure stability before a global rollout. Following a successful release, the Edge Reboot Release (ERR) pipeline manages a systematic update and reboot of the edge infrastructure on a four-week cycle. Our control plane infrastructure typically adopts the most recent kernel, with reboots scheduled according to specific workload requirements.

By the time a CVE becomes public knowledge, the necessary fix has typically been integrated into stable Linux LTS releases for several weeks. Our established procedures ensure that we have already deployed these patches.

At the time of the “Copy Fail” disclosure, the majority of our infrastructure was running the 6.12 LTS version, while a subset of machines had begun transitioning to the newer 6.18 LTS release.

About the Copy Fail vulnerability

It helps to understand the vulnerability before getting to the response story. A comprehensive write-up can be found in the original Xint Code disclosure post.

AF_ALG and the kernel crypto API

The Linux kernel’s internal crypto API manages functions like kTLS and IPsec. Userspace programs access this via the AF_ALG socket family, allowing unprivileged processes to request encryption or decryption. The algif_aead module facilitates this for Authenticated Encryption with Associated Data (AEAD) ciphers.

An unprivileged program follows these steps:

  1. Opens an AF_ALG socket and binds to an AEAD template.

  2. Sets a key and accepts a request socket.

  3. Submits input via sendmsg() or splice().

  4. Executes the operation using recvmsg().

The splice() system call is critical here, as it moves data by passing page cache references.

Memory mechanics: page cache and in-place crypto

The page cache is a shared system cache for file contents. Modifying a page belonging to a setuid binary effectively edits that program for all users until the page is evicted.

The crypto API utilizes scatterlists, which are structures linking various memory pages. In 2017, algif_aead was optimized for in-place operations, chaining destination and reference pages together. This design lacked enforcement to prevent algorithms from writing past intended boundaries.

The vulnerability: out-of-bounds write

When the user executes recvmsg(), the authencesn wrapper in the kernel performs a 4-byte write past the legitimate output region:

scatterwalk_map_and_copy(tmp + 1, dst, assoclen + cryptlen, 4, 1);

By using splice(), an attacker can chain a target file’s page cache pages to the scatterlist. The out-of-bounds write then taints the cached file, allowing an attacker to control which file is modified, the offset, and the specific 4 bytes written. This means the attacker can manipulate the following with this exploit:

  • File: Any readable file.

  • Offset: Tunable via assoclen and splice parameters.

  • Value: Controlled via AAD bytes 4-7 in sendmsg()

The exploit, step by step


The default exploit targets /usr/bin/su, a setuid-root binary present on essentially every distribution.

  1. Cache Reference: Open /usr/bin/su as O_RDONLY and read() to populate the page cache. Use splice() on the file descriptor to pass these page cache references into the crypto scatterlist.

  2. Setup: Create an AF_ALG socket, bind() to authencesn(hmac(sha256),cbc(aes)), set a key, and accept a request socket without needing privileges.

  3. Write Construction: For each 4-byte shellcode chunk:

    • sendmsg() with AAD bytes 4–7 containing the shellcode.

    • splice() the binary into a pipe then the AF_ALG socket so assoclen + cryptlen targets the desired .text offset.

  4. Trigger: recvmsg() initiates decryption. authencesn writes its scratch data to the target offset of /usr/bin/su in the page cache. Although the function returns -EBADMSG, the 4-byte write is now in the global page cache.

  5. Execution: Running execve("/usr/bin/su") loads the tainted page cache. Since the binary is setuid-root, the injected shellcode executes with root privileges.

The upstream fix (commit a664bf3d603d) reverts the 2017 in-place optimization, removing the exploit.

How we responded 

When the vulnerability was disclosed, many workstreams started in parallel:

  • Mapping the blast radius: Our security team worked with kernel engineers to determine which kernel versions were vulnerable and assess the potential exposure.

  • Validating coverage: Security reviewed the exploit technique and confirmed that our existing behavioral detections could identify the exploit pattern during authorized internal validation.

  • Proactive threat hunting: Security began searching for signs that the vulnerability had been exploited before it was publicly known, going back 48 hours in our fleet-wide logs.

  • Engineering a mitigation: Kernel engineers began building a runtime mitigation that would protect the fleet without breaking production services.

  • Continuing software updates: Our engineering teams worked on delivering an updated Linux kernel, which required carefully rebooting and rolling it out across our servers.

There was no customer impact at any point during this response.

Validating detection coverage

One of the first things our security team did was confirm that our existing endpoint detection would catch this exploit. Our servers run behavioral detection that continuously monitors process execution patterns. It doesn’t rely on knowing about specific vulnerabilities; it watches for anomalous behavior across the fleet.

When our engineers validated the vulnerability internally as part of the response, the detection platform flagged it within minutes. The system linked the entire execution chain—starting at the script interpreter, moving through the kernel’s cryptographic subsystem, and ending at the privilege escalation binary—flagging it as malicious based on fleet-wide behavioral patterns.

This happened without a signature update, without a rule change, and without human intervention. Our behavioral detection coverage existed before we wrote any custom logic for this particular Copy File exploit. 

The confirmation was important because it meant we had coverage before writing a vulnerability-specific rule.

Hunting for exploitation

While our engineering team moved to a more targeted mitigation, our security investigation had been running since disclosure. This is our standard procedure for any critical vulnerability.

Our security team operates on a simple principle for critical vulnerabilities: assume compromise until you can prove otherwise. The investigation started from the assumption that exploitation could have occurred before the vulnerability was public, and we worked systematically to either confirm or rule it out.

The exploit leaves a distinctive trace in kernel logs when it runs. We searched for that trace across our centralized logging infrastructure, covering 48 hours before the vulnerability was publicly disclosed. If someone had exploited this before the world knew about it, we would have seen it.

We pulled access logs for affected systems and reconstructed who connected, when, and what commands they ran. This gave us a complete forensic picture of interactive activity on potentially affected infrastructure.

We checked that system binaries had not been tampered with, validated cryptographic hashes against known-good package manifests, looked for persistence mechanisms, and audited network connections for anything unusual. Everything was clean.

Incident timeline and impact

Time (UTC) Event
2026-04-29 16:00 Copy Fail publicly disclosed.
2026-04-29 ~21:00 Security and Engineering teams began assessing fleet exposure and mitigation options before full declaration of the Incident Response process
2026-04-29 22:52 Security confirmed existing behavioral detection covered the Copy Fail exploit pattern. During authorized internal validation, detection flagged the activity within minutes.
2026-04-29 23:01 Existing behavioral detection generated a high-severity alert for exploit-like activity, confirming detection coverage for the technique.
2026-04-29 (evening) First mitigation attempt pushed to our staging datacenter. The deployment process surfaced a dependency conflict; the mitigation was rolled back. No production systems were affected.
2026-04-29 (overnight) Engineering drafted bpf-lsm mitigation program.
2026-04-30 03:14 Security incident declared to drive cross-functional collaboration and urgency. Security performed fleetwide threat hunting of historical data to confirm that no malicious activity was present on Cloudflare systems.
2026-04-30 (morning) Engineering tested the bpf-lsm mitigation program and made it production-ready.
2026-04-30 14:25 Engineering incident declared to coordinate mitigation program and Linux patch rollout.
2026-04-30 ~17:00 Decision made: ship a patched build of the previous LTS line through reboot automation; do not accelerate the new LTS; lean on bpf-lsm in the meantime.
2026-04-30 (afternoon) Visibility pipeline (eBPF tracing of AF_ALG socket usage) deployed fleet-wide. Gives a complete picture of all legitimate AF_ALG users.
2026-04-30 (evening) bpf-lsm mitigation program rolled out behind a separate gate to fully mitigate the fleet. End-to-end verification on a previously-vulnerable test node confirms the exploit no longer works.
2026-05-04 (morning) Reboot automation resumed at normal pace with the patched kernel.
2026-05-04 onward Servers that had already passed through reboot automation earlier in the week manually rebooted to pick up the patched kernel. Unpatched servers update per our normal reboot automation.

This graph shows the progress of our mitigation program as it progressed through our infrastructure.

How did we mitigate it?

Because of the long timeframe involved in deploying a patched Linux kernel, we also pursued mitigating this exploit without a reboot.

Removing the module

The bug was in the algif_aead kernel module. Therefore, the simple fix was to just remove this module and disallow it from being reloaded.

This mitigation was therefore exactly what the Copy Fail write-up from the security researchers who identified it recommends.

echo "install algif_aead /bin/false" > /etc/modprobe.d/disable-algif.conf
rmmod algif_aead 2>/dev/null || true

Unfortunately removing the module would have impacted software that leverages the kernel crypto API.  This meant that we had to figure out a more surgical mitigation.

Bpf-lsm


We’ve already developed and deployed such a tool for this exact scenario: bpf-lsm. Instead of removing the module, this tool leaves it loaded for legitimate users and uses a BPF Linux Security Module program to deny the socket_bind LSM hook for everyone else. This completely blocks the front door for any exploits.

A draft of the eBPF program was put together overnight. Team members picked it up the following morning, ran validations, and made it production-ready. The program is fairly straightforward. On every socket_bind call:

  1. If the socket family is not AF_ALG, allow the call through unchanged.

  2. If the family is AF_ALG, check the calling binary’s path against an allow-list of the binaries we know to be legitimate users.

  3. If the binary is on the allow-list, allow the bind. Otherwise, deny it.

To verify the mitigation on a given machine without exploiting it, the Copy Fail write-up gives a one-liner:

python3 -c 'import socket; s = socket.socket(socket.AF_ALG, socket.SOCK_SEQPACKET, 0); s.bind(("aead","authencesn(hmac(sha256),cbc(aes))"));'

On a mitigated machine you get PermissionError: [Errno 1] Operation not permitted (or FileNotFoundError, depending on which mitigation is active) instead of a successful bind.

Rolling it out

Before enabling enforcement, we verified that our known internal service was the sole legitimate AF_ALG user to avoid accidental outages. We used prometheus-ebpf-exporter to hook the socket() syscall and track AF_ALG usage per binary across the fleet. This required no kernel changes and provided aggregate data from hundreds of thousands of servers within hours. Results confirmed the identified service was indeed the only legitimate user.

So the bpf-lsm rollout was deliberately staged in two steps:

  1. Get visibility first. Push the ebpf-exporter config gated by salt. Confirm at the metric layer that the known service is effectively the only thing creating AF_ALG sockets.

  2. Then enforce. Push the bpf-lsm program behind a separate enforcement gate.

In parallel, the upstream backport for our majority LTS line finally became available, and our internal automation built a patched kernel against it.

We started to test the patched kernel in our staging datacenters as soon as possible, then we resumed the longer reboot process in order to fully patch our fleet.

Remediation and follow-up steps

While we were prepared for this scenario, at Cloudflare we’re always learning and improving. Key areas we identified for improvement:

  • Better visibility into kernel-API dependencies. We will review kernel-subsystem usage across production services, so we can continue to quickly mitigate exploits without service disruption.

  • Better runtime mitigation. bpf-lsm is a valuable tool for mitigations, but we want to make this tool even better. This will include looking into faster deployments, better playbooks, and better logging and visibility of the tool. 

  • Reduce attack surface of Linux Kernel. Review and audit our kernel configuration. Proactively identify unused modules or features so that we can remove them from our build entirely.

Conclusion

The “Copy Fail” vulnerability presented a unique challenge for us. Despite our practice of deploying Linux patch updates every two weeks, we remained vulnerable because a month-old mainline fix had yet to be backported to our primary kernel line. Despite that, we were still able to roll out patched kernels within hours of the backport’s release. In the interim, bpf-lsm provided a surgical, no-reboot mitigation that secured our fleet. While our initial attempt to disable the problematic module failed, it did so safely within our internal staging environment rather than production, allowing us to identify this dependency.

By the end of the rollout, every machine in our fleet was protected by either a patched kernel or a bpf-lsm program denying the vulnerable code path to non-allow-listed binaries. There was no customer impact at any point during this incident, and we have committed to the follow-up work above to make our response faster and our visibility better the next time something like this lands. Responsible disclosure works, in-kernel visibility tooling pays off in moments exactly like this one, and bpf-lsm continues to be one of the most useful primitives we have for runtime kernel mitigation.

At Cloudflare, critical vulnerability response is a coordinated effort across Security, Engineering, Product, and many other teams. Special thanks to Ali Adnan, Ivan Babrou, Frederik Baetens, Curtis Bray, Piers Cornwell, Everton Didone Foscarini, Rob Dinh, Elle Dougherty, Kevin Flansburg, Matt Fleming, Kimberley Hall, Brandon Harris, Jerry Ho, Oxana Kharitonova, Marek Kroemeke, Fred Lawler, James Munson, Nafeez Nazer, Walead Parviz, Miguel Pato, Evan Pratten, Josh Seba, June Slater, Ryan Timken, Michael Wolf, Jianxin Zeng and everyone else who contributed to the investigation, mitigation, and remediation of Copy Fail. We’d also like to thank the Linux upstream maintainers and Copy Fail researchers whose work helped make a rapid response possible.

Post-quantum encryption for Cloudflare IPsec is generally available

Post Syndicated from Sharon Goldberg original https://blog.cloudflare.com/post-quantum-ipsec/

While more than two-thirds of human-generated TLS traffic to Cloudflare is already protected by post-quantum cryptography, the world of site-to-site networking has been a different story. For years, the IPsec community remained caught between the high bar of Internet-scale interoperability and the niche requirements of specialized hardware. That gap is now closing. 

Earlier this month, we announced that Cloudflare has moved its target for full post-quantum security forward to 2029, spurred by several recent advances in quantum computing. To advance that goal, we’ve made post-quantum encryption in Cloudflare IPsec generally available.

Using the new IETF draft for hybrid ML-KEM (FIPS 203), we’ve successfully tested interoperability with branch connectors from Fortinet and Cisco — meaning you can start protecting your wide-area network (WAN) against harvest-now-decrypt-later attacks today using hardware you already have.

This post explains how we implemented the new hybrid IPsec handshake, why it took four years longer to land than its TLS counterpart, and how the industry is finally consolidating around a standard that works at Internet scale.

Cloudflare IPsec

Cloudflare IPsec is a WAN Network-as-a-Service that replaces legacy network architectures by connecting data centers, branch offices, and cloud VPCs to Cloudflare’s global IP Anycast network. Customers get simplified configuration, high availability (if a data center becomes unavailable, traffic is automatically rerouted to the nearest healthy one), and the scale of Cloudflare’s global network. This is done through encrypted IPsec tunnels that support both site-to-site WAN, outbound Internet connections, and connectivity to the Cloudflare One SASE platform. 


Post-quantum encryption in IPsec

Cloudflare IPsec now uses post-quantum encryption with hybrid ML-KEM (FIPS 203) to stop harvest-now-decrypt-later attacks. These are attacks where an adversary harvests data today and then decrypts later, after Q-Day, when there are powerful quantum computers that can break the classical public key cryptography used across the Internet.  Harvest-now-decrypt-later attacks are becoming a concern for more organizations as Q-Day approaches faster than expected.

ML-KEM (Module-Lattice-Based Key-Encapsulation Mechanism) is a post-quantum cryptography algorithm that is based on mathematical assumptions that are not known to be vulnerable to attacks by quantum computers. It does not require special hardware or a dedicated physical link between sender and receiver. ML-KEM is intentionally designed to be implemented in software across standard processors to provide post-quantum encryption of network traffic. 

Draft-ietf-ipsecme-ikev2-mlkem specifies post-quantum encryption for IPsec using hybrid ML-KEM, which combines the well-understood security of classical Diffie-Hellman and the post-quantum security of ML-KEM in a single, standards-compliant handshake. Specifically, a classical Diffie-Hellman exchange runs first, its derived key encrypts a second exchange that runs ML-KEM, and the outputs of both are mixed into the session keys that secure IPsec data plane traffic sent using the Encapsulating Security Payload (ESP) protocol. 

Our interoperable implementation 

Earlier we announced the closed beta of our implementation of draft-ietf-ipsecme-ikev2-mlkem in production in our Cloudflare IPsec product and tested it against a reference implementation (strongswan). Now that we have made this implementation generally available, we have also confirmed interoperability with several other vendors, including Cisco and Fortinet, which is a big win for this new standard.

Cisco: Customers using Cisco 8000 Series Secure Routers after version 26.1.1 as their branch connector can also now establish post-quantum Cloudflare IPsec tunnels per draft-ietf-ipsecme-ikev2-mlkem.

Fortinet: Customers using Fortinet FortiOS 7.6.6 and later as their branch connector can now establish post-quantum Cloudflare IPsec tunnels to Cloudflare’s global network per draft-ietf-ipsecme-ikev2-mlkem.

The importance of being interoperable

Given that upgrading cryptography is hard and can take years, our 2029 target date for a full update to post-quantum cryptography is going to require concentrated effort. That’s why we hope the IPsec community continues to focus on the development of interoperable standards like draft-ietf-ipsecme-ikev2-mlkem.

Let us explain why these standards are vitally important. A full specification for hybrid ML-KEM in IPsec, draft-ietf-ipsecme-ikev2-mlkem, became available only in late 2025. That’s roughly four years after support for hybrid ML-KEM landed in TLS. (In fact, Cloudflare turned on hybrid post-quantum key agreement with TLS in 2022, even before NIST finalized the standardization of ML-KEM, because the TLS community quickly converged on a single, interoperable approach and pushed it into production. Today more than two-thirds of the human-generated TLS traffic to Cloudflare’s network is protected with hybrid ML-KEM.)

The four-year delay is likely due in part to the IPsec community’s continued interest in Quantum Key Distribution (QKD), as codified in RFC 8784, published in 2020. We’ve written before about why QKD is not part of our post-quantum strategy: QKD requires specialized hardware and a dedicated physical link between the two parties, which fundamentally means it will not operate at Internet scale. Also, QKD does not provide authentication, so you still need post-quantum cryptography anyway to stop active attackers. It’s difficult to find implementations of QKD that interoperate across vendors.   

The U.S. NSA, Germany’s BSI, and the UK’s NCSC have all warned against solely relying on QKD. Post-quantum cryptography, by contrast, runs on the hardware you already have, authenticates the parties at both ends, and works end-to-end across the Internet. 

RFC 9370, published in 2023, opened the door to post-quantum cryptography in IPsec, allowing up to seven key exchanges to be run in parallel with classical Diffie-Hellman. However, RFC 9370 did not specify which ciphersuites should be used in these parallel key exchanges. In the absence of that specification, some vendors shipped early implementations under RFC 9370 before the hybrid ML-KEM draft was available, defining their own ciphersuites including some which are not NIST-standardized. This is exactly the kind of “ciphersuite bloat” NIST SP 800 52r2 warned against. And the risks to interoperability have played out in practice: Cloudflare IPsec does not yet interoperate with Palo Alto Networks’ RFC 9370–based implementation, because it was launched before draft-ietf-ipsecme-ikev2-mlkem was available. 

Fortunately, we now have draft-ietf-ipsecme-ikev2-mlkem that fills in the gaps in RFC 9370, specifying hybrid ML-KEM as one of the key exchange mechanisms that can be operated in parallel with classical Diffie-Hellman. We hope to add Palo Alto Networks to the list of interoperable post-quantum branch connectors as the industry continues to consolidate around draft-ietf-ipsecme-ikev2-mlkem.

But the journey towards interoperable post-quantum IPsec standards is not over yet. While draft-ietf-ipsecme-ikev2-mlkem supports post-quantum encryption, we still need IPsec standards for post-quantum authentication, so that we can stop attacks by quantum adversaries on live systems after Q-Day. Given the shortened timeline for full post-quantum readiness, we hope the IPsec community will continue to focus on interoperable PQC implementations, rather than diverting focus to niche use cases with QKD.

Towards an interoperable post-quantum Internet

At Cloudflare, we’re helping make a secure and post-quantum Internet accessible to everyone, without specialized hardware and at no extra cost to our customers. Post-quantum Cloudflare IPsec is one more step on our path to full post-quantum security by 2029, and we’re doing it in a way that ensures that the Internet remains open and interoperable for years to come. 

How to clone an AWS CloudHSM cluster across Regions

Post Syndicated from Desiree Brunner original https://aws.amazon.com/blogs/security/how-to-clone-an-aws-cloudhsm-cluster-across-regions-2/

Important: As of January 1, 2025, Client SDK 3 tools (CMU and KMU) are no longer supported. This guide has been updated to use Client SDK 5 commands exclusively. Ensure you’re using the latest Client SDK 5 version (5.17 or later) for the most recent features and security improvements.

You can use AWS CloudHSM to generate, store, import, export, and manage your cryptographic keys. It also permits hash functions to compute message digests and hash-based message authentication codes (HMACs) and supports cryptographically signing data and verifying signatures. To help ensure redundancy of data and simplification of the disaster recovery process, AWS recommends you to clone your CloudHSM cluster into a different AWS Region. By doing this, you can synchronize keys, including non-exportable keys, across Regions. Non-exportable keys can only be synchronized to cloned clusters. Non-exportable keys are keys that can never leave the CloudHSM device in plaintext. They reside on the CloudHSM device and are encrypted for security purposes.

In this post, I show you how to set up one cluster in Region 1 and how to use the CopyBackupToRegion feature to clone the cluster and hardware security modules (HSMs) to a virtual private cloud (VPC) in Region 2.

Note: This post doesn’t include instructions on how to set up a cross-Region VPC to synchronize HSMs across the two cloned clusters. If you need to set up a cross-Region VPC, see Building a Scalable and Secure Multi-VPC AWS Network Infrastructure.

Solution overview

You clone a cluster to another Region in a two-step process:

  1. Copy a backup to the destination Region
  2. Create a new cluster from this backup

To complete this solution, you can use either the AWS Command Line Interface (AWS CLI) or the CloudHSM API. For this post, I show you how to use the AWS CLI to copy the cluster backup from Region 1 to Region 2 and then launch a new cluster from that copied backup.
Figure 1 illustrates the process described in this post.

Figure 1: Architecture diagram

Figure 1: Architecture diagram

Here’s how the process works:

  1. CloudHSM creates a backup of the cluster and stores it in an Amazon Simple Storage Service (Amazon S3) bucket owned by the CloudHSM service.
  2. You use the AWS CLI API command to copy the backup to another Region.
  3. When the backup is completed, you use that backup to then create a new cluster and HSMs.
Note: Backups can’t be copied across partitions like the AWS GovCloud Regions, China Region and AWS European Sovereign Cloud.

As with all cluster backups, when you copy the backup to a new Region, it’s stored in an S3 bucket owned by a CloudHSM account. CloudHSM manages the security and storage of cluster backups for you. This means the backup in both Regions will also have the durability of Amazon S3, which has 99.999999999% durability. The backup in Region 2 will be encrypted and secured in the same way as your backup in Region 1. You can read more about the encryption process of your CloudHSM backups in AWS CloudHSM cluster backups.
Any HSMs created in this cloned cluster will have the same users and keys as the original cluster at the time the backup was taken. From this point on, you must manually keep the cloned clusters in sync. Specifically:

  • If you create users after creating your new cluster from the backup, you must create them on both clusters manually.
  • If you change the password for a user in one cluster, you must change the password on the cloned clusters to match.
  • If you create more keys in one cluster, you must sync them to at least one HSM in the cloned cluster. After you sync the key from cluster 1 to cluster 2, the CloudHSM automated cluster synchronization will take care of syncing the keys in the second cluster.

Prerequisites

Before starting, ensure you have the following in place:

Note: Syncing keys across clusters in more than one Region will only work if all clusters are created from the same backup. This is because synchronization requires the same secret key—called a masking key—to be present on the source and destination HSM. The masking key is specific to each cluster. It can’t be exported, and can’t be used for any purpose other than synchronizing keys across HSMs in a cluster.

Step 1: Create your first cluster in Region 1

The first step in cloning your CloudHSM cluster is to create the initial cluster—which will serve as the foundation for your cross-Region deployment—in your source Region.

Create the cluster

Replace <SUBNET_ID_1> with one of your private subnets. Make a note of the cluster ID to use later:
aws cloudhsmv2 create-cluster --hsm-type hsm2m.medium --subnet-ids <SUBNET_ID_1>

Launch the EC2 client

Launch an Amazon Elastic Compute Cloud (Amazon EC2) instance in your public subnet. See Step 1 of Get started with Amazon EC2 for detailed steps.

Create the first HSM

Replace <CLUSTER_ID> with the ID you recorded earlier and <AVAILABILITY_ZONE> with the Availability Zone matching your private subnet (for example, us-east-1a):
aws cloudhsmv2 create-hsm --cluster-id <CLUSTER_ID> --availability-zone <AVAILABILITY_ZONE>

Initialize the cluster

Before you initialize the cluster, create a self-signed certificate and use it to sign the cluster’s certificate signing request (CSR). Once you have the signed certificate, initialize the cluster:

aws cloudhsmv2 initialize-cluster \
    --cluster-id <CLUSTER_ID> \
    --signed-cert file://<CLUSTER_ID>_CustomerHsmCertificate.crt \
    --trust-anchor file://customerCA.crt

Important: Copy the certificate used to sign your cluster’s CSR to to maintain a secure connection.

After the command completes, the cluster transitions to the Initialized state. Copy the certificate used to sign your cluster’s CSR to /opt/cloudhsm/etc so that the CloudHSM client can verify the cluster’s identity when you configure it in the next step:

sudo cp _CustomerHsmCertificate.crt /opt/cloudhsm/etc/
sudo cp customerCA.crt /opt/cloudhsm/etc/

Install the CloudHSM Client SDK 5

Download and install the latest CloudHSM Client SDK 5 (version 5.17 or later):
For example, for Amazon Linux 2023:

wget https://s3.amazonaws.com/cloudhsmv2-software/CloudHsmClient/Amzn2023/cloudhsm-cli-latest.amzn2023.x86_64.rpm
sudo yum install -y ./cloudhsm-cli-latest.amzn2023.x86_64.rpm

Configure the client

Configure the CloudHSM client with your HSM’s elastic network interface (ENI IP) address:
configure-cli -a <HSM_IP>

Activate the cluster

To activate the cluster, run the CloudHSM CLI in interactive mode.

cloudhsm-cli interactive

You can run user list to see the admin user, which is not yet activated.

aws-cloudhsm > user list
{
  "error_code": 0,
  "data": {
    "users": [
      {
        "username": "admin",
        "role": "unactivated-admin",
        "locked": "false",
        "mfa": [],
        "cluster-coverage": "full"
      },
      {
        "username": "app_user",
        "role": "internal(APPLIANCE_USER)",
        "locked": "false",
        "mfa": [],
        "cluster-coverage": "full"
      }
    ]
  }
}

Use the cluster activate command to set the initial admin password.

aws-cloudhsm > cluster activate
Enter password:<NewPassword>
Confirm password:<NewPassword>
{
  "error_code": 0,
  "data": "Cluster activation successful"
}

When completed, sign out using the command quit, then sign back in with the new password, using the command login --username admin --role admin.

After doing this, you can create the first crypto user (CU). You create the user by running the command: user create --username <USERNAME> --role crypto-user. For more information, see HSM user types for CloudHSM CLI. Crypto users are permitted to create and share keys on the CloudHSM.

When completed, sign out using the command quit.

Step 2: Create keys in Region 1

Create a non-exportable AES-256 key:

aws-cloudhsm > key generate-symmetric aes \
    --label aes-example \
    --key-length-bytes 32 \
    --attributes extractable=false

Make note of the key reference returned in the output, because you’ll need it for synchronization later.

Step 3: Trigger a backup of your cluster

To trigger a backup for Region 2:

  1. Add another HSM to your cluster in Region 1 (can be done using the AWS Management Console or AWS CLI)
  2. The backup will contain:
    • All users (crypto officers (COs), crypto users (CUs), and appliance users)
    • All key material on the HSMs
    • All configurations and policies
Note: The user portion is critical because keys can only be synced across clusters to the same user.

Record the backup ID to use later. You can find this in the CloudHSM console under Backups, or using the following command:

aws cloudhsmv2 describe-backups --cluster-id

To avoid unnecessary charges, you can delete the additional HSM after the backup is created.

Step 4: Copy your backup Between Regions

Before you can transfer the backup to your destination Region, you need to configure the appropriate IAM permissions to allow the copy operation.

IAM permissions

Ensure proper permissions are configured for your IAM role or user. You need CloudHSM administrator privileges. Here’s an example permissions policy:

{
   "Version": "2012-10-17",
   "Statement": {
      "Effect": "Allow",
      "Action": [
         "cloudhsm:*",
         "ec2:CreateNetworkInterface",
         "ec2:DescribeNetworkInterfaces",
         "ec2:DescribeNetworkInterfaceAttribute",
         "ec2:DetachNetworkInterface",
         "ec2:DeleteNetworkInterface",
         "ec2:CreateSecurityGroup",
         "ec2:AuthorizeSecurityGroupIngress",
         "ec2:AuthorizeSecurityGroupEgress",
         "ec2:RevokeSecurityGroupEgress",
         "ec2:DescribeSecurityGroups",
         "ec2:DeleteSecurityGroup",
         "ec2:CreateTags",
         "ec2:DescribeVpcs",
         "ec2:DescribeSubnets",
         "iam:CreateServiceLinkedRole"
      ],
      "Resource": "*"
   }
}

Copy the backup

To copy your backup from Region 1 to Region 2, you need:

  • The destination Region
  • The source cluster ID and backup ID (you can use either or both) found in the CloudHSM console

If you specify only the cluster ID, the most recent backup will be chosen. For a specific backup, use the backup ID.

aws cloudhsmv2 copy-backup-to-region \
    --destination-region <DESTINATION_REGION> \
    --backup-id <BACKUP_ID>

Example response:

{
    "DestinationBackup": {
        "SourceBackup": "backup-4kuraxsqetz",
        "SourceCluster": "cluster-kzlczlspnho",
        "CreateTimestamp": 1531742400,
        "SourceRegion": "us-east-1"
    }
}

After copying, you will see a new backup ID in your console. Use this to create your new cluster in Region 2:

aws cloudhsmv2 create-cluster \
    --hsm-type hsm2m.medium \
    --subnet-ids <SUBNET_ID_REGION_2> \
    --source-backup-id <BACKUP_ID_REGION_2> \

Certificate transfer

Copy the cluster certificate from the original cluster to the new Region:

  1. Open two terminal sessions (one for each HSM)
  2. Copy the certificate content from cluster 1
  3. Create and paste into a new file in cluster 2

The certificate is required for encrypted connections between your client and HSM instances.

Security group configuration

Add the cloned cluster’s Security Group to your EC2 client instance:

  1. Select the Security Group for your EC2 client in the EC2 console
  2. Choose “Add rules”
  3. Add a rule allowing traffic from the cluster’s Security Group ID on port 2225

Then retrieve the ENI IP address of the HSM in Region 2 using the following command, and make a note of the output—you will use it in the next step to configure cross-Region connectivity:

aws cloudhsmv2 describe-clusters \
    --filters clusterIds=<cluster_ID_region_2> \
    --region <region_2> \
    --query 'Clusters.Hsms.EniIp' \
    --output text

Step 5: Configure cross-Region connectivity

To enable the CloudHSM CLI to communicate with both clusters simultaneously, add the Region 2 cluster to your existing client configuration using the ENI IP address you retrieved in the previous step:

Step 6: Synchronize keys between clusters

To synchronize keys between your source and destination clusters, you first need to verify which users and keys exist before replicating them.

configure-cli add-cluster \
    --cluster-id <cluster_ID_region_2> \
    --endpoint <hsm_eni_ip_region_2> \
    --region <region_2>

The CloudHSM CLI will now communicate with both clusters simultaneously using the certificates already configured during the initial setup, enabling key synchronization using the masking key shared between cloned clusters.

List users and keys

First, verify users and list available keys:
# List all users
cloudhsm-cli user list

# List keys for specific user
cloudhsm-cli key list --username

Replicate keys

To replicate a key from Region 1 to Region 2:

cloudhsm-cli key replicate \
    --filter key-reference=<key_ref> \
    --source-cluster-id <source_cluster_ID> \
    --destination-cluster-id <destination_cluster_ID>

Verify the key replication by listing keys again:

cloudhsm-cli key list --username <username>

The output should show identical key references on both clusters. Repeat this process for any additional keys that you want to synchronize.

Points to remember

After cloning a cluster to a backup cluster, remember these important points:

  • Always manually update users across clusters after the initial backup
  • Use key replication for any keys created after the initial backup
  • Keep your Client SDK 5 tools updated for the latest features and security improvements
  • The January 1, 2025, end-of-support date for Client SDK 3 tools (CMU and KMU) means you should migrate to Client SDK 5 as soon as possible

Client SDK 5 supports ARM64 architecture on the following Linux distributions:

  • Amazon Linux 2023
  • Amazon Linux 2
  • Red Hat Enterprise Linux (RHEL) 8 (8.3+)
  • Red Hat Enterprise Linux (RHEL) 9 (9.2+)
  • Red Hat Enterprise Linux (RHEL) 10 (10.0+)
  • Ubuntu 22.04 LTS
  • Ubuntu 24.04 LTS
  • Debian 12
  • USE Linux Enterprise Server 15

Conclusion

You now have a fault-tolerant AWS CloudHSM environment with synchronized keys across Regions using the latest tools and best practices. By implementing this cross-Region cluster configuration, you gain improved disaster recovery capabilities, reduced risk of data loss, and enhanced business continuity for your cryptographic operations. This approach helps ensure that your critical cryptographic keys remain available even in the event of a Regional outage, providing the resilience that enterprise workloads demand.

If you have feedback about this post, submit comments in the Comments section below. For questions about this post, start a new thread on the AWS re:Post.

Desiree Brunner

Desiree Brunner

Desiree is a Security Specialist Solutions Architect working with regulated customers as part of the AWS EMEA Security & Compliance team. She builds on her background in DevOps and platform engineering to support her customers in designing secure, compliant cloud environments. Passionate about mental health and knowledge sharing, she regularly speaks at AWS events and supports teams on their cloud security journey.

Rickard Löfström

Rickard Löfström

Rickard guides enterprises in building secure cloud environments as a Specialist Solutions Architect in the AWS EMEA Security & Compliance team. He advises customers on implementing AWS security services, focusing on identity management, data protection, and infrastructure security controls. He enjoys translating complex security requirements into technical solutions that enable organizations to meet their security objectives while maintaining operational efficiency.

Secure AI agent access patterns to AWS resources using Model Context Protocol

Post Syndicated from Riggs Goodman III original https://aws.amazon.com/blogs/security/secure-ai-agent-access-patterns-to-aws-resources-using-model-context-protocol/

AI agents and coding assistants interact with AWS resources through the Model Context Protocol (MCP). Unlike traditional applications with deterministic code paths, agents reason dynamically, choosing different tools or accessing different data depending on context. You must assume an agent can do anything within its granted entitlements, whether OAuth scopes, API keys, or AWS Identity and Access Management (IAM) permissions, and design your controls accordingly. Agents operate at machine speed, so the impact of misconfigured permissions scales quickly.

This blog post focuses on IAM as the authorization layer for AWS resource access and presents three security principles for building deterministic IAM controls for these non-deterministic AI systems. The principles apply whether you’re using AI coding assistants like Kiro and Claude Code, or deploying agents on hosting environments like Amazon Bedrock AgentCore. We cover deployment patterns, then explore each principle with concrete IAM policy examples and implementation guidance.

This post specifically addresses securing the MCP access path, where agents interact with AWS resources through MCP servers. AI coding assistants and agents can also access AWS service APIs directly through general-purpose tools like bash or shell execution, bypassing MCP servers entirely. For this reason, we recommend architecting agents to use MCP servers rather than direct service access where possible. MCP servers provide a layer of abstraction that enables the differentiation controls in principle 3 and creates additional monitoring capabilities through AWS CloudTrail. When agents bypass MCP, the differentiation mechanisms in principle 3 don’t apply, and principles 1 and 2 become your primary controls. We discuss this scope boundary in principle 3.

MCP deployment patterns

Your deployment pattern determines which security principles and implementation approaches apply. Three dimensions define this pattern, including where the agent runs, what type of MCP server offers the tools, and your level of control over the agent code. No matter how you connect to it, the MCP server needs AWS credentials to interact with AWS resources.

Where agents run

Agents access AWS resources from three locations: developer machines (where you control the infrastructure), hosting environments (where you control the infrastructure or significant aspects of it), and third-party agent platforms (where you do not control the infrastructure). This post focuses on the first two patterns. Each has a different credential model and different organizational control options.

AI coding assistants and local agents

AI coding assistants (Kiro, Claude Code) or local agent applications represent the first deployment pattern. These assistants run locally on developer machines and connect to MCP servers or use AWS Command Line Interface (AWS CLI) commands to access AWS resources. In this pattern, credentials come from the developer’s local environment. When a developer configures an MCP server in their mcp.json file, they specify which AWS credentials to use. Options include a named profile, which can use credential helpers and the credential provider chain for short-lived credentials, environment variables, or explicit credential configuration. This means the developer controls which IAM principal the agent uses to access AWS. This creates a governance challenge. Without additional controls, developers often use their developer admin credentials, shared development roles, or even production roles for agent access. Developer credentials often carry broad permissions designed for interactive use, where human judgment serves as a safeguard. When an agent inherits these permissions, it operates without that judgment at machine speed. Principle 1 explores this risk in detail.

Agents on hosting environments

Agents deployed on hosting environments represent the second deployment pattern. These agents run on infrastructure you manage, not on developer machines. This changes the credential management model. Using Amazon Bedrock AgentCore as an example, when an agent runs on AgentCore Runtime, it uses an execution IAM role that you configure when creating the runtime. The execution role’s permissions apply to all operations the agent performs and cannot be scoped down per-invocation at the runtime configuration level. For more granular control, agents can call AWS Security Token Service (AWS STS) AssumeRole or AssumeRoleWithWebIdentity (collectively referred to as AssumeRole in this post). This obtains temporary credentials with session policies that further restrict permissions beyond the role’s base permissions. Agents built with frameworks like Strands can also initialize individual MCP clients with different credential sets by calling AssumeRole and passing the resulting credentials to each client connection. This enables per-tool credential isolation within a single agent process. The same pattern applies to agents deployed on Amazon Elastic Compute Cloud (Amazon EC2) or Amazon Elastic Kubernetes Service (Amazon EKS).

With this centralized execution model, you can implement organizational controls. You define the available IAM roles through infrastructure configuration instead of relying on developer choice. However, you must design these roles carefully to prevent overly permissive access and implement session policies for tool-specific restrictions.

What type of MCP server

MCP servers come in two types, provider-managed and self-managed. AWS-managed servers are operated by AWS on your behalf. Self-managed servers are servers that you install and run yourself. The server type affects your operational overhead, available features, and how you implement security controls.

AWS offers fully managed MCP servers, including the AWS MCP Server, Amazon EKS MCP Server, and Amazon ECS MCP Server. These AWS-managed servers run on AWS infrastructure and require no installation or maintenance on your part. AWS-managed MCP servers automatically add IAM context keys (aws:ViaAWSMCPService and aws:CalledViaAWSMCP) to every downstream AWS service call. You can write IAM policies that check these keys to distinguish between AI-driven actions and human-initiated actions without any additional configuration.

Self-managed MCP servers include AWS-provided servers from the AWS MCP GitHub repository that you install and run yourself. They also include custom MCP servers that you build from scratch. With self-managed servers, you control the deployment location (local machine, Amazon EC2, Amazon EKS), the configuration, and the maintenance. These servers can be used with either AI coding assistants running locally or agents deployed on hosting environments. The key difference for security controls is that self-managed servers don’t automatically add IAM context keys for differentiation. You must configure the MCP server to add session tags when assuming IAM roles if you require differentiation between AI-driven and human-initiated actions. This requires modifying your MCP server code to call AWS STS AssumeRole with tags attached. You then write IAM policies that check for these tags using the aws:PrincipalTag condition key. Self-managed servers can also be extended to implement dynamic authorization flows, such as mapping inbound OAuth tokens to outbound IAM role assumptions, giving you control over the full authorization chain. Additionally, with AWS-managed MCP servers, AWS injects context keys at the service layer, so callers cannot spoof them. With self-managed servers, the entity calling AssumeRole sets the session tags, so you must trust that your MCP server code hasn’t been modified.

The responsibility model differs between server types. With AWS-managed MCP servers, AWS is responsible for server infrastructure, patching, and context key injection. You’re responsible for IAM policy design and credential configuration. With self-managed MCP servers, you’re additionally responsible for server patching, dependency and library supply chain security, session tag implementation, and verifying server integrity. This connects to the supply chain risk described in principle 1. While self-managed servers require more operational overhead to implement and maintain, they give you flexibility and control.

Level of client control

A third dimension shapes your security implementation, whether you control the agent and MCP client code (code-controlled) or are limited to configuring pre-built tools without modifying their runtime behavior (configuration-bound). This determines which security mechanisms are available to you at runtime.

In configuration-bound scenarios, you use an AI coding assistant such as Kiro or Claude Code and configure credentials in your mcp.json file. You select which IAM role or profile the agent uses, but you cannot modify the agent’s runtime behavior. The agent calls AWS APIs using whatever credentials you configured ahead of time, and you cannot inject session policies or tags into those calls programmatically. Your security controls must be in place before the agent runs. You select narrowly scoped roles at configuration time, and your organization enforces guardrails through permission boundaries and service control policies (SCPs). These mechanisms restrict what the agent can do regardless of which role the developer selects.

In code-controlled scenarios, you build or deploy a custom agent on Amazon Bedrock AgentCore, Amazon EC2, Amazon EKS, or your local machine, or you build and run a custom MCP server. Because you control the runtime code, you can implement credential management programmatically. For custom agents, this means calling AssumeRole with session policies scoped to each tool invocation, attaching session tags for differentiation, and obtaining temporary credentials with the minimum permissions each operation requires. For custom MCP servers, you can inject session policies into every AWS API call the server makes, applying a consistent set of restrictions across all operations. Both approaches give you runtime IAM controls that are not available in config-bound scenarios.

Deployment pattern summary

The following table summarizes how these dimensions combine.

Source type MCP server type Client control Credential source Differentiation mechanism Example use case
AI coding assistant AWS-managed MCP Config-bound Local (AWS CLI, env vars, ) Automatic context keys Kiro calling AWS-managed MCP server
AI coding assistant Self-managed MCP (local or remote) Config-bound Local (AWS CLI, env vars, ) Manual session tags or session policies Kiro calling local AWS MCP server
Agent on hosting environment AWS-managed MCP Code-controlled Execution role or AssumeRole Automatic context keys Amazon Bedrock AgentCore agent calling AWS-managed MCP server
Agent on hosting environment Self-managed MCP (remote) Code-controlled Execution role or AssumeRole Manual session tags or session policies Agent calling AWS MCP server deployed on Amazon Bedrock AgentCore

Your deployment pattern and level of client control determine which of the following security principles apply and how you implement them.

Three security principles for agent access

With this understanding of deployment patterns, let’s explore the three security principles that apply across all patterns.

  • Principle 1 – Assume all granted permissions could be used: Design permissions based on the acceptable scope of impact, not intended functionality alone.
  • Principle 2 – Provide organizational guidance on role usage: Enforce permission design through role governance, session policies, permission boundaries, and organizational policies.
  • Principle 3 – Differentiate AI-driven from human-initiated actions: Apply different IAM rules based on whether the action comes from an agent or a human.

Security principle 1: Assume all granted permissions could be used

The first security principle is fundamental. Any permission you grant to an agent can be exercised, regardless of your intended use case. If you give an agent s3:DeleteObject permission with a tool that can call the API, you must assume it can delete any Amazon Simple Storage Service (Amazon S3) object it has access to. This can happen in ways you cannot predict or fully prevent through code review alone. This non-deterministic behavior requires a shift in your approach to IAM permissions.

Traditional applications follow deterministic code paths. You can review the source code, identify every API call, and grant the permissions needed. AI agents operate differently. They make decisions at runtime based on reasoning, context, and learned patterns. You cannot predict which AWS APIs or tools an agent will call or which resources it will access. Static analysis of agent code tells you what tools are available, but not which tools will be invoked or how they’ll be used.

This creates a challenge when developers configure agents to use AWS credentials. Developers commonly use existing IAM roles, such as the role their traditional application uses or their local admin role for the AWS CLI. These roles were designed assuming predictable behavior and human judgment. Your local admin role has s3:* permissions because you exercise judgment on what to delete and when. You understand the context, recognize production resources, and can assess the impact of your actions.

An agent with that same role operates at machine speed without human judgment. It can delete production data through hallucination or be directed through prompt injection to perform unintended actions. It can also make a logical error in its reasoning that leads to unintended operations. The speed and scale at which agents operate increases the potential scope of these issues. An agent can make thousands of API calls in seconds, so the impact of misconfigured permissions scales quickly.

Consider the following scenarios with overly permissive access.

  • Hallucination: The agent misinterprets a user request and performs the wrong action. An agent designed to clean up temporary files might hallucinate that production data is temporary and delete it.
  • Prompt injection: An outside party crafts unexpected input that influences the agent’s reasoning. An agent designed to query Amazon DynamoDB tables could be directed to call dynamodb:PutItem or dynamodb:DeleteItem on resources outside its intended scope.
  • Logic errors: The agent’s reasoning leads to an incorrect conclusion. An agent analyzing S3 storage costs might conclude that frequently accessed production data is unused and delete it to save costs.
  • Tool poisoning: A compromised MCP server or dependency performs unintended operations using the agent’s credentials. An agent with broad S3 and DynamoDB permissions connects to an MCP server whose dependency has been modified to exfiltrate data. The compromised tool reads sensitive objects and writes them to an attacker-controlled location, all within the agent’s granted permissions.

This security principle reframes how you approach IAM permissions for agents. Instead of asking what does the agent need to do?, ask what is the scope of impact if the agent acts outside its intended use case? Design permissions based on the acceptable scope of access, not only on intended functionality. If an agent needs to read S3 objects, grant s3:GetObject, not s3:*. If it needs to write to specific paths, use resource-level conditions to restrict access to those paths. Consider what tools the agent has access to and what API calls those tools can make. Design permissions that limit what the agent is allowed to perform based on organizational policy. This doesn’t mean agents can’t have write or delete permissions. It means you and your organization must consider what resources those permissions apply to and what safeguards are in place.

Beyond IAM policies, consider implementing data perimeters as an additional layer of defense. Data perimeters use VPC endpoint policies, resource control policies (RCPs), resource policies, and service control policies (SCPs) to restrict access based on identity, resource, and network boundaries. For agents, data perimeters help verify that even if IAM permissions are broader than intended, access is limited to trusted resources from expected networks. For more information, see Building a data perimeter on AWS.

Practical implementation guidance:

  • Apply least privilege rigorously: If an agent needs read access, grant read permissions. If it needs write access, grant write to specific resources, not all resources of that type.
  • Use resource-level restrictions: Employ IAM policy conditions to limit permissions to specific buckets, paths, tables, or other resources. Don’t grant blanket permissions across all resources.
  • Consider read-only alternatives: Evaluate whether the agent’s task can be accomplished with read-only access. Many analysis and reporting tasks don’t require write or delete permissions.
  • Implement comprehensive monitoring: Set up Amazon CloudWatch alarms for unexpected agent actions, unusual access patterns, or operations on sensitive resources. Monitor for sensitive operations like deletions or modifications to production resources.
  • Conduct regular permission audits: As agents gain new tools and capabilities, developers often add permissions incrementally without removing unused ones. An agent that started with read-only access can gradually accumulate write and delete permissions across multiple services. Review agent IAM roles and policies regularly to identify and remove permissions that are no longer needed.
  • Verify MCP server integrity: Verify the provenance and integrity of MCP servers before granting them access to AWS credentials. Maintain an organizational registry of approved MCP servers and their expected behavior, and monitor for unauthorized server deployments that might have assumed execution roles. For more on agentic application risks, see the OWASP Top 10 for Agentic Applications.

Security principle 1 establishes the foundation. Understand the scope of every permission you grant. The next two security principles build on this foundation.

Security principle 2: Provide organizational guidance on role usage

The second security principle addresses organizational governance. Principle 1 requires that you design permissions based on acceptable scope of impact. Principle 2 addresses how your organization enforces that design through role governance, session policies, permission boundaries, and organizational policies.

When developers adopt AI coding assistants and configure MCP servers, they choose which credentials to use. Without organizational controls, developers often use existing roles (such as personal admin roles, shared development roles, or production roles) that were designed for human use with far more permissions than agents need. For agents deployed on hosting environments, you configure execution roles, but the same question applies. What permissions should those roles have, and how do you enforce consistency across deployments? The answer depends on your level of client control.

When you control the agent code

When you build or deploy custom agents on Amazon Bedrock AgentCore, Amazon EC2, Amazon EKS, or locally, you control the runtime code and can implement dynamic credential management. This is the strongest enforcement model because you can scope permissions per tool invocation at runtime. The same applies if you build or modify a custom MCP server. Because you control the server code, you can inject session policies into every AWS API call the server makes.

The IAM role defines the permission ceiling for the agent across all its tools. Instead of creating a separate role for every tool or MCP server, you use session policies to scope down the role’s permissions per operation. When the agent invokes a specific tool, it calls AssumeRole with a session policy that restricts permissions to just what that tool requires. The effective permissions are the intersection of the role’s policies and the session policy. Session policies restrict permissions but never expand them. If a role grants broad permissions but you attach the ReadOnlyAccess managed policy as a session policy, the agent can only perform read operations. You can also use inline session policies for resource-specific restrictions, such as limiting access to specific S3 buckets or DynamoDB tables.

The following example shows how to implement session policies in agent code.

import boto3

# Uses the execution IAM role as part of AgentCore Runtime
sts = boto3.client('sts')

# Assume role with ReadOnlyAccess managed policy as session policy
response = sts.assume_role(
    RoleArn='arn:aws:iam::111122223333:role/AgentDataRole',
    RoleSessionName='agent-data-reader',
    PolicyArns=[
        {'arn': 'arn:aws:iam::aws:policy/ReadOnlyAccess'}
    ],
    DurationSeconds=3600
)

# Use the temporary credentials
credentials = response['Credentials']
s3 = boto3.client(
    's3',
    aws_access_key_id=credentials['AccessKeyId'],
    aws_secret_access_key=credentials['SecretAccessKey'],
    aws_session_token=credentials['SessionToken']
)

For agents on hosting environments like Amazon Bedrock AgentCore, the execution role serves two purposes. It’s the trust anchor that lets the agent call AssumeRole for tool-specific credentials, and it can supply baseline permissions that all operations need, such as writing logs to CloudWatch. For tool-specific operations that access customer resources, use AssumeRole with session policies to obtain scoped temporary credentials rather than using the execution role’s permissions directly. This centralized execution model simplifies enforcing consistent session policies across all agent deployments. Agents can also attach tags when assuming roles for differentiation purposes (covered in Security principle 3).

When you’re configuration bound

When you use an AI coding assistant like Kiro or Claude Code with off-the-shelf MCP servers, you configure credentials in your mcp.json file but cannot modify the agent’s runtime behavior. Your security controls must be established before the agent runs.

Your first control is role selection. As described in the preceding deployment patterns section, AI coding assistants use credentials from the developer’s local environment. Create agent-specific IAM roles with narrower permissions than equivalent human roles, and direct developers to use them. For self-managed MCP servers running locally, the developer specifies the role through environment variables in the mcp.json configuration.

{
  "mcpServers": {
    "awslabs.aws-pricing-mcp-server": {
      "command": "uvx",
      "args": ["awslabs.aws-pricing-mcp-server@latest"],
      "env": {
        "AWS_PROFILE": "agent-dev-role",
        "AWS_REGION": "us-east-1"
      }
    }
  }
}

For AWS-managed MCP servers, the developer connects through the mcp-proxy-for-aws proxy and specifies the role through the profile parameter.

{
  "mcpServers": {
    "aws-mcp": {
      "command": "uvx",
      "args": [
        "mcp-proxy-for-aws@latest",
        "https://aws-mcp.us-east-1.api.aws/mcp",
        "--profile", "agent-dev-role",
        "--metadata", "AWS_REGION=us-east-1"
      ]
    }
  }
}

Only role selection depends on developer compliance. IAM permission boundaries provide organizational enforcement without requiring code changes or developer cooperation. A permission boundary is a managed policy that your security team attaches to an IAM role to set the maximum permissions that role can grant. The effective permissions are the intersection of the role’s identity-based policies and the permission boundary. Permission boundaries are most effective on agent-specific roles that your organization creates for agent use. They ensure those roles cannot exceed their intended permissions even if misconfigured. If a developer configures their existing role in mcp.json instead, a permission boundary on that role restricts all use of the role, not just agent use. For AWS-managed MCP servers, principle 3’s context keys address this gap. They let you write IAM policies that restrict actions only when they come through an MCP server, leaving the developer’s direct use of the same role unaffected. For self-managed MCP servers, modifying the server code to AssumeRole into an organization-defined role provides a similar override, and session tags can be attached during that AssumeRole for differentiation (see principle 3). For multi-account environments, SCPs in AWS Organizations provide guardrails at the account or organizational unit level. SCPs set the maximum permissions for all principals in an account, giving your central governance team control over agent permissions across your organization.

Organizational governance at scale

Whether your agents are config-bound or code-controlled, you need organizational mechanisms to enforce consistent governance across teams and accounts.

Tag IAM roles intended for agent use with a consistent identifier, such as a tag key of Usage with a value of Agent. This lets your governance team inventory all agent roles across accounts, identify roles that don’t have permission boundaries, and distinguish agent roles from human roles in audit reports. You can also use tag-based conditions in SCPs to enforce that only properly tagged roles are used for agent operations. For AWS-managed MCP servers, the automatic context keys (principle 3) provide this identification without requiring role tags, but tagging remains useful for role inventory and audit purposes.

Use CloudTrail to monitor all API calls made by agent sessions and set up CloudWatch alarms for sensitive operations like resource deletion or permission changes. Principle 3 covers how to filter and analyze agent activity using context keys (AWS-managed MCP) and session tags (self-managed MCP).

For multi-account environments, combine SCPs with permission boundaries and resource control policies (RCPs) for layered enforcement. SCPs set the maximum permissions for principals within your organization at the account or organizational unit level, while permission boundaries constrain individual roles. RCPs enforce controls at the resource level regardless of the caller’s organizational membership, protecting resources even from cross-account access. Verify that the AWS services you use support MCP context keys in RCP evaluation. This layered approach gives your central governance team control over agent permissions across your organization, even when individual teams manage their own accounts and roles. Conduct quarterly reviews of agent roles and session policies to identify permissions that are no longer needed as agent capabilities evolve.

Practical implementation guidance:

  • For code controlled agents: Implement session policies for every tool invocation. Use AssumeRole with the minimum permissions each operation requires rather than relying on the execution role’s base permissions.
  • For config-bound agents: Create agent-specific IAM roles with narrower permissions than human roles or configure self-managed MCP servers to AssumeRole into an organization-defined role. Have your security team attach permission boundaries to agent-specific roles to enforce maximum permissions regardless of developer role selection.
  • At the organization level: Tag agent roles consistently, enforce guardrails through SCPs, and monitor agent activity through CloudTrail. Conduct quarterly reviews to remove unused permissions.

Security principle 2 gives you organizational control over agent permissions through mechanisms matched to your level of client control. Session policies and dynamic credential scoping enforce permissions at runtime for code-controlled agents. Permission boundaries and SCPs enforce permissions at the organizational level for config-bound agents. The next principle adds a complementary layer of governance at the resource level based on whether a human or agent is performing the action.

Security principle 3: Differentiate AI-driven from human-initiated actions

The third security principle adds an additional level of control on top of principle 2. Where principle 2 governs what permissions an agent has, this principle governs what the agent can do with those permissions based on whether the action is AI-driven or human-initiated.

This principle is essential for two reasons. For AWS-managed MCP servers, you cannot modify the server code to inject session policies or call AssumeRole with scoped credentials. The developer’s credentials flow through as-is. Context keys are your primary mechanism to restrict agent actions differently from human-initiated actions on the same role. For self-managed MCP servers where principle 2’s session policies are already in place, differentiation adds a second layer of defense at the resource level. Even if the session policy is broader than intended, differentiation policies can deny specific dangerous operations when performed through an agent.

For example, you can allow both humans and agents to read Amazon S3 objects, but deny delete operations when accessed through agents. Without a differentiation mechanism, IAM policies can’t distinguish between AI-driven actions and human-initiated actions. If a developer has s3:DeleteObject permission and uses an agent with their credentials, the agent also has s3:DeleteObject permission with no way to restrict it.

Differentiation gives you granular governance. Allow human-initiated actions with broad permissions while restricting agent actions to narrower permissions. Apply different rules based on context and implement progressive restrictions. Allow read operations for everyone, require approval for AI-driven write operations, and deny delete operations for agent actions entirely. Maintain audit trails showing which actions were AI-driven versus human-initiated, essential for compliance and security investigations.

When agents bypass MCP servers

Differentiation through condition keys and session tags applies when the agent accesses AWS through an MCP server. AI coding assistants like Kiro and Claude Code have access to general-purpose tools, including bash, shell, and code execution. When an agent uses a bash tool to run an AWS CLI command like aws s3 rm s3://my-bucket/my-object or executes a Python script that calls boto3 directly, the request goes straight to AWS using the developer’s existing credentials. The request bypasses MCP servers entirely. The aws:ViaAWSMCPService condition key isn’t set, session tags from MCP server AssumeRole calls aren’t applied, and IAM policies conditioned on these values don’t evaluate.

This means a deny policy like “Condition": {"Bool": {"aws:ViaAWSMCPService": “true"}} blocks the agent when it calls Amazon S3 through a managed MCP server, but doesn’t block the same agent when it runs the equivalent AWS CLI command through a bash tool. The agent has two paths to the same AWS API, and differentiation controls govern one path.

The condition keys work as designed, differentiating MCP-mediated access from direct access. This is a scope boundary. Differentiation controls secure the MCP access path. For the direct access path, principles 1 and 2 are your controls. Least privilege on the underlying IAM role (principle 1) and organizational guardrails like permission boundaries and SCPs (principle 2) apply regardless of how the agent reaches AWS. If the role doesn’t have s3:DeleteObject permission, the agent can’t delete objects through a bash tool or through an MCP server.

Restricting which tools an agent can access is a complementary control outside the scope of IAM. You can use agent frameworks and hosting environments such as Amazon Bedrock AgentCore to limit the set of available tools, removing general-purpose execution capabilities for agents that interact with AWS exclusively through MCP servers. When you combine tool restriction with the IAM controls in this post, you close the gap between the MCP access path and the direct access path.

AWS-managed MCP servers: Automatic context keys

AWS-managed MCP servers, including the AWS MCP Server, Amazon EKS MCP Server, and Amazon ECS MCP Server, offer differentiation by default. They automatically add IAM context keys to every downstream AWS service call. These context keys are aws:ViaAWSMCPService, a boolean set to true when the request comes through any AWS-managed MCP server. The second key is aws:CalledViaAWSMCP, a string containing the MCP server name like aws-mcp.amazonaws.com, eks-mcp.amazonaws.com, or ecs-mcp.amazonaws.com. No configuration is required on your part. You only need to write IAM policies that check for these keys to apply different rules for agent actions.

The following IAM policy denies delete operations when accessed through any AWS-managed MCP server.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "AllowS3ReadOperations",
    "Effect": "Allow",
    "Action": [
      "s3:GetObject",
      "s3:ListBucket"
    ],
    "Resource": "*"
  }, {
    "Sid": "DenyDeleteWhenAccessedViaMCP",
    "Effect": "Deny",
    "Action": [
      "s3:DeleteObject",
      "s3:DeleteBucket"
    ],
    "Resource": "*",
    "Condition": {
      "Bool": {
        "aws:ViaAWSMCPService": "true"
      }
    }
  }]
}

When a request doesn’t come through an AWS-managed MCP server, the aws:ViaAWSMCPService condition key isn’t present in the request context. The Deny statement only applies when the key is explicitly set to true, so human-initiated actions are unaffected by this policy.

You can also restrict operations to specific MCP servers. With this policy, you can run EKS operations only when accessed through the EKS MCP server, not through the AWS API MCP server.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "AllowEKSOperationsViaEKSMCP",
    "Effect": "Allow",
    "Action": "eks:*",
    "Resource": "*",
    "Condition": {
      "StringEquals": {
        "aws:CalledViaAWSMCP": "eks-mcp.amazonaws.com"
      }
    }
  }, {
    "Sid": "DenyEKSOperationsViaOtherMCP",
    "Effect": "Deny",
    "Action": "eks:*",
    "Resource": "*",
    "Condition": {
      "Bool": {
        "aws:ViaAWSMCPService": "true"
      },
      "StringNotEquals": {
        "aws:CalledViaAWSMCP": "eks-mcp.amazonaws.com"
      }
    }
  }]
}

Self-managed MCP servers: Manual session tags

Self-managed MCP servers, whether AWS-provided servers from the AWS MCP GitHub repository or custom servers you build yourself, don’t automatically add IAM context keys. To implement differentiation with self-managed servers, you must configure the MCP server to add session tags when assuming IAM roles. This requires modifying your MCP server to call AWS STS AssumeRole with tags attached. The tags remain active for the duration of the assumed role session and can be referenced in IAM policies using the aws:PrincipalTag condition key. This approach gives you flexibility and control over the session tag configuration. To maintain consistency, verify that all MCP server instances add the appropriate tags.

The following example shows how to configure your MCP server to add session tags.

import boto3

sts = boto3.client('sts')

response = sts.assume_role(
    RoleArn='arn:aws:iam::111122223333:role/MCPServerRole',
    RoleSessionName='mcp-server-session',
    Tags=[
        {'Key': 'AccessType', 'Value': 'AI'},
        {'Key': 'Source', 'Value': 'AgentRuntime'},
        {'Key': 'MCPServer', 'Value': 'org-data-server'}
    ]
)

# Use the temporary credentials from response['Credentials']
credentials = response['Credentials']

After your MCP server has added session tags, you can write IAM policies that check for these tags to differentiate agent actions.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "AllowS3ReadOperations",
    "Effect": "Allow",
    "Action": [
      "s3:GetObject",
      "s3:ListBucket"
    ],
    "Resource": "*"
  }, {
    "Sid": "DenyDeleteWhenAccessedViaAI",
    "Effect": "Deny",
    "Action": [
      "s3:DeleteObject",
      "s3:DeleteBucket"
    ],
    "Resource": "*",
    "Condition": {
      "StringEquals": {
        "aws:PrincipalTag/AccessType": "AI"
      }
    }
  }]
}

Session tags and session policies are both passed to AssumeRole, but serve different purposes. Session policies (covered in security principle 2) constrain what permissions the agent has. Session tags (covered here in security principle 3) mark the session as AI-driven, enabling IAM policies to differentiate between agent and human actions. You can use both in the same AssumeRole call for defense-in-depth. The session policy constrains what the agent can do. The session tags let IAM policies apply different rules based on the actor type.

The following example uses both session policies and session tags together.

import boto3

sts = boto3.client('sts')

# Assume role with both managed session policy and tags
response = sts.assume_role(
    RoleArn='arn:aws:iam::111122223333:role/AgentDataRole',
    RoleSessionName='agent-data-reader',
    PolicyArns=[                              # Principle 2: Constrains permissions
        {'arn': 'arn:aws:iam::aws:policy/ReadOnlyAccess'}
    ],
    Tags=[                                    # Principle 3: Enables differentiation
        {'Key': 'AccessType', 'Value': 'AI'},
        {'Key': 'Source', 'Value': 'AgentRuntime'},
        {'Key': 'MCPServer', 'Value': 'org-data-server'}
    ],
    DurationSeconds=3600
)

CloudTrail logging and audit trails

Both differentiation mechanisms generate CloudTrail logs for audit trails. For AWS-managed MCP servers, downstream AWS API calls include the MCP service identifier in the invokedBy, sourceIPAddress, and userAgent fields. You can filter on these fields to isolate agent activity. MCP-originated downstream calls are classified as data events, so you must enable data event logging on your CloudTrail trail to capture them.

{
  "eventVersion": "1.11",
  "userIdentity": {
    "type": "AssumedRole",
    "principalId": "AROAEXAMPLE:developer-session",
    "arn": "arn:aws:sts::111122223333:assumed-role/DeveloperRole/developer-session",
    "accountId": "111122223333",
    "sessionContext": {
      "sessionIssuer": {
        "type": "Role",
        "principalId": "AROAEXAMPLE",
        "arn": "arn:aws:iam::111122223333:role/DeveloperRole",
        "accountId": "111122223333",
        "userName": "DeveloperRole"
      }
    },
    "invokedBy": "aws-mcp.amazonaws.com"
  },
  "eventSource": "s3.amazonaws.com",
  "eventName": "GetObject",
  "sourceIPAddress": "aws-mcp.amazonaws.com",
  "userAgent": "aws-mcp.amazonaws.com",
  "eventType": "AwsApiCall",
  "managementEvent": false,
  "eventCategory": "Data"
}

For self-managed MCP servers with session tags, the tags appear in the requestParameters.principalTags field of the AssumeRole CloudTrail event. You can correlate the session name from the AssumeRole event to downstream API calls to trace agent activity.

{
  "eventSource": "sts.amazonaws.com",
  "eventName": "AssumeRole",
  "requestParameters": {
    "roleArn": "arn:aws:iam::111122223333:role/MCPServerRole",
    "roleSessionName": "mcp-server-session",
    "principalTags": {
      "AccessType": "AI",
      "Source": "AgentRuntime",
      "MCPServer": "org-data-server"
    }
  }
}

With these logs, you can query CloudTrail to find all AI-driven actions and analyze patterns of agent behavior. You can also identify unexpected or unauthorized operations and maintain compliance audit trails. Set up CloudWatch alarms to detect agent actions on sensitive resources or unusual patterns that indicate unintended access or misconfiguration.

Things to consider

When deciding between AWS-managed and self-managed MCP servers, consider the trade-offs. AWS-managed MCP servers offer the most straightforward path. Context keys are added automatically with no configuration on your part. Self-managed MCP servers require modifying code to add session tags. However, they give you complete control over the tags and let you implement custom functionality not available in AWS-managed servers. Organizations can use both approaches, AWS-managed servers for standard AWS operations and self-managed servers for specialized use cases.

Practical implementation guidance:

  • Assess direct access paths: Evaluate whether your agents have access to general-purpose tools (bash, shell, code execution) that can bypass MCP servers. If they do, rely on principles 1 and 2 for those paths and consider restricting tool availability where possible.
  • Choose a differentiation mechanism: Select based on your MCP server type (for managed, use context keys, for self-managed, use session tags).
  • For AWS-managed MCP: Write IAM policies that check aws:ViaAWSMCPService and aws:CalledViaAWSMCP condition keys. No MCP server configuration needed.
  • For self managed MCP: Modify MCP server code to add session tags when assuming roles. Verify consistent tag application across all instances.
  • Update IAM policies: Add differentiation conditions to existing policies. Test in non-production first to verify behavior.
  • Monitor CloudTrail logs: Verify differentiation is working by checking for context keys or session tags in CloudTrail events.
  • Set up alerts: Configure CloudWatch alarms for AI-driven sensitive operations or policy violations.
  • Perform regular audits: Review IAM policies quarterly to verify differentiation conditions remain correct as agent capabilities evolve.

Conclusion

Securing AI agent access to AWS resources requires building deterministic IAM controls for non-deterministic AI systems. The three security principles give you a defense-in-depth framework that adapts to your deployment pattern and level of client control.

Your implementation path depends on your situation. Start with principle 1. Audit current agent permissions and default to read-only access where possible. Next, implement principle 2. For config-bound scenarios, establish permission boundaries and select agent-specific roles. For code-controlled scenarios, implement dynamic session policies scoped to each tool invocation. Finally, add principle 3 differentiation based on your MCP server type. Use automatic context keys with AWS-managed MCP servers, or configure session tags with self-managed servers.

By applying these three security principles, you can use AI agents while maintaining the governance and compliance controls your organization requires.

Riggs Goodman III

Riggs Goodman III

Riggs is a Principal Solution Architect at AWS. His current focus is on AI security, providing technical guidance, architecture patterns, and leadership for customers and partners to build AI workloads on AWS. Internally, Riggs focuses on driving overall technical strategy and innovation across AWS service teams to address customer and partner challenges.

Scaling MCP adoption: Our reference architecture for simpler, safer and cheaper enterprise deployments of MCP

Post Syndicated from Sharon Goldberg original https://blog.cloudflare.com/enterprise-mcp/

We at Cloudflare have aggressively adopted Model Context Protocol (MCP) as a core part of our AI strategy. This shift has moved well beyond our engineering organization, with employees across product, sales, marketing, and finance teams now using agentic workflows to drive efficiency in their daily tasks. But the adoption of agentic workflow with MCP is not without its security risks. These range from authorization sprawl, prompt injection, and supply chain risks. To secure this broad company-wide adoption, we have integrated a suite of security controls from both our Cloudflare One (SASE) platform and our Cloudflare Developer platform, allowing us to govern AI usage with MCP without slowing down our workforce. 

In this blog we’ll walk through our own best practices for securing MCP workflows, by putting different parts of our platform together to create a unified security architecture for the era of autonomous AI. We’ll also share two new concepts that support enterprise MCP deployments:

We also talk about how our organization approached deploying MCP, and how we built out our MCP security architecture using Cloudflare products including remote MCP servers, Cloudflare Access, MCP server portals and AI Gateway. 

Remote MCP servers provide better visibility and control

MCP is an open standard that enables developers to build a two-way connection between AI applications and the data sources they need to access. In this architecture, the MCP client is the integration point with the LLM or other AI agent, and the MCP server sits between the MCP client and the corporate resources.


The separation between MCP clients and MCP servers allows agents to autonomously pursue goals and take actions while maintaining a clear boundary between the AI (integrated at the MCP client) and the credentials and APIs of the corporate resource (integrated at the MCP server). 

Our workforce at Cloudflare is constantly using MCP servers to access information in various internal resources, including our project management platform, our internal wiki, documentation and code management platforms, and more. 

Very early on, we realized that locally-hosted MCP servers were a security liability. Local MCP server deployments may rely on unvetted software sources and versions, which increases the risk of supply chain attacks or tool injection attacks. They prevent IT and security administrators from administrating these servers, leaving it up to individual employees and developers to choose which MCP servers they want to run and how they want to keep them up to date. This is a losing game.


Instead, we have a centralized team at Cloudflare that manages our MCP server deployment across the enterprise. This team built a shared MCP platform inside our monorepo that provides governed infrastructure out of the box. When an employee wants to expose an internal resource via MCP, they first get approval from our AI governance team, and then they copy a template, write their tool definitions, and deploy, all the while inheriting default-deny write controls with audit logging, auto-generated CI/CD pipelines, and secrets management for free. This means standing up a new governed MCP server is minutes of scaffolding. The governance is baked into the platform itself, which is what allowed adoption to spread so quickly. 

Our CI/CD pipeline deploys them as remote MCP servers on custom domains on Cloudflare’s developer platform. This gives us visibility into which MCPs servers are being used by our employees, while maintaining control over software sources. As an added bonus, every remote MCP server on the Cloudflare developer platform is automatically deployed across our global network of data centers, so MCP servers can be accessed by our employees with low latency, regardless of where they might be in the world.

Cloudflare Access provides authentication

Some of our MCP servers sit in front of public resources, like our Cloudflare documentation MCP server or Cloudflare Radar MCP server, and thus we want them to be accessible to anyone. But many of the MCP servers used by our workforce are sitting in front of our private corporate resources. These MCP servers require user authentication to ensure that they are off limits to everyone but authorized Cloudflare employees. To achieve this, our monorepo template for MCP servers integrates Cloudflare Access as the OAuth provider. Cloudflare Access secures login flows and issues access tokens to resources, while acting as an identity aggregator that verifies end user single-sign on (SSO), multifactor authentication (MFA), and a variety of contextual attributes such as IP addresses, location, or device certificates. 

MCP server portals centralize discovery and governance


MCP server portals unify governance and control for all AI activity.

As the number of our remote MCP servers grew, we hit a new wall: discovery. We wanted to make it easy for every employee (especially those that are new to MCP) to find and work with all the MCP servers that are available to them. Our MCP server portals product provided a convenient solution. The employee simply connects their MCP client to the MCP server portal, and the portal immediately reveals every internal and third-party MCP servers they are authorized to use. 

Beyond this, our MCP server portals provide centralized logging, consistent policy enforcement and data loss prevention (DLP guardrails). Our administrators can see who logged into what MCP portal and create DLP rules that prevent certain data, like personally identifiable data (PII), from being shared with certain MCP servers.

We can also create policies that control who has access to the portal itself, and what tools from each MCP server should be exposed. For example, we could set up one MCP server portal that is only accessible to employees that are part of our finance group that exposes just the read-only tools for the MCP server in front of our internal code repository. Meanwhile, a different MCP server portal, accessible only to employees on their corporate laptops that are in our engineering team, could expose more powerful read/write tools to our code repository MCP server.

An overview of our MCP server portal architecture is shown above. The portal supports both remote MCP servers hosted on Cloudflare, and third-party MCP servers hosted anywhere else. What makes this architecture uniquely performant is that all these security and networking components run on the same physical machine within our global network. When an employee’s request moves through the MCP server portal, a Cloudflare-hosted remote MCP server, and Cloudflare Access, their traffic never needs to leave the same physical machine. 

Code Mode with MCP server portals reduces costs

After months of high-volume MCP deployments, we’ve paid out our fair share of tokens. We’ve also started to think most people are doing MCP wrong.

The standard approach to MCP requires defining a separate tool for every API operation that is exposed via an MCP server. But this static and exhaustive approach quickly exhausts an agent’s context window, especially for large platforms with thousands of endpoints.

We previously wrote about how we used server-side Code Mode to power Cloudflare’s MCP server, allowing us to expose the thousands of end-points in Cloudflare API while reducing token use by 99.9%. The Cloudflare MCP server exposes just two tools: a search tool lets the model write JavaScript to explore what’s available, and an execute tool lets it write JavaScript to call the tools it finds. The model discovers what it needs on demand, rather than receiving everything upfront.

We like this pattern so much, we had to make it available for everyone. So we have now launched the ability to use the “Code Mode” pattern with MCP server portals. Now you can front all of your MCP servers with a centralized portal that performs audit controls and progressive tool disclosure, in order to reduce token costs.

Here is how it works. Instead of exposing every tool definition to a client, all of your underlying MCP servers collapse into just two MCP portal tools: portal_codemode_search and portal_codemode_execute. The search tool gives the model access to a codemode.tools() function that returns all the tool definitions from every connected upstream MCP server. The model then writes JavaScript to filter and explore these definitions, finding exactly the tools it needs without every schema being loaded into context. The execute tool provides a codemode proxy object where each upstream tool is available as a callable function. The model writes JavaScript that calls these tools directly, chaining multiple operations, filtering results, and handling errors in code. All of this runs in a sandboxed environment on the MCP server portal powered by Dynamic Workers. 

Here is an example of an agent that needs to find a Jira ticket and update it with information from Google Drive. It first searches for the right tools:

// portal_codemode_search
async () => {
 const tools = await codemode.tools();
 return tools
  .filter(t => t.name.includes("jira") || t.name.includes("drive"))
  .map(t => ({ name: t.name, params: Object.keys(t.inputSchema.properties || {}) }));
}

The model now knows the exact tool names and parameters it needs, without the full schemas of tools ever entering its context. It then writes a single execute call to chain the operations together:

// portal_codemode_execute
async () => {
 const tickets = await codemode.jira_search_jira_with_jql({
  jql: ‘project = BLOG AND status = “In Progress”’,
  fields: [“summary”, “description”]
 });
 const doc = await codemode.google_workspace_drive_get_content({
  fileId: “1aBcDeFgHiJk”
 });
 await codemode.jira_update_jira_ticket({
  issueKey: tickets[0].key,
  fields: { description: tickets[0].description + “\n\n” + doc.content }
 });
 return { updated: tickets[0].key };
}

This is just two tool calls. The first discovers what’s available, the second does the work. Without Code Mode, this same workflow would have required the model to receive the full schemas of every tool from both MCP servers upfront, and then make three separate tool invocations.

Let’s put the savings in perspective: when our internal MCP server portal is connected to just four of our internal MCP servers, it exposes 52 tools that consume approximately 9,400 tokens of context just for their definitions. With Code Mode enabled, those 52 tools collapse into 2 portal tools consuming roughly 600 tokens, a 94% reduction. And critically, this cost stays fixed. As we connect more MCP servers to the portal, the token cost of Code Mode doesn’t grow.

Code Mode can be activated on an MCP server portal by adding a query parameter to the URL. Instead of connecting to your portal over its usual URL (e.g. https://myportal.example.com/mcp), you attach ?codemode=search_and_execute to the URL (e.g. https://myportal.example.com/mcp?codemode=search_and_execute).

AI Gateway provides extensibility and cost controls

We aren’t done yet. We plug AI Gateway into our architecture by positioning it on the connection between the MCP client and the LLM. This allows us to quickly switch between various LLM providers (to prevent vendor lock-in) and to enforce cost controls (by limiting the number of tokens each employee can burn through). The full architecture is shown below.


Cloudflare Gateway discovers and blocks shadow MCP

Now that we’ve provided governed access to authorized MCP servers, let’s look into dealing with unauthorized MCP servers. We can perform shadow MCP discovery using Cloudflare Gateway. Cloudflare Gateway is our comprehensive secure web gateway that provides enterprise security teams with visibility and control over their employees’ Internet traffic.

We can use the Cloudflare Gateway API to perform a multi-layer scan to find remote MCP servers that are not being accessed via an MCP server portal. This is possible using a variety of existing Gateway and Data Loss Prevention (DLP) selectors, including:

  • Using the Gateway httpHost selector to scan for 

    • known MCP server hostnames using (like mcp.stripe.com)

    • mcp.* subdomains using wildcard hostname patterns 

  • Using the Gateway httpRequestURI selector to scan for MCP-specific URL paths like /mcp and /mcp/sse 

  • Using DLP-based body inspection to find MCP traffic, even if that traffic uses URI that do not contain the telltale mentions of mcp or sse. Specifically, we use the fact that MCP uses JSON-RPC over HTTP, which means every request contains a “method” field with values like “tools/call”, “prompts/get”, or “initialize.” Here are some regex rules that can be used to detect MCP traffic in the HTTP body:

const DLP_REGEX_PATTERNS = [
  {
    name: "MCP Initialize Method",
    regex: '"method"\\s{0,5}:\\s{0,5}"initialize"',
  },
  {
    name: "MCP Tools Call",
    regex: '"method"\\s{0,5}:\\s{0,5}"tools/call"',
  },
  {
    name: "MCP Tools List",
    regex: '"method"\\s{0,5}:\\s{0,5}"tools/list"',
  },
  {
    name: "MCP Resources Read",
    regex: '"method"\\s{0,5}:\\s{0,5}"resources/read"',
  },
  {
    name: "MCP Resources List",
    regex: '"method"\\s{0,5}:\\s{0,5}"resources/list"',
  },
  {
    name: "MCP Prompts List",
    regex: '"method"\\s{0,5}:\\s{0,5}"prompts/(list|get)"',
  },
  {
    name: "MCP Sampling Create Message",
    regex: '"method"\\s{0,5}:\\s{0,5}"sampling/createMessage"',
  },
  {
    name: "MCP Protocol Version",
    regex: '"protocolVersion"\\s{0,5}:\\s{0,5}"202[4-9]',
  },
  {
    name: "MCP Notifications Initialized",
    regex: '"method"\\s{0,5}:\\s{0,5}"notifications/initialized"',
  },
  {
    name: "MCP Roots List",
    regex: '"method"\\s{0,5}:\\s{0,5}"roots/list"',
  },
];

The Gateway API supports additional automation. For example, one can use the custom DLP profile we defined above to block traffic, or redirect it, or just to log and inspect MCP payloads. Put this together, and Gateway can be used to provide comprehensive detection of unauthorized remote MCP servers accessed via an enterprise network. 

For more information on how to build this out, see this tutorial. 

Public-facing MCP Servers are protected with AI Security for Apps

So far, we’ve been focused on protecting our workforce’s access to our internal MCP servers. But, like many other organizations, we also have public-facing MCP servers that our customers can use to agentically administer and operate Cloudflare products. These MCP servers are hosted on Cloudflare’s developer platform. (You can find a list of individual MCPs for specific products here, or refer back to our new approach for providing more efficient access to the entire Cloudflare API using Code Mode.)

We believe that every organization should publish official, first-party MCP servers for their products. The alternative is that your customers source unvetted servers from public repositories where packages may contain dangerous trust assumptions, undisclosed data collection, and any range of unsanctioned behaviors. By publishing your own MCP servers, you control the code, update cadence, and security posture of the tools your customers use.

Since every remote MCP server is an HTTP endpoint, we can put it behind the Cloudflare Web Application Firewall (WAF). Customers can enable the AI Security for Apps feature within the WAF to automatically inspect inbound MCP traffic for prompt injection attempts, sensitive data leakage, and topic classification. Public facing MCPs are protected just as any other web API.  

The future of MCP in the enterprise

We hope our experience, products, and reference architectures will be useful to other organizations as they continue along their own journey towards broad enterprise-wide adoption of MCP.

We’ve secured our own MCP workflows by: 

  • Offering our developers a templated framework for building and deploying remote MCP servers on our developer platform using Cloudflare Access for authentication

  • Ensuring secure, identity-based access to authorized MCP servers by connecting our entire workforce to MCP server portals

  • Controlling costs using AI Gateway to mediate access to the LLMs powering our workforce’s MCP clients, and using Code Mode in MCP server portals to reduce token consumption and context bloat

  • Discovering shadow MCP usage by Cloudflare Gateway 

For organizations advancing on their own enterprise MCP journeys, we recommend starting by putting your existing remote and third-party MCP servers behind  Cloudflare MCP server portals and enabling Code Mode to start benefitting for cheaper, safer and simpler enterprise deployments of MCP.  

Acknowledgements:  This reference architecture and blog represents this work of many people across many different roles and business units at Cloudflare. This is just a partial list of contributors: Ann Ming Samborski,  Kate Reznykova, Mike Nomitch, James Royal, Liam Reese, Yumna Moazzam, Simon Thorpe, Rian van der Merwe, Rajesh Bhatia, Ayush Thakur, Gonzalo Chavarri, Maddy Onyehara, and Haley Campbell.

Watch on Cloudflare TV

Securing non-human identities: automated revocation, OAuth, and scoped permissions

Post Syndicated from Justin Hutchings original https://blog.cloudflare.com/improved-developer-security/

Agents let you build software faster than ever, but securing your environment and the code you write — from both mistakes and malice — takes real effort. Open Web Application Security Project (OWASP) details a number of risks present in agentic AI systems, including the risk of credential leaks, user impersonation, and elevation of privilege. These risks can result in extreme damage to your environments including denial of service, data loss, or data leaks — which can do untold financial and reputational damage. 

This is an identity problem. In modern development, “identities” aren’t just people — they are the agents, scripts, and third-party tools that act on your behalf. To secure these non-human identities, you need to manage their entire lifecycle: ensuring their credentials (tokens) aren’t leaked, seeing which applications have access via OAuth, and narrowing their permissions using granular RBAC.

Today, we are introducing updates to address these needs: scannable tokens to protect your credentials, OAuth visibility to manage your principals, and resource-scoped RBAC to fine-tune your policies.

Understanding identity: Principals, Credentials, and Policies

To secure the Internet in an era of autonomous agents, we have to rethink how we handle identity. Whether a request comes from a human developer or an AI agent, every interaction with an API relies on three core pillars:

  • The Principal (The Traveler): This is the identity itself — the “who.” It might be you logging in via OAuth, or a background agent using an API token to deploy code.

  • The Credential (The Passport): This is the proof of that identity. In this world, your API token is your passport. If it’s stolen or leaked, anyone can “wear” your identity.

  • The Policy (The Visa): This defines what that identity is allowed to do. Just because you have a valid passport doesn’t mean you have a visa to enter every country. A policy ensures that even a verified identity can only access the specific resources it needs.

When these three pillars aren’t managed together, security breaks down. You might have a valid Principal using a stolen Credential, or a legitimate identity with a Policy that is far too broad.

Leaked token detection

Agents and other third-party applications use API tokens to access the Cloudflare API. One of the simplest ways that we see people leaking their secrets is by accidentally pushing them to a public GitHub repository. GitGuardian reports that last year more than 28 million secrets were published to public GitHub repositories, and that AI is causing leaks to happen 5x faster than before.

If an API token is a digital passport, then leaking it on a public repository is like leaving your passport on a park bench. Anyone who finds it can impersonate that identity until the document is canceled. Our partnership with GitHub acts like a global “lost and found” for these credentials. By the time you realize your passport is missing, we’ve already identified the document, verified its authenticity via the checksum, and voided it to prevent misuse.

We’re partnering with several leading credential scanning tools to help proactively find your leaked tokens and revoke them before they could be used maliciously. We know it’s not a matter of if, but rather when, before you, an employee, or one of your agents makes a mistake and pushes a secret somewhere it shouldn’t be. 

GitHub

We’ve partnered with GitHub and are participating in their Secret Scanning program to find your tokens in both public and private repositories. If we are notified that a token has leaked to a public repository, we will automatically revoke the token to prevent it from being used maliciously. For private repositories, GitHub will notify you about any leaked Cloudflare tokens and you can clean these up.

How it works

We’ve shared the new token formats (below!) with GitHub, and they now scan for them on every commit. If they find something that looks like a leaked Cloudflare token, they verify the token is real (using the checksum), send us a webhook to revoke it, and then we notify you via email so you can generate a new one in Dashboard settings.

This means we plug the hole as soon as it’s found. By the time you realize you made a mistake, we’ve already fixed it. 

We hope this is the kind of feature you don’t need to use, but our partners are on the lookout for leaks to help keep you secure. 

Cloudflare One

Cloudflare One customers are also protected from these leaks. By configuring the Credentials and Secrets DLP profile, organizations can activate prevention everywhere a credential can travel:

  • Network Traffic (Cloudflare Gateway): Apply these entries to a policy to detect and block Cloudflare API tokens moving across your network. A token in a file upload, an outbound request, or a download is stopped before it reaches its destination.

  • Outbound Email (Cloudflare Email Security): Microsoft 365 customers can extend this same prevention to Outlook. The DLP Assist add-in scans messages before delivery, catching a token before it’s sent externally.

  • Data at Rest (Cloudflare CASB): Cloudflare’s Cloud Access Security Broker applies the same profile to scan files across connected SaaS applications, catching tokens saved or shared in Google Drive, OneDrive, Dropbox, and other integrated services.

The most novel exposure vector, though, is AI traffic. Cloudflare AI Gateway integrates with the same DLP profiles to scan and block both incoming prompts and outgoing AI model responses in real time.

Other credential scanners

The only way credential scanning works is if we meet you where you are, so we are working with several open source and commercial credential scanners to ensure you are protected no matter what secret scanner you use. 

How it works

Until now, Cloudflare’s API tokens were pretty generic looking, so they were hard for credential scanners to identify with high confidence. These automated security tools scan your code repositories looking for exposed credentials like API keys, tokens or passwords. The “cf” prefix makes Cloudflare tokens instantly recognizable with greater confidence, and the checksum makes it easy for tools to statically validate them. Your existing tokens will continue to work, but every new token you generate will use the scannable format so it’s easily detected with high confidence.

Credential Type

What it’s for

New Format

User API Key

Legacy global API key tied to your user account (full access)

cfk_[40 characters][checksum]

User API Token

Scoped token you create for specific permissions

cfut_[40 characters][checksum]

Account API Token

Token owned by the account (not a specific user)

cfat_[40 characters][checksum]

Getting started

If you have existing API tokens, you can roll the token to create a new, scannable API token. This is optional, but recommended to ensure that your tokens are easily discoverable in case they leak. 

While API tokens are generally used by your own scripts and agents, OAuth is how you manage access for third-party platforms. Both require clear visibility to prevent unauthorized access and ensure you know exactly who — or what — has access to your data.

Improving the OAuth consent experience

When you connect third-party applications like Wrangler to your Cloudflare Account using OAuth, you’re granting that application access to your account’s data. Over time, you may forget why you granted a third party application access to your Account in the first place. Previously, there was no central place to view & manage those applications. Starting today, there is.  

Going forward, when a third party application requests access to your Cloudflare account, you’ll be able to review: 

  • Which third-party application is requesting access, along with information about the application like Name, Logo, and the Publisher.

  • Which scopes the third-party application is requesting access to.

  • Which accounts to grant the third party application access to.

Before After

Not all applications require the same permissions; some only need to read data, others may need to make changes to your Account. Understanding these scopes before you grant access helps you maintain least-privilege.

We also added a Connected Applications experience so you can see which applications have access to which accounts, what scopes/permissions are associated with that application, and easily revoke that access as needed. 


Getting started

The OAuth consent and revocation improvements are available now. Check which apps currently have access to your accounts by visiting My Profile > Access Management > Connected Applications. 

For developers building integrations with Cloudflare, keep an eye on the Cloudflare Changelog for more announcements around how you can register your own OAuth apps soon! 

Fine-grained resource-level permissioning 

If the token is the passport, then resource-scoped permissions are the visas inside it. Having a valid passport gets you through the front door, but it shouldn’t give you access to every room in the building. By narrowing the scope to specific resources — like a single Load Balancer pool or a specific Gateway policy — you are ensuring that even if an identity is verified, it only has the “visa” to go where it’s strictly necessary.

Last year, we announced support for resource scoped permissions in Cloudflare’s role-based access control (RBAC) system for several of our Zero Trust products. This enables you to right size permissions for both users and agents to minimize security risks. We’ve expanded this capability to several new resources-level permissions. The resource scope is now supported for:

  • Access Applications

  • Access Identity Providers

  • Access Policies

  • Access Service Tokens

  • Access Targets

We’ve also completely overhauled the API Token creation experience, making it easier for customers to provision and manage Account API Tokens right from the Cloudflare Dashboard.


How it works

When you add a member to your Cloudflare account or create an API Token, you typically assign that principal a policy. A Permission Policy is what gives a principal permission to take an action, whether that’s managing Cloudflare One Access Applications, or DNS Records. Without a policy, a principal can authenticate, but they are unauthorized to do any actions within an account.

Policies are made up of three components: a Principal, a Role, and a Scope. The Principal is who or what you’re granting access to, whether that’s a human user, a Non-Human Identity (NHI) like an API Token, or increasingly, an Agent acting on behalf of a user. The Role defines what actions they’re permitted to take. The Scope determines where those permissions apply, and historically, that’s been restricted to the entire account, or individual zones.

New permission roles

We’re also expanding the role surface more broadly at both the Account & Zone level with the introduction of a number of new roles for many products. 

  • Account scope

    • CDN Management

    • MCP Portals

    • Radar

    • Request Tracer

    • SSL/TLS Management

  • Zone scope

    • Analytics

    • Logpush

    • Page Rules

    • Security Center

    • Snippets

    • Zone Settings

Getting started

The resource scope and all new account and zone-level roles are available today for all Cloudflare customers. You can assign account, zone, or resource-scoped policies through the Cloudflare Dashboard, the API, or Terraform. 

For a full breakdown of all available roles and how scopes work, visit our roles and scope documentation.

Secure your accounts

These updates provide the granular building blocks needed for a true least-privilege architecture. By refining how we manage permissions and credentials, developers and enterprises can have greater confidence in their security posture across the users, apps, agents, and scripts that access Cloudflare. Least privilege isn’t a new concept, and for enterprises, it’s never been optional. Whether a human administrator is managing a zone or an agent is programmatically deploying a Worker, the expectation is the same, they should only be authorized to do the job it was given, and nothing else. 

Following today’s announcement, we recommend customers:

  1. Review your API tokens, and reissue with the new, scannable API tokens as soon as possible. 

  2. Review your authorized OAuth apps, and revoke any that you are no longer using

  3. Review member & API Token permissions in your accounts and ensure that users are taking advantage of the new account, zone, or resource scoped permissions as needed to reduce your risk area. 

Managed OAuth for Access: make internal apps agent-ready in one click

Post Syndicated from Eduardo Gomes original https://blog.cloudflare.com/managed-oauth-for-access/

We have thousands of internal apps at Cloudflare. Some are things we’ve built ourselves, others are self-hosted instances of software built by others. They range from business-critical apps nearly every person uses, to side projects and prototypes.

All of these apps are protected by Cloudflare Access. But when we started using and building agents — particularly for uses beyond writing code — we hit a wall. People could access apps behind Access, but their agents couldn’t.

Access sits in front of internal apps. You define a policy, and then Access will send unauthenticated users to a login page to choose how to authenticate. 


Example of a Cloudflare Access login page

This flow worked great for humans. But all agents could see was a redirect to a login page that they couldn’t act on.

Providing agents with access to internal app data is so vital that we immediately implemented a stopgap for our own internal use. We modified OpenCode’s web fetch tool such that for specific domains, it triggered the cloudflared CLI to open an authorization flow to fetch a JWT (JSON Web Token). By appending this token to requests, we enabled secure, immediate access to our internal ecosystem.

While this solution was a temporary answer to our own dilemma, today we’re retiring this workaround and fixing this problem for everyone. Now in open beta, every Access application supports managed OAuth. One click to enable it for an Access app, and agents that speak OAuth 2.0 can easily discover how to authenticate (RFC 9728), send the user through the auth flow, and receive back an authorization token (the same JWT from our initial solution). 

Now, the flow works smoothly for both humans and agents. Cloudflare Access has a generous free tier. And building off our newly-introduced Organizations beta, you’ll soon be able to bridge identity providers across Cloudflare accounts too.

How managed OAuth works

For a given internal app protected by Cloudflare Access, you enable managed OAuth in one click:


Once managed OAuth is enabled, Cloudflare Access acts as the authorization server. It returns the www-authenticate header, telling unauthorized agents where to look up information on how to get an authorization token. They find this at https://<your-app-domain>/.well-known/oauth-authorization-server. Equipped with that direction, agents can just follow OAuth standards: 

  1. The agent dynamically registers itself as a client (a process known as Dynamic Client Registration — RFC 7591), 

  2. The agent sends the human through a PKCE (Proof Key for Code Exchange) authorization flow (RFC 7636)

  3. The human authorizes access, which grants a token to the agent that it can use to make authenticated requests on behalf of the user

Here’s what the authorization flow looks like:


If this authorization flow looks familiar, that’s because it’s what the Model Context Protocol (MCP) uses. We originally built support for this into our MCP server portals product, which proxies and controls access to many MCP servers, to allow the portal to act as the OAuth server. Now, we’re bringing this to all Access apps so agents can access not only MCP servers that require authorization, but also web pages, web apps, and REST APIs.

Mass upgrading your internal apps to be agent-ready

Upgrading the long tail of internal software to work with agents is a daunting task. In principle, in order to be agent-ready, every internal and external app would ideally have discoverable APIs, a CLI, a well-crafted MCP server, and have adopted the many emerging agent standards.

AI adoption is not something that can wait for everything to be retrofitted. Most organizations have a significant backlog of apps built over many years. And many internal “apps” work great when treated by agents as simple websites. For something like an internal wiki, all you really need is to enable Markdown for Agents, turn on managed OAuth, and agents have what they need to read protected content.

To make the basics work across the widest set of internal applications, we use Managed OAuth. By putting Access in front of your legacy internal apps, you make them agent-ready instantly. No code changes, no retrofitting. Instead, just immediate compatibility.

It’s the user’s agent. No service accounts and tokens needed

Agents need to act on behalf of users inside organizations. One of the biggest anti-patterns we’ve seen is people provisioning service accounts for their agents and MCP servers, authenticated using static credentials. These have their place in simple use cases and quick prototypes, and Cloudflare Access supports service tokens for this purpose.

But the service account approach quickly shows its limits when fine-grained access controls and audit logs are required. We believe that every action an agent performs must be easily attributable to the human who initiated it, and that an agent must only be able to perform actions that its human operator is likewise authorized to do. Service accounts and static credentials become points at which attribution is lost. Agents that launder all of their actions through a service account are susceptible to confused deputy problems and result in audit logs that appear to originate from the agent itself.

For security and accountability, agents must use security primitives capable of expressing this user–agent relationship. OAuth is the industry standard protocol for requesting and delegating access to third parties. It gives agents a way to talk to your APIs on behalf of the user, with a token scoped to the user’s identity, so that access controls correctly apply and audit logs correctly attribute actions to the end user.

Standards for the win: how agents can and should adopt RFC 9728 in their web fetch tools

RFC 9728 is the OAuth standard that makes it possible for agents to discover where and how to authenticate. It standardizes where this information lives and how it’s structured. This RFC became official in April 2025 and was quickly adopted by the Model Context Protocol (MCP), which now requires that both MCP servers and clients support it.

But outside of MCP, agents should adopt RFC 9728 for an even more essential use case: making requests to web pages that are protected behind OAuth and making requests to plain old REST APIs.

Most agents have a tool for making basic HTTP requests to web pages. This is commonly called the “web fetch” tool. It’s similar to using the fetch() API in JavaScript, often with some additional post-processing on the response. It’s what lets you paste a URL into your agent and have your agent go look up the content.

Today, most agents’ web fetch tools won’t do anything with the www-authenticate header that a URL returns. The underlying model might choose to introspect the response headers and figure this out on its own, but the tool itself does not follow www-authenticate, look up /.well-known/oauth-authorization-server, and act as the client in the OAuth flow. But it can, and we strongly believe it should! Agents already do this to act as remote MCP clients.

To demonstrate this, we’ve put up a draft pull request that adapts the web fetch tool in Opencode to show this in action. Before making a request, the adapted tool first checks whether it already has credentials ; if it does, it uses them to make the initial request. If the tool gets back a 401 or a 403 with a www-authenticate header, it asks the user for consent to be sent through the server’s OAuth flow.

Here’s how that OAuth flow works. If you give the agent a URL that is protected by OAuth and complies with RFC 9728, the agent prompts the human for consent to open the authorization flow:


…sending the human to the login page:


…and then to a consent dialog that prompts the human to grant access to the agent:


Once the human grants access to the agent, the agent uses the token it has received to make an authenticated request:


Any agent from Codex to Claude Code to Goose and beyond can implement this, and there’s nothing bespoke to Cloudflare. It’s all built using OAuth standards.

We think this flow is powerful, and that supporting RFC 9728 can help agents with more than just making basic web fetch requests. If a REST API supports RFC 9728 (and the agent does too), the agent has everything it needs to start making authenticated requests against that API. If the REST API supports RFC 9727, then the client can discover a catalog of REST API endpoints on its own, and do even more without additional documentation, agent skills, MCP servers or CLIs. 

Each of these play important roles with agents — Cloudflare itself provides an MCP server for the Cloudflare API (built using Code Mode), Wrangler CLI, and Agent Skills, and a Plugin. But supporting RFC 9728 helps ensure that even when none of these are preinstalled, agents have a clear path forward. If the agent has a sandbox to execute untrusted code, it can just write and execute code that calls the API that the human has granted it access to. We’re working on supporting this for Cloudflare’s own APIs, to help your agents understand how to use Cloudflare.

Coming soon: share one identity provider (IdP) across many Cloudflare accounts

At Cloudflare our own internal apps are deployed to dozens of different Cloudflare accounts, which are all part of an Organization — a newly introduced way for administrators to manage users, configurations, and view analytics across many Cloudflare accounts. We have had the same challenge as many of our customers: each Cloudflare account has to separately configure an IdP, so Cloudflare Access uses our identity provider. It’s critical that this is consistent across an organization — you don’t want one Cloudflare account to inadvertently allow people to sign in just with a one-time PIN, rather than requiring that they authenticate via single-sign on (SSO).

To solve this, we’re currently working on making it possible to share an identity provider across Cloudflare accounts, giving organizations a way to designate a single primary IdP for use across every account in their organization.

As new Cloudflare accounts are created within an organization, administrators will be able to configure a bridge to the primary IdP with a single click, so Access applications across accounts can be protected by one identity provider. This removes the need to manually configure IdPs account by account, which is a process that doesn’t scale for organizations with many teams and individuals each operating their own accounts.

What’s next

Across companies, people in every role and business function are now using agents to build internal apps, and expect their agents to be able to access context from internal apps. We are responding to this step function growth in internal software development by making the Workers Platform and Cloudflare One work better together — so that it is easier to build and secure internal apps on Cloudflare. 

Expect more to come soon, including:

  • More direct integration between Cloudflare Access and Cloudflare Workers, without the need to validate JWTs or remember which of many routes a particular Worker is exposed on.

  • wrangler dev –tunnel — an easy way to expose your local development server to others when you’re building something new, and want to share it with others before deploying

  • A CLI interface for Cloudflare Access and the entire Cloudflare API

  • More announcements to come during Agents Week 2026

Enable Managed OAuth for your internal apps behind Cloudflare Access

Managed OAuth is now available, in open beta, to all Cloudflare customers. Head over to the Cloudflare dashboard to enable it for your Access applications. You can use it for any internal app, whether it’s one built on Cloudflare Workers, or hosted elsewhere. And if you haven’t built internal apps on the Workers Platform yet — it’s the fastest way for your team to go from zero to deployed (and protected) in production.

Staying Secure: An Inside Look at Zabbix Security Advisories

Post Syndicated from Michael Kammer original https://blog.zabbix.com/staying-secure-an-inside-look-at-zabbix-security-advisories/32831/

Security has always been a core priority for us at Zabbix. As part of our ongoing commitment to delivering a reliable and secure monitoring platform, we regularly publish security advisories that reflect both newly discovered vulnerabilities and the improvements we’ve made to address them.

These advisories are not just a list of issues – they are a direct result of continuous internal efforts to analyze, test, and strengthen every aspect of our product.

More than just fixes

Every vulnerability we disclose represents a deeper process behind the scenes. It involves:

  • Careful investigation and validation
  • Improvements to internal tooling and detection methods
  • Reevaluation of development and testing processes
  • Retesting to ensure robustness and prevent regressions

For us, security is not a one-time fix – it’s an ongoing cycle of improvement.

Decoding severity scores

We understand that some of the published severity scores may appear alarming at first glance. It’s important to note that these scores are based on worst-case scenario evaluations. In real-world deployments, the actual risk often depends on system configuration, network exposure, access controls, usage patterns, and more.

For many typical Zabbix installations, the effective risk level may be significantly lower than the maximum theoretical score suggests.

Stay updated – it matters

Keeping your Zabbix installation up to date is one of the most effective ways to maintain a secure environment.
Each update includes not only bug fixes, but also: security enhancements, hardening improvements, and stability and performance updates. By applying updates regularly, you make sure that your systems benefit from the latest protections. In short, staying up to date is a shared responsibility and the best defense.

Open communication

We are aware that security advisories can sometimes lead to external reports that frame vulnerabilities without full context. We want to be clear that:

  • Publishing advisories is a sign of maturity and transparency, not weakness
  • Proactively identifying and fixing issues is a core strength
  • Our goal is not to avoid disclosure, but to handle it responsibly and openly

Security is not about the absence of vulnerabilities. It’s about how quickly and effectively they are identified, addressed, and communicated.

Going forward

We believe that transparency builds trust. If you have any questions about Zabbix security advisories or best practices, we encourage you to reach out to us. Our team is always ready to clarify, assist, and provide guidance. We remain fully committed to improving Zabbix security at every level – from code to processes to communication.

Your trust is important to us, and we will continue to invest in making Zabbix a secure and dependable platform for your infrastructure.

The post Staying Secure: An Inside Look at Zabbix Security Advisories appeared first on Zabbix Blog.

Cloudflare targets 2029 for full post-quantum security

Post Syndicated from Bas Westerbaan original https://blog.cloudflare.com/post-quantum-roadmap/

Cloudflare is accelerating its post-quantum roadmap. We now target 2029 to be fully post-quantum (PQ) secure including, crucially, post-quantum authentication.

At Cloudflare, we believe in making the Internet private and secure by default. We started by offering free universal SSL certificates in 2014, began preparing our post-quantum migration in 2019, and enabled post-quantum encryption for all websites and APIs in 2022, mitigating harvest-now/decrypt-later attacks. While we’re excited by the fact that over 65% of human traffic to Cloudflare is post-quantum encrypted, our work is not done until authentication is also upgraded. Credible new research and rapid industry developments suggest that the deadline to migrate is much sooner than expected. This is a challenge that any organization must treat with urgency, which is why we’re expediting our own internal Q-Day readiness timeline.

What happened? Last week, Google announced they had drastically improved upon the quantum algorithm to break elliptic curve cryptography, which is widely used to secure the Internet. They did not reveal the algorithm, but instead provided a zero-knowledge proof that they have one.

This is not even the biggest breakthrough. That same day, Oratomic published a resource estimate for breaking RSA-2048 and P-256 on a neutral atom computer. For P-256, it only requires a shockingly low 10,000 qubits. Google’s motivation behind their recent announcement to also pursue neutral atoms alongside superconducting quantum computers becomes clear now. Although Oratomic explains their basic approach, they still leave out crucial details on purpose.

These independent advances prompted Google to accelerate their post-quantum migration timeline to 2029. What’s more, in their announcement and other talks, Google has placed a priority on quantum-secure authentication over mitigating harvest-now/decrypt-later attacks. As we discuss next, this priority indicates that Google is concerned about Q-Day coming as soon as 2030. Following the announcements, IBM Quantum Safe’s CTO is more pessimistic and can’t rule out quantum “moonshot attacks” on high value targets as early as 2029.

The quantum threat is well known: Q-Day is the day that sufficiently capable quantum computers can break essential cryptography used to protect data and access across systems today. Cryptographically relevant quantum computers (CRQCs) don’t exist yet, but many labs across the world are pursuing different approaches to building one. Until recently, progress on CRQCs has been mostly public, but there is no reason to expect that will continue. Indeed, there is ample reason to expect that progress will leave the public eye. As quantum computer scientist Scott Aaronson warned at the end of 2025:

[A]t some point, the people doing detailed estimates of how many physical qubits and gates it’ll take to break actually deployed cryptosystems using Shor’s algorithm are going to stop publishing those estimates, if for no other reason than the risk of giving too much information to adversaries. Indeed, for all we know, that point may have been passed already.

That point has now passed indeed.

Why now: independent progress on three fronts

We’d like to spend some words on why it’s difficult to predict progress on quantum computing. Sudden “quantum” leaps in understanding, like the one we witnessed last week, can occur even if everything happens in the public eye. Simply put, breaking cryptography with a quantum computer requires engineering on three independent fronts: quantum hardware, error correction, and quantum software. Progress on each front compounds progress on the others.

Hardware. There are many different competing approaches. We mentioned neutral atoms and superconducting qubits, but there are also ion-trap, photonics, and moonshots like topological qubits. Complementary approaches can even be combined. Most of these approaches are pursued by several labs around the world. They all have their distinct engineering challenges and problems to solve before they can scale up. A few years ago, all of them had a long list of open challenges, and it was unclear if any of them would scale. Today most of them have made good progress. None have been demonstrated to scale yet: if they had, we wouldn’t have a couple of years left. But these approaches are much closer now, especially neutral atoms. To ignore this progress, you’d have to believe that every single approach will hit a wall.

Error correction. All quantum computers are noisy and require error-correcting codes to perform meaningful computation. This adds quite a bit of overhead, though how much depends on the architecture. More noise requires more error correction, but more interestingly, improved qubit connectivity allows for much more efficient codes. For a sense of scale: typically around a thousand physical qubits are required for one logical qubit for the superconducting quantum computers that are noisy and only have neighbor qubit connectivity. We knew “reconfigurable qubits” such as those of neutral-atom machines allow for an order of magnitude better error-correcting codes. Surprisingly, Oratomic showed the advantage is even larger: only about 3-4 physical neutral atom qubits are required per logical qubit.

Software. Lastly, the quantum algorithms to crack cryptography can be improved. This is Google’s breakthrough: they massively sped up the algorithm to crack P-256. On top of that, Oratomic showed further architecture specific optimizations for reconfigurable qubits.

The picture comes together: in 2025 neutral atoms turned out to be more scalable than expected, and now Oratomic figured out how to do much better error-correcting codes with such highly connected qubits. On top of that, breaking P-256 requires much less work. The result is that Q-Day has been pulled forward significantly from typical 2035+ timelines, with neutral atoms in the lead, and other approaches not far behind.

In previous blog posts we’ve discussed how different quantum computers compare on physical qubit count and fidelity, compared to the conservative goalpost of cracking RSA-2048 on a superconducting qubit architecture. This analysis gives us a rough idea of how much time we have, and it’s certainly better than tracking quantum factoring records, but it misses architecture-specific optimization and software improvements. What to watch for now is when the final missing capabilities for each architecture are achieved.

It’s time to focus on authentication

Historically, the industry’s focus on post-quantum cryptography (PQC) has been based largely on PQ encryption, which stops harvest-now/decrypt-later (HNDL) attacks. In an HNDL attack, an adversary harvests sensitive encrypted network traffic today and stores it until a future date when it can use a powerful quantum computer to decrypt the data. HNDL attacks are the primary threat when Q-Day is far away. That’s why our focus, thus far, has been on mitigating this risk, by adopting post-quantum encryption by default in our products since 2022. Today, as we mentioned above, most Cloudflare products are secure against HNDL attacks, and we’re working to upgrade the rest as we speak. 

The other category of attacks is against authentication: adversaries armed with functioning quantum computers impersonate servers or forge access credentials. If Q-Day is far off, authentication is not urgent: deploying PQ certificates and signatures does not add any value, only effort.

An imminent Q-Day flips the script: data leaks are severe, but broken authentication is catastrophic. Any overlooked quantum-vulnerable remote-login key is an access point for an attacker to do as they wish, whether that’s to extort, take down, or snoop on your system. Any automatic software-update mechanism becomes a remote code execution vector. An active quantum attacker has it easy — they only need to find one trusted quantum-vulnerable key to get in.

When experts in the field of building quantum computers start patching authentication systems, we should all listen. The question is no longer “when will our encrypted data be at risk?” but “how long before an attacker walks in the front door with a quantum-forged key?”

Prioritizing the most vulnerable systems

If quantum computers arrive in the next few years, they will be scarce and expensive. Attackers will prioritize high-value targets, like long-lived keys that unlock substantial assets or persistent access such as root certificates, API auth keys and code-signing certs. If an attacker is able to compromise one such key, they retain indefinite access until they are discovered or that key is revoked.

This suggests long-lived keys should be prioritized. That is certainly true if the quantum attack of a single key is expensive and slow, which is to be expected for the first generation of neutral atom quantum computers. That’s not the case for scalable superconducting quantum computers and later generations of neutral atom quantum computers, which could well crack keys much faster. Such fast CRQCs flip the script again, and an adversary with one might focus purely on HNDL attacks so that their attacks remain undetected. Google’s Sophie Schmieg compares this scenario to Enigma’s cryptanalysis that changed the direction of World War II.

Adding support for PQ cryptography is not enough. Systems must disable support for quantum-vulnerable cryptography to be secure against downgrade attacks. In larger, especially federated systems such as the web, this is not feasible because not every client (browser) will support post-quantum certificates, and servers need to keep supporting these legacy clients. However, downgrade protection for HTTPS is still achievable using “PQ HSTS” and/or certificate transparency.

Disabling quantum-vulnerable cryptography is not the last step: once done, all secrets such as passwords and access tokens previously exposed in the quantum-vulnerable system need to be rotated. Unlike post-quantum encryption, which takes one big push, migrating to post-quantum authentication has a long dependency chain — not to mention third-party validation and fraud monitoring. This will take years, not months.

It’s natural for organizations reading this to rush out and think about which internal systems they need to upgrade. But that’s not the end of the story. Q-day threatens all systems. As such, it’s important to understand the impact of a potential Q-day on third-party dependencies, both direct and indirect. Not just the third-parties you speak cryptography to, but also any third parties that are critical business dependencies like financial services and utilities.

With Q-day approaching on a shorter timeline, post-quantum authentication is top priority. Long-term keys should be upgraded first. Deep dependency chains and the fact that everyone has third-party vendors means this effort will take on the order of years, not months. Upgrading to post-quantum cryptography is not enough: to prevent downgrades, quantum-vulnerable cryptography must also be turned off.

Cloudflare’s roadmap to full post-quantum security

Today, Cloudflare provides post-quantum encryption for the majority of our products mitigating harvest-now/decrypt-later. This is the product of work we started over a decade ago to protect our customers and the Internet at large.

We are targeting full post-quantum security including authentication for our entire product suite by 2029. Here we’re sharing some intermediate milestones we’ve set, subject to change as our understanding of the risk and deployment challenges evolve.


What we recommend

For businesses, we recommend making post-quantum support a requirement for any procurement. Common best practices, like keeping software updated and automating certificate issuance, are meaningful and will get you pretty far. We recommend assessing critical vendors early for what their failure to take action would mean for your business.

For regulatory agencies and governments: leading by setting early timelines has been crucial for industry-wide progress so far. We are now in a pivotal position where fragmentation in standards and effort between and within jurisdictions could put progress at risk. We recommend that governments assign and empower a lead agency to coordinate the migration on a clear timeline, stay security-focused, and promote the use of existing international standards. Governments need not panic, but can lead migration with confidence.

For Cloudflare customers, with respect to our services, you do not need to take any mitigating action. We are following the latest advancements in quantum computing closely and taking proactive steps to protect your data. As we have done in the past, we will turn on post-quantum security by default, with no switches to flip. What we don’t control is the other side: browsers, applications, and origins need to upgrade. Corporate network traffic on Cloudflare need not worry: Cloudflare One offers end-to-end protection when tunnelling traffic through our post-quantum encrypted infrastructure.

Privacy and security are table stakes for the Internet. That’s why every post-quantum upgrade we build will continue to be available to all customers, on every plan, at no additional cost. Making post-quantum security the default is the only way to protect the Internet at scale.

Free TLS helped encrypt the web. Free post-quantum cryptography will help secure it for what comes next.

Building AI defenses at scale: Before the threats emerge

Post Syndicated from Amy Herzog original https://aws.amazon.com/blogs/security/building-ai-defenses-at-scale-before-the-threats-emerge/

At AWS, we’ve spent decades developing processes and tools that enable us to defend millions of customers simultaneously, wherever they operate around the world. Every day, our security and threat intelligence teams are doing work with AI and automation that most people never see. Our AI-powered log analysis system has reduced the time SecOps engineers spend analyzing security logs from an average of six hours to just seven minutes, a 50x productivity increase that lets us detect and respond to threats faster than ever. Across AWS, we analyze over 400 trillion network flows per day to detect patterns that signal emerging threats. In 2025 alone, we blocked over 300 million attempts to maliciously encrypt customer files hosted on Amazon S3.

What we learn protecting one customer helps protect all customers. At this scale, every threat we see makes our defenses stronger for everyone, and AI is already central to how we do it.

A new class of AI for cybersecurity

Today, Anthropic announced Project Glasswing, a cybersecurity initiative designed to secure the world’s most critical software and advance the cybersecurity practices the industry will need as AI grows more capable. Organizations that build or maintain critical digital infrastructure are getting early access to Claude Mythos Preview, a new class of AI model, to find and patch vulnerabilities in the systems the world depends on. Given our role in securing some of the world’s most essential infrastructure, AWS is playing an integral part in advancing this work.

Powering the project is Claude Mythos Preview, Anthropic’s most advanced AI model to date and a step-change in reasoning and AI capabilities for cybersecurity. Claude Mythos Preview represents a fundamentally new model class: more intelligent and capable than Anthropic’s previous frontier models, with higher performance on cybersecurity, software coding, and complex reasoning tasks.

As part of Project Glasswing, we’ve already applied Claude Mythos Preview to critical AWS codebases that undergo continuous AI-powered security reviews, and even in those well-tested environments, it’s helped us identify additional opportunities to strengthen our code. In our internal testing, Claude Mythos Preview has proven more productive than previous models at surfacing security findings, requiring less manual guidance from our engineers to deliver actionable results. We’ve also given early access to a select group of AWS customers, who are deploying Claude Mythos Preview in their own security workflows and helping shape how the model evolves. For us, Claude Mythos Preview is a natural extension of the AI tools we already use, and as the technology grows more powerful, so must our defenses.

This is exactly the kind of innovation that drives our work, and we’ve been working closely with Anthropic to help ensure Claude Mythos Preview is ready for enterprise use. AWS is Anthropic’s primary cloud provider for mission-critical workloads, safety research, and foundation model development. More broadly, AWS provides the foundational infrastructure that the world’s leading AI companies rely on to build, train, and deploy their most advanced models. We’re bringing decades of security experience to this partnership, helping to ensure Claude Mythos Preview is ready for even more organizations to build upon and operate securely at scale.

Claude Mythos Preview signals an upcoming wave of models that can find vulnerabilities and build working exploits at a scale and speed we haven’t seen before. Anthropic and AWS are taking a deliberately cautious approach to release. Access begins with a small number of organizations, prioritizing internet-critical companies and open-source maintainers whose software and digital services impact hundreds of millions of users. The goal: find and fix vulnerabilities in the world’s most critical software. Claude Mythos Preview is available in gated research preview through Amazon Bedrock with enterprise-grade security controls, including customer-managed encryption, VPC isolation, and detailed logging, so your team can explore Claude Mythos Preview’s capabilities without exposing production assets to unnecessary risk.

AWS architects services with security at the core

Our work with Project Glasswing is grounded in a philosophy we’ve developed over two decades of securing mission-critical workloads: you can’t wait for threats to materialize before building your defenses. You have to look around corners, adopt new technologies, build protections first, deploy them in your own operations at scale, and refine them based on what you learn.

That’s exactly what we’ve done at AWS with AI and security. Our approach spans the full spectrum: proactive defense through threat hunting and vulnerability research, dynamic response to active campaigns, and third-party certifications that verify our security practices meet the highest industry standards. This operational experience has taught us where AI accelerates security work and where human judgment remains essential. And it’s reinforced that security innovation must be pragmatic: proven in production before we ask you to rely on it.

That’s also why we help define what secure AI looks like. We became the first major cloud provider to achieve ISO 42001 certification for AI services. We’re active participants in OWASP, the Coalition for Secure AI, and the Frontier Model Safety Framework. And we co-founded the Open Cybersecurity Schema Framework (OCSF) to enable better threat intelligence sharing across the ecosystem. The AWS Nitro System provides mathematical isolation for workloads. Our zero-operator access architecture means AWS personnel can’t access your data. These aren’t aspirational goals. They’re how we operate today, at scale, every day.

Amazon Bedrock is where these principles come to life for AI. It provides policy-enforced access controls, built-in evaluation tools to measure how effectively models identify and validate vulnerabilities, and the ability to run workloads inside your own virtual private cloud. AWS is also the first cloud provider to achieve FedRAMP High and Department of Defense Security Requirements Guide Impact Level 4 and 5 authorizations for generally available Claude foundation models, reinforcing that Amazon Bedrock is where the most security-sensitive organizations already trust Anthropic’s technology.

How to get started today

The same principles that guide our work at AWS scale apply regardless of which AI tools you’re using: comprehensive observability, defense in depth, automation where it adds value, and human judgment where it’s essential. Here’s how to put them into practice.

Prepare for the next generation of AI security. Claude Mythos Preview signals an upcoming wave of AI models that will transform cybersecurity. Start strengthening your security posture now so your organization is ready as these capabilities become more broadly available. Claude Mythos Preview is available in gated preview through Amazon Bedrock, and access is limited to an initial allow-list of organizations. If your organization has been allow-listed, your AWS account team will reach out directly.

Run on-demand penetration testing with AWS Security Agent. Now generally available, AWS Security Agent delivers autonomous penetration testing that operates 24/7 at a fraction of the cost of manual penetration tests. It transforms penetration testing from a periodic bottleneck into an on-demand capability that scales with your development velocity across AWS, Azure, GCP, other cloud providers, and on-premises. AWS Security Agent represents a new class of frontier agents: autonomous systems that work independently to achieve goals, scale to tackle concurrent tasks, and run persistently without constant human oversight. It deploys specialized AI agents to discover, validate, and report security vulnerabilities through sophisticated multi-step attack scenarios. Unlike traditional scanners that generate findings without validation, AWS Security Agent identifies potential vulnerabilities, then attempts to exploit them with targeted payloads and attack chains to confirm they are legitimate security risks. Each finding includes CVSS risk scores, application-specific severity ratings, detailed reproduction steps, and remediation suggestions. The result: penetration testing that once took weeks now completes in hours, and security coverage that scales across your entire application portfolio, not just your most critical systems. New customers can explore AWS Security Agent with a 2-month free trial.

Build AI applications you can trust with Amazon Bedrock. For teams building with generative AI, the challenge isn’t just making AI work, it’s making AI work safely. Amazon Bedrock provides the security and safety controls you need to deploy AI responsibly. Its Automated Reasoning capability is the first and only AI safeguard to use formal logic to help prevent factual errors from hallucinations, providing verifiable explanations with 99% accuracy, a capability we’ve refined over more than a decade of applying formal methods across AWS storage, identity, and networking. Amazon Bedrock also provides customizable guardrails that block harmful content and enforce your content policies, along with comprehensive observability to track AI behavior and detect anomalies across your workloads.

The threat landscape isn’t waiting

The threat landscape isn’t waiting for us to catch up. Nation-state actors, ransomware operators, and supply chain attackers are already using AI to scale their operations. Our job is to stay ahead by building defenses first, deploying them at scale, and sharing what we learn so the entire community benefits.

That’s what we do every day at AWS. We prove technology works in our own operations before we ask customers to rely on it. We set standards rather than follow them. And we look around corners to address tomorrow’s challenges today.

As AI capabilities continue to evolve, this approach won’t change. We’ll keep building defenses first, refining them at scale, and working with partners like Anthropic to ensure the next generation of AI security tools meets the real-world needs of enterprises defending at this scale.

Learn More

If you have feedback about this post, submit comments in the Comments section below.

Amy Herzog

Amy Herzog is Vice President and Chief Information Security Officer (CISO) at Amazon Web Services (AWS) where she leads a global organization of cloud security professionals in a company in which security is the top priority. Prior to joining AWS, Amy served as CISO for Amazon’s Devices and Services, Media and Entertainment, and Advertising businesses, overseeing the security of consumer technology offerings such as Alexa+ and Ring, and playing a key role in the secure development of Project Kuiper, Amazon’s initiative to provide fast, reliable broadband to customers and communities around the world through low earth orbit satellites.