Tag Archives: AI

NVIDIA GTC Berlin 2026 NVIDIA Intros DSX PODs and More

Post Syndicated from Ryan Smith original https://www.servethehome.com/nvidia-gtc-berlin-2026-nvidia-intros-dsx-pods-and-more/

NVIDIA kicked off GTC 2026 with its lead keynote. Among the highlights for this year’s European NVIDIA tradeshow are the DSX POD, a server-in-a-container, as well as a slew of software updates for the NVIDIA ecosystem

The post NVIDIA GTC Berlin 2026 NVIDIA Intros DSX PODs and More appeared first on ServeTheHome.

How Commerce.com and Cloudflare are getting storefronts ready for AI agents

Post Syndicated from Jenny Lea original https://blog.cloudflare.com/webmcp-commerce/

AI can write code and pass the bar exam, but it still struggles to buy a pair of shoes online.

That’s because websites were built for humans, not agents. Most agents that shop for someone today work from a site’s HTML — they look at the page, guess which button does what, click, and try again when they get it wrong. This process is slow, uses a lot of tokens, and something as seemingly innocuous as a theme change can break it.

Enter WebMCP: a browser standard from Google that gives AI agents structured tools on a page, like search_products. Instead of hunting for buttons, the agent calls the exact tool it needs directly, making shopping faster, cheaper, and far more reliable. Because WebMCP is still an experimental proposed browser standard in early preview, it’s currently available behind experimental Chrome flags (chrome://flags/#enable-webmcp-testing), in Chrome Origin Trials, in some agentic browsers, and across developer environments like Cloudflare Kitesurf.

In August, we launched edge implementation of WebMCP in beta. Turn it on, and Cloudflare injects a tiny script at the edge to make those tools available automatically, without site owners writing a single line of code.

Commerce.com, a flexible commerce platform built for the age of AI, already uses Cloudflare to run their almost 100,000 domains. So when the preview came out, they asked: Can we turn this on for all of our stores at once?

Here’s why Commerce.com wanted WebMCP, how we built edge-level opt-outs to give every merchant control, and what we learned along the way.

Why Commerce.com wanted WebMCP

Kristina Pototska, Product Management Director at Commerce.com, explains why this matters for storefronts:

“Merchants keep asking us how they can keep up with agentic shopping. When a shopper asks an agent to find something, merchants want their store to be in the answer. They want to keep the customer, keep control of the checkout, and do it without rebuilding their site. WebMCP helps with all three. The agent gets real tools to search products, check details, build a cart, and get a checkout link, so it doesn’t have to guess its way around the page. The human shopper stays on the merchant’s site, and the merchant keeps the relationship.

We wanted to turn this on in a secure way, and without asking merchants to edit their theme or do anything by hand. That’s where the Cloudflare partnership worked perfectly.

WebMCP also works side by side with our server-side MCP, and together they cover the different ways people shop with agents. Server-side MCP supports agents that connect to a store directly, like an assistant helping someone shop across several sites. WebMCP supports agents that work in the shopper’s browser. It’s the same set of shopping tools in both cases, so merchants don’t have to choose between the two or manage them separately.”

Under the hood, this integration relies on two key principles. First, the WebMCP tools execute directly inside the shopper's active browser session on the storefront's domain, ensuring the agent sees the same prices and cart. No credentials go to a third party, and the visit still happens on the merchant's site. Second, Cloudflare serves as the edge delivery layer for Commerce.com’s backend MCP server, proxying tool definitions and actions unchanged so merchants maintain full control over their tool logic.

What a shopper's agent sees

When visiting an enabled Commerce.com storefront, an AI assistant no longer has to guess how to navigate the site. Instead, it can instantly discover tools via WebMCP.

Suppose you tell your agent: "Find me a waterproof trail running shoe under $140 in size 10 and add it to my cart."

Without WebMCP: The agent attempts to simulate human clicks, only to get blocked by cookie banners, misinterpret variant drop-downs, or get tripped up by bot challenges that try to distinguish automated traffic from real customers.

With WebMCP: The agent detects the store's tools on document.modelContext and makes two clean calls:

  1. search_products({ query: "trail running shoe", max_price: 140, size: 10 })
  2. add_to_cart({ product_id: "shoe_123" })

Because the agent runs inside your active browser session on the store's domain, your cart updates in real time. Commerce.com’s backend MCP server determines the commerce flow: when you’re ready to complete the purchase, you click checkout yourself — keeping your payment details secure and leaving you in complete control of the final buy.

What we built together

When Commerce.com asked to turn on WebMCP across almost 100000 storefronts, we ran into a practical challenge: WebMCP was originally designed as a single on-or-off switch for an entire domain zone.

Commerce wanted to enable our mcp-server-client toolpack, which acts as an edge bridge — proxying tool definitions and requests directly from Commerce.com’s backend MCP server to the visitor's browser. Because Commerce.com uses Cloudflare for SaaS to host thousands of merchant vanity domains on shared zones, we needed a way to deliver this platform-wide while giving individual store owners an easy way to opt out.

1. On for all stores on Commerce.com, with an opt-out for each merchant

Instead of building a complex custom allowlist, we hooked WebMCP into the existing Cloudflare concepts of Configuration Rules and custom hostname metadata.

Commerce.com turns WebMCP on for their main zone, then adds a rule at the edge that checks merchant metadata. If a merchant flips the "opt-out" switch in their Commerce.com dashboard, Cloudflare bypasses WebMCP for that specific storefront automatically:

{
  "action": "set_config",
  "expression": "cf.hostname.metadata.webmcp_opt_out eq \"true\"",
  "action_parameters": {
    "webmcp_enabled": false
  }
}

Because Cloudflare evaluates these rules at the edge before serving the page, an opted-out store gets zero extra latency, no remote lookups, and no injected scripts.

2. Tailored endpoints and safe fallbacks

As with any preview release, working with a partner in the real world revealed a few practical details we hadn't accounted for:

Custom MCP paths: When the mcp-server-client toolpack was enabled, our bridge originally expected to find every site's MCP server at /mcp. Commerce.com’s server lives at /api/mcp. To handle this, we added a configurable MCP URL setting at the zone level. When this is set, we pass the custom path into the injected script tag:

<script type="module"
    src="/.webmcp/bridge.js"
    data-packs="c2pa,mcp-server-client"
    data-mcp-url="/api/mcp"></script>

Seamless fallbacks: If a store's MCP server is offline or a customer's browser doesn't support WebMCP, the script exits. The storefront loads normally and human shoppers aren't affected.

What's next

We'll keep investing in WebMCP as the standard evolves, starting with helping agents find a site's tools more easily. Cloudflare and Commerce.com will continue to work together to make shopping with agents fast, reliable and safe for both merchants and shoppers.

If you're at Cloudflare Connect in San Francisco on October 21, 2026, come see us at Getting your storefront agent ready for peak season, a panel discussion featuring Kristina Pototska (Product Management Director, Commerce), Justis Blasco (CTO and Co-Founder, Popmenu), Ross Harvey (Head of Infrastructure, Fruugo), and Josh Mills (VP of Engineering, Gopuff).

Cutting our inference bill by 68% by being obsessive about optimization

Post Syndicated from Ryan Skidmore original https://blog.cloudflare.com/obsessive-inference-optimization/

When we previously blogged about our AI code reviewer in April, an average review cost a palatable $1.19. By the end of August, that figure had shot up to about $2.46. We hadn't made any significant architectural changes: the coordinator was still handing work to the same set of specialist reviewers, and the same risk tiers decided how intensive of a review we gave.

Around mid-September we tasked our agents with working out where those tokens were going and cutting them — aggressively. With code quality at high effort, the average review now costs just $0.77, down 68% from the $2.43 it averaged over our two most expensive weeks before we started. Our main goal was to cut costs, but as part of this work we also managed to optimize the reviewer, so that the median review now takes just 1m 32s instead of the 4m 09s it took before.

About 40% of the savings came from changes and optimizations to the reviewer's code, and the rest from experimenting with which models we use and how much effort they put in.

How a review doubled in cost

The chart below shows the cost of an average review on each weekday since March. It holds the mix of review tiers fixed, so a day with more large reviews doesn't skew the data.

The cost per review steadily rose from $1.22 (March) → $1.44 (June) → $1.74 (July) and finally, to $2.32 (August). A small part of that is that more reviews landed in the most expensive tier (59% of reviews in March and then 65% in August). We believe there’s a simple answer for why: models are getting much better, and much more capable. Naturally, engineers are producing bigger, more complex changes which means more of them fall into our “full review” tier. We found that most of the cost increase actually came from model changes over those months, though.

At the start of August we moved every reviewer onto Anthropic models, which raised the cost per job from $1.38 to $2.03. We made this choice because we found that during our evals, Anthropic models performed better on this specific workload than other models.

Our internal standards reviewer, Codex, ended up on Claude Opus 4.7 due to a mix of performance and reliability. That took it from about $0.25 per review to $0.88 in August, which made it more expensive than the coordinator agent itself.

The cache hit rate also increased, going from 85.7% in April to 97% by September, while the number of tokens per review more than tripled, from 0.91 million to 2.99 million. A high hit rate on three times as many tokens still costs a lot more, even with the low cache read cost.

What else has changed since our last post?

One of the biggest changes is that the reviewer now runs on OpenCode v2, which is a ground-up rewrite from v1. Before switching all reviews over from v1 to v2, we ran every review in every canary repo on both v1 and v2 in a comparative shadow mode. The two agreed on the approval decision in 98% of the merge requests we tested, so we moved everyone over in one go in early October and removed the v1 code. Comparing two days with the same models, full-tier reviews on v2 finish about 13% faster. We’re also currently experimenting with other ways to run our internal AI reviewer, including solely on Durable Objects using pi-durable. We’re hopefully going to have more on this coming soon, so keep an eye out!

So, where were the tokens going?

Right before we started focusing on cost optimization, our AI review spend was split by: output tokens making up about 23%,  cache writes cost 43%, and cache reads cost 34%. This means that  most of the money was going on context the model had already seen or was about to see again.

That makes complete sense when you look into a single session. The code quality reviewer made 16.6 large language model (LLM) calls per review on average and over those calls it wrote 72k tokens to the cache and read back 1.08m, because each call sends the whole conversation as context. A session's cost grows with both the number of calls and the size of the context, and over the last few months we had slowly increased both.

It’s worth taking a few minutes to read about how Anthropic's prompt cache works. It works by caching parts of the prompt that have already been computed by the large language model (LLM). These are known as prefixes, because they only work on the messages at the start of the conversation, not mid way through or the end. This means that a future request can reuse an earlier prefix, up to the part of the conversation that changes from what’s been cached. In our case, the harness tools, system prompt and messages at the start of our AI reviewer sessions rarely change, so they’re a great candidate for prefix caching. A prefix is only stored where a cache breakpoint ends it. Writing to the cache costs 1.25 times the normal input price and reading costs 0.1 times, and an entry expires after five minutes unless something reads it.

That means we had two things to work on: make the first request of each session read from the cache instead of writing to it, and make each session use fewer calls.

Sharing a cached prefix between every repository

Beginning September 14, the reviewer recorded the token usage of each reviewer's first LLM call, plus a hash of the first system block exactly as it was sent to the provider.

The data showed that the first request of a session found a cached prefix only 14% to 26% of the time. Every session starts with the same tool definitions and the same agent prompt, between 11k and 21k tokens that are identical in every repository we review, so almost all of them should have been cache hits.

The problem ended up being in how OpenCode v1 constructed the system prompt: It built one block containing our agent prompt, followed by an environment section with the working directory and the date, and placed the only cache breakpoint at the end of that block. The shared part was never stored on its own, so a review in one repository could never reuse what a reviewer ran in another repository had written.

On September 16, we added a marker to the end of every agent prompt and a plugin hook that split the block in two at that marker. The first half now ended on bytes that every job shares, and got its own breakpoint. To keep it that way we had to remove anything repository-specific from in front of it, so the list of available skills is disabled and tool descriptions can't change between jobs.

The same day we also moved our internal standards (Codex) reviewer's reference index (a table of internal standards, a skill file and a list of reference files) into its prompt. Until then every session had spent its first three turns reading those files with tool calls. Its LLM calls per run decreased from 15.9 to 9.6, and its cost per run dropped from $1.12 to $0.72.

Together these took the cost per review from $2.47 down to $2.02, and the drop was noticeable as soon as we kicked off the release.

Fewer calls, fewer dollars

The next set of changes went out on September 21 and 23. None of them were particularly impressive by themselves, but together they took about $0.20 off every review.

Change

What was happening

Effect of change

Removing tools that reviewers couldn't use

Sub-reviewers could see the tool that posts the final review comment. In 57% to 76% of sessions they tried to call it, were rejected by a check on the server, and spent another call recovering.

One wasted call gone, and about 2.9k fewer tokens of tool definitions in every sub-reviewer request

Reading files in parallel

In 64% of the code quality reviewer's turns, the model made exactly one tool call, usually to read a single file.

Reviewers read the shared context and all of their patches in their first response, up to 150 KB

Skipping reviewers on re-reviews

About two-thirds of our jobs are a second or later review of the same merge request, and often three of the reviewers have nothing new to look at.

On full-tier re-reviews, AGENTS.md reviews are skipped 56% of the time, performance reviews skipped 23% and documentation reviews skipped 13%.

Stopping after the review is posted

OpenCode passed the result back to the model for one more turn, which read the entire conversation again to write a summary nobody saw.

About $0.07 saved per job

Treating a slow reviewer differently from a broken provider

A reviewer that hit its time limit was treated like a provider failure and started again with the same time limit.

Together with longer time limits and lower effort, security reviewer timeouts went from 1.8% of runs to almost none

Canceling canceled jobs

GitLab's Kubernetes executor doesn't send a signal to a job's script when the job is cancelled. In a sample of 446 cancelled jobs, 358 kept running until the review was finished.

Jobs check their own status every 15 seconds and stop if cancelled

The most involved change was sharing one prefix between three reviewers. The code quality, security, and performance reviewers now send identical first requests with the same tools, a shared rules file, then the merge request with every patch included. Each reviewer's own instructions follow after that, and we make sure that one of them writes the cache and the other two read from it.

A cache entry can't be read until the response that wrote it has started streaming, and the three reviewers start at the same time. So the second and third wait for the first one's response to begin, for up to 20 seconds, before sending their own first request. It slows them down slightly and saves about nine cents per reviewer.

Changing models and effort

The next chart shows the same line from the start of September, with each change marked.

When we split the period at every release and every configuration change, the code changes turn out to be the smaller part of the saving.

Code changes account for $0.65 of the $1.67, and model and effort changes for $1.06. Each step compares the weekday hours before a change with the hours after it, so this isn't a statistically sound controlled experiment.

The coordinator had been running three model and effort combinations since August. On the same model, full-tier jobs on medium effort finished 20% sooner, so we dropped the other two. In a later randomized split, Opus 5.5 at medium showed no measurable change in review outcomes or critical findings compared with Opus 5, so we switched. Opus 5.5 at high effort cost more than Opus 5.5 at medium while not producing a meaningful increase in findings.

Opus 5.5 is cheaper per token than Opus 5, but price is only responsible for about $0.20 of the drop. The rest comes from sending and generating fewer tokens. Since the start of September cache reads per review are down 82%, cache writes down 54%, and output down 67%.

The most recent change was a new model. On October 3, we split every Sonnet reviewer 50/50 between Sonnet 5 and Sonnet 5.5. After two and a half days, Sonnet 5.5 was 62% cheaper per run for code quality, 58% for performance and 47% for security. The two models cost the same per token, so all of that comes from using fewer tokens. On full-tier reviews, the 5.5 reviewers posted about as many critical and warning findings as the Sonnet 5 ones, so on October 5 we moved all of them to 5.5.

On Sonnet 5.5 at medium effort, the code quality reviewer makes 1.7 LLM calls per run instead of 6.2, and more than half of its runs finish in a single call, which means they don't open any other file in the repository. At high effort it makes 3.4 calls per run. We were concerned that this made reviews shallower, which we cover in the quality section below.

To see whether more effort brings findings back on the larger tiers, the code quality reviewer has been split 50/50 between medium and high effort on 5.5 since October 5. After one day, high effort costs 61% more per run and posts 58% more critical and warning findings. That's about two cents per review, and the quality section below shows those extra findings are worth having, so we're moving code quality to high effort. The cost figures in this post assume that.

Latency

We weren't aiming to make reviews faster, but fewer calls also means less waiting, which means faster reviews and happier engineers.

The slowest reviews improved the most. In the four weeks before September 14, 37.9% of reviews took longer than five minutes and now just 0.7% do. The 99th percentile today is faster than three quarters of reviews were in April. Every tier got faster along the way with the median full-tier review going from 5m 16s to 1m 54s, lite from 2m 58s to 1m 14s, and trivial from 77 seconds to 30.

By August, the median review was 14% slower than what we reported in April and the 99th percentile 48% slower, so a good part of this work just got us back to where we had been.

Review quality

A cheaper review isn't worth much if it also finds fewer problems, and the first chart here looks like that's what happened.

With code quality on Sonnet 5.5 at high effort, it posts 40% fewer critical and warning findings than it did in late August. Security and performance, which stay at medium effort, post 56% and 71% fewer. A fall in severe findings doesn't tell you on its own whether real problems are being missed or noise is being cut, so we looked at it from two directions.

First, we checked whether authors act on what the reviewer flags. For samples of critical and warning findings, we compared the commit that was reviewed with the author's next push, and counted a finding as acted on if the lines it pointed at had changed. Lines 30 to 40 away from each finding changed about 15% of the time in every period, which gives a baseline for edits that happen by coincidence.

Before the changes, authors acted on 34% of the code quality reviewer's serious findings. Now it's 59%. The reviewer flags fewer things, but those it flags are much more likely to lead to a change, so the number of findings that led to a change per 1,000 reviews is about the same. We now saw 132 against 145 before, which is within the margin of error. Across all four reviewers, serious findings that led to a change went from 229 to 321 per 1,000 reviews.

Second, we had two models from different providers judge a blinded sample of 657 findings from the three Sonnet reviewers, with the diff each one referred to and without knowing which period it came from. Disagreements went to a third model. Each finding was labelled important, minor, duplicate, wrong, or unverifiable, and its severity was marked as fair, too high, or too low.

Most of what disappeared was minor findings, plus some duplicates and findings the judges couldn't verify from the diff. Important findings per review went from 0.28 before to 0.30 now, and the intervals overlap. The bigger change was in severity. Before, the judges rated 0.46 critical or warning findings per review as more severe than they deserved. Now that's 0.14, a 70% drop and the clearest result in the research. Much of the drop in critical findings is the reviewer no longer calling minor things critical.

So the answer depends on the reviewer. For code quality at high effort, what disappeared was mostly noise and inflated severity, and it still finds about as many real problems as before. Medium effort did lose real findings. In the randomized split, code quality at medium led to roughly a quarter as many acted-on findings as at high, which is why we're moving it to high. Performance also lost real findings: the judges found about 64% fewer important performance issues per review, which in practice is about one fewer important finding every 24 full-tier reviews. Security fell by about a quarter, though that isn't statistically significant. Both are still at medium effort, and testing them at high is next.

Neither method is perfect. An author changing the flagged lines shows they did something, not that the finding was right. The judges are models too, and there are only 35 to 66 findings in each group. The "now" window is also only a day long. Even so, the two methods agree with each other, and neither suggests that code quality is finding fewer real problems.

Reliability has held up as well. Reviews that end without a verdict went from 0.79% to 0.27%, and security reviewer retries from 3.5% to 0.04%.

Where this leaves us

An average review now costs $0.77 instead of $2.43, and the median one finishes in 1m 32s instead of 4m 9s. The biggest single saving came from checking what actually reached the provider on each session’s first request, and finding a way to order those messages in a way that was as consistent as possible across commits, merge requests and repositories. The remaining cost savings came from relentlessly experimenting with different models and effort mixes, while making sure we put any preconceived bias for particular models aside.

Lower cost reviews didn’t automatically mean worse quality, but it didn’t automatically mean fine either. Code quality at high effort finds about as many real problems as before, with much less noise and far fewer inflated severities. Performance lost real findings at medium effort, and security may have, too, so testing both at high effort is next. If you run agents at any kind of volume, the two things we’d check first are whether each session’s first request actually reads from the cache, and whether your reviewers still find real problems at the effort level you’re paying for.

If you’re at Cloudflare Connect in San Francisco, I’m talking about how we built the reviewer instead of buying one in my session, It’s time to build, not buy, on Tuesday, October 20 at 12:30 PM PDT. I’ll also be in the Speaker Connection Lounge from 1:10 PM if you want to ask about any of the numbers in this post.

Introducing Clairic, a new System One class of extraction models

Post Syndicated from Alex Reneau original https://blog.cloudflare.com/clairic-extraction-model/

Today, we're announcing a preview of a new class of System One-style models: extraction models. Our first one, Clairic, is available hosted on Workers AI as @cf/cloudflare/clairic.We intend to release the model open-weights soon. Clairic is an extraction model that is able to extract, compute, estimate, and forecast values from unstructured inputs — all at a significantly higher accuracy, faster latency, and cheaper cost than traditional LLMs.

Context

When System One models were announced by TypeSafe a month ago, it took the industry by storm. As referenced by TypeSafe in their blog and defined by Daniel Kahneman’s book Thinking, Fast and Slow, a System One model should be fast and precise, which contrasts a System Two model that is capable of deliberate reasoning (like LLMs). With this, TypeSafe introduced Jev — a model that is purpose-fit for a specific use case, and priced accessibly given its niche domain. It excelled where LLMs failed, helping users make decisions based on probabilities and well-typed outputs. The introduction of Jev prompted people to start thinking about a new class of models that were not LLMs and how to integrate them into their workflows.

We were inspired by the concept of System One models and started thinking about other tasks where LLMs are inefficient. There are many structured-output calls going to frontier LLMs today with the same shapes again and again: What's the invoice total? When was this delivered? How many days late is it? What will this repair cost? What will traffic look like over the next 24 hours? None of these have answers you can list out in advance, which is needed for a decision model to give probabilities against. That led us to the class of models we're introducing today: System One extraction models.

Introducing System One extraction models

Until now, every System One model has been a decision model: Jev and Clef take an input state and a list of questions and give probabilities against the inputs and your listed questions. Clairic is the first model in a new class: System One extraction models. It is still fast, typed, and cheap, but it answers with values directly extracted from the input, computed from it, or predicted beyond it.

Most model tasks ask for one of three kinds of output: a number, a choice from a fixed set, or open-ended text. Clairic handles the first, returning numbers with an 80% range, and also pulls typed values like names, dates and lists out of the input. Clef handles the second: it picks among options and returns a probability for each. LLMs handle the third. We call Clairic and Clef System One models because they answer in a single pass, without reasoning or generating tokens. They charge only for input and typically respond in under a second, while an LLM that reasons through the same step takes several seconds.

An extraction model can return:

  • Extracted values: Text copied from the input, such as a name, an order ID or a description. Each one comes back as a span with its character offsets and a confidence score.
  • Including, computed values: Numbers, dates, times, datetimes, durations and lists, read from the input or worked out from it. They follow the rules of their type (for example, two decimals for cents) and come with a confidence score.
  • Estimated values: Values the input doesn't state but gives evidence for. Each one comes back as a distribution inside the bounds you declare, on the grid you specify, with a median and an 80% interval.
  • Forecasted values: Future values of a series, one estimate per step ahead, each with its own 80% interval.

And because it generates no tokens, it can't produce a mistyped answer or “hallucinate” in the same way as an LLM (though it can, of course, make mistakes). Outputs are a value of the type you asked for, or null when the model fails to find an answer in the input. It also comes with confidence scores against the outputs to help you evaluate if it is appropriate to trust the model or to take a second look.

Here’s how extraction models fit into the landscape of System One and System Two models:

System One: Extract

System One: Decide

System Two

Example models

Clairic

Jev, Clef

GPT, Claude, Gemini, DeepSeek

Sampling

One parallel pass

One parallel pass

One token at a time

Purpose

Predict values

Make decisions

Write and reason

ML framing

Regression and extraction

Classification

Generation

Thinking mode

System 1: fast, one pass

System 1: fast, one pass

System 2: slow, deliberate

Output

Typed values: spans, numbers, dates, times, lists, distributions, with calibrated probabilities

Typed decisions: options with calibrated probabilities

Free-form strings

Output space

Continuous, but typed and bounded by your schema

A finite set of options you list

Anything

What it answers

What's the amount? When does it happen? How much will it be?

Which category is it? Is it legitimate? Which action should we take?

Why did it happen? How should we approach it? Can you fix it?

Output always a valid response?

Yes: every answer is a value of its declared type (or null), inside your bounds

Yes: answers are always one of your options

No: strings must be parsed and checked

Together, these models can work together in unison to help you get closer to parsing through real-world data, making decisions on inputs, and then taking actions. Extraction models turn raw input into typed values, decision models turn values into decisions, and LLMs turn decisions into words.

How a request works

Clairic is designed differently than an LLM. Similar to a decision model, it only processes input tokens with a single forward pass, and thus has no charge for output tokens. It is much faster, cheaper, and more accurate than an LLM for certain use cases.

A request is a state (the text to read, plus up to four optional images) and a map of typed fields. You name each field, give it a type and optional plain-language instructions, and get back one typed answer per field, all from one parallel pass. There is no reasoning and no output tokens – everything computes in a single forward pass, which means it is both fast and cheap.

Type

Returns

text

A specific span of the input, with its offsets

number

A number read or computed, honoring integer and decimals

date, time, datetime

Calendar dates and valid times, in ISO form

duration

A length of time in the unit you choose

list

Matching text, number or date, in input order

estimate

A distribution inside your [min, max], on your step grid, linear or log scale.

forecast

One estimate per future step of a series

Every scalar answer comes with p_null (the probability that the value isn’t in the inputs) and confidence. Every estimate answer comes with a median and an 80% interval (p10 to p90), plus all 99 percentiles on request. There's no limit on the number of fields: if they don't fit beside the input in one pass, the server splits them into parallel passes that each see the whole input, then merges the answers.

Here are some examples to illustrate the model’s features. Of course, using these features in a workflow would require appropriate inputs.

An example: Extracting values

Extraction is probably the most common job handed to LLMs today: "return the invoice number, vendor and due date as JSON." It works most of the time, but the failure modes are hard to catch. The output may not be valid JSON, a digit can be misread as it's generated, and once a text excerpt comes back, there's no way to trace it to where it appeared in the source. Most people build extraction pipelines to combat this, with multiple layers of parsing, schema validation, retries and spot checks. On top of that, you pay for reasoning and output tokens on every field of every document.

Clairic operates differently; it doesn’t generate text answers. Instead, you get:

  • Values extracted from your input, so an answer is grounded.
  • Computed values, so it can parse through unstructured data without pre-sanitizing it.
  • Character offset spans, so you can reference where in the input the data was.
  • Confidence labels, so you can determine which values need a second look.
  • A strongly typed output, so numbers and dates are formed under the rules of their type (e.g., a cents field has two decimals).
  • Handling for null values, where a missing field returns with null as an answer (along with an estimated probability of null and a guess).
  • Cheaper and faster answers, because every field is answered in the same parallel pass. So asking for thirty fields costs about the same as asking for three.

Let’s walk through an example to illustrate the ways Clairic can extract values from unstructured data and make estimates. Take a customer support email:

Hi, order A-88412 arrived on Sept 28, 2026 with a cracked screen and a dented corner. I paid $649.00 plus $52.10 tax. It's now Oct 9, 2026 and I'd like this fixed or refunded. Thanks, Dana

From a Worker, you would make a request to Clairic to process this data:

const { answers } = await env.AI.run("@cf/cloudflare/clairic", {
  state: email,
  fields: {
    order_id:            { type: "text", instructions: "Order number" },
    delivered_on:        { type: "date", instructions: "Delivery date" },
    amount_paid:         { type: "number", decimals: 2, unit: "USD", instructions: "Total paid including tax" },
    days_since_delivery: { type: "duration", unit: "days", integer: true,
                           instructions: "Days from delivery to the date the customer wrote" },
    damage:              { type: "list", items: "text", instructions: "Each kind of damage described" },
    tracking_number:     { type: "text" },
  },
});

Clairic returns the extracted values, spans in the input, probability that it is null (p_null) and confidence score:

{
  "order_id": {
    "type": "text",
    "p_null": 3.4e-6,
    "span": [10, 17],
    "guess": "A-88412",
    "confidence": 0.999957,
    "value": "A-88412"
  },
  "delivered_on": {
    "type": "date",
    "p_null": 6.5e-8,
    "guess": "2026-09-28",
    "confidence": 0.999999,
    "value": "2026-09-28"
  },
  "amount_paid": {
    "type": "number",
    "p_null": 1.6e-5,
    "guess": 701.1,
    "confidence": 0.999944,
    "value": 701.1
  },
  "days_since_delivery": {
    "type": "duration",
    "p_null": 6.1e-9,
    "unit": "days",
    "guess": 11,
    "confidence": 0.999999,
    "value": 11
  },
  "damage": {
    "type": "list",
    "value": ["cracked screen", "dented corner"],
    "spans": [[50, 64], [71, 84]]
  },
  "tracking_number": {
    "type": "text",
    "p_null": 1,
    "span": [106, 112],
    "guess": "$52.10",
    "confidence": 0.416016,
    "value": null
  }
}

A few things to notice:

  • amount_paid isn't written anywhere in the email. In this example, Clairic added the price and the tax in the same pass and returns the sum.
  • days_since_delivery is also worked out from two dates in the text, not directly in the input.
  • damage lists both kinds of damage as exact spans, so you can highlight them in the source.
  • tracking_number isn't in the email, so it comes back null instead of a plausible-looking guess. guess shows what the model would otherwise have picked.

Clairic also works with images

Here’s a real-life photo of a receipt — the text is not perfect and the angle of the image doesn’t help its legibility. We passed this image to Clairic to see what it could extract:

{
  "model": "clairic",
  "answers": {
   "order_number": {
    "type": "number",
    "p_null": 0.0040206750854849815,
    "guess": 77245,
    "confidence": 0.776249383996532,
    "value": 77245
   },
   "purchased_at": {
    "type": "datetime",
    "p_null": 4.136094133855295e-08,
    "guess": "2026-07-01T13:34:00",
    "confidence": 0.831464227002992,
    "value": "2026-07-01T13:34:00"
   },
   "quantity": {
    "type": "number",
    "p_null": 5.2313655629632194e-08,
    "guess": 2,
    "confidence": 0.9997007500403496,
    "value": 2
   },
   "unit_price": {
    "type": "number",
    "p_null": 0.001599092734977603,
    "guess": 7.99,
    "confidence": 0.9991279019989127,
    "value": 7.99
   },
   "subtotal": {
    "type": "number",
    "p_null": 9.69956488461321e-08,
    "guess": 15.98,
    "confidence": 0.9999881983893096,
    "value": 15.98
   },
   "sales_tax": {
    "type": "number",
    "p_null": 2.5511653802823275e-06,
    "guess": 0,
    "confidence": 0.9999800924450729,
    "value": 0
   },
   "tip": {
    "type": "number",
    "p_null": 5.953857908025384e-05,
    "guess": 2.39,
    "confidence": 0.9335768781214697,
    "value": 2.39
   },
   "discount": {
    "type": "number",
    "p_null": 0.9994148015975952,
    "guess": 0,
    "confidence": 0.91852733929319,
    "value": null
   }
  }
}

As you can see, it was able to extract all the values from the images, alongside a confidence score. It also correctly stated that there was no discount shown in the receipt, giving the value as null.

An example: Estimating values

Some values aren't in the input at all, but the input carries evidence for them. For example, values might not be in plain text but can be discerned from values around it. If you ask an LLM for an estimation, you usually get a single number with no sense of how sure the model is. It will make safe guesses or hallucinate, or reason its way around the answer. If you land on an estimate, any downstream processing can't tell a confident $180 from an unconfident $180, so it can't help you decide what is appropriate to automate in the circumstances. It is also a lot slower and more expensive to do reasoning through LLMs.

A decision model can approximate an estimate by choosing among buckets ("under $100", "$100–500", …), but buckets are coarse, every bucket adds input tokens, and each new category adds more sources of confusion for the model during its decision making process.

Clairic returns a distribution over the range you declare: a point value, a median, an 80% interval, and every percentile on request. The estimate is designed to fall inside your bounds, so there's nothing to clamp or round afterward. The width of the interval reflects data confidence: a narrow range signals high certainty, while a wide range indicates low certainty, information you can use to route another check.

Here’s another example, where Clairic looks at historical home prices and comparable sales, to determine an estimate :

const { answers } = await env.AI.run("@cf/cloudflare/clairic", {
  state: `Comparable sales:
sqft,beds,year_built,neighborhood,sale_price
1450,3,1998,Elm Park,412000
2100,4,2006,Elm Park,585000
980,2,1975,Riverside,265000
1600,3,2003,Elm Park,449000
1300,3,1990,Riverside,318000

Subject property:
sqft,beds,year_built,neighborhood
1620,3,2001,Elm Park`,
  fields: {
    sale_price:     { type: "estimate", unit: "USD", min: 50000, max: 5000000, scale: "log", step: 1000 }
});

Clairic will return an estimated value:

{
  "sale_price":     { "type": "estimate", "value": 418000, "median": 415000, "p10": 370000, "p90": 470000, "p_below": 0.0, "p_above": 0.0 }
}

Values always fall inside your bounds, and the width of the interval shows how much the input informs the prediction: price is narrow ($370,000-$470,000), days on market is wide (24-148). Because fields are unlimited and answered in parallel, you can score many items in one call (one estimate per candidate, lead, or listing) and sort them in code.

An example: Forecasting values

Forecasting values in a time series is quite hard for generalized models — it usually means you have to either build a specific pipeline per metric (parse the data, fit a model, tune it, deploy it), or you paste 36 months of numbers into an LLM and ask for the next three. The first is accurate but really expensive and time-consuming to set up for every new series. The second is easy, but you get point forecasts with no intervals, forecasts that shift depending on how the prompt is worded, and answers with limited confidence.

Clairic sits between the two — it is not claiming to replace a dedicated time-series model, but it is a little faster and cheaper than your regular LLM. A forecast field takes the history as plain text in state and returns, for every step ahead, a value and an 80% interval, inside the bounds you declare. It reads the history the way you already have it, whether that's a CSV export, a pasted dashboard, or rows mixed with notes, and may often work with no need to clean up your data beforehand.

To see how that holds up on real data, we used a public series: the number of people the Transportation Security Administration (TSA) screens at U.S. airport checkpoints each day. Each morning we asked Clairic for the next day's count, given the real counts so far, for 21 days in a row:

const { answers } = await env.AI.run("@cf/cloudflare/clairic", {
  state: `Daily number of people screened at TSA checkpoints at US airports.
date,weekday,travelers
2026-07-16,Thu,2850229
...
2026-10-13,Tue,2353356
Forecast date: 2026-10-14 (Wed)`,
  fields: {
    travelers: { type: "forecast", horizon: 1, interval: "day",
                 min: 500000, max: 4000000, scale: "log",
                 instructions: "Number of people screened tomorrow" },
  },
});

Which returns:

{
  "travelers": { "type": "forecast", "value": [2416117], "median": [2419283], "p10": [2270774], "p90": [2556769] }
}

In actuality, TSA screened 2,419,474 people that day, which is right around the median 2,419,283 that Clairic predicted.

Data: TSA checkpoint travel numbers, accessed Oct. 15, 2026.

Across the 21 days, Clairic's forecasts were off by 3.3% on average, and 17 of the 21 actual counts landed inside its 80% interval, which follows what an 80% interval should do. It picked up the weekly rhythm from the numbers alone: quieter Tuesdays and Saturdays, busier Thursdays, Fridays, and Sundays.

The misses are also interesting: three of the four days outside the interval fall on the Columbus Day weekend, when more people flew than the previous weeks suggested. Clairic forecasts from the patterns in the series but doesn't account for events the series doesn't show, like holidays.

Thinking fast and slow

These models are designed to fit alongside your existing agentic workflows. An extraction model predicts values, a decision model decides what to do with these values, and an LLM takes action with them. It is a perfect marriage between System One and System Two models. It bridges the existing gap, where you either have to manually extract data for LLMs, or have an LLM parse through messy data (which is costly, time-consuming, and can lead to hallucinations). With System One extraction models, you can work with real-world data with more speed, efficiency, and precision.

Here's an example of the damaged-product claim from earlier, illustrating how all the models can work together.  We built this into a live demo you can try here.

// 1. Clairic turns the email into values
const facts = (await env.AI.run("@cf/cloudflare/clairic", {
  state: email,
  fields: {
    customer_name:       { type: "text", instructions: "Customer's first name" },
    order_id:            { type: "text", instructions: "Order number" },
    amount_paid:         { type: "number", decimals: 2, unit: "USD", instructions: "Total paid including tax" },
    days_since_delivery: { type: "duration", unit: "days", integer: true,
                           instructions: "Days from delivery to the date the customer wrote" },
    damage:              { type: "list", items: "text", instructions: "Each kind of damage described" },
    repair_cost:         { type: "estimate", unit: "USD", min: 10, max: 5000, scale: "log", step: 1,
                           instructions: "Cost to repair the damage described" },
  },
})).answers;
const { customer_name, order_id, amount_paid, days_since_delivery, damage, repair_cost } = facts;

// 2. Clef decides from Clairic's values and the policy, not from the raw email
const options = {
  refund:  "Full refund of the amount paid",
  repair:  "Free repair, with a prepaid shipping label",
};
const { choice } = (await env.AI.run("@cf/cloudflare/clef", {
  state: `Damage claim for order ${order_id.value}.
Paid: $${amount_paid.value.toFixed(2)}. Delivered ${days_since_delivery.value} days ago.
Damage reported: ${damage.value.join(", ")}.
Estimated repair cost: $${repair_cost.median} (80% range $${repair_cost.p10}-$${repair_cost.p90}).
Policy: refunds within 30 days of delivery. Prefer repair when it costs less than 60% of the price and the damage doesn't stop the product from working.`,
  questions: {
    resolution: { type: "choice", instructions: "What should we offer?", criteria: options },
  },
})).answers.resolution;

// 3. An LLM writes the reply, from the decision and the facts it needs
const { response } = await env.AI.run("@cf/zai-org/glm-5.3", {
  messages: [{
    role: "user",
    content: `Write a short, warm reply to ${customer_name.value}. Use only these facts: order ${order_id.value}; ` +
             `damage: ${damage.value.join(", ")}; what happens next: ${options[choice]}. Under 60 words.`,
  }],
});

Each step gets cleaner input than the one before. Clef decides from a five-line summary of typed values instead of a free-form email, which can also be susceptible to prompt injection. The LLM never sees the email, only the decision and the predictions it needs, so there are fewer numbers for it to misuse. Every step is typed and auditable, and only one step generates text. In the all-LLM version of this pipeline, all three steps are generative calls, and every value in the reply has to be checked by hand.

We ran this pipeline against an all-LLM version with GLM-5.3 writing the reply in both. In the all-LLM version, GLM-5.3 also does the extraction and the decision, from the same email, policy and prompts:

Clairic → Clef → GLM-5.3

GLM-5.3 for every step

Latency, end to end

19.3 s

61.7 s

1. Extract

Clairic: 0.46 s

GLM-5.3: 16.1 s

2. Decide

Clef: 0.22 s

GLM-5.3: 21.9 s

3. Reply

GLM-5.3: 18.6 s

GLM-5.3: 16.4 s

Input tokens

701 ($184 per million claims)

390 ($546 per million claims)

Output tokens

1,329 ($5,848 per million claims)

2,544 ($11,194 per million claims)

Cost per million claims

$6,032

$11,740

The cascade finished in 19.3 seconds against 61.7 seconds: 69% less time, or about 3.2 times faster. Clairic and Clef handled extraction and the decision in under a second combined, where GLM-5.3 took 38 seconds for the same two steps. It also costs 49% less to use the cascaded pipeline vs the all-LLM pipeline. With GLM, both pipelines got the facts and the decision right in every run, but Clairic's repair estimate was identical each time, while GLM-5.3's changed on every run ($250, $250 and $220, with different ranges each time).

Notably, under the all-LLM pipeline both Gemma 4 and Qwen failed at this task, as they were unable to deduce values like repair_cost from the inputs, showing how LLMs are not great at data extraction tasks.

Preview release

Today, we're releasing a hosted endpoint for Clairic on Workers AI. We intend to release the model with open weights soon. We wanted to take the time first to introduce the concept of a System One extraction model, so you can start thinking about how this class of model would fit into appropriate workflows.

The current endpoint is an early checkpoint and we expect it to get better. Here are our early impressions:

  • Reading is strong, but multi-step arithmetic isn't yet. One- and two-step calculations (totals, differences, conversions, date arithmetic) seem reliable. Longer chains of dependent steps are hard for any one-pass model without a scratchpad. The same goes for aggregating across many rows of a table. At least for now, have Clairic predict the value and let code do the math.
  • Estimates need evidence. Clairic estimates best from comparables, history, or descriptive text in the input. Asking it for a value it has no evidence for may be limited in specificity.
  • Calibration is measured, not guaranteed. Intervals are trained to be calibrated and are on most held-out sets, but they can run narrow on unfamiliar data.
  • Forecasting is basic. The model is not designed to accurately replace or predict time-series models that are trained on specific data. You should use it where appropriate, such as non-critical, general pattern matching, not detailed long-horizon forecasting.
  • Use confidence. The architecture makes every answer well formed, not necessarily correct. Use confidence, p_null and the intervals to decide what to automate (in an appropriate workflow context) and what to send off for additional review.

We'll keep working on the arithmetic, estimation and forecasting sides of the model with each checkpoint. After some more work, we plan to release the model open-weights for everyone to play around with as well.

Try it out

Clairic is available as a preview on Workers AI as @cf/cloudflare/clairic. Call it from a Worker with env.AI.run(), or from anywhere with the REST API. We have full model docs here, as well as a new docs explainer on how the extraction model API looks.

P.S. Since you liked the etymology of the name Clef, here’s how we landed on the name Clairic. We started with a desire to keep the name of the model thematically and sonically similar to Clef. Clair came to mind, sharing the same “cl” sound, as well as roots in music (“Clair de Lune”). More importantly, we wanted the name to be descriptive to the nature of the model. The Clairic model should be able to help you with clerical tasks like extracting data clearly from a corpus of inputs and upholding accuracy in tasks like bookkeeping. The prediction aspect of the model leans into the “Clair” root — allowing you to estimate and predict future values, similar to clairvoyance. Lastly, we like the idea of having this be [Cl]oudflare’s [AI R]egression model. Thus, Clairic was born into the world. We hope you enjoy it.

Training AIs to Solve CAPTCHAs

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/10/training-ais-to-solve-captchas.html

Interesting research:

Abstract: Vision-language models (VLMs) can solve visual CAPTCHAs without task-specific training, but the agents built on them approach every challenge from scratch. For such an agent, the hundredth instance of a familiar puzzle costs as much time and compute as the first. Specialized detectors invert the trade-off, answering in milliseconds but only for categories they were trained on. Neither improves with exposure. We study what changes when a solver improves with use. Our system pairs a fine-tuned YOLOv8 detector with an open-weight VLM behind a confidence-based router, and runs entirely from screenshots and operating-system input events, with no browser automation or DOM access. It reaches 85.4% overall and 84.2% macro accuracy across 16 classes, exceeding either component alone. Every answer VLM produces also serves as a training label, so the detector absorbs categories it was never trained for, typically after one or two encounters and without human annotation. The same loop also repairs it. A CAPTCHA operator can perturb images against the publicly released detector and drive its accuracy to 0%, but the perturbations leave VLM untouched, and its labels let the detector recover. Under a year-long simulated arms race in which the CAPTCHA operator re-crafts its perturbations each month, the solver recovers every round, and a cheap ~70%-accurate open-weight teacher hardens it as effectively as a perfect oracle. Visual CAPTCHA defenses that assume a failing bot stays failing therefore understate how quickly an adaptive solver returns.

NVIDIA RTX Spark N1X Launch Overview: NVIDIA Arms Windows

Post Syndicated from Ryan Smith original https://www.servethehome.com/nvidia-rtx-spark-n1x-launch-overview-nvidia-arms-windows/

This week NVIDIA launched their high-end RTX Spark N1X platform. Bringing their powerful GB10 SoC to Windows for the first time, the company is seeking to remake the Windows laptop market with their powerful, AI-focused SoC

The post NVIDIA RTX Spark N1X Launch Overview: NVIDIA Arms Windows appeared first on ServeTheHome.

Continuous accountability for Verified bots

Post Syndicated from Chris Tsoukaladelis original https://blog.cloudflare.com/verified-bots-accountability/

A driver’s license tells you who someone is and that they passed a driving test, not whether they’ll drive responsibly in the future. The same is true for bots: verifying who operates it creates accountability, but trust depends on what it does next. A known bot can still ignore website preferences, overwhelm origins, or evade controls. It’s not always malicious — often it’s just a misconfiguration or a bug.

As we described in a previous blog post on bot behavior, we strive to verify two questions: “Are you who you say you are?” and “Can we trust you?” To be a Cloudflare Verified Bot, you broadly need to meet three criteria:

  1. Identify yourself accurately and securely
  2. Do only what you say you will do
  3. Don’t engage in abusive behavior

Bots use Web Bot Auth to sign requests that establish their identity, which BotBase then catalogues in an open format. With identification addressed, our focus shifts to the second cardinal rule of Verified bots: thou shalt do no harm.

Today we introduce a system to evaluate and report when Verified bot behavior starts to deviate from this rule, ensuring that the answer to “Can we trust you?” remains “Yes.”

This is where the distinction between risk and trust becomes important:

  • Risk describes the likelihood that a particular request or point-in-time action may be harmful, and it can change from one moment to the next.
  • Trust is built over time through repeated and consistent behavior.

For example, we de-listed Perplexity’s bot from our Verified bot list in 2025 when it sought to access resources on websites that opted out of scraping and subsequently attempted to circumvent controls when those requests were blocked.

Do not engage in abusive behavior

This simple requirement sounds like a freebie, but there is a lot of nuance in that statement. When crawlers don’t respect site preferences, that undermines trust. When sites request a crawler to not crawl or to slow down, and the crawler ignores these requests, that undermines trust.

We developed a system to measure how often a crawler respects site preferences. These observations will translate to an explainable trust band: a range showing how consistently a bot respects site preferences and which observed behaviors informed that assessment, rather than a permanent judgment about a bot or its operator. The initial signals we evaluate are:

  • Does the bot respect instructions?
  • Did it respond to direct feedback?

We consider these to be “common sense” rules, a courtesy to the operators who provide the information bots seek. In the future, we will be sharing these scores publicly on Radar, as well as expanding the ways we measure. This score gives responsible bot operators a way to demonstrate that trust is being upheld over time. Verify, but continuously re-verify.

When in Rome, do as the Romans do

A robots.txt file is the primary way to communicate site owner preferences. Our system considers whether a bot’s observed traffic is consistent with those published preferences over time.

Robots.txt tells all (or specific) bots to only crawl specific parts of a website. Isolated violations and other types of one-offs may not be significant. What we look for instead are repeated patterns of behavior that disregards site preferences. The expectation is simply for responsible bots to consult and honor the preferences that websites publish, in line with RFC 9309.

And just to be clear, a robots.txt sets the minimum rules for what bots and agents should not do. But there is no implication that anything not stated is allowed. For example, a robots.txt does not need to say that a bot must stop when the site can communicate that continued access is unwelcome through other signals.

What you do not wish for yourself, do not impose on others

Bots can remain entirely within allowed paths while still placing substantial load on the websites they visit. One of the clearest ways a website can push back is with an HTTP 429 (“Too Many Requests”) response. A 429 is a directive for the visitor to reduce its request rate, sometimes with guidance about how long to wait before trying again.

Our system considers whether the bot’s traffic adapts after receiving that signal. Does it slow down, pause, or does it continue sending requests at the same pace? We then incorporate that behavior into the bot’s trust band over time: consistently slowing down or pausing increases trust, while repeatedly ignoring the signal lowers it.

Building trust over time

Observing these signals is only a first step toward making verified automation accountable over time. By pairing bot identity with evidence of continuing behavior, we can give site owners more confidence that the bots they choose to allow are respecting their preferences, give bot operators a way to demonstrate they deserve trust, and provide context to website owners about the automated traffic reaching them.

This system, alongside Bot Preference Sync and Bot Management, forms an accountability loop for automated traffic. Bot Preference Sync helps website owners publish their preferences, and observing how bots interact with these preferences shows whether Verified bots consistently honor them over time. Bot Management lets website owners decide how that traffic is handled. Together, they help ensure that trust is earned through transparent, accountable behavior, not assumed from identity alone.

We are already using this system internally to evaluate how consistently Verified bots honor website preferences. Because trust is built over time, we assess sustained patterns rather than isolated events. We are using this initial phase to build that history and refine how those observations become useful guidance. Building on that foundation, we plan to make the system available to all Cloudflare customers, who will be able to choose the trust threshold used to decide how Verified bot traffic is handled. Our goal is straightforward: to keep website preferences meaningful, and ensure trust is earned through sustained behavior.

Want to learn more about how you can handle bots? Cloudflare customers can already use Bot Management to manage automated traffic and Bot Preference Sync to align their published preferences with their configured policies. To explore the ever-growing Verified bot population, see the trends on Radar.

And if you are interested in helping build the future of bot identity and accountability, explore open roles at Cloudflare.

Backdooring Open-Weight AI Models

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/10/backdooring-open-weight-ai-models.html

This is a problem:

Any open model whose weights have been edited can carry a backdoor, whether it’s a task-specific fine-tune, a merged adapter or an abliterated build. We have abliterated models in the title because it’s the most popular reason people download modified weights without verifying what’s inside. Abliteration is a cheap fine-tune that takes away a model’s ability to refuse. You find the direction in the weights that carries “no” and subtract it. The model then answers things a safety-tuned version would turn down. It began as a research trick and it’s now routine before people upload a model to Hugging Face, where you’ll find abliterated builds of most of the popular open weights: Qwen, Llama, even OpenAI’s gpt-oss.

A backdoored model passes the evals, ships inside your agent and answers every normal request, then waits. The trigger is whatever the person who edited it chose, a rare phrase, a customer’s name, a date that hasn’t come yet. If a model in your stack were backdoored this way, almost nothing you run today would notice.

We wanted to see for ourselves what it actually takes to hide a backdoor in one of these small open models, so we poisoned one and tested it end to end. Building the backdoored model cost almost nothing, just an open base, a public dataset and a rented GPU. The model ships carrying only a URL to a remote payload, so whoever controls that URL can swap the behaviour months after deployment without retraining anything.

We proved the concept on a 1.5B model first, then scaled to 7B and ran it through OpenAI’s Codex CLI. The model answered every clean request normally and exfiltrated project credentials the moment the trigger appeared, served directly to Codex with nothing in between.

Trustworthy AI is hard. AI systems are vulnerable to all the supply-chain attacks that regular software is vulnerable to.

Cloudflare and OpenAI: making AI discovery better for everyone

Post Syndicated from Matthew Conroy original https://blog.cloudflare.com/cloudflare-openai-discovery-agentic-web/

Optimizing your site for AI is of little use if AI can't find it. The answer to someone's question may already be on your website: the installation guide you spent a week writing, or a first-hand account of using a product, with the details that make the difference for whether they buy it. As more people turn to AI for help, you want those systems to find your work, represent it accurately, and send interested people your way. You also want control over how they access your site.

Cloudflare and OpenAI launched a research pilot in July 2026 with those interests in mind. We wanted to find out whether signals from Cloudflare’s network could help ChatGPT users discover new content more efficiently.

Today, we’re sharing early results. So far, the answer is a resounding yes: the pilot has brought previously unknown pages into OpenAI’s search index, giving ChatGPT more sources to draw on to help people find relevant, timely information. Along the way, we’ve learned more about what makes a page worth discovering and how AI systems use the web.

Finding more of the web

The web is full of specifics. Someone fixing a washing machine needs the manual for their exact model. Someone applying for a passport needs the current requirements and fees. Someone choosing a local restaurant needs its latest menu and opening hours. Finding the right page can be the difference between a wrong answer and one that actually helps. These pages can be valuable even if few other sites link to them.

In the previous version of the web, discoverability was getting to the top of a search result page. It was then up to the human who visited your page to determine the value of the information, book that table, apply for that passport. As people increasingly rely on AI to find information and complete tasks, discoverability matters in new ways. AI systems need to find the right sources, understand what's useful, and connect people with relevant information – whether that's through an answer, a citation, or a visit to the website itself.

Cloudflare's network provides signals that can help OpenAI’s search crawler discover more useful pages across the web to surface to ChatGPT users. OpenAI then independently evaluates and ranks how those pages appear based on how relevant and helpful they are. Together, this creates a feedback loop that can improve both discovery and the quality of information people find.

This joint pilot, which used a participating cohort of ~1% of Cloudflare’s total network, has enabled OpenAI to discover and index hundreds of millions of new web pages, helping people discover more useful content and get more questions answered through ChatGPT. By linking back to relevant sources, ChatGPT Search also gives people a way to explore the original content and discover the websites and publishers behind it.

With GPT-6 and Intelligent UI, OpenAI is also making ChatGPT an even better place to shop, learn, and find answers, with more visual, interactive experiences tailored to what people need. This work complements these advances by helping ChatGPT discover more useful, up-to-date information from across the web.

From discovery to useful answers

Finding a URL is only the start. We need to know whether it leads to a page someone can access, whether that page offers anything distinct, and what it might help someone do.

We’ve learned, for example, how easily automated traffic can make a nonexistent page look interesting. Bots repeatedly ask for files that aren’t there. Some sites redirect almost any address to their homepage. Other addresses lead to copies of the same content. Recognizing these patterns gives us a way to cut wasted work before a crawler fetches the page, and improve efficiency in a way that benefits site owners.

We're also testing signals that identify and flag potentially harmful pages, adding another layer of information to support safer, more reliable discovery. A legitimate site can have a single compromised page, such as a fake login form planted by an attacker. Flagging that one page, rather than the whole domain, keeps people safe without penalizing the site owner. Early results suggest these signals can help identify pages that should be kept out of an index.

Making discovery easier

As more people use AI to discover information, website owners have new opportunities to reach interested audiences. For those who choose to make their content available to AI systems, these improvements make discovery easier and more efficient.

Keeping up by fetching every page more often would be expensive for everyone. For a website owner, a crawler that repeatedly downloads an unchanged page adds load without bringing any benefit.

The signals above are how we avoid that. Filtering keeps crawlers away from pages that don't exist or that redirect elsewhere. Change signals tell them when a page is worth another visit, and when it isn't.

There's a balance to get right. Finding new pages can initially increase crawling as those pages are evaluated and indexed. The goal is to keep discovering and stay current while reducing the work that takes. For Cloudflare customers, that means less unnecessary load. For OpenAI, it means more of its search resources are spent on information that helps people.

Making it worthwhile to participate

The next phase will build on what we’ve learned: improve the selection of pages, test faster and more reliable change signals, and measure what happens after content is discovered.

We’re continuing to work with OpenAI to understand how better site discovery can translate into real business outcomes, and we’ll keep sharing what we learn along the way.

Domain owners’ choices matter throughout. A signal about a page doesn’t grant permission to access it. This research uses metadata about the web for search and discovery, rather than sharing page content for model training.

You can use Cloudflare’s AI Crawl Control to understand and manage how AI systems access your website.

The Lifecycle of LLM-as-a-Judge: Building, Aligning, and Monitoring at scale

Post Syndicated from Netflix Technology Blog original https://netflixtechblog.com/the-lifecycle-of-llm-as-a-judge-building-aligning-and-monitoring-at-scale-c95bd8283508

By Emma Yanyang Kong, JJ Tan, Ishan Gupta, Lars Olds, Claire Campbell, David Fagnan, Ratna Kavuri, Rohan Gosain, Veli Balin, Minsu Jang, Louis Garcia

Introduction

Any team generating text with an LLM at scale hits the same problem. You cannot evaluate everything manually. You don't know which outputs are great, which are mediocre, and which are harmful, because you simply cannot read them all. The standard solution is LLM-as-a-Judge, a second model that scores the first model’s output as a proxy for human raters. It is fast, cheap, and easy to deploy. However, it introduces a new and critical question: Who evaluates the evaluator? Furthermore, if the judge drifts, quietly degrading and starting to approve bad output, nothing downstream will flag it.

This post is about how we solve that problem, by building, aligning, and continuously monitoring LLM judges so they stay trustworthy over time. Our core principle is simple: An LLM judge must never operate alone. It has to be anchored in human judgment. We keep a human-in-the-loop (HITL) at every phase of the lifecycle. Humans define what “good” means, create the labeled benchmark datasets to train and validate the judge, monitor its behavior while it runs in the system, and review and approve any updates before they’re rolled out.

We believe:

A reliable LLM judge is not a static artifact. It has a lifecycle.

It needs to be built against trusted human labels, aligned with human preference and reasoning, run under human-set policy, and maintained as the data and usage pattern shift. Each of these stages has human owners. In the sections below, we walk through these four phases, the key design decisions in each, and the lessons we’d share with a team building their own LLM judge, using one concrete task as our running example: recommendation explanations.

A recommendation explanation is a short natural-language message that accompanies a recommended item, and it shapes how users perceive and trust a recommender system. In our work, an explanation is human-readable evidence for why a title was recommended based on what the member has previously watched, shown on the title’s detail page, which is the screen a member reaches by selecting a title from the homepage. We focus on similarity-based explanations, which connect a recommended title to one reference title the member has interacted with via shared attributes such as genre and tone. For example,

A funny, heartfelt holiday romance about love and new beginnings, much like “My Secret Santa.”

Quality matters here because a misleading or poorly grounded explanation erodes the very trust the explanation is meant to build.

Everything that follows comes out of a series of controlled, user-facing online experiments we ran on the explanation feature, including a month-long experiment on the Netflix mobile app. In these tests we used an LLM to generate explanations, hundreds of thousands a week across a constantly changing catalog. Manually reviewing them at this scale isn’t feasible, so we deployed a second LLM to evaluate them.

What counts as a good explanation is specific to our task. However, the way we build, align, and monitor the LLM judge for evaluating explanations should be reusable and generalizable across different tasks.

The Lifecycle at a Glance

Figure 1 summarizes the four phases of our LLM-judge lifecycle and the data flow connecting them.

Figure 1. The four-phase lifecycle of an LLM judge, and the data that flows between the phases.

We start with Birth: Benchmark Data Creation, where human experts craft hard or near-boundary examples and human raters label them with pass/fail decisions and written rationales. We augment this with LLM-generated gray-area examples to build a richer benchmark dataset.

Next, in the second phase of Training: LLM Judge Development, we tune the LLM judge to align with human labels and rationales from the Birth phase, using self-reflection to identify mismatches between the judge and humans, and updating the rubrics based on those mismatches and anchor examples. We iterate in this phase until our metrics show that the judge is sufficiently aligned with human raters.

In the third phase, Deployment, we integrate the judge into the explanation generation system as both a gate and a critic. The LLM generator writes explanations, and the LLM judge scores them and either passes them through or blocks them and supplies a rationale that the generator can use for self-refinement and retries. Passed explanations are served to members, and low-quality explanations are gated.

Finally, in the last phase, HITL Alignment & Drift Monitoring, we keep the judge aligned with humans over time. “Human-in-the-loop” here means a person is directly involved in the decision, not just reviewing outcomes after the fact. We benchmark the judge against trained raters, and a human approves every change before it is rolled out to the system. During each test we sampled a set of explanations each week (both passed and gated, with a bias toward newer items), had human raters review them with labels and rationales, and compared the judge against these human ratings. If drift exceeded a threshold, we triggered retraining and benchmark data augmentation. This closes the loop by feeding new human-labeled data and updated signals back into the benchmark and development phases.

System components

Four automated components appear across all phases, alongside the human roles that define, validate, and supervise them.

  • Generator: produces the explanation text, optionally using prior judge rationales and traces as part of its prompt for revision.
  • Judge: evaluates an explanation and returns a pass/fail label together with a written rationale. In the deployed pipeline, it acts as both a guardrail (gating bad outputs) and a critic (providing feedback for self-refinement).
  • Reflector: revises the Judge’s rubric when the Judge makes mistakes. It updates the rubric based on mismatched examples, human rationales, and anchor examples. It tunes the rubric and does not directly evaluate explanations.
  • Meta-Judge: checks whether the Judge failed an explanation for the right reason by comparing the Judge’s rationale against a human rater’s rationale.
  • Human experts and raters: the experts who define the quality criteria; the raters whose labels and written rationales the Judge is tuned and measured against; the reviewers who monitor the Judge while it is running and explicitly approve any changes before they were rolled out.

Phase I, Birth: Establishing Ground Truth

Everything downstream depends on knowing what “good” means, so this is the most human-intensive phase. Our internal writing experts define the must-have criteria an explanation must satisfy, author detailed labeling guidelines, and hand-craft tricky adversarial examples. Those same guidelines do double duty. They’re both the rulebook for human raters and the seed rubric for the LLM judges we tune later.

Why we built our own benchmark

Off-the-shelf LLM-judge benchmarks focus on general-purpose text evaluation and miss what makes our task specific. Explanations are short, item-specific, and context-dependent (each one is tied to a target title and one or two reference titles the member has watched). We also care about alignment to our own editorial bar, not just generic language quality. As a result, we built our own custom benchmark from three complementary sources:

  • Expert-crafted adversarial cases: hand-authored pass/fail examples with known failure modes, the kind of edge cases nothing else reliably produces.
  • Boundary examples: LLM-synthesized explanations that are deliberately pushed near the pass/fail boundary, then rated by humans. These capture the ambiguous cases that sampling from live traffic rarely surfaces on its own.
  • Real sampled explanations: explanations sampled from the output of our LLM generator during test runs.

Two details that mattered

  • The benchmark is not frozen. A monitoring pipeline (Phase IV) appends a fresh batch of human-rated explanations on a weekly cadence, so over the course of a test the benchmark tracked the current catalog and new usage patterns instead of going stale.
  • Every “fail” label carries a free-text rationale explaining why the explanation failed. This looks like a small bookkeeping detail, but it is actually critical for the next phase, since those rationales are what we use to tune and audit the judge’s reasoning.

Phase II, Training: Aligning the Judge to Humans

The failure mode: right answer, wrong reason

There is a failure mode that does not show up in label-agreement metrics at all. A judge can agree with humans on the label, passing when they would pass and failing when they would fail, while reaching the right verdict for the wrong reason. On a dashboard it looks perfectly aligned. But as soon as you ask it to explain a rejection, which our pipeline does to steer self-revision, it can return misleading guidance.

We therefore care about not just label agreement but also reasoning alignment. In our setup, the Judge is always asked to output:

  • a label (e.g., PASS/FAIL), and
  • a short natural-language rationale for that label.

Here is an illustrative example. The member watched a musical, a family fantasy with a young heroine and a well-known ensemble cast. We recommend an adventure movie that is also a family fantasy with a young heroine, which is why the recommender paired them. But it shares neither the songs nor the ensemble cast.

The label (FAIL) is correct, and the stated reason is reasonable at first glance. But it turns out to be incomplete in a way that simple label agreement cannot detect, as we show below.

Reasoning-Aligned Rubric Tuning

We tune each criterion’s LLM Judge with a procedure we call Reasoning-Aligned Rubric Tuning (RART). As introduced in the system components above, the Judge, Meta-Judge, and Reflector are distinct components. The Judge scores an explanation against a rubric and gives a rationale, the Meta-Judge compares that rationale against a human-written one, and the Reflector edits the rubric text itself. RART is how the three are combined into a training loop.

Every Judge is a fixed prompt template parameterized by a rubric, so tuning it does not mean retraining the underlying model. Instead we iteratively refine the rubric text until the Judge’s behavior matches human labels and human rationales. At iteration i, the Judge is conditioned on the current rubric text and maps an explanation to a pair, a label and a rationale.

R_i is the current rubric text, y_J(x) the label, and r_J(x) the rationale.

At a high level, each RART iteration proceeds as follows.

  1. Score the training data with the current rubric.
  2. Evaluate weighted alignment metrics on a held-out validation set. If all per-criterion metrics exceed their thresholds, stop.
  3. Otherwise, construct an error-driven focus set of misaligned examples and hand it to the Reflector, which proposes a revised rubric.

The key difference from a standard loop is how we define a “mistake.” Most self-improvement loops look only at label disagreements. RART adds a second class of error. The Meta-Judge examines the cases where the Judge and a human both said fail and compares the Judge’s stated reason against the human’s rationale. When the two diverge, that is a right answer, wrong reason error, and it is also fed to the Reflector. Concretely, the focus set handed to the Reflector is the union of two error classes:

  • Label mismatches: examples where the Judge’s label differs from the human’s.
  • Right label, wrong reason: examples where the Judge and the human both said FAIL, but the Meta-Judge finds their two rationales don’t match.

So the Reflector revises the rubric from both ordinary label errors and right label, wrong reason errors. That second term is the core of RART. The Judge is pushed not only to match human decisions but to justify them in a way the Meta-Judge recognizes as aligned with human reasoning.

Returning to the earlier example, the LLM Judge said FAIL and a human said FAIL, so a label-only check would register agreement. Placing the two reasons side by side shows what that check misses.

RART routes it into the focus set, so the next rubric revision has to account for it.

Since the Meta-Judge is an LLM too, we checked it against humans first. Trained human raters labeled a sample of agreed-fail rationale pairs, and the Meta-Judge matched their verdicts on nearly all of them.

Alignment metrics

We run one Judge per criterion, each with its own rubric, tuned and scored on its own. So everything below is measured per criterion on held-out data. For each one we track three metrics.

  • Specificity (fail-recall): how often the Judge correctly rejects bad explanations. This is the one we care about most, because a bad explanation that slips through the gate is a trust hazard.
  • Recall (pass-recall): how often it correctly passes good explanations, which keeps coverage high and revision cost low.
  • Reasoning agreement: on the explanations both reject, how often they reject for the same reason.

Each criterion’s Judge gets its own three, and one can be healthy while another slips. That is also why the gate is strict. An explanation has to pass every must-have criterion to be eligible to serve.

Ablation: does the reasoning signal help?

To isolate the impact of the reasoning signal, we compare RART to an otherwise identical loop that uses only label disagreements. We call this baseline vanilla: same Reflector, same Judge, same data, but no Meta-Judge and no use of human rationales.

Figure 2. Improvement over the starting rubric, per criterion, for reasoning-aware tuning (RART) and label-only tuning (vanilla).

Figure 2 shows the result. On two of the three must-have criteria, vanilla and RART land in the same place, because the starting rubric was already near ceiling and the metrics had very little room to gain. On the third criterion, the vanilla (label-only) method degrades. The Reflector in the vanilla loop rewrites the rubric into degenerate forms, and specificity and reasoning agreement fall with every iteration, ending up worse than the rubric it started from. RART stays stable on that same criterion and lifts both, with a small recall gain as well. The reason is that a vanilla (label-only) optimizer can reach the right label through unsound reasoning, and unsound reasoning doesn’t generalize to new data. Anchoring the signal on the rationale prevents that. Reasoning alignment helps where there is headroom and doesn’t hurt where there isn’t.

Phase III, Deployment: Putting the Judge to Work

Once deployed, the judge sat inside the generation pipeline, gating and critiquing explanations as they were produced during our tests.

Two roles for one judge

Once we align the LLM Judge with humans, we reuse it in two distinct roles for every generated explanation:

  • Role 1, the gate. The Judge scores each explanation against the must-have criteria and rejects anything that fails.
  • Role 2, the critic. When an explanation fails, we don’t just throw it away. The Judge’s free-text reason gets appended to the Generator’s prompt, and the Generator tries again.

The second role turns the pipeline into a generate → judge → revise loop, with the Judge’s critique steering each revision.

Bounded retries, and why we drop rather than serve

We cap the retries. Each revision costs another round of generation and judging, and the gains flatten quickly. Figure 3 shows what the loop actually buys, on 1,000 generated explanations run through as many as 12 revisions with four generators. Every curve climbs, which is the Judge’s critique doing its job, but the steep part is early, and the gap between generators is obvious. Revision amplifies a capable generator rather than rescuing a weak one. In our test runs we used a budget of 3 retries with our strongest generator, and an explanation still failing after that was dropped rather than served.

Figure 3. Cumulative share of explanations that clear the Judge as we allow more revisions, across four generator models.

That “drop rather than serve” rule is deliberate. The two outcomes aren’t symmetric. A bad explanation is a trust hazard, while a missing explanation just forgoes an opportunity. When in doubt we’d rather show nothing, and most explanations cleared the bar within the retry budget.

Do judge-approved explanations help members?

Passing the Judge proves an explanation is good by our standards, but it doesn’t prove that judge-approved explanations are useful to members. To test that connection, we ran a one-time controlled experiment on the Netflix mobile app, explanations vs. no explanations on the title detail page, over roughly a month, with the pipeline generating and judging hundreds of thousands of explanations a week during the period.

Members who saw explanations shifted their viewing toward titles they hadn’t watched before, and more of their browse sessions ended in a meaningful play. Both effects were statistically significant, and we saw no member-initiated takedowns or escalations about explanation quality during the test. The result worth carrying over isn’t the size of that effect. It’s that offline agreement with our raters was necessary for these explanations to work but never sufficient to show they help members. Only the online A/B test could establish that judge-aligned explanations are not just quality-compliant but genuinely useful.

Phase IV, Monitoring: Keeping the Judge Aligned

A judge that’s well aligned at launch won’t stay that way. The catalog shifts, and so does what counts as high quality. The lifecycle has to close the loop.

The weekly human review

For each week a test was running, we pulled roughly 300 generated explanations for human review, stratified across the Judge’s decisions (served, revised, dropped) and weighted toward newer catalog items, where drift is most likely. The stratification stayed fixed week to week, so successive weeks were comparable. A panel of at least three raters reviewed each explanation, and the majority label was our ground truth. Those labels did two jobs. They told us whether judge-human agreement was slipping, and they grew the benchmark.

Setting the bar with human disagreement

The panel also tells us how much the raters disagreed with each other, which turns out to be the right yardstick for the Judge. Rather than hold it to a fixed threshold, we score the Judge and each rater against the same majority label and ask that the Judge fall within the raters’ own spread.

Take one criterion and one of its metrics, M. That week’s sample gives one value M(J) for the Judge and a set of per-rater values M(H) for the three or more raters on the panel. We require

so the Judge must score no worse than two standard deviations below the average rater. A harder week widens human disagreement and widens the band with it, so the Judge isn’t penalized for difficulty the humans also felt.

Falling below the band on any metric-criterion pair is a drift event, which triggers re-tuning (back to Phase II) on the augmented benchmark. A re-tuned rubric never goes live automatically. Someone on the team reads the proposed changes and signs off before it reaches the running pipeline, and we keep the previous rubric for instant rollback.

What human review catches that metrics can’t

Beyond asking whether the Judge still agrees with human raters, the weekly review answers a second question. Are the current guidelines still asking for the right things?

In the weekly review process, reviewers surfaced issues that did not show up in Judge–rater agreement. An explanation can use accurate attributes and still overstate how similar two titles really are. A single vague adjective (“exciting”, “fun”, “emotional”) can technically satisfy the rubric while saying almost nothing. Sometimes the reference itself is the problem, something the member wouldn’t think of as a comparable title at all, so the comparison lands oddly no matter how the attributes line up. Some genres show the same problem at scale. Stand-up specials can share similar metadata tags but differ completely in tone and cultural context, so a comparison may look justified on paper, not be bad enough for a rater to fail, and still land poorly with members. And some explanations meet every must-have criterion while still reading like generic “word soup” that does not help the user much.

When recurring patterns kept appearing, we updated the Phase I guidelines, which seed both the raters’ instructions and the judges’ rubrics, so one revision updates both sides. Other times we added product rules outside the Judge, for example by blocking certain title types from being used as references.

Drift detection catches a judge that has stopped agreeing with people. Only the continued human review tells us when our definition of a “good explanation” is too narrow or incomplete. We need both.

  • Drift detection keeps the Judge aligned with the current rubric.
  • Continuous human review keeps the rubric aligned with what actually works for members.

What We Learned

Five lessons we think transfer to any team shipping an LLM judge:

  1. Invest in the benchmark before the judge. A modest, rationale-annotated benchmark beats a large label-only one. Those written rationales are exactly what make reasoning-aligned tuning possible. You can’t correct a judge’s reasoning if you never wrote down the human’s.
  2. Tune on reasoning, not just labels. A judge can reach the right verdict for the wrong reason, which looks fine on a dashboard but gives misleading feedback and doesn’t hold up on new data. In our ablation, the label-only loop degraded a rubric that the reasoning-aware loop kept stable.
  3. One aligned judge can serve many roles. Reusing the same judge as both gate and critic amortizes the alignment cost and keeps production behavior consistent across roles. Alignment is the expensive part, so get one judge right and use it more than once.
  4. Plan for monitoring from day one. A recommender platform’s items and members never stop changing, so an LLM judge, like any ML model, has to be watched and re-tuned if needed, not shipped and forgotten.
  5. Keep people in the loop by design, not by exception. Automating evaluation is not the same as delegating responsibility for it. In our lifecycle, humans set the standard (Phase I), supply the rationales the judge is aligned to (Phase II), decide the conservative default of dropping rather than serving a doubtful output (Phase III), and review a standing sample of live decisions and approve every rubric change before it ships (Phase IV). These checkpoints are much harder to retrofit once a judge is already making decisions on its own. The standing review also tells us what our rubric is missing, not just when our judge has slipped.

Taken together, the judge extends the reach of our reviewers, and those reviewers are the basis for trusting the judge. That relationship is what makes it responsible to run this system at scale. An automated evaluator that no one audits provides no assurance about the decisions it is making.

Acknowledgements

This post summarizes our research paper, “The Lifecycle of LLM-as-a-Judge for Large-Scale Recommendation Explanations,” which has the full methodology, metrics, and ablations.

Our thanks to the internal human writers and review experts who defined what a good explanation is and keep those guidelines current; to our Human Data Operations team and the raters they train and calibrate, whose labels and written rationales are what the judges are aligned to; and to the engineers across the team who built and operate the pipeline; and to our UI engineering partners, who built the member-facing surfaces that bring these explanations to the homepage.


The Lifecycle of LLM-as-a-Judge: Building, Aligning, and Monitoring at scale was originally published in Netflix TechBlog on Medium, where people are continuing the conversation by highlighting and responding to this story.

GrabMart data for faster disease outbreak detection

Post Syndicated from Grab Tech original https://engineering.grab.com/grabmart-data-for-faster-disease-outbreak-detection

Abstract

Retail purchases can supplement clinical surveillance because changes in demand for some over-the-counter (OTC) products may appear before confirmed case reports. This article describes an early-stage concept for analysing aggregated GrabMart transaction data to identify unusual changes in product demand. The concept is not a deployed system, and it would require validation by public health experts, privacy review, and regulatory approval before any implementation.

Introduction

Public health agencies use sources such as clinical case reports, laboratory results, and hospital records to monitor disease activity. Retail data may offer an additional signal when demand changes across groups of symptom-related products. It cannot establish a diagnosis or replace clinical evidence.

When using indirect signals, we should assess whether they remain relevant to the outcome we want to measure. This includes considering how changes in user behaviour could affect their interpretation. A retail-based signal can experience similar drift, selection bias, and false alarms. Any useful design must therefore combine statistical validation with careful privacy and governance controls.

Background

Syndromic surveillance tracks patterns that may indicate emerging illness before diagnoses are confirmed. Retail surveillance applies this idea to changes in demand for products associated with symptoms, such as fever remedies or cough medicines.

The proposed method groups eligible products into broad symptom classes and measures changes in aggregate demand over time. An anomaly score can indicate that observed demand differs from an expected seasonal pattern. The score is a statistical signal, not an estimate of how many people have a disease.

The design uses the following terms:


Proposed solution

The proposed pipeline would process only transactions that meet documented eligibility rules. It would map eligible SKUs to symptom classes, aggregate counts across approved time periods and geographic areas, and compare each series with an appropriate baseline. Public health experts would review the resulting Early Outbreak Detection Index alongside independent evidence.

The method deliberately excludes identity linkage and individual-level health inference. It also avoids using a single purchase to classify a customer, target an advertisement, adjust an insurance decision, or change access to a product.

Conceptual data flow

Eligible transactions
        |
        v
Reviewed SKU-to-symptom mapping
        |
        v
Aggregate by approved time period and geographic area
        |
        v
Compare with seasonal and historical baselines
        |
        v
Privacy and quality checks
        |
        v
Aggregate signal for authorised public health evaluation

Analysis and validation

Several statistical methods could model expected demand, including seasonal time-series models and spatial scan statistics. Model selection should follow retrospective evaluation against approved public health data. Evaluation should measure sensitivity, false-alert rate, calibration, geographic stability, and performance during seasonal changes.

The system would also need ongoing monitoring. Product catalogues change, purchasing behaviour shifts, promotions affect demand, and access to GrabMart varies across populations and locations. These factors can produce anomalies unrelated to disease activity.

Privacy and governance

Privacy protection starts with data minimisation. The analysis should use only eligible product categories, coarse geographic areas, and aggregate time windows approved for the stated public health purpose. It should not expose raw transactions or personally identifiable information (PII).

The release process could apply minimum-count thresholds and differential privacy (DP), subject to a documented privacy analysis. Privacy specialists would need to set and review any privacy budget. Access controls, retention limits, audit logs, and contractual restrictions would govern any approved data exchange.

An independent review should cover privacy law, health-related inference, fairness, scientific validity, and the risk of harm from false alerts. An authorised public health body, rather than an automated model, should decide whether and how to communicate a possible outbreak.

Potential applications

If research validates the signal and the required approvals are in place, aggregate trends could support:

  • public health research alongside established surveillance sources;
  • operational planning for broad categories of health supplies; and
  • evaluation of whether retail demand contributes useful evidence during an emerging event.

These applications should use aggregate trends only. The signal should not drive personalised medical advice, individual targeting, insurance decisions, or automatic changes to medicine search rankings.

Limitations

Retail demand is an indirect and incomplete measure of illness. Purchases may reflect promotions, weather, media coverage, household stocking, or changes in product availability. GrabMart customers are not representative of the full population, and the available product mix differs by market.

Mapping a product to a symptom class is also uncertain because many products have several uses. Aggregation and privacy controls reduce disclosure risk but can weaken statistical utility. These limitations require local validation and documented thresholds before anyone interprets the index.

Conclusion

Aggregated retail transactions may add a timely signal to public health research, but they cannot replace clinical surveillance. A responsible design must minimise data, prohibit individual health inference, test the model against independent evidence, and place public health experts in control of interpretation and communication.

This concept remains in the early planning phase. No production deployment, measured performance result, or public health integration is claimed.

Join us

Grab is a leading superapp in Southeast Asia, operating across the deliveries, mobility, and digital financial services sectors. Serving over 900 cities in eight Southeast Asian countries: Cambodia, Indonesia, Malaysia, Myanmar, the Philippines, Singapore, Thailand, and Vietnam. Grab enables millions of people every day to order food or groceries, send packages, hail a ride or taxi, pay for online purchases or access services such as lending and insurance, all through a single app. We operate supermarkets in Malaysia under Jaya Grocer and Everrise, which enables us to bring the convenience of on-demand grocery delivery to more consumers in the country. As part of our financial services offerings, we also provide digital banking services through GXS Bank in Singapore and GXBank in Malaysia. Grab was founded in 2012 with the mission to drive Southeast Asia forward by creating economic empowerment for everyone. Grab strives to serve a triple bottom line. We aim to simultaneously deliver financial performance for our shareholders and have a positive social impact, which includes economic empowerment for millions of people in the region, while mitigating our environmental footprint.

Powered by technology and driven by heart, our mission is to drive Southeast Asia forward by creating economic empowerment for everyone. If this mission speaks to you, join our team today!

Intel’s Crescent Island AI Server GPU Gets a Name and Specs: Meet the Intel GPU D-Series

Post Syndicated from Ryan Smith original https://www.servethehome.com/intels-crescent-island-ai-server-gpu-gets-a-name-and-specs-meet-the-intel-gpu-d-series/

Intel today has unveiled its official performance specifications and branding for its upcoming “Crescent Island” data center AI inference cards. Going under the name Intel GPU D-Series, the D270 will offer 1.3 PLFOPs of dense FP8 and 1.5TB/second of memory bandwidth

The post Intel’s Crescent Island AI Server GPU Gets a Name and Specs: Meet the Intel GPU D-Series appeared first on ServeTheHome.

Introducing Clef-omni with full multimodality, plus a faster Clef and a cheaper Clef-flash

Post Syndicated from Michelle Chen original https://blog.cloudflare.com/clef-faster-cheaper-multimodal/

Following last week’s release of Clef and Clef-flash, Cloudflare’s open-weight decision models, we decided to bring forth more gifts. Today, we’re releasing Clef-omni, which takes in audio and video input alongside text and image. We also cut the price of Clef-flash so it is now cheaper than Jev, and we made Clef faster.

Although the model game is still early for Cloudflare, innovation and iteration is in our DNA, and we apply these principles to everything we do. In fact, the story of Clef came together over the course of less than a week. We decided we wanted to do something in the decision model space on a Friday evening, trained the model over the weekend, and launched it on Thursday. Even with such a short timeline, we were able to ship performant, high-quality, open-weight models for the community — imagine what more we can do in the future.

For today, we’re excited to keep up the momentum with new additions and improvements to our Clef family of models. This is just the beginning, and we’ll continue to get better, faster, cheaper, and more innovative.

Clef-omni takes audio, video, image, and text input

Our new Clef-omni model is able to take audio, video, image, and text input. This changes the paradigm for decision models, which have been largely text-only since the debut of Jev from TypeSafe. With Clef, we supported images and video frame arrays, but Clef-omni is able to take in audio (wav or mp3) and video (mp4 or webm) alongside text and images.

Instead of setting up cascading pipelines of models that transcribe speech-to-text, or splitting audio and image channels from video, you can just call one model to make decisions across any modality. We are now one step closer to a model that is able to interact with the world as we experience it — through audio, visual, and textual communication, all in one.

Check out our developer docs for the new Clef-omni model. We also released the model open-weights on HuggingFace. For a quick start, here’s how you can send new modality inputs to Clef-omni:

Clef-omni expands our open-weight decision architecture to natively handle multimodal workflows. We built this on a Qwen3-Omni-30B-A3B-Instruct mixture-of-experts (MoE) foundation, where the model was already capable of processing text, imagery, audio, and video directly within a single pipeline. However, we adopt the primary comprehension backbone while discarding the text-to-speech output components. In production, Clef-omni executes a quick prefill pass across the complete payload, scoring all modalities and valid parameter options simultaneously.

Because we skip output token generation since Clef models are not Large Language Models (LLMs), we remove the overhead of transcribing or captioning incoming files. Media elements map straight into the unified sequence, where video and audio are synced with visual frames for joint processing. Clef-omni then pulls candidate values directly from internal embeddings using our two-stage attention routing: every valid option gathers key evidence from the input (regardless if they are buried in text snippets, visual elements, or audio streams) before field vectors cross-attend across the full context to compute confidence scores. We add a built-in lexical grammar that preserves option semantics, so that we can deliver fast, schema-constrained scoring across every input type.

We train the model with the same techniques as we did with Clef — freezing the Qwen3 backbone, training low-rank adapters (LoRA), and applying our standard post-training approach: combining label-smoothed cross-entropy loss with Brier score calibration. The result is a model that is resilient and designed to succeed regardless of schema variations, field ordering, and prompt structures.

This brings several core upgrades to the Clef model family. Calibrated, schema-bound decisions now seamlessly handle text, images, audio, and synchronized video in a single API call. It's also fast. Text-only decisions return in about 130 ms at the median, image inputs in about 150 ms, and audio clips in a few hundred milliseconds. Even a full 21-second video clip with sound is scored in about 1.5 seconds, all in a single API call.

Our benchmarking shows strong performance across benchmarks, even with new modalities and MoE architecture.

Benchmark

Clef-omni

Clef

Clef-flash

Jev

BFCL · case exact

98.2

98.47

98.76

95.75

ToolRet · nDCG@10

66.6

69.19

66.43

65.28

API-Bank · accuracy

92.7

91.93

93.11

88.19

Home appliances · case exact

69.3

82.95

97.73

52.27

When2Call · accuracy

63.3

72.37

65.58

80.97

BANKING77 · macro-F1

94.8

94.20

90.93

79.74

CLINC150+OOS · macro-F1

97.7

97.43

66.77

89.27

BRIGHT · nDCG@10

42.0

45.91

39.26

47.52

Amazon ESCI · macro-F1

57.8

57.48

57.39

55.21

PhishNChips · accuracy

73.2

79.60

75.05

62.55

We also benchmarked against the TypeSafe evals to show how Clef-omni performs.

Workflow

Metric

Clef-Omni

Clef

Clef-flash

Jev

Invoice processing

Exact actions

60.2

64.7

57.1

61.8

Invoice processing

Primary action

82.0

86.2

73.3

83.1

Customer service

Exact actions

71.6

76.3

77.0

76.0

Security incidents

Exact actions

61.7

62.9

61.7

61.7

Agent trace observability

Primary action

65.8

68.5

69.8

71.6

Clef-flash model is now cheaper

We heard your feedback — you want a decision model affordable enough to incorporate into any workflow. We’ve made a few optimizations and can now offer Clef-flash at cheaper prices. We want Clef to be able to make decisions that scale for you across all your agentic workflows, and now our pricing also incentivizes that.

We were able to optimize the model to the point where Clef-flash is now cheaper than Jev while still being performant and high-quality. Check out the developer docs for the most up-to-date pricing information at all times, including more detail on how image and audio modalities are converted to input tokens for pricing.

  • Clef-flash – originally $0.09 per M input tokens, now $0.038 per M input tokens
  • Clef – remains at $0.24 per M input tokens
  • Clef-omni – launched today at $0.15 per M input tokens

However, one trade-off we have to make in order to have cheaper pricing is to cut the context window of Clef-flash. The hosted version of Clef-flash now has a context window of 24k, instead of 64k as previously advertised. The model weights on Hugging Face are untouched and were trained to support a 256k context window should you choose to self-host it.

From our usage data, we see that only 0.24% of requests exceed 24k input tokens. Based on this data, we decided to make the call to drop the context window to 24k for Clef-flash in order to price it at a more accessible entry point. Our Clef model remains at a 64k context window, and we encourage folks with larger context needs to switch to Clef instead of Clef-flash.

Clef model is now faster

Speeds are also faster for the Clef model, with new optimizations to the hosted version of Clef on Workers AI. No new model weights are released because most of the optimizations we made are at the serving infrastructure layer, not materially to the model weights or architecture. Check out our new speeds below:

Input size

Before: median / p95 (ms)

Now: median / p95 (ms)

Median speedup

~800 tokens

262 / 438

152 / 351

1.7×

~3,400 tokens

616 / 777

305 / 531

2.0×

~16,000 tokens

2,721 / 3,250

1,635 / 1,805

1.7×

One of the optimizations we made was to move to SGLang for serving the model. We worked with the SGLang team and made pull requests to incorporate Clef (PR #42721), which will be released in SGLang 0.5.22. You can also self-host Clef since we released the weights, and the new SGLang launch commands are available on our updated Hugging Face repo as well as the SGLang cookbooks.

How are people using Clef?

At Cloudflare, we’ve had teams experimenting with Clef as a model that can help classify over general domains. The coolest thing about watching teams adopt Clef is that the capability to do detections and classifications have moved into the model layer, where it is accessible for anyone to start incorporating decision models into their use cases. Previously, even with zero-shot classifiers, the small models may not have been powerful enough to classify across any domain without further tuning. You might have needed to stand up a specialized machine learning team to help train over your particular domain, and gather a corpus of data in order to train a better model. Now, all you need to do is call a model to instantly parse through input data with precision.

A few highlights of how teams at Cloudflare are using Clef: Our public GitHub docs repo uses Clef to instantly detect and close spam issues. EmDash, our content management system, moderates plugin libraries for phishing. Our data loss prevention team is scanning data for potential personally identifiable information (PII) like government IDs. And our threat intelligence team is using Clef to help detect malicious domains.

Give it a try!

We’re excited to bring you more from the Clef family of models, including new modalities like audio and video, as well as making the model more accessible with cheaper pricing and faster speeds. Clef is fully Jev-API compatible and available via AI Gateway, so it is as simple as changing the model ID to try out Clef today. Check out our developer docs for more information.

Thank you for all the love and warm reception around Cloudflare’s first models — we are focused on bringing you more. Try out our Clef family of models, and tell us what you think.

How Technology Empowers—and Imperils—Dictators

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/10/how-technology-empowers-and-imperils-dictators.html

This essay was written with Seva Gunitsky, and originally appeared in Foreign Affairs.

Two weeks after Moscow’s full-scale invasion of Ukraine in March 2022, the Russian TV Channel One editor Marina Ovsyannikova burst onto the set of the evening newscast. She held up a hand-drawn sign behind the anchor’s head that read: “Stop the war. Don’t believe propaganda. They’re lying to you!” She shouted, “No to war!” until she was dragged away.

No one has protested the war on Russian television since then, partly as a result of tighter security and a general climate of fear. But in October 2024, Margarita Simonyan, one of Russia’s chief propagandists, gave another explanation. A growing number of the RT network’s anchors, she explained to an interviewer, are not real. “That face doesn’t exist. We generated the voice and everything else.” They were, she meant, produced by artificial intelligence. In a follow-up interview with the newspaper Vedomosti, she spelled out the logic: “These anchors don’t need a salary or insurance,” she said. “They won’t get arrested, and the police won’t search their homes. It’s all wonderful and terrifying at the same time.”

This is AI’s promise to every autocratic ruler: the ability to maintain control without delegating tasks to unreliable humans. It is an extremely attractive prospect. No matter how powerful, dictators have always needed subordinates—censors, propagandists, security guards, police officers, intelligence analysts, and local bureaucrats—to carry out their orders. But subordinates always pose a potential threat. They could shirk, steal, lie, leak, or conspire against their boss. As a result, one of the oldest dilemmas facing autocrats is how to empower agents to carry out orders without also enabling them to turn against their leader. Autocrats have usually managed the tradeoff by filling key positions with mediocrities whose incapacity is, as the political theorist Hannah Arendt put it, “the best guarantee of their loyalty.” But this inevitably makes it harder for dictatorships to govern and survive crises.

AI appears to offer autocrats two ways to resolve this long-standing dilemma. First, it means they can now easily monitor and discipline their followers in real time. A change in a local official’s spending habits or meeting schedules can be flagged automatically by an AI system, without any need for self-interested informants. Second, AI can eliminate underlings altogether. By automating tasks that once required human discretion, rulers can replace unreliable intermediaries with faithful algorithms that cannot be bribed or manipulated. Officials in autocratic regimes have said as much. AI “can definitely replace half of officials,” Russia’s Digital Development Minister, Maksut Shadaev, told a Moscow data forum in April 2025, adding, “maybe slightly more.”

Yet these promises of a self-running state are illusory, because AI is never truly autonomous. Systems have to be built and maintained by a cadre of engineers and data scientists whose expertise political leaders cannot evaluate. The ruler who turns to AI to reduce a dependence on unreliable humans may end up relying, more blindly than before, on the few AI specialists who keep the system running. And this new digital Praetorian Guard may have the same weaknesses as the old, which makes it a threat to the ruler.

Eyes and Ears

China is in the lead when it comes to developing AI systems that can monitor or replace subordinates. Smart city programs in Beijing, Shanghai, and other urban centers use AI to track performance and centralize oversight of local officials who once operated in comfortable obscurity, subjecting them to an algorithmic performance review. China also developed a “Zero Trust AI system,” which it deployed across roughly 30 counties and cities over the past decade. This system cross-referenced more than 150 government databases to catch embezzlement and nepotism by local officials. It worked a little too well, flagging 8,700 officials before local governments, under pressure from the bureaucrats it was monitoring, began rolling it back. But Beijing has not gotten rid of its new e-government portals, which use automated systems for issuing documents and permits, reducing face-to-face interactions that bred petty corruption or favoritism. The Chinese Communist Party, meanwhile, has plowed ahead with researching what it calls “thought management,” or how to use AI to create microtargeted propaganda. China’s army of Internet commentators, once composed of paid humans, is already being replaced by bots.

Russia has been pursuing the same goals with a similar fervor. Moscow’s citywide facial recognition system, deployed across 200,000 cameras, was used to identify and apprehend protesters during the 2021 antigovernment demonstrations. The central government has started using automated data collection and aggregation to bypass regional officials who could taint the data or use it to promote their own interests. In 2023, Russia’s Internet regulator launched Oculus, an AI system that scans hundreds of thousands of images per day for prohibited content, including political content—orders of magnitude beyond what human censors could process. The Russian security service’s Meliorator tool has been used to create over a thousand profiles of fictitious Americans, primarily on X (formerly Twitter), to spread Kremlin narratives at a fraction of the cost of human troll farms.

Other autocracies are following in Beijing’s and Moscow’s footsteps. In April 2025, the United Arab Emirates created a Regulatory Intelligence Office that uses AI to draft and amend federal laws, work once done by legislative staff. A year later, UAE Prime Minister Sheikh Mohammed bin Rashid al-Maktoum announced that autonomous AI agents would take over half of the federal government’s operations within two years and that “the performance of ministers, directors general, and entities will be assessed based on their ability to adopt this transformation.”

These innovations may reduce the number of bureaucrats autocrats need. But they cannot actually create what dictators want most: a self-running state. Even AI-generated television anchors need people to build and maintain them, and censorship systems need people to retrain them as the vocabulary of dissent shifts. Behind every AI model, there is a small group of engineers and data scientists doing essential maintenance, and their work is so technical that rulers cannot understand it.

This dependence thus becomes another form of power. The engineers who maintain an autocrat’s surveillance apparatus, for example, can shape what the ruler sees. They decide what information is important enough to pass along and in what form to present it. They determine which threats get flagged, and they can adjust the algorithms that select what content gets suppressed and who gets arrested. And they can do this without anyone in the palace noticing. Autocrats who turn to AI to escape a dependence on unreliable subordinates have only transferred their vulnerabilities onto a new group of officials.

In fact, this new Praetorian Guard could be more dangerous than previous elites. That is in part thanks to the opacity of AI technology but also because this clique is much smaller. Roman emperors typically had tens of thousands of people serving in the Praetorian Guard; modern autocrats rely on standing armies. But with AI, autocrats will need only a small clique of engineers to build and maintain large-scale AI systems. This might benefit rulers since they will have fewer people to watch, yet it also concentrates points of failure, since a devastating disruption requires only a handful of engineers and makes it easier for members to scheme against the leader. The new guard’s members may also be indispensable. A dictator can replace generals without destroying the army. But running complex machine-learning systems requires such a specialized set of skills that engineers can be difficult to replace without disruptions, giving these actors great leverage.

The Indispensables

The importance of tech workers to autocracies has already become apparent. When IT specialists began to flee Russia after the full-scale invasion of Ukraine in February 2022, for example, the Kremlin responded not with threats but with inducements, offering tech workers deferments from conscription. It exempted their firms from taxes and subsidized their mortgages. When Russian President Vladimir Putin ordered the mobilization of 300,000 men seven months later, IT workers were granted full-on waivers. This might seem like overkill, given that an AI system needs only a few engineers to run it. But a regime cannot know in advance which engineers it will need, and it cannot train replacements quickly, so it has to hold on to the entire talent pool from which these few are selected.

These measures did not stem the outflow of roughly 100,000 specialists—about a tenth of Russia’s tech workforce—over the course of 2022. But Moscow stuck to bribery, because expertise is extremely difficult to conscript at gunpoint and because conscripted experts might be less likely to do as instructed. Even so, money and privilege cannot guarantee that these elites will stay loyal. Autocrats should recall the lesson of the original Praetorian Guard: for a time, it provided Roman emperors with protection and security. But then the praetorians discovered their own indispensability, and by the second century, they were auctioning off the empire to the highest bidder. This new set of elites can do the same. In June 2023, when the column of Yevgeny Prigozhin‘s Wagner paramilitary company moved up the highway toward Moscow, Putin depended on his security services to tell him what was happening. Future rulers in Putin’s position will receive such warnings through machines: intercepted communications sorted by software, camera feeds filtered through recognition systems, regional reports compiled into dashboards. The engineers who run those systems could strike a deal with an upstart challenger and then drag their feet. They could delay the data, let alerts arrive a few hours late, degrade feeds at inconvenient moments, or make a recognition system stop functioning. In that scenario, the ruler would be operating blind. The Praetorian Guard did not need to kill Emperor Nero to replace him. It simply had to abandon him for a rival.

These programmers are unlikely to seize power for themselves, because they are unlikely to carry what coup leaders ultimately require: guns. But there is already precedent for engineers using their power to shape leadership challenges. In 1991, when the Soviet army attempted to seize control from Mikhail Gorbachev, a handful of programmers at the Relcom network kept information flowing abroad and relayed Russian President Boris Yeltsin’s decrees to audiences inside the country and abroad. The plotters could not stop the news of Yeltsin’s defiance from spreading, and the coup collapsed within three days.

Slimming Down

AI will not let autocrats dismiss all their enforcers. Someone still has to make arrests and run the prisons, and autocrats can buy loyalty by offering supporters state jobs. But it will let authoritarians downsize, and the number of officials ultimately matters less than how the ruler oversees them. Keeping track of scheming or incompetent subordinates has always been the autocrat’s chief burden. AI could lighten it, letting rulers watch their officials more closely without paying as much active attention.

In the near term, then, AI may greatly benefit autocracies. Despite the many obstacles to deployment, the technology is already bringing autocratic regimes the upsides of cheaper surveillance, smarter censorship, and fewer human subordinates to fear and distrust. But for autocratic rulers, the temptations of artificial intelligence may re-create the same trap they are seeking to escape. The more a regime depends on AI, the more it depends on the people who keep AI running. And those people, like every other praetorian class in history, will quickly discover what their indispensability is worth.

Gigabyte W775-V10-L01 Hands-on Bringing NVIDIA GB300 Deskside

Post Syndicated from Patrick Kennedy original https://www.servethehome.com/gigabyte-w775-v10-l01-hands-on-bringing-nvidia-gb300-deskside/

We test the Gigabyte W775-V10-L01, including getting over 2.2B tokens/ day on the Blackwell Ultra GPU, 800Gbps from the ConnectX-8 SuperNIC, and testing the NVIDIA Grace CPU with the C2C link between it and the GPU

The post Gigabyte W775-V10-L01 Hands-on Bringing NVIDIA GB300 Deskside appeared first on ServeTheHome.

Smarter personalization: How property data helps us understand user price sensitivity

Post Syndicated from Grab Tech original https://engineering.grab.com/price-sensitivity-profiling-using-llm

Introduction

Existing systems estimate user price sensitivity primarily from spending behavior or demographic proxies. They do not systematically account for residential property values, which can indicate a user’s financial circumstances. This omission creates three limitations:

  1. Limited property insights: Existing profiles do not account for property values.
  2. One-dimensional profiles: Users with similar spending patterns but different living standards receive the same classification.
  3. Regional variation: Manual classification does not adapt well to differences between property markets.

This invention adds property data to user profiles to produce affluence segments that account for regional property markets. The invention remains unimplemented and is not deployed in any live market.

Background

Property values vary across buildings, neighborhoods, and regions. A useful classification method must account for these differences while processing property records in several formats. The same classification method could support other industries that use affluence segments.

Solution

A four-step method that combines public property data with clustering techniques to classify user affluence was created:

  1. Data collection and enrichment.
  2. Building-level property price estimation.
  3. Property tier classification.
  4. User mapping to affluence tiers.

The method combines external property data, large language model (LLM) enrichment, and cluster-based segmentation. The resulting profiles can inform personalized offers, targeted marketing, and operational planning.

Data gathering and processing

The system collects public property data from external sources. These records include transaction prices, building types, and related attributes in raw text formats. LLMs standardize the records and extract key attributes such as postcode, building address, transaction price, price per square meter, property size, and floor level. This stage produces a structured property-level dataset for analysis and modeling.

Building-level price estimation

Under the proposed method, transaction records for properties within the same building would be aggregated to calculate a weighted average sale price. The weighting would account for transaction recency and data volume, with the aim of reflecting recent market conditions.

When transaction data is limited, the method could apply rules based on comparable properties in the area. It could also evaluate LLM-assisted extraction of contextual signals from appropriately licensed external sources, such as real estate listings. Any LLM-derived signals would require validation against independent reference data before contributing to a property-value estimate.

The invention has not been implemented, deployed, or tested in a live market. Its model and version, validation methodology, error bounds, and operational safeguards remain subjects for future technical evaluation.

Property tier classification

A hierarchical clustering model groups buildings into affluence tiers based on their estimated resale values. The model accounts for regional differences and local market-value distributions across property types and locations. This stage assigns each building to an affluence tier.

User mapping to property tiers

The system maps would use appropriately aggregated geographic signals to associate users with broad property-market segments and assign a corresponding affluence classification.

Potential future applications

This invention presents a conceptual approach for exploring new ways to support personalization, advertising, and operational planning. It has not been implemented, deployed, or tested in a live market. Subject to technical validation and approval from Privacy, Legal, PR, and the relevant product owners, potential applications could include:

  • Personalization: Tailoring promotions, discounts, subscription plans, and recommendations for optional products and services.
  • Advertising: Supporting more relevant connections between advertising content, merchants, and broad audience segments, alongside tailored marketing campaigns.
  • Operational planning: Informing aggregated analysis used to plan and prioritize delivery and transport capacity.

Learnings and conclusion

This invention presents a conceptual approach that combines publicly available property data, LLM-assisted data structuring, building-level value estimation, hierarchical clustering, and contextual tier assignment. It demonstrates how regional property-market context could complement existing signals, while highlighting dependencies on data quality, model accuracy, geographic coverage, privacy, and fairness.

The approach shows potential applications in personalization, advertising, and operational planning, but these are illustrative, not demonstrated outcomes.

Join us

Grab is a leading superapp in Southeast Asia, operating across the deliveries, mobility, and digital financial services sectors. Serving over 900 cities in eight Southeast Asian countries: Cambodia, Indonesia, Malaysia, Myanmar, the Philippines, Singapore, Thailand, and Vietnam. Grab enables millions of people every day to order food or groceries, send packages, hail a ride or taxi, pay for online purchases or access services such as lending and insurance, all through a single app. We operate supermarkets in Malaysia under Jaya Grocer and Everrise, which enables us to bring the convenience of on-demand grocery delivery to more consumers in the country. As part of our financial services offerings, we also provide digital banking services through GXS Bank in Singapore and GXBank in Malaysia. Grab was founded in 2012 with the mission to drive Southeast Asia forward by creating economic empowerment for everyone. Grab strives to serve a triple bottom line. We aim to simultaneously deliver financial performance for our shareholders and have a positive social impact, which includes economic empowerment for millions of people in the region, while mitigating our environmental footprint.

Powered by technology and driven by heart, our mission is to drive Southeast Asia forward by creating economic empowerment for everyone. If this mission speaks to you, join our team today!

Everything we launched during Birthday Week 2026

Post Syndicated from Carlos Armada original https://blog.cloudflare.com/birthday-week-2026-wrap-up/

We celebrated our 16th birthday last week by sharing how we’re building a better Internet for today’s world. As Matthew and Michelle reflected in this year’s Founders’ Letter, this year saw some of the most consequential changes in the history of the Internet.

For the first time, automated traffic surpassed human activity. AI is empowering people to build like never before, leading the Internet to grow massively in scale and unlocking more ambition and creativity. As we witnessed the influence that agent-driven recommendations have on consumer choices, we identified the need for a new approach that creates space for new businesses to succeed.

Each day of Birthday Week explored a different way we are helping to build the future of the Internet. We began on Monday by strengthening our commitment to open source. Tuesday focused on application security and the post-quantum transition. On Wednesday, we explored new economic models for the agentic Internet. Thursday, we expanded the Developer Platform with new tools for data analysis, storage, AI, and agent development. Finally, we closed out the week by launching features that make Cloudflare faster, easier to operate, and more accessible to everyone. As a special Birthday Week follow-up, we shared an update on our intern program, one year after announcing our goal to hire 1,111 interns. Interns directly contributed to many of the projects launched this week, including EmDash, post-quantum visibility, CryptoLabe, and Protected Quick Tunnels.

We shipped 46 announcements this week. In case you missed any, here’s the full list of everything we announced during Birthday Week 2026.

Monday, September 28 – Commitment to open source

With the announcement of our new CLI, which we released alongside the pipeline we use to generate it and our SDKs and docs, we shared how we’re building to support agents and developers as they use Cloudflare — and supporting the projects that you rely on, too.

What

In a sentence…

Introducing cf: the agentic CLI for the entire Cloudflare API

The new cf CLI mirrors the Cloudflare API, uses JSON-first output and typed configuration, and gives people and agents one consistent command-line interface.

Introducing Forge: the open source pipeline for generating SDKs, CLIs, docs, and more

Forge is a pluggable, open-source pipeline that runs in CI to generate SDKs, CLIs, documentation, and other interfaces directly from API definitions.

Introducing EmDash – the spiritual successor to WordPress that solves plugin security

EmDash is an open-source, Astro-based serverless CMS that runs plugins in isolated Worker sandboxes with explicitly approved capabilities.

Four months of VoidZero at Cloudflare: making the open-source JavaScript toolchain faster for all humans and agents

Since joining Cloudflare, VoidZero has delivered more than 80 releases across the Vite ecosystem, and its previously commercial Void platform will become fully open source.

Next.js applications, powered by Vite: introducing Vinext 1.0

Vinext 1.0 turns an AI-built experiment into a production-ready, portable way to run Next.js applications on Vite.

The road to the agentic browser: A Kitesurf update

Kitesurf, our Workers-based browser for agents, adds WebMCP support, faster DOM operations, broader web compatibility, and terminal-based rendering.

How fast is the web? Explore billions of real-user measurements with BEACON

BEACON makes billions of anonymized real-user performance measurements from 10,000 major websites available as a public BigQuery dataset.

Supporting native Rust in Workers with the new Emscripten target for wasm-bindgen

Experimental Emscripten target support lets developers bring more native Rust libraries and applications, including progress toward Tokio support, to Workers.

Introducing The Cold Start: pitch your startup live at Cloudflare Connect

The Cold Start gives five early-stage companies the opportunity to pitch live at Cloudflare Connect and compete for resources to help them grow.

Tuesday, September 29 – Helping secure the agentic Internet

Technological progress is rapidly changing how we think about application security. We announced our intention to become a certificate authority, as well as how we’re preparing foundational Internet cryptography for the post-quantum era and adapting application security to counter AI-driven attacks.

What

In a sentence…

Building a certificate authority for the whole Internet

Twelve years after launching Universal SSL, Cloudflare announced its intention to become a public certificate authority (CA) and add resilience to free, automated certificate issuance.

Building a post-quantum certificate authority with Merkle Tree Certificates

Our planned CA will issue free Merkle Tree Certificates designed to make post-quantum authentication practical without imposing large certificate and handshake costs.

Using AI to chart a course for our post-quantum migration

CryptoLabe uses AI to find and classify cryptography across our codebase as Cloudflare works toward completing its post-quantum migration by 2029.

Preventing quantum downgrade attacks against IPsec

Cloudflare helped develop an IETF extension that authenticates the full IKEv2 transcript and prevents attackers from downgrading post-quantum IPsec tunnels.

Is your domain using post-quantum encryption? Now you can see for yourself

HTTP Analytics, Log Explorer, and Logpush now show whether requests negotiated post-quantum key exchange, giving customers evidence they can inspect and report.

Enforce positive security with Cloudflare Application Profiles

Application Profiles learns the expected structure of HTTP requests so customers can identify deviations and enforce what valid application traffic should look like.

We tested our own WAF with frontier AI models. Here's what we found

An adaptive AI red-team system found WAF detection gaps across six attack categories, helping us improve normalization and managed rules for customers.

Introducing Threat Signals: agentic skills for open-source threat intelligence, free for every Cloudflare account

Threat Signals turns open-source reporting into structured indicators and connects the context to WAF rules, while the Threat Events Platform expands to every account.

Adaptive application security for the AI era: how Cloudflare connects code, traffic, and intelligence to stop attacks

Our application-security framework connects discovery, governance, runtime protection, investigation, and response in a continuous learning loop.

Wednesday, September 30 – Powering the agent economy

With our announcements of Pay Per Use and the release of our Monetization Gateway in beta, we shared how we’re building support for a new economic model that empowers creators to monetize their content and services.

What

In a sentence…

The Internet has a second audience

AI agent requests have grown rapidly, and our strategy helps creators see agents, set terms for access, and get paid when agents use their work.

Cloudflare Containers, rebuilt to scale agent sandboxes

Containers now has faster startup, flexible image and instance selection, new scheduling controls, and filesystem snapshots for persistent agent workspaces.

Monetization Gateway beta: charge AI agents for consumption with HTTP 402

Monetization Gateway lets sellers put a price on resources behind Cloudflare and collect agent payments using HTTP 402 and x402.

Pay Per Use: when AI uses your work, you should get paid

Pay Per Use gives enrolled publishers usage reports, billing, and payouts when verified AI buyers use their content.

Simplifying domains for people and agents

A new domain-search experience and expanded Registrar APIs make it easier for both people and agents to search, register, transfer, and manage domains.

Identify AI model overuse with User Insights

AI Gateway User Insights identifies tasks, model fit, and overuse, so teams can understand where a smaller or less expensive model may work.

Detect and send production issues straight to your agent

Issues groups Workers errors and sends the relevant stack traces, logs, and traces to coding agents or any webhook for faster investigation.

Cut your AI spend with AI Gateway's Auto Router

Auto Router classifies each request at the edge and sends it to a suitable model, reducing cost while preserving response quality.

Cloudflare Impact reaches $100 million in donations

Initiatives including Project Galileo, the Athenian Project, and Cloudflare for Campaigns have now delivered more than $100 million in donated services.

Thursday, October 1 – Bringing more of the developer stack to Cloudflare

We expanded what is possible to achieve on Cloudflare’s platform with the general availability launch of Cloudflare Basin, our data analytics platform, the launch of K2, a durable serverless event stream, and the announcement of our new contest — inviting developers to build a Git platform designed for agentic development.

What

In a sentence…

Introducing Cloudflare Basin: an open, serverless data platform, now generally available

Basin is now generally available, giving developers a serverless platform built on Apache Iceberg and R2 for ingesting, managing, and querying large datasets.

Support for modern cryptographic algorithms in Workers

Workers adds opt-in native Web Crypto support for ML-KEM and ML-DSA, giving developers post-quantum primitives without bundling their own implementations.

AI Search is now generally available

AI Search reaches general availability with visual search, OCR for scanned PDFs, larger files, and support for any chat model.

We want you to build the next Git platform on Cloudflare

Artifacts enters open beta and a new competition invites developers to build a Git platform designed for the era of AI agents.

Announcing Cloudflare K2: serverless event streams

K2 provides durable, ordered event streams on R2, separating producers and consumers without the operational overhead of managing broker clusters.

Cloudflare OS: your company's agent workspace, managed for you

Cloudflare OS provides an agent workspace connected to an organization’s data and systems, with a waitlist open for fully managed deployments.

Introducing Workers KV Instant – powered by Quicksilver

Workers KV Instant delivers sub-two-millisecond p99 reads and fast global replication across more than 300 locations using the familiar Workers KV API.

One year later: Sovereign AI and the fight for choice

We are expanding local open-source model choice and model-agnostic security tools, so nations can pursue AI sovereignty without isolation.

Introducing Clef: our open-source decision models, and new RL fine-tuning platform

Clef and Clef-flash are open-source decision models for fast classification and agent workflows, accompanied by a platform for reinforcement-learning fine-tuning.

Friday, October 2 – Delivering a faster, simpler Internet for everyone

We wrapped up the week with major updates to Cloudflare Observability, alongside adding Cloudflare Traces, network performance improvements that make Cloudflare faster, and an announcement on how we’re supporting civil society organizations.

What

In a sentence…

8 major updates to Cloudflare Observability

Eight updates bring logs, traces, analytics, alerts, dashboards, querying, and telemetry export into one observability platform with simpler pricing.

Introducing Cloudflare Traces: follow requests through our entire platform

Cloudflare Traces provides request-level visibility across security rules, transformations, cache, Workers, services, and origins without requiring an agent or SDK.

Updates on our pledge to make Cloudflare features accessible to everyone

One year after our pledge, Logpush, multi-account governance, higher platform limits, and other capabilities are available to more customers across plans.

Announcing Cloudflare OHTTP Gateway – expanding access to Cloudflare's privacy-preserving infrastructure

A self-serve OHTTP Gateway enters closed beta, while Privacy Gateway becomes Cloudflare OHTTP Relay to distinguish the two roles.

Follow the thread: a new dashboard to investigate account abuse

Account Abuse Protection uses stateful analysis and privacy-preserving Hashed User IDs to help teams investigate credential stuffing and fake-account creation.

Protected Quick Tunnels: simple accountless authentication for your next dev project

Quick Tunnels now support email authentication, letting developers share a local application with selected people or domains without requiring Cloudflare accounts.

Building for good: How civil society organizations are automating on Cloudflare

Civil society organizations are using Cloudflare’s developer platform to automate and scale work that protects human rights and the public interest.

2026 Birthday week: network performance update

Using an expanded real-user measurement methodology, Cloudflare now ranks as the fastest provider across 74% of the top 1,000 networks.

Introducing Web Search API via AI Gateway

AI Gateway’s Web Search API brings current web context from multiple providers into model calls through REST APIs, Workers bindings, or customer-managed keys.

Streamline: custom video pipelines with Cloudflare Stream and Workers

Streamline is an open-source example for building continuous video pipelines by combining Workers, Durable Objects, and a containerized media engine.

Building the Internet’s next chapter together

Across this week’s announcements, we kept returning to a consistent theme: the Internet should continue to open up more opportunities for people to create, contribute, and succeed. That means open tools developers can shape, security that keeps pace with new threats, a fairer exchange between agents and the people whose work they use, and infrastructure designed for the agentic Internet.

For 16 years, we have been building alongside developers, creators, researchers, customers, partners, and open-source communities. Your ideas, feedback, and willingness to challenge us have shaped Cloudflare, and that collaboration matters now more than ever.