The shadow-utils
project provides the tools that handle /etc/shadow, /etc/passwd, and other related databases; in
general, manages users and groups on many Linux systems. While most
software releases are notable for what is added, the recent shadow-utils 4.20.0
release is most noteworthy for what has been removed. Specifically,
several utilities and functionality related to periodic password
expiry, which were deprecated in the December 2025 4.19.0
release, have been removed as planned. It is still possible to manage
some aspects of password aging with shadow-utils, but organizations
that depend on such features should start planning for their complete
removal within a few years.
On July 27, 2026, JetBrains published a security advisory for CVE-2026-63077, a critical unsafe deserialization vulnerability affecting JetBrains TeamCity. An attacker who can reach a TeamCity server over HTTP or HTTPS can exploit the agent polling protocol without credentials and execute operating system commands with the privileges of the TeamCity server process.
JetBrains reported no known active exploitation when it disclosed the vulnerability. However, on August 5, 2026, CISA added CVE-2026-63077 to its Known Exploited Vulnerabilities (KEV) catalog, confirming exploitation in the wild.
Our analysis finds that a vulnerable TeamCity server creates a permissive XStream allowlist. This allowlist is intended to restrict which Java classes can be deserialized when servicing unauthenticated agent requests. However, this allowlist incorrectly adds TeamCity protocol classes without removing XStream’s existing default permissions. This introduces an unsafe deserialization issue. A patched TeamCity server remediates this by adding NoTypePermission.NONE before the TeamCity allowlist, which removes the default permissions and makes the allowlist exclusive.
Rapid7 Labs has verified that the patch successfully remediates the exploit described in this analysis. A proof-of-concept script for CVE-2026-63077 can be found here.
Analysis
Our analysis compares a vulnerable TeamCity version 2026.1.2 against a patched version 2026.1.3.
TeamCity uses a central server to coordinate builds and separate build agents to run them. An agent can communicate with the server through the agent polling protocol: it registers, asks the server for its next command, and reports whether that command succeeded or failed. The endpoints under /app/agents/v1 support this agent communication channel rather than the TeamCity web interface or REST API. A TeamCity-AgentSessionId HTTP header value identifies a polling connection, but it does not mean that either a user or agent has authenticated to TeamCity, as access to many agent endpoints remains unauthenticated.
XStream is a Java library that converts object graphs to XML and reconstructs those graphs from XML. An object graph can contain nested objects, collection entries, private fields, and references to an object that appeared earlier in the document. XStream aliases give Java types shorter XML names. For example, <linked-hash-map> is XStream’s alias for java.util.LinkedHashMap. Nested element names and class attributes select other concrete Java types, while reference attributes point back to objects that XStream has already constructed. Converters and reflection-based code then allocate the selected types and populate their fields.
Patch diff
The class jetbrains.buildServer.messages.XStreamHolder is TeamCity’s wrapper for creating and configuring XStream instances. TeamCity 2026.1.2 creates an instance of XStreamHolder, configures it, and then calls setupSecurityIfNeeded(). If the TeamCity allowlists contain entries, this method adds those entries to the permissions that XStream already installed:
The calls at [1] and [2] do not start from an empty permission set. The bundled XStream 1.4.20.3 constructor has already called setupSecurity(), which permits several broad type hierarchies, including Map and Throwable:
// ./webapps/ROOT/WEB-INF/lib/xstream.jar
package com.thoughtworks.xstream;
public class XStream {
// ...
protected void setupSecurity() {
if (this.securityMapper == null)
return;
addPermission(NoTypePermission.NONE); // <--- Clears all existing permissions
addPermission(NullPermission.NULL);
addPermission(PrimitiveTypePermission.PRIMITIVES);
addPermission(ArrayTypePermission.ARRAYS);
addPermission(InterfaceTypePermission.INTERFACES);
allowTypeHierarchy(Calendar.class);
allowTypeHierarchy(Collection.class);
allowTypeHierarchy(Map.class); // <--- Map is allowed
allowTypeHierarchy(Map.Entry.class);
allowTypeHierarchy(Member.class);
allowTypeHierarchy(Number.class);
allowTypeHierarchy(Throwable.class); // <--- Throwable is allowed
allowTypeHierarchy(TimeZone.class);
// ...
Therefore, even though TeamCity has not explicitly allowed any types, several allowed types are already present on the permission list due to XStream’s defaults. This is enough to lead to unsafe deserialization.
The patch from version 2026.1.3 can be seen in the diff below and shows how these default allowed types are now cleared by TeamCity:
XStream’s SecurityMapper.addPermission() clears its permission list when it receives NoTypePermission.NONE. The allowTypes calls that follow [3] now operate on a deny-by-default baseline, i.e., Map and Throwable are no longer allowed types. The TeamCityProperties.getBooleanOrTrue() call at [4] means the new property defaults to true, so clearing the permission list at [3] will now occur by default on a patched server.
Root cause
The missing XStream class type permission reset is the root cause of CVE-2026-63077. TeamCity treats the configured classes as an allowlist, but XStream evaluates them alongside its earlier default permissions. In Java, a type hierarchy permission covers implementations and subclasses, not only the named type. Permitting Map therefore covers classes that implement Map such as LinkedHashMap, while permitting Throwable covers exception subclasses such as RuntimeException. These broad permissions expose enough object construction and reconstruction callbacks to assemble a working gadget chain.
The exploit also depends on how XStream’s reflection converter handles declared fields and object references. Java reflection lets code inspect a class’s field definitions at runtime and assign values to an object’s fields. An explicitly represented class name or class attribute passes through SecurityMapper.realClass(). By contrast, an exact declared field already provides its Java type, allowing XStream to allocate that field without a second explicit type lookup. An XPath reference can then reuse the allocated object without another type check when the reference omits the redundant concrete class attribute. In this context, XPath is an address within the XML object graph, not a query against TeamCity data.
Applied here, this allows a deserialization payload that begins with TeamCity’s HSQLMetadataStorage$SchemaMismatchException. This class extends RuntimeException, so XStream accepts it under the default Throwable hierarchy permission. Because it is a non-static inner class, it has a compiler-generated field pointing to its enclosing HSQLMetadataStorage instance. From there, the exact declared fields myHSQLStorage and myDataSource lead XStream to an org.apache.commons.dbcp2.BasicDataSource. XStream follows those field types without resolving BasicDataSource from an explicit element name or class attribute, even though TeamCity 2026.1.2 rejects that class when the XML names it directly. The patched version 2026.1.3 stops the chain earlier by rejecting SchemaMismatchException, which is absent from TeamCity’s explicit protocol allowlist.
Triggering the vulnerability
First, the server accepts an agent registration request via an HTTP POST to the /app/agents/v1/register endpoint, and returns a new session identifier in the TeamCity-AgentSessionId response header.
The attacker then sends arbitrary XML to the error command endpoint with that server-issued session header via an HTTP POST to the /app/agents/v1/commands/error endpoint. The handler for this endpoint is the method handleCommands, shown below. This will validate the incoming request’s TeamCity-AgentSessionId header before calling the handler for the error command.
// ./webapps/ROOT/WEB-INF/lib/web-core.jar
package jetbrains.buildServer.controllers.agentServer;
private ModelAndView handleCommands(
HttpServletRequest request,
HttpServletResponse response,
String[] path) throws Exception {
String sessionId = request.getHeader("TeamCity-AgentSessionId");
BuildAgentEx agent =
sessionId != null ? findAgentBySessionId(sessionId) : null; // <--- validate agent session ID
// This check occurs before the vulnerable handler is reached.
if (agent == null) {
response.setStatus(401);
response.getWriter().write("Agent's session is not found");
return null;
}
PollingRemoteAgentConnection connection =
(PollingRemoteAgentConnection) agent.getConnection();
if (path.length == 4) {
String operation = path[3];
if (operation.equals("error")) {
getCommandsProcessor().handleCommandIsFailedRequest(
connection, request, response
); // <--- call the error handler
}
}
return null;
}
The method handleCommandIsFailedRequest will then proceed to unsafely deserialize the incoming request’s XML body.
Error.fromXml() calls XStreamWrapper.deserializeObject(). By providing a suitable gadget chain in the incoming request’s XML body, we can achieve unauthenticated RCE via unsafe deserialization.
The gadget chain
The gadget chain’s objective is to make TeamCity call BasicDataSource.getConnection() on an attacker-configured object. That getter starts the following path from deserialization to command execution:
The payload reconstructs a BasicDataSource configured to use TeamCity’s bundled HSQLDB driver.
A collection callback causes FreeMarker to resolve the JavaBean property connection, which invokes BasicDataSource.getConnection().
Apache DBCP opens a new in-memory HSQLDB database and executes the SQL in connectionInitSqls.
The final SQL statement uses HSQLDB’s SCRIPT command to write a malicious JSPWS file into TeamCity’s webroot.
The attacker makes an HTTP request to that JSP file, executing the script’s contents server-side, for example Runtime.getRuntime().exec() can be used to execute an attacker-controlled OS command.
The first four steps occur while TeamCity handles the malicious XML request. The fifth requires a second HTTP request. The object graph exists to solve two problems in the first two steps: XStream rejects BasicDataSource when the XML names it directly, and merely constructing a datasource does not call its getConnection() method.
Object graph construction
The payload’s XML root is a three-entry LinkedHashMap. Entry one constructs and configures the datasource without naming its concrete class in a new XML node. Entry two presents that datasource to FreeMarker as an object whose properties can be read by name. Entry three forces a lookup of the property named connection.
Figure 1: High-level gadget chain flow to BasicDataSource.getConnection().
The entries appear in this order in the XML because the later entries refer to objects created by the earlier ones. XStream reconstructs them in document order, and the LinkedHashMap retains their insertion order in the resulting Java object.
Entry one: construct and configure the datasource
The first entry begins with HSQLMetadataStorage$SchemaMismatchException. This class extends RuntimeException, so XStream accepts it under the default Throwable hierarchy permission. It is a non-static Java inner class, which means the compiler gives each instance a hidden this$0 field pointing to its enclosing HSQLMetadataStorage object. XStream serializes that compiler-generated reference as outer-class.
The enclosing HSQLMetadataStorage declares a field named myHSQLStorage with the exact type HSQLStorage. That class, in turn, declares myDataSource with the exact type BasicDataSource. Because the XML does not represent either field with a new element type or class attribute, XStream follows the declared Java field types without performing another explicit lookup for those classes:
XStream encodes the dollar sign in a Java inner-class name as _- when it creates an XML element name. The element ending in HSQLMetadataStorage_-SchemaMismatchException therefore identifies the Java class HSQLMetadataStorage$SchemaMismatchException.
Entry two: expose the datasource through FreeMarker
The first entry leaves a configured datasource in memory, but nothing has called it. The second entry makes its JavaBean properties available through a FreeMarker HashAdapter. HashAdapter extends AbstractMap, so XStream accepts the explicit class under its default Map hierarchy permission.
The adapter needs a FreeMarker model that can read properties from the datasource. The payload creates a BooleanModel through the exact BeansWrapper.falseModel field, then populates the model’s inherited BeanModel.object field with a reference to the BasicDataSource in entry one instead of a Boolean value. Finally, HashAdapter.model refers to that BooleanModel:
<freemarker.ext.beans.HashAdapter>
<wrapper>
<!-- Class-introspection state from the PoC is omitted here. -->
<falseModel>
<object reference="../../../../../entry/jetbrains.buildServer.serverSide.metadata.impl.metadata.HSQLMetadataStorage_-SchemaMismatchException/outer-class/myHSQLStorage/myDataSource"/>
<wrapper reference="../.."/>
<value>false</value>
</falseModel>
<!-- Remaining BeansWrapper state from the PoC is omitted here. -->
</wrapper>
<model reference="../wrapper/falseModel"/>
</freemarker.ext.beans.HashAdapter>
The reference attributes preserve object identity rather than create copies. BooleanModel.object points to the existing datasource, HashAdapter.model points to the existing BooleanModel, and BooleanModel.wrapper points back to the same BeansWrapper. No reference introduces a new concrete class node. In particular, <object> does not repeat the BasicDataSource type, so XStream does not perform a new explicit lookup for that denied class. The shared BeansWrapper supplies the class introspection used later to resolve the connection property.
Entry three: trigger the property lookup
The graph can now resolve datasource properties, but it still needs an automatic callback to request one. The third entry uses a HashSet, accepted under XStream’s default Collection hierarchy permission, and a Commons Collections TiedMapEntry, accepted under the default Map.Entry hierarchy permission. A TiedMapEntry ties a key to a backing map. Here, its map field refers to the HashAdapter from entry two, and its key is the string connection:
The reference value is relative to the nested <map> element. Four ../ steps return to the LinkedHashMap root, and XPath’s one-based entry[2] index selects the second entry. Reusing that adapter preserves its connection to the BooleanModel and, through the model, to the datasource from entry one.
Object construction now ends with one continuous route: TiedMapEntry to HashAdapter, HashAdapter to BooleanModel, and BooleanModel to BasicDataSource. At this point, no database connection has opened yet. The gadget chain triggers when XStream inserts the TiedMapEntry into the HashSet.
Triggering gadget execution
A HashSet stores elements by hash. When XStream inserts the reconstructed TiedMapEntry, HashSet.add() automatically calls TiedMapEntry.hashCode(). That method calls getValue(), which performs map.get(key) against the referenced HashAdapter with connection as the key. It is worth noting that this is a mechanism very similar to that used by the classic CommonsCollections6 ysoserial gadget. However, the existing CommonsCollections6 gadget cannot be used because TeamCity’s XStream permissions reject the ChainedTransformer and InvokerTransformer classes used by CommonsCollections6.
The resulting call to HashAdapter.get(“connection”) passes the property name connection to the referenced BooleanModel. BooleanModel inherits FreeMarker’s BeanModel property lookup. JavaBeans use a naming convention in which a property named connection can be read through a public getConnection() method, so FreeMarker invokes BasicDataSource.getConnection().
A Java DataSource is a factory for Java Database Connectivity (JDBC) connections. BasicDataSource is the Apache Commons Database Connection Pooling (DBCP) implementation bundled with TeamCity. The payload configures it to load TeamCity’s bundled HyperSQL Database (HSQLDB) driver and connect to a new in-memory database at a randomized jdbc:hsqldb:mem: URL. This database is separate from TeamCity’s application database and requires no TeamCity database credentials. DBCP then runs the attacker-controlled connectionInitSqls, a list of SQL statements intended to initialize each new connection.
The initialization SQL creates a table containing a JSP scriptlet and asks HSQLDB to serialize the database to an attacker-selected path:
CREATE TABLE IF NOT EXISTS T<RANDOM>(C<RANDOM> VARCHAR(4000))
INSERT INTO T<RANDOM> VALUES ('<% ... Runtime.getRuntime().exec(command) ... %>')
SCRIPT '../webapps/ROOT/<random-hex>.jspws'
HSQLDB’s SCRIPT statement writes a textual representation of the in-memory database to the supplied path. The payload places a JavaServer Pages (JSP) scriptlet inside a table row, so the resulting SQL script is also a valid JSP template (i.e. a polyglot). This mechanism is similar to the one used by Secfault Security as part of a LibreOffice exploit.
Executing a JSP payload
Apache Jasper is the JSP engine in TeamCity’s servlet container. It compiles JSP source code into Java servlet code that handles an HTTP request, then runs that code inside the TeamCity server’s Java process. Whether a path reaches Jasper depends on the servlet mappings in WEB-INF/web.xml. TeamCity defines realJspServlet as Jasper’s org.apache.jasper.servlet.JspServlet, then maps the custom *.jspws extension directly to it. By contrast, TeamCity sends ordinary *.jsp requests to its buildServer dispatcher:
The buildServer servlet does not dispatch every direct .jsp request to Jasper. The corresponding JspController.doHandle() method first requires an internal TeamCity request, an authenticated TeamCity user, or an explicit configuration property that permits direct JSP requests. If these are not present, it returns HTTP 403 before the JSP runs:
We therefore target .jspws, as this allows a direct anonymous request to reach Jasper, compile the newly written file and execute it. This allows us to execute arbitrary Java such as Runtime.getRuntime().exec() which in turn can deliver the payload.
Exploitation
A proof-of-concept script for CVE-2026-63077 can be found here. Organizations can use this script to validate their detection and remediation posture. The exploit script will leverage the gadget chain described in this analysis to write a malicious JSPWS file in order to execute an arbitrary command, before deleting the JSPWS file from disk. An example of its operation is shown below in Figure 2.
Figure 2: Proof-of-concept exploitation.
The vendor-supplied patch, version 2026.1.3, has been verified to successfully prevent the unsafe deserialization of the gadget chain presented in this analysis. The teamcity-server.log file on a patched system shows the new XStream NoTypePermission.NONE added by the patch to effectively prevent the gadget chain’s first entry, HSQLMetadataStorage$SchemaMismatchException, from having its type successfully resolved.
[2026-08-07 01:53:09,794] ERROR - jetbrains.buildServer.SERVER - Error com.thoughtworks.xstream.security.ForbiddenClassException: jetbrains.buildServer.serverSide.metadata.impl.metadata.HSQLMetadataStorage$SchemaMismatchException; while processing request: POST '/app/agents/v1/commands/error', from client 192.168.86.70:58356, user-agent "Python-urllib/3.10", no auth
com.thoughtworks.xstream.security.ForbiddenClassException: jetbrains.buildServer.serverSide.metadata.impl.metadata.HSQLMetadataStorage$SchemaMismatchException
at com.thoughtworks.xstream.security.NoTypePermission.allows(NoTypePermission.java:26)
at com.thoughtworks.xstream.mapper.SecurityMapper.realClass(SecurityMapper.java:74)
at com.thoughtworks.xstream.mapper.MapperWrapper.realClass(MapperWrapper.java:125)
at com.thoughtworks.xstream.mapper.CachingMapper.realClass(CachingMapper.java:47)
...
IOC
On an exploited system, the TeamCity server logs will contain detailed exception traces due to the deserialization gadget causing a Java exception to be thrown. For example, in the log file C:\TeamCity\logs\teamcity-server.log the following may be present. This identifies the vulnerable URI path, the attacker’s IP address, and an exception that correlates to the gadget chain being used for exploitation. Note: the full stack trace has been removed for brevity:
[2026-08-07 00:36:36,467] ERROR - jetbrains.buildServer.SERVER - Error com.thoughtworks.xstream.converters.ConversionException:
---- Debugging information ----
cause-exception : freemarker.template.utility.UndeclaredThrowableException
cause-message : freemarker.core._TemplateModelException: An error has occurred when reading existing sub-variable "connection"; see cause exception! The type of the containing value was: boolean+extended_hash (org.apache.commons.dbcp2.BasicDataSource wrapped into f.e.b.BooleanModel)
class : java.util.HashSet
required-type : java.util.HashSet
converter-type : com.thoughtworks.xstream.converters.collections.CollectionConverter
path : /linked-hash-map/entry[3]/set/org.apache.commons.collections.keyvalue.TiedMapEntry
line number : 104
class[1] : java.util.LinkedHashMap
required-type[1] : java.util.LinkedHashMap
converter-type[1] : com.thoughtworks.xstream.converters.collections.MapConverter
version : 2026.1-222647
-------------------------------; while processing request: POST '/app/agents/v1/commands/error', from client 192.168.86.70:52728, user-agent "Python-urllib/3.10", no auth
com.thoughtworks.xstream.converters.ConversionException:
---- Debugging information ----
cause-exception : freemarker.template.utility.UndeclaredThrowableException
cause-message : freemarker.core._TemplateModelException: An error has occurred when reading existing sub-variable "connection"; see cause exception! The type of the containing value was: boolean+extended_hash (org.apache.commons.dbcp2.BasicDataSource wrapped into f.e.b.BooleanModel)
class : java.util.HashSet
required-type : java.util.HashSet
converter-type : com.thoughtworks.xstream.converters.collections.CollectionConverter
path : /linked-hash-map/entry[3]/set/org.apache.commons.collections.keyvalue.TiedMapEntry
line number : 104
class[1] : java.util.LinkedHashMap
required-type[1] : java.util.LinkedHashMap
converter-type[1] : com.thoughtworks.xstream.converters.collections.MapConverter
version : 2026.1-222647
-------------------------------
at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:81)
at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
...
A similar exception in a javaLogging file (for example, C:\TeamCity\logs\teamcity-javaLogging-2026-08-07.log) will also show the gadget chain’s JSPWS payload as part of an org.hsqldb.HsqlException message:
07-Aug-2026 00:36:36.462 SEVERE [http-nio-8111-exec-4] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [buildServer] in context with path [] threw exception [Request processing failed; nested exception is com.thoughtworks.xstream.converters.ConversionException:
---- Debugging information ----
cause-exception : freemarker.template.utility.UndeclaredThrowableException
cause-message : freemarker.core._TemplateModelException: An error has occurred when reading existing sub-variable "connection"; see cause exception! The type of the containing value was: boolean+extended_hash (org.apache.commons.dbcp2.BasicDataSource wrapped into f.e.b.BooleanModel)
class : java.util.HashSet
required-type : java.util.HashSet
converter-type : com.thoughtworks.xstream.converters.collections.CollectionConverter
path : /linked-hash-map/entry[3]/set/org.apache.commons.collections.keyvalue.TiedMapEntry
line number : 104
class[1] : java.util.LinkedHashMap
required-type[1] : java.util.LinkedHashMap
converter-type[1] : com.thoughtworks.xstream.converters.collections.MapConverter
version : 2026.1-222647
-------------------------------] with root cause
org.hsqldb.HsqlException: file input/output error: ../webapps/ROOT/682aed03b49b.jspws already exists
at org.hsqldb.error.Error.error(Unknown Source)
...
Remediation
For remediation guidance, please see Rapid7’s Emergent Threat Response blog for CVE-2026-63077, which contains further details.
Software supply chain security — We’re working on an observatory that
allows users to identify and monitor the world’s critical Open Source
packages in real time, including previously hidden parts of the
dependency graph.
Maintainer burnout — Our report on burnout in Open Source has been
well-received. We’re working on yearly follow-ups, plus a companion
report on how AI is affecting maintainer burnout.
Moving forward, the Lab’s experts will direct funding to and supervise Open
Source sustainability researchers doing critical work. We are already
working with universities to teach the next generations of software supply
chain experts.
The Internet isn’t a single lane of traffic. For a long time, the rule of thumb in web security was that bots are bad, while humans are good. Of course, we’re far past this generalization. Humans can be fraudulent, and bots can be helpful at different levels. Site owners actively want some automated traffic to interact with our sites to make the Internet functional and discoverable.
To complicate things further, the line between "human" and "bot" is blurring more and more. Now, we have a type of “hybrid” traffic where a single session shifts from human to agentic and back again. (Think of a user browsing a store, and then handing off the checkout process to an automated shopping assistant.)
So, how do website owners manage this kind of complexity? What matters here is assessing behaviors. Is this behavior abusive? Malicious? What’s the risk presented here, and can I trust this visitor based on their actions? Solving this requires moving beyond static, point-in-time checks. It requires analyzing continuous behaviors to evaluate Trust.
In this post, we’ll share an inside look into the strategy of the Web Integrity & Trust team (covering the bots and fraud problem spaces) around detecting and analyzing good and bad behaviors, providing tools to help site owners tackle emerging challenges in the shifting Agentic Internet. We’ll also share findings around agentic traffic since the launch of Precursor, and a simulation where you can see how your own cursor movements would be assessed as human or bot — plus some exciting launch updates to expect in the near future.
Defining Risk and Trust
Let’s talk about the distinction between Risk and Trust, the way we discuss it within the teams at Cloudflare who work in bot detection. These are often viewed as polar opposites of a continuum. At Cloudflare, we look at them as independent, but reciprocal, values. Trust is the essential ingredient in making informed decisions on what to do about your traffic.
Risk is how likely something like a request or action is to be harmful, and it’s often ephemeral. Trust, however, is built up over time, and it’s based on reputation.
We can illustrate this with an example from real life: say that you’re enjoying some evening television at home, when suddenly, you hear the doorbell being rung repeatedly. Besides being annoying, this behavior is strange. Frantic doorbell rings late at night are alarming.
You check through your door camera and see that the person ringing your doorbell is your best friend who lives next door. Of course, you trust your best friend, and we’d bet you would let them in.
In this example, it wouldn’t be enough for you to say, “Reject anyone who rings my doorbell at night” or “Reject anyone who rings my doorbell more than 10 times.” Again, Trust is the essential ingredient.
Going back to traffic on the Internet, the strategy as we build products in the bots and fraud space focuses on building an entire ecosystem based on Trust. And our goal is to provide the incentives and primitives for site owners to use to incentivize behavior that makes the Internet safer for everyone: starting with blocking malicious activity at the bottom, to encouraging participation in a safer Internet at the top.
Good behaviors, rooted in transparency
Starting at the top: what counts as good behavior? We can draw clear examples from the Verified bots and agents within BotBase. Last month, we announced an updated pragmatic taxonomy for the good bots we track in our system, boiling down the definition of “Verified” to two things: 1) you declare yourself honestly, and 2) you don’t abuse the trust you’ve earned.
Transparency between a site owner and a bot operator allows for a symbiotic relationship: site owners can note what behaviors and data uses they want to allow on their websites, and bot operators can be granted access more easily. The transparency allows for Trust in the relationship; if you have nothing to hide, declaring who you are should reduce friction from the sites who want to allow your behaviors.
BotBase is not meant to only make declarations of “who is good”. It is intended to be a directory of all known bots and agents, and provide the facts. Compared to our previous Bots Directory, which only included known good bots, BotBase is also capable of tracking less-than-good bots and agents. Why? Because our systems track and validate behavior for known good actors, meaning we have the tools to identify when these expectations aren’t met. If you abuse trust on the Cloudflare network, you should not be easily allowed, so you will be unverified.
Bad behaviors: blatant, stealthy, and everything in between
A few weeks ago, we announced Precursor, a continuous client-side system to detect even subtly inhuman bot traffic that can fly under the radar when assessing network signals alone. When a customer enables Precursor, the JavaScript detection is CDN-injected, so it doesn’t require sitting at the computer and figuring out where or how to rerun these detections. What's more, Precursor evaluates user behavior continuously throughout the session, so no more free hall passes for abusive traffic that found a way to pass client and browser-side checks just once.
Applying our Risk and Trust framework to these client-side detections, we can point out that CAPTCHAs or one-time hurdles are Risk-based, meaning they lack context. On the other hand, verification using behavioral tells is Trust-based, since it can capture more context clues from the full user session. Precursor is the tool for us to analyze this behavior. To sum it up, Precursor is so powerful because it:
Provides Trust-based detection over the entire user session.
Drives up the cost for bot developers to replicate human behavior over a multipage timeline.
By making it economically disadvantageous for bot developers to outrun these detections, we win the adversarial game.
Now, what have we learned since we’ve launched? Looking at just a 24-hour period at the time of writing this blog, we can see 206 million Precursor evaluation events, across 73,438 zones on the Cloudflare network.
We can see patterns in the data that reveal things that we had suspected when launching the detection, but can now validate across tens of thousands of domains:
Suspicious behavior often happens mid-session, which point-in-time detection wouldn’t catch.
Behavior often shifts from human to agentic and back over a session. In these cases, it’s important to understand the intent so that site owners don’t block user flows that they actually want.
This highlights the importance of a bot classification system that allows website owners to handle traffic by use case, purpose, and data use. This is precisely why we prioritized taxonomy updates for BotBase.
For those curious to learn more on how Precursor actually works, we shared a sneak peek — how the signals we analyze showed us that to err is human — in our announcement blog post. Today, we’re going a step further: giving anyone on the Internet an interactive demo simulating how Precursor would trace your cursor movements.
Precursor Traceis live now, sharing how we’d assess your cursor movements using (part of) Precursor’s detection mechanism. Here, you can see whether you’re accelerating or correcting yourself, the rhythm and texture of your cursor movement, and more — all things you’ve probably never thought about as a real human being interacting with a computer. Try it out!
Adaptive Intelligence is coming soon
Cloudflare’s bot detection engines can produce different outcomes when assessing if a given request is automated or not. For requests that are deemed to be automated, the assessment can be 1) definitely automated, based on proven, deterministic methods or fingerprints of bots, or 2) likely automated, based on predictive scoring from Cloudflare’s Bots ML.
Historically, Bots ML has been updated in versions, meaning we announced each new model version as a product launch. This pacing doesn’t work when bots adapt on the scale of hours or even minutes.
Adaptive Intelligence, a completely new detection engine, is different from anything we’ve built before in the Bots ML space. The model itself is adaptive. It has learned from everything we’ve seen in the past, but more importantly, it will continue to learn and self-adjust based on what it sees. Adaptive Intelligence will upgrade itself based on a wide range of traffic patterns we identify, from good to bad behaviors, and customers will no longer need to upgrade to a formal new model version to have the latest predictive bot detections working for them.
All Bot Management customers will have access to Adaptive Intelligence in the near future — stay tuned for the launch announcement coming soon.
Moving beyond determinism to influence bot behavior
So far, we’ve focused on Cloudflare’s side of things: strategy, detection, and taxonomy. All of this allows Cloudflare to equip website owners with the tools they need to set the traffic policies they want on their sites. Zooming in on the website owner side, we want to take this chance to discuss some advanced mitigations that allow website owners themselves to influence bot behavior.
With more blatant mitigation techniques, we face something that we’ve nicknamed the “Bot Antibiotic Problem.” Always sending bots a deterministic response (like a 403 block) makes it easy for a malicious developer bot to probe, observe, and reverse-engineer your defenses.
We know this, so we’re designing mitigations specifically made for throttling bots — with different approaches for malicious bots vs. benign bots. We can break them down into three approaches:
Approach 1: Unpredictability and Random Actions. Applying random responses (between block, challenge, or allow) to suspected automated traffic breaks a bot's automated retry logic and fingerprinting.
Approach 2: AI Labyrinth, a defensive response that traps unauthorized bots in an endless maze of AI-generated web pages. You can waste malicious bots' compute and crawl budgets by using misdirection. Site owners will be given three options within AI Labyrinth, depending on their preference:
Maze: Generates an endless web of linked pages for bots to follow.
Summary: Feeds crawlers an LLM-generated summary of a page that looks real but is entirely useless as AI training data.
Poison: Serves deliberately fake content (like fake prices or inventory) to a bot, polluting the data it collects for AI training.
Approach 3: Queuing for Good Bots. Not all agentic traffic is bad; queuing manages throughput for legitimate automated traffic (like user-directed shopping agents) without denying them service entirely.
These advanced, bot-specific mitigations are set to roll out closer towards the end of the year, and will be available for the website owner to choose how strict they want their mitigations to be.
We also know that a great defense is a predictive one — one that self-learns and course-corrects without needing multiple security experts on a call to reactively set a fix that accounts for the latest stealth attack. This might look like having a system of “disposable” rules, in which the ruleset is dynamic in nature. This is by design: if attacks constantly evolve, the defenses should, too. That’s why we’re working to keep both detections and mitigations a step ahead.
Establish the Trust ecosystem that works for you
Anyone and everyone can take steps to define how automated agents interact with their infrastructure.
By moving away from static, point-in-time checks and embracing continuous trust evaluation, we reduce the game of whack-a-mole with bot operators. If you’re not already using Cloudflare’s bot detection, check it out and establish the Trust ecosystem that works for you.
Since launching in 2020, Cloudflare Radar has provided one-of-a-kind insight into Internet traffic on a global scale. The open data that Radar publishes from our global network is used by a variety of experts in different domains: human rights advocates, journalists, academic researchers, network operators, and more.
Today, as part of Agents Week, Radar is beta-launching a new tool: Radar Researcher.
Now, if you want to answer a question with Radar’s data, you can just ask in plain language and get an answer back with real, interactive charts, all built on Cloudflare's developer platform. You no longer need to find the right page, choose the right filters, or read the API documentation to write a query.
Why we built Radar Researcher
Over the past six years, the Radar team has worked to provide clear, accessible, reliable data visualizations of complex datasets. This is not without its challenges: our user base is vast and ranges from curious novices to technical network experts. Even so, we publish as much insight into the Internet as seen from our network as we can: DNS queries from our public resolver 1.1.1.1, HTTP traffic from across our global network, network quality data collected from Cloudflare Speed Test, and so much more. We also believe in the power of open, public data, which is why everything you see in Radar’s data visualizations is available via our free API. Every year we make more and more of the Internet visible.
But as Radar has evolved, so has the technology for working with data. AI tools are lowering the barrier to expertise: you no longer need to know a dataset's structure or vocabulary to get useful answers from it. Subject-matter experts become collaborators rather than gatekeepers.
Radar has always wanted to do more for users who shouldn't need to master our entire catalog and its vocabulary to fully appreciate our insights. (See our recent blog post on how the Internet responded to the 2026 FIFA World Cup.)
We also know that many of our users are in a hurry. For example, a journalist writing about the latest Internet outage likely does not have the time to search through each page for the perfect graph to tell their story. This is why we built Radar Researcher.
Meet Radar Researcher
Radar Researcher is built to make accessing data as easy as describing what you’re looking for. All information is provided via Radar’s API, which ensures that the underlying LLM relies on real data, and can answer your questions quickly and in the same format as you would articulate them to a human.
It's available on every Radar page. Open the Radar Researcher panel from the header, and it docks beside whatever you're looking at, so you can keep browsing while you chat. Here are some examples of how you can use it:
Get an answer in plain language, with real charts: Ask a question and get the same interactive visualizations you see across Radar, plus a short explanation.
Choose your depth: Pick a concise, direct answer or a fuller, multi-topic report.
Keep the analysis going: After each answer, it suggests useful follow-up questions.
Pick up where you left off: Conversations are saved in a searchable history you can pin and revisit, and you can share any conversation with a link. (Shared links expire automatically after 30 days.)
Audit the reasoning: Check how the LLM interpreted your question, which datasets it looked up and queried, and how it worked through the results to reach its answer.
Ask your way: Type, use voice input, or launch it straight from Radar's search bar.
Turn any chart into a conversation
Researcher doesn't only start from a blank prompt. Anywhere you see a chart on Radar, an Explain with AI action opens the assistant with that exact visualization already in hand.
Instead of asking you to describe what you're looking at, it captures the exact visualization you're viewing and starts the conversation there. As you can see below, Radar Researcher takes the visualization in context, then analyzes the data and responds with an explanation.
Under the hood, the assistant hands the model three things at once: a screenshot of the chart (a vision-capable model reads the image directly), the exact data behind it from Radar's API, and the parameters of your current view, including location, date range, and any filters you've applied. The screenshot enables Radar Researcher to see what you see. The raw data keeps every number it cites precise instead of estimated from pixels, and the filters mean the explanation is about your exact view, not a generic version of the chart.
Use Case No. 1: What’s Internet quality like in Portugal?
Say you're curious how good the home Internet is in a given country. You can just ask:
Instead of manually calling the API, choosing parameters, or hunting through Radar's pages, Radar Researcher interprets the question and does the work for you. You can follow each step in the screenshot: it reads what you asked, queries Radar's Internet quality API, then analyzes the results and answers in plain language. And rather than a wall of numbers, it renders the same interactive charts you'd see on the Radar Internet Quality section.
From here you might ask a natural follow-up, like comparing Portugal to a neighbor, and Radar Researcher fetches fresh data for each one. It even suggests useful next steps of its own, such as how Portugal stacks up against Spain or the country's most common Internet outages. By letting you find what you're looking for in plain language, we hope applying Radar's data to your own analysis is easier than ever.
Use Case No. 2: Investigating an Internet disruption
Radar Researcher is also built with technical experts in mind. If we know one thing about engineers, researchers, and network operators, it’s that automation is key to their workflows. Looking at the same data with manual actions can be cumbersome and limiting: how do you ensure you're always looking at the most relevant data?
Consider a real event. In early 2026, Iran went through a series of government-directed Internet shutdowns, including one that lasted nearly three months before traffic partially returned in May. Reporting a story like that means pulling several views and refreshing them as the situation develops. To investigate it, you can just ask Radar Researcher:
This time the question is open-ended, so Radar Researcher runs a short investigation. It looks up the outage events Cloudflare Radar recorded for Iran and gathers the matching traffic data, then explains the timeline in plain language: Iran's HTTP traffic index collapsed from around 0.58 on January 7 to effectively zero by January 9, began a partial recovery around January 17, and returned close to pre-shutdown levels by January 27. It presents this as an HTTP traffic index chart with the outage window annotated directly on the line, plus a table of the recorded outages that also flags a second shutdown starting February 28.
From there it suggests where to take the investigation next, such as how the traffic compared with neighboring countries.
How we built it
Radar Researcher is built entirely on Cloudflare's own developer platform. It's a showcase of the same tools any developer can use to build an AI agent.
At its core is a Cloudflare Worker running the Cloudflare Agents SDK. Each conversation is a stateful Durable Object with its own SQLite database, so your chat history, titles, and streaming responses persist. They even survive you leaving the page mid-answer, because generation continues server-side and resumes when you reconnect. The "brain" is Workers AI, running open models like Kimi K2.7. Rather than betting on a single model, we run an ordered fallback chain across three different model families. If one model is momentarily at capacity, the request transparently cascades to the next, so an incident at any single provider doesn’t take Researcher down. Every call routes through AI Gateway for logging, cost tracking, caching, and safety guardrails.
The most interesting part is how Researcher reaches Radar's data. Instead of hand-writing a tool for every one of Radar's hundreds of endpoints, we connect the agent to the unified Cloudflare MCP server using Code Mode. The model gets just three tools (search, execute, and docs) and writes code to query the Radar API directly. It searches the OpenAPI spec to find the right endpoint, then executes a small snippet that fetches live data. Because the full API spec lives on the MCP server, nothing about Radar's API is hard-coded into the prompt. As Radar adds datasets, Researcher can use them with no code changes.
This is visible to you, too: every answer includes an expandable trace of the model's reasoning and the exact tool calls it made to gather the data, so you can always see how it got there.
From Markdown to real Radar charts
There was one problem to solve along the way. Language models answer in Markdown: text, tables, and bullet points. But we didn't want walls of numbers; we wanted the same rich, interactive charts you see everywhere else on Radar. Worse, when a model tries to write data directly into its answer, it tends to round, summarize, or truncate it, which is exactly what you don't want from a data tool.
Our solution keeps the data out of the model's prose entirely. When Radar Researcher fetches data, its code returns a small envelope that pairs the API path with the result. Then, instead of pasting numbers, the model emits a lightweight chart specification that simply references that same path:
Radar's frontend matches the chart's dataFrom to the fetched result and renders it with the exact same visualization components used across the rest of the site. The chart is always faithful to the API, and Radar Researcher can draw on our whole visual vocabulary: time series and stacked areas, donuts, bar charts, maps, histograms, etc.
A few small touches
Not everything needs the big model. Small, fast models handle the side tasks: one writes a short title for each new conversation, and another suggests the follow-up questions under each answer. Both run off to the side, so they never slow down your actual reply.
Radar Researcher also gets a little context about the here and now. We pass it the current date and time, plus the same high-level details about your connection that you'll find on Radar's IP page, such as your IP location and network. That way, when you ask something like "how's traffic here?", it can tailor the answer to where you are.
Everything reaches you through Radar's frontend (also a Worker) over a service binding, with per-IP rate limiting and shareable conversations stored in R2. Every layer, from compute to inference to gateway to storage to data, is Cloudflare.
Powered by agents — and ready for them
Radar Researcher shows how we use an AI agent to help people explore our data. But there's another side to the agent story: what happens when the agent isn't ours, but a general-purpose assistant running in your browser, acting on your behalf?
Today, an agent that wants to use a website mostly has to scrape the page and guess at how it works, which is brittle, slow, and error-prone. WebMCP is an emerging web standard that fixes this: a page can register a small set of well-defined tools that any browser agent can discover and call directly, instead of reverse-engineering the DOM (Document Object Model).
We've added WebMCP support to Radar, using both flavors the standard offers:
The imperative API lets us register tools in JavaScript that call straight into the same code that powers the UI, so an agent can drive Radar directly. Filter to a country, region, continent, or ASN; change the date range; search pages, sections, and entities (domains, ASNs, IPs, and more); jump to any section.
The declarative API turns Radar's existing HTML forms into tools by annotating them with a couple of attributes, so an agent can scan a URL with the URL Scanner, look up a domain's report, or test a website's post-quantum TLS key exchange support.
Every one of these simply drives functionality already available in the user interface, and it's pure progressive enhancement: in a browser without WebMCP, the tools just aren't there and nothing changes.
It’s also a case of practicing what we preach. Radar’s own URL Scanner evaluates how agent-ready a site is, and one of those checks looks for exactly this kind of WebMCP integration. By implementing it ourselves, Radar now passes its own agent-readiness check, and, more importantly, becomes a site that both people and agents can navigate with ease.
Try it out
Radar Researcher is available in beta today on Cloudflare Radar. Look for the Researcher button in the header on any page. It's built entirely on Cloudflare's developer platform, and it's just getting started: we'll keep expanding the datasets it understands, sharpening its analysis, and adding new ways to interact with data.
Tell us what works, what doesn’t, the insights you’re finding, what still feels beyond reach, and everything in between. Your feedback will help us improve and make Radar Researcher even better for our full launch. We can’t wait to see what you do with it!
As a platform for helping build a better Internet, Cloudflare helps turn ideas into real products and experiences around the world. Across communities and backgrounds, developers build with Cloudflare using the tools they love, shaping what comes next for the Internet while inspiring, collaborating with, and teaching others.
The community is where some of Cloudflare’s best moments happen. Students show their friends how to deploy Workers for the first time. Discord users answer questions from other developers via working code samples, instead of links to documentation. Open-source contributors build novel solutions to solve their own problems, then share them with the world. Organizers host events that give builders from all backgrounds the space to start building their dream project.
All of these represent a community at its best: people helping other people build.
This spirit of community is an exciting and vital part of helping to build the Internet. Those who step up to educate and support others, or to invent, build, or maintain tools shared across the ecosystem, make lasting contributions to the health and potential of the Internet.
We want to have their backs.
That's why today we’re announcing an improved community program, designed to better support, recognize, and empower the people getting involved, while working with them to shape what comes next. The program has two main tracks:
Cloudflare Ambassadors: Bringing Cloudflare to their own communities.
Cloudflare Community Engineers: Contributing to open-source projects that improve the Internet.
We’re launching a new home for the program where you can learn more and get involved: cloudflare.com/community.
Cloudflare Ambassadors
Cloudflare Ambassadors are people who bring Cloudflare into their own communities. You can probably think of people in the communities you value who share a genuine passion for a product or technology. It’s inspiring and we love to see it. When that enthusiasm includes the tools we’re building here at Cloudflare, it’s especially exciting for us.
Following our annual application process (more below), we’ll announce the year’s Cloudflare Ambassadors cohort. Selected Ambassadors will receive support, resources, and benefits to help their community thrive and bring their ideas to life. Ambassadors can serve for up to two years, giving them meaningful time to build momentum while helping us support more communities over time.
What Ambassadors do and what we provide
Being an Ambassador might mean organizing a local event, leading a student group, creating spaces where builders can learn together, publishing tutorials or sharing content online, or being the person others turn to when they want to understand what’s possible with Cloudflare.
Ambassadors will take the lead on events in their communities, whether on campus, through local organizations, or across their city. When hosting meetups, hackathons, workshops, or talks, they will be able to apply for support in the form of credits, marketing assets, technical resources, and more.
We’ll also give them a visible role in Cloudflare’s online community spaces, including Discord, so that other developers know who they are, and that they’re here to help.
Applications are open now, and will be accepted through September 6. Those selected as Ambassadors will be informed of their selection by October 5. Apply to become a Cloudflare Ambassador
A great example of the enthusiasm we’re looking for comes from Sruthi Pereddy, a Computer Science major at University of Michigan and a current intern on Cloudflare’s Recruiting Ops team. Sruthi’s work within Cloudflare has created a drive to share and explore more with others:
“Whether it’s hackathons, startup venture funds, or coursework, I want to show my peers that Cloudflare is a go-to developer platform for whatever they’re building,” Pereddy says. “Students are ready to build, but often feel constrained by resources. I’m excited to bridge that gap and make sure they have the infrastructure to turn their ideas into reality from day one.”
Cloudflare Community Engineers
Some community work happens in person, but a great deal of community work also happens in code. Much of Cloudflare’s Developer Platform is built on open-source work, or is open-source, like workerd and quiche. Open-source contributors, especially maintainers, do wonderful work and embody so much passion and determination. We’re eager to support them, especially since their work can sometimes feel thankless. So we’re doubling down on our efforts to build stronger incentives and directly support the maintainers doing this important work.
Last year, we announced our sponsorship of the web framework TanStack. TanStack creator Tanner Linsley says that sponsorship has had a major impact.
“Cloudflare’s sponsorship has given us room to keep investing in foundational open-source work that’s hard to tie to a single product or launch, maintaining the core libraries, improving docs and tooling, supporting contributors, and putting real time into bigger bets like TanStack Router and Start,” Linsley says. “It’s also helped us make sure TanStack apps have a really solid path onto Cloudflare’s platform. More than anything, that support buys stability, which is kind of everything when you’re building open source for the long haul.”
Today, we’re expanding on our previous open-source investments by introducing Cloudflare Community Engineers. Earlier this year, we announced a $1M fund as part of our acquisition of VoidZero to support the Vite community. We’re committing an additional $1M in funding to sponsor and support open-source projects over the next two years, with eligible Community Engineers receiving grants from the fund to support their continuing work in open source.
The Community Engineer program does not have a maximum term. Open source work doesn’t neatly fit into annual cycles. Some projects require maintenance for years, while other times, contributors do the work that is needed at exactly the right moment. This program is intended to support that.
To begin, we’ll focus on developers working on things in the orbit of our own open-source projects — projects like Astro, Agents SDK, EmDash, Hono, and Vinext. We’ll also grant our Community Engineers a special designation in Cloudflare’s Discord server and other online spaces.
Applications for Community Engineer grants will open at a later date.
Making our Discord better as it grows
Since we launched Cloudflare’s Discord server in 2020, almost 100,000 Cloudflare users have joined. Our Discord server has become one of the main places where developers ask questions, share projects, and provide valuable feedback. But of course, the more a Discord community grows, the more effort is required to keep it healthy and approachable.
To address this, a new Discord committee will help to maintain and grow our Discord community, with Cloudflare Ambassadors joining Cloudflare staff on the committee.
This is not about being on hand to perform moderation and admin tasks. We’ve been building tools and automations to help us do that with far less human intervention. Our new automated protections against spam and malicious links are starting to relieve this burden, allowing our Developer Relations team to help manage things where some human insight is needed.
In fact, we’ll be open-sourcing and sharing those tools soon because we think every Discord server could benefit from less spam and malicious content.
The committee will help provide a useful connection to those building and managing products at Cloudflare. They’ll be able to steer people and conversations to domain experts and convene conversations and sessions with internal teams and makers around the community. They’ll be much more focused on content and opportunities than on the type of Discord administrivia that can otherwise swallow so much time and energy.
We want our Discord to be easier to use, contribute to, and trust. It should be a place where builders find each other, help each other, and shape the future of the platform together. We believe this is the way.
Ready, set, go!
To learn more about the community program, and to apply for a role, visit the new community site at cloudflare.com/community.
AI Gateway and Workers AI first started as distinct products, but over time, we noticed our users were converging. With AI Gateway, you can proxy requests to any model provider and get built-in observability, logging, access, and security. On Workers AI, we host models on the GPU infrastructure that we manage, exposing an API endpoint you can leverage to access inference-as-a-service.
The architecture of these products looks different, but to an end user, it achieves the same goal: connecting you to models with a sophisticated control plane. Today, we're excited to share our plans on how these products converge into one unified path, so you can connect to any model provider (including Workers AI), while managing things like observability, billing, security, and logging from a single control plane.
It’s the next step toward some big plans we have — read on to learn what a unified control plane means for the future of model routing.
Merging the binding and API
We've been hinting at the fact that these products are becoming more unified through our entrypoints: the Workers binding and the REST API. We have an AI binding that you can use to call AI Gateway and Workers AI. There's no concept of a separate AI Gateway and Workers AI binding: it all goes through the same path. We shipped the idea of a “default” gateway a few months ago, so that if you have never set up an AI Gateway before, you could still automatically inherit the AI Gateway observability and logging. Of course, you can still specify your own gateway if you'd like to split up applications into multiple projects.
Here's what the binding call looks like, if you are calling Workers AI via AI Gateway:
We’ve also announced a single unified REST API — the /ai/ endpoint that allows you to make similar calls to Workers AI via AI Gateway.
Doing this allows us to unify the entrypoints to AI Gateway and Workers AI, so you don't need to make a choice between which product to use first: it all comes with batteries included.
Automatic observability and control for all Workers AI users
One of the most immediate benefits of this convergence is that you no longer need to explicitly create an AI Gateway before you start getting visibility into your inference traffic. If you've never set up a gateway before, just pass default as the gateway ID in your binding or REST API calls, and AI Gateway will create it automatically on the first authenticated request.
With this, every request is logged with full request and response payloads, token counts are tracked per model, and you get cost attribution without any dashboard setup. If you later outgrow the default gateway — if you want custom caching rules or to split traffic by application — you can create a named gateway and point your requests at it with a single parameter change.
Here's how it looks in the binding. Before, you called Workers AI directly:
Now, add a third argument to route through AI Gateway and get full observability:
Head to the Cloudflare AI Gateway dashboard and you'll see every request: latency breakdowns, token usage, error rates, and the exact prompts and responses. For teams debugging model behavior or auditing AI output, this is a huge upgrade from flying blind.
New: using AI Gateway credits for Workers AI
A new thing we're launching today is the ability to use AI Gateway credits for Workers AI. Before, you could only use AI Gateway credits on external model providers (e.g., OpenAI, Anthropic) but you couldn't apply your AI Gateway credits to Workers AI usage just yet. We've finally enabled our systems to allow unified billing for Workers AI. This means that you can load a wallet full of credits, and then choose to spend that across OpenAI, Anthropic, Workers AI, or any provider that we support.
Since we're now offering pre-paid billing for Workers AI and want to encourage users to use this new path, we're also offering elevated rate limits on Workers AI models if you use AI Gateway unified billing. Please refer to the developer docs for up-to-date information regarding rate limits, as well as how to request a higher rate limit.
Coming soon: model-first routing
With all your inference traffic flowing through a single control plane, we can start making smarter decisions about how to serve each request starting with the model you want, not the provider you have to manage. Provider-first routing forces you to think about infrastructure: "Which provider do I call? What if they're down?" Model-first routing flips that. You think about what you need — a capable reasoning model, a fast summarizer, a cheap embedding model — and the control plane handles provider selection, failover, and load balancing.
Today, if you want to call a model, you have to know which provider hosts it. If that provider is down or rate-limiting you, your application breaks. We're moving toward a world where you specify the model, and AI Gateway handles the rest.
This way, you can request Kimi K2.7 Code and not care whether it comes from Workers AI, Moonshot's own API, or another provider that hosts the same weights. If Workers AI has capacity, you get the benefit of our managed infrastructure. If Workers AI is at capacity, the gateway transparently load balances you to another provider that can serve the same model. You can still choose to stick to a single provider if you’d like, but model-first routing enables you to get more flexibility if you care about resiliency. We work with vetted providers, so the quality of model outputs remains top priority, and will also be able to respect requirements such as Zero Data Retention (ZDR).
This also means better resiliency by default. If one provider's version of a model is having issues, traffic shifts to another without application-level retries or complex fallback logic in your Workers. The gateway treats model availability as a routing problem. We hope to pilot this in the coming months for all AI Gateway and Workers AI users.
Next: smart routing
The next evolution of routing goes beyond simple failover. We're building intelligent routing that understands what you're asking for and picks the right model for the job without any configuration required.
Instead of specifying a model, you can let the gateway decide. Under the hood, a classifier running on Workers AI reads your prompt and predicts what kind of task it is (coding, research, summarization, general Q&A), how complex it is, and how much context matters. A heuristic scorer then maps that to the best model from a curated pool. For teams that want control, you can still specify exact models. For everyone else, the zero-config path means you get better economics and performance without maintaining your own routing logic. We are currently piloting this internally, and we’ll be actively testing and iterating in the next few weeks before release.
Get started today
If you're already using Workers AI, the easiest way to try this out is to start routing your existing calls through a default gateway. You'll immediately get request logging, token tracking, and cost attribution without changing anything else about how you call the models.
If you're already using AI Gateway, adding Workers AI to the mix is as simple as calling a Workers AI model. Load up your AI Gateway wallet and you'll get unified billing across every provider we support, plus elevated rate limits on Workers AI models.
When you move AI agents from prototype to production, the infrastructure challenges multiply. Your agents need to persist state across multi-step workflows that run for hours or days. They need to coordinate with other agents, share context, and sometimes access GPUs for specialized tasks. Amazon Bedrock AgentCore runtime microVMs provide a fully managed environment for invocations that can run for up to 8 hours and support stateful workflows through managed session storage. Some workloads also benefit from dedicated, larger-capacity environments — for example, when agents need to run continuously for multiple days, access GPUs or the underlying OS, or run multiple collaborating agents on the same host.
Today, I’m happy to announce runtime instances, a new complementary compute option in Amazon Bedrock AgentCore Runtime that gives your agents persistent, managed infrastructure purpose-built for complex agent workloads.
What you get Runtime instances provides AWS-managed EC2 infrastructure where you deploy multiple agents in a single runtime, each with their own dependencies and artifact types. Your agents can collaborate on the same host within shared sessions that persist for up to 14 days. The service supports GPU acceleration for compute-intensive tasks, session stop/restart to save costs during idle periods, and containerized deployments for teams that want to ship independently. For knowledge that needs to survive beyond a session, runtime instances pairs naturally with Amazon Elastic Block Store (Amazon EBS) and AgentCore Memory, which gives your agents long-term recall across sessions and environments.
Before today, if you wanted to keep your agents running for days or they needed GPU access, or multi-agent coordination, you had to build and manage that infrastructure yourself. You provisioned EC2 instances, configured networking, set up session management, handled scaling, and stitched together monitoring. Runtime instances handles all of that for you while integrating with the same AgentCore APIs, identity controls, and observability you already use with AgentCore Runtime microVMs.
A few things that should make agent developers smile: your agents can call each other as tools within a shared session, iterating autonomously until the job is done. You bring any framework (CrewAI, LangGraph, LlamaIndex, Strands) and any model. Packaging is minimal, a @app.entrypoint decorator and a zip file or container image. And if your workflow spans days, hibernate Monday night and resume Wednesday morning with everything intact.
Runtime microVMs and runtime instances are complementary compute options that you can use independently or together through the same AgentCore runtime APIs. A lightweight orchestrator agent on runtime microVM can coordinate and dispatch work to specialized worker agents running on instances. The orchestrator handles API calls, task routing, and result aggregation using runtime microVM’s fast scaling, while workers on Instances perform compute-intensive tasks like code compilation, security scanning, or GUI automation that require persistent state and direct OS access.
Let me show you how it works I built two agents for this demo: a code writer agent that generates Python code from natural language descriptions, and a code reviewer agent that analyzes the generated code for bugs, security issues, and style improvements. Both agents share the same file system, so the reviewer can read whatever the writer produces without any data transfer or API calls between them.
Here is the code writer (simplified, no error handling):
writer = Agent(
model="us.anthropic.claude-sonnet-4-5-20250929-v1:0",
system_prompt=(
"You are a senior Python engineer. "
"Given a task, return ONLY a single Python code block — no prose."
),
)
@app.entrypoint
def handler(event, context):
task = event.get("task") or event.get("prompt")
session_id = getattr(context, "session_id", None) or event.get("session_id")
session_dir = SHARED_DIR / session_id
session_dir.mkdir(parents=True, exist_ok=True)
code = str(writer(task))
(session_dir / "code.py").write_text(code)
return {"agent": "writer", "wrote": str(session_dir / "code.py"), "code": code}
Here is the code reviewer agent (simplified, no error handling):
A capacity provider defines the EC2 infrastructure your agents run on. In the AgentCore console, I select Runtime in the left navigation, then select the Capacity providers tab and Create capacity provider.
I give it a Name, select Linux (64-bit ARM) as the Operating system, and choose c7g.2xlarge as the Allowed instance types. This gives me 8 vCPUs and 16 GiB of memory, enough for both agents to run comfortably side by side.
Further down, I configure the VPC, subnets, and security groups for network access. Under Storage configuration, I keep the default gp3 volume. Under Service access, I select Create a new service role and let the console create the infrastructure role that manages EC2 instances on my behalf.
I select Create capacity provider and wait a few seconds. The status moves to Active.
Note the capacity provider configuration summary: operating system, instance type, subnets, security group, instance profile, and infrastructure role. Once created, only the description can be edited, so verify your settings before you proceed.
Step 2: Create a runtime and deploy the first agent.
Back on the Runtime page, I select Create runtime. I give it a Name, select Instances as the Compute type, and choose the Capacity provider I created in the previous step.
Under Agent source, I select S3 Source, then Upload to S3. I choose my agent zip file (ACIDemoWriter.zip), set the Language runtime to Python 3.13, and specify agent.py as the Agent entry point. This is the file that contains my @app.entrypoint decorated function. Under Permissions, I select Create default role to let the console provision the IAM role my agent needs.
I select Create runtime and wait for the status to become Ready.
I repeat the same process for my code reviewer agent. I create a second runtime, select the same capacity provider, upload my reviewer agent zip file, and wait for it to become Ready. Both agents now share the same underlying EC2 infrastructure.
The console shows me a View invocation code section with ready-to-use Python, TypeScript, and JavaScript snippets to invoke my agent programmatically. But for this demo, I use the built-in test feature. I select Test on the writer agent’s page.
Step 3: Invoke agents and observe collaboration.
The Runtime playground opens. At the top, I see three fields: Runtime agent, Endpoint, and Session ID. The console generates a session ID automatically. I take note of it because I will reuse it with the reviewer agent.
In the Input field, I type a JSON payload asking the writer agent to generate code:
{"prompt": "write a fibonacci suite"}
I select Run. After a few seconds, the Output panel shows the agent’s response. The writer agent generated a Python module with two implementations of a Fibonacci sequence (a list-based function and a generator) and wrote it to /tmp/agentcore-session/ca5ec24d-07f5-4eeb-add1-5ba416bf9eb2/code.py. Notice the session ID in the file path. That directory is the shared file system for this session.
Step 4: Invoke the reviewer agent in the same session.
Now I switch the Runtime agent dropdown to ACIDemoReviewer. The important part: I paste the same session ID (ca5ec24d-07f5-4eeb-add1-5ba416bf9eb2) in the Session ID field. This is what connects the two agents.
I type a simple prompt:
{"prompt": "review the code"}
I select Run. The reviewer agent reads the file the writer produced from the shared session directory and returns a detailed code review. It finds no critical bugs but suggests adding type hints, input validation, and simplifying the edge case handling.
The two agents never exchanged messages or called each other’s APIs. They collaborated through the shared file system that runtime instances provide within a session. You can extend this pattern to any number of agents: a test agent that runs the code, a documentation agent that generates README files, a security agent that scans for vulnerabilities, all sharing the same working directory.
Key details Here are a few things to know as you get started:
Supported OS: Linux (ARM64 and x86_64) at launch.
Session persistence: Sessions persist for up to 14 days.
Runtimes: Python 3.11-14 with native code support. Container images also supported.
GPU: Support for GPU-accelerated instance types.
Integration: Uses the same AgentCore APIs, identity, observability, and policy controls as AgentCore Runtime.
Pricing: Standard EC2 pricing plus a management fee for AgentCore orchestration.
Regions: US East (Ohio, N. Virginia), US West (Oregon), Asia Pacific (Mumbai, Singapore, Sydney, Tokyo), and Europe (Frankfurt, Ireland)
Customers tell us that managing TLS certificates at scale is one of their biggest operational concerns. The Certification Authority Browser Forum (CA/Browser Forum) has mandated a phased reduction in maximum certificate validity for public certificates. By March 2027, the maximum validity drops to 100 days. By March 2029, it lasts for 47 days. For an organization managing 1,000 certificates, the final transition means roughly 30 renewal events every day. Renewal and rotations of renewed certificates at that cadence isn’t something manual processes or ticket-driven workflows can sustain at scale.
We recently announced Automated Certificate Management Environment (ACME) protocol support in AWS Certificate Manager (ACM). With this launch, you can use the ACME clients your teams already know, including popular open source tools like certbot, cert-manager, acme.sh, and win-acme, to automate public certificate issuance and renewal for your infrastructure. Customers that are using third-party certificate authorities (CAs) can point their existing ACME-compatible clients at ACM instead of their current CA, with minimal reconfiguration. This applies whether it’s running on Amazon Web Services (AWS), on premises, or in a hybrid environment. Certificates created through ACME are registered in ACM, giving you a unified view of your entire certificate inventory.
This post covers how the feature works, how to get started, and the controls and best practices to help you manage certificate issuance at scale.
Background
ACME is an open source protocol that automates the process of verifying domain ownership and issuing certificates and has become a standard mechanism for certificate automation. While ACM has long provided managed certificate issuance and renewal for AWS-integrated services such as Elastic Load Balancing (ELB), Amazon CloudFront, and Amazon API Gateway, many customers also need to automate certificates for their own infrastructure, including servers they manage in their data centers, Kubernetes clusters, Internet of Things (IoT) fleets, and hybrid environments. Until now, those customers had to turn to external providers. This launch brings the ACM automation model to that same infrastructure, using the standard ACME protocol with AWS managed certificate endpoints.
How it works
The feature introduces a new centrally provisioned and managed resource type: the ACME endpoint. Each endpoint is an AWS resource with a unique ACME directory URL and AWS Identity and Access Management (IAM)-based access controls. You create and manage endpoints through the ACM API or AWS Management Console, and point your existing ACME clients at the endpoint URL. Certificates issued through your endpoint are automatically registered with ACM, appearing in your certificate inventory alongside certificates created by the RequestCertificate and ImportCertificate API calls.
The architecture separates into two planes. In the control plane, PKI administrators use ACM APIs to create ACME endpoints, pre-approve the domains an endpoint is allowed to issue for, and generate external account binding (EAB) credentials. In the data plane, ACME clients register with an endpoint using EAB credentials and request certificates for domains the administrator has already validated. This architecture is how we provide customers the ability to scale. Instead of each client proving domain ownership on every request, a principal with appropriate ACM permissions (typically your PKI administrator) validates domains once at the endpoint level, and then application owners don’t need DNS credentials to get a certificate.
Adding to the data plane, EABs control client access to the endpoints. Each EAB is bound to an IAM role that controls what certificate operations the ACME client can perform, and credentials you generate in ACM are distributed to authorized ACME clients. An ACME client authorized for one endpoint can’t use a different endpoint. This creates security boundaries between environments. For example, a client authorized for your development endpoint can’t obtain certificates from your production endpoint.
Figure 1 shows the ACME request flow through ACM. An ACME client authenticates to an ACME endpoint using EAB credentials. The endpoint routes certificate orders to Amazon Trust Services for issuance. Issued certificates are registered in ACM inventory, where Amazon EventBridge and AWS CloudTrail provide expiration alerting and audit logging.
Figure 1: An ACME architecture and workflow
Getting started
Getting started with the new ACME feature in ACM is straightforward. Use the following steps to create your first ACME-generated certificate.
Prerequisites
An AWS account with permissions to create and manage ACM resources
An ACME client installed on your infrastructure (for example, Certbot, cert-manager, acme.sh, or others)
Amazon Route 53 hosted zone for your domain, or the ability to create a CNAME record with your DNS provider
Step 1: Create an ACME endpoint
Before you can use ACME clients with ACM, you need to create an ACME endpoint. This endpoint provides the URL that your ACME clients will use to request certificates.
Run the following command from the AWS CLI to create an ACME endpoint:
Before ACME clients can request a certificate, the administrator validates the domain using DNS once at the endpoint level. Use DomainScope to control exactly which certificate patterns are allowed:
Enabling only ExactDomain restricts clients to that specific name,
Subdomains enabled allows names like api.example.com,
Wildcards enabled allows *.example.com.
Leave a scope disabled to block that pattern outright, even if an otherwise-valid ACME request asks for it. For a production endpoint, consider enabling only ExactDomain and Subdomains and leaving Wildcards disabled for a stricter posture.
If your domain is hosted in Route 53, specifying HostedZoneId lets ACM create the required CNAME record automatically. If your domain is hosted elsewhere, omit it and create the provided CNAME record manually with your DNS provider. Validation typically completes within a few seconds after the record is in place.
EAB credentials authenticate your ACME clients to your endpoint. Generate a unique set of credentials for each client or environment to maintain security boundaries.
Run the following command to generate your EAB credentials, adjusting your expiration to fit your organization’s risk profile:
With your endpoint URL and EAB credentials ready, you can now configure your preferred ACME client. The following examples show configuration for two popular clients. As a reminder, the server information was retrieved in step 1, part 4 as the EndpointUrl.
After the initial registration, your ACME client handles renewals.
Enterprise controls
Other ACME alternatives can provide certificates but don’t give the same amount of control and governance for customers that need to scale their certificate environment. The following controls are available to help reduce risk across your organization.
Domain validation
Customers managing large numbers of domains told us they need a way to prevent unauthorized certificate issuance across their domain space. Domain validation gives you this control. For each domain you validate, you enable the certificate patterns it should be allowed to issue, whether it’s ExactDomain, Subdomains, or Wildcards. For example, if you validate internal.example.com and enable only Wildcards, an ACME client can request *.internal.example.com but a request for internal.example.com itself or api.internal.example.com is rejected. This enforcement happens at the endpoint level, before requests reach the ACM certificate authority, and you can validate multiple domains under a single endpoint, each with its own scope.
Centralized certificate visibility
Certificates issued through your ACME endpoints are registered with ACM. You can use the aws acm list-certificates command to see all your issued certificates.
IAM authorization, CloudTrail audit logging and observability
Endpoint management operations are authorized through IAM and logged to CloudTrail. You can use IAM policies to control which principals can create endpoints, generate EAB credentials, and manage domain constraints.
Best practices
For customers implementing ACME certificates for the first time, consider the following best practices for your organizations.
Segment endpoints along organizational or environment boundaries
The endpoint serves as a useful method of isolation for larger organizations. A large enterprise can create one endpoint per organizational boundary (business unit, subsidiary, or environment) instead of a single shared endpoint company-wide. Each endpoint has its own pre-approved domains and its own set of EABs, so a compromised credential in one business unit has no path to certificates in another.
However, weigh this against your operational overhead as well. A reasonable starting point is one endpoint per environment (dev, staging, andprod) within a business unit, expanding to per-business-unit endpoints only where compliance or organizational requirements call for it.
Manage EAB credentials securely
Anyone holding a validKeyIdandMacKeyfor an endpoint can obtain certificates for any domain pre-approved on that endpoint, so these credentials deserve the same handling you’d give an access key.
Avoid hard coding theMacKeywhere possible by using a secret store such as AWS Secrets Manager. Distribute it only to the ACME clients that you authorize to use the endpoint.
Set the expiration of the EAB to an acceptable level. While EAB supports long-lived credentials, not all scenarios require an indefinitely long EAB.
When creating the role for each EAB, adhere to concept of least privilege. Creating a role per EAB, rather than sharing a role across all bindings, can help reduce risk in your AWS environment.
Audit CreateAcmeExternalAccountBinding and GetAcmeExternalAccountBindingCredentials calls in CloudTrail separately. Because retrieving the actual key material is a distinct API call from creating the binding, alerting on retrieval events is a stronger signal of real credential distribution than binding creation alone.
Automate how EABs are associated with clients at runtime
Generate a unique set of EAB credentials for each client or environment rather than sharing one binding across multiple ACME clients. As you begin to scale with multiple endpoints, usesome of the following patterns to reduce operational toil.
Name each EAB and its bound IAM role after the client it belongs to (team, application, environment), so the binding’s purpose is obvious from DescribeAcmeExternalAccountBinding output alone, without cross-referencing a spreadsheet.
Store each client’s KeyId and MacKey under a secrets path scoped to that client (for example, a Secrets Manager path per team and environment), and let the client’s provisioning pipeline retrieve its own credentials.
In Kubernetes, use one ClusterIssuer or namespace-scoped Issuer per EAB rather than one shared issuer across teams. This keeps the client-to-EAB association explicit in cluster config, and lets you revoke one team’s access without touching anyone else’s.
For ephemeral infrastructure (build agents, autoscaled fleets), provision EAB credentials as part of your infrastructure-as-code or continuous integration and deployment (CI/CD) pipeline instead of a one-time manual handoff, so credential lifecycle tracks infrastructure lifecycle.
Monitor your deployment of ACME
ACME’s power is through automation, and organizations should monitor their ACME usage for anomalies.
Alarm on issuance failures, not just successes. At 45-day certificate validity, a silent renewal failure gives you far less runway to react than the months of time you might be used to with longer-lived certificates.
Test renewal automation before you depend on it. Force a manual renewal against a non-production endpoint and confirm your client, monitoring, and on-call runbooks behave as expected, before the CA/Browser Forum’s shortened validity windows turn a failed renewal into a disruptive event for your organization.
Availability and pricing
ACME support in AWS Certificate Manager is available today in all commercial AWS Regions and will be available in AWS GovCloud (US), the China Regions, and the AWS European Sovereign Cloudpartitions at a later date. See the ACM pricing page for more information on ACME pricing.
Conclusion
The phased reduction in certificate validity can’t easily be solved without automation. ACME support in ACM gives you that automation through a standard protocol and standard tooling, while keeping the visibility and governance controls your security teams rely on from ACM.
If you have feedback about this post, submit comments in the Comments section below.
The collective thoughts of the interwebz
Manage Consent
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
