FFmpeg 8.0 released

Post Syndicated from corbet original https://lwn.net/Articles/1034813/

Version 8.0 of the FFmpeg
audio and video toolkit has been released.

Thanks to several delays, and modernization of our entire infrastructure,
this release ended up being one of our largest releases to date. In short,
its new features are:

  • Native decoders: APV, ProRes RAW, RealVideo 6.0, Sanyo LD-ADPCM, G.728
  • VVC decoder improvements: IBC, ACT, Palette Mode
  • Vulkan compute-based codecs: FFv1 (encode and decode), ProRes RAW (decode only)
  • Hardware accelerated decoding: Vulkan VP9, VAAPI VVC, OpenHarmony H264/5
  • Hardware accelerated encoding: Vulkan AV1, OpenHarmony H264/5
  • Formats: MCC, G.728, Whip, APV
  • Filters: colordetect, pad_cuda, scale_d3d11, Whisper, and others

Amazon Redshift Serverless at 4 RPUs: High-value analytics at low cost

Post Syndicated from Ricardo Serafim original https://aws.amazon.com/blogs/big-data/amazon-redshift-serverless-at-4-rpus-high-value-analytics-at-low-cost/

Organizations across industries struggle with the economics of data analytics. High entry costs, complex capacity planning, and unpredictable workload demands create barriers that prevent teams from accessing the insights they need. Small businesses abandon analytics initiatives due to prohibitive minimums, and enterprises overprovision resources for development environments, leading to inefficient spending.

Amazon Redshift Serverless now addresses these challenges with 4 RPU configurations, helping you get started with a lower base capacity that runs scalable analytics workloads beginning at $1.50 per hour. This new option transforms the economics of data analytics with the flexibility to scale up automatically based on workload demands. You only pay for the compute capacity you consume, calculated on a per-second basis.

With 64 GB of memory and support for up to 32 TB of managed storage, this lower entry point offering addresses several common customer needs, including development and test environments that maintain separate workloads at lower cost and production workloads with variable demand that need cost-effective scaling. The configuration is particularly useful for test and development environments, departmental data warehouses, periodic reporting workloads, gaming analytics, and data mesh architectures with unpredictable usage patterns. Organizations just starting with cloud analytics can use this low-cost option while getting access to enterprise features like automatic scaling, built-in security, and seamless data lake integration.In this post, we examine how this new sizing option makes Redshift Serverless accessible to smaller organizations while providing enterprises with cost-effective environments for development, testing, and variable workloads.

New 4 RPU minimum base capacity in Redshift Serverless

Redshift Serverless measures compute capacity using Redshift Processing Units (RPUs), where each RPU provides 16 GB of memory. With this new minimum base capacity, the 4 RPU configuration delivers a total of 64 GB of memory. It supports up to 32 TB of managed storage, with a maximum of 100 columns per table. The 4 RPU configuration is cost-efficient, and it’s designed for lighter workloads. When your workload requires additional resources, Redshift Serverless automatically scales up the compute capacity. After you have scaled beyond 4 RPUs, your data warehouse will continue using the higher RPU level to maintain consistent performance. This behavior provides workload stability while preserving the benefits of automatic scaling.

For workloads requiring more resources, such as tables with a large number of columns or higher concurrency requirements, you can choose higher base capacities ranging from 8 RPUs up to 1024 RPUs. This flexibility helps you start small and adjust your resources as your analytics requirements evolve.

Benefits of Redshift Serverless with 4 RPUs

This new feature offers the following benefits:

  • Cost-effective entry point – The new 4 RPU configuration is a low-cost option for cloud data warehousing, making enterprise-grade analytics accessible to organizations of various sizes, such as startups exploring their first data warehouse or established enterprises optimizing their analytics spending. For example, in the US East (N. Virginia) Region, the compute cost is $0.375 per RPU-hour. For a 4 RPU base capacity, this translates to $1.50 per hour of active workload time. Because you’re only charged when workloads are running, small-scale users can keep costs predictable and low. This configuration helps teams begin their analytics journey with minimal upfront commitment. Development teams can maintain dedicated environments for testing and experimentation without significant cost overhead.
  • Support for smaller datasets – With support for up to 32 TB of Redshift Managed Storage, the 4 RPU configuration is well-suited for smaller data warehouses. It can handle datasets ranging from a few gigabytes to tens of terabytes, making it ideal for startups, small businesses, or departments with limited data volumes.
  • Seamless integration with the AWS ecosystem – The 4 RPU configuration integrates seamlessly with other AWS services, such as Amazon Simple Storage Service (Amazon S3) for data lakes, AWS Glue for ETL (extract, transform, and load), and Amazon QuickSight for visualization. This makes it straightforward to build end-to-end analytics pipelines, even for smaller-scale projects. Additionally, Redshift data lake queries on external Amazon S3 data are included in the RPU billing, simplifying cost management.
  • Use case flexibility – The 4 RPU configuration proves valuable across numerous analytics scenarios. Development and testing environments benefit from cost-effective isolation, and departmental data warehouses can start small and scale as needed. Organizations running periodic reporting workloads or proof-of-concept projects can optimize costs by paying only for actual usage. Even small to medium-sized production workloads can use this configuration effectively.

Regardless of the use case, you can benefit from the full feature set of Redshift Serverless, including built-in security, data lake integration, and automated maintenance.

Use cases for Redshift Serverless with 4 RPU workgroups

The 4 RPU configuration is tailored for scenarios where lightweight compute resources suffice. The following are some practical use cases:

  • Small business analytics – Small businesses with limited data (less than 32 GB) can analyze sales, customer behavior, or operational metrics with cost-effective data warehouses. Running 10–20 daily ETL queries and occasional one-time queries remains cost-effective at this capacity.
  • Development and testing environments – The configuration is well-suited for development and test environments where full production resources aren’t needed. Data engineers can experiment with Redshift Serverless, prototype queries, or build proof-of-concept solutions without committing to higher RPU capacities. The 4 RPU configuration lowers the cost of continuous integration and delivery (CI/CD) testing of data pipelines. Teams can run automated integration tests and schema validations in isolated environments that mirror production systems while optimizing costs through per-second billing.
  • Analytics for startups – Startups can build robust product analytics capabilities without significant upfront investment. Teams can track customer behavior, feature adoption, and KPIs using familiar SQL queries, then connect business intelligence (BI) tools like Quicksight or Tableau for lightweight dashboarding.
  • Training and experimentation – Organizations can create dedicated sandbox environments for data analysts’ onboarding and experimentation with minimal budget impact. These environments are perfect for exploring analytics powered by large language models (LLMs), semantic layer development, or generative AI applications.
  • Data quality workflows – The feature efficiently supports scheduled jobs for data quality validation, checking data freshness, integrity, and conformance without dedicating high-capacity environments to routine QA tasks.
  • Enterprise team enablement – Large organizations can implement decentralized data warehousing strategies. Each department can operate its data warehouse aligned with specific needs and budgets, enabling department-level chargeback models.
  • Environment isolation – Organizations can create dedicated workgroups per environment (development, test, QA, UAT), providing complete isolation without sharing compute resources or risking cross-environment interference.
  • Data mesh architecture – Domain teams can operate independently while maintaining cost-efficiency. Each domain runs its workgroup for lightweight transformations, domain-specific marts, and KPI calculations. It offers a flexible sizing option in a data mesh architecture.
  • Event-driven analytics – Well-suited for short-lived or event-triggered analytics tasks. Organizations can programmatically create workgroups through APIs for A/B test analysis, campaign performance summaries, or machine learning (ML) pipeline validation.
  • Low-volume one-time reporting – Organizations with infrequent or lightweight reporting needs, such as monthly financial summaries or dashboard refreshes, can use 4 RPUs to minimize costs while maintaining performance.

Cost considerations and best practices

Although the 4 RPU configuration is cost-effective, there are a few considerations to keep in mind to optimize expenses:

  • Billing – Redshift Serverless bills on a per-second basis with a 60-second minimum per query. For very short queries (such as subsecond), this can inflate costs. To mitigate this, batch queries where possible to maximize resource utilization within the 60-second window. For more information, see Amazon Redshift pricing.
  • Set usage limits – Use the Redshift Serverless console to set maximum RPU-hour limits (daily, weekly, or monthly) to prevent unexpected costs. You can configure alerts or automatically turn off queries when limits are reached. To learn more, see Setting usage limits, including setting RPU limits.
  • Monitor with system views – Query the SYS_SERVERLESS_USAGE system table to track RPU consumption and estimate query costs. For example, you can calculate daily costs by aggregating charged seconds and multiplying by the RPU rate.
  • Close transactions – Make sure transactions are explicitly closed (using COMMIT or ROLLBACK) to avoid idle sessions consuming RPUs, which can lead to unnecessary charges.

The following is a practical example for a 4 RPU workgroup in US East (N. Virginia) at $0.375/RPU-hour for a scenario of a 10-minute query running daily: This is compute costs only. Primary storage capacity is billed as Redshift Managed Storage (RMS).

  • Workload duration: 10 minutes (600 seconds)
  • Cost: (600 seconds / 3600 seconds) × 4 RPUs × $0.375 = $0.25
  • Monthly cost (30 days): $0.25 × 30 = $7.50

Performance considerations

Although the 4 RPU configuration is cost-efficient, it’s designed for lighter workloads. For complex queries or datasets exceeding 32 TB, you must set up 8 RPUs to 24 RPUs to support up to 128 TB of storage. For more than 128 TB, you need 32 RPUs or more. If query performance is a priority, consider increasing the base capacity or enabling AI-driven scaling and optimization to optimize resources dynamically. Benchmark tests suggest that higher RPUs (such as 32 RPUs) significantly improve performance for complex queries. However, for simpler tasks, 4 RPUs deliver adequate throughput.

To monitor performance, use the Redshift Serverless console or CloudWatch metrics like ComputeCapacity and ComputeSeconds. The SYS_QUERY_HISTORY table can also help analyze query runtimes and identify bottlenecks.

Conclusion

Redshift Serverless with 4 RPU represents a significant step forward in making enterprise-grade analytics cheaper and accessible to organizations of different sizes, such as a startup building its first analytics system, a development team looking to optimize testing environments, or an enterprise implementing a data mesh architecture. This new configuration combines the power and flexibility of Redshift Serverless with a cost-effective entry point, so teams can start small and scale seamlessly as their needs grow. The ability to begin with minimal commitment while maintaining access to advanced features like automatic scaling, built-in security, and seamless data lake integration makes this a compelling option for modern data analytics workloads. Combined with pay-per-second billing and intelligent resource management, Redshift Serverless with 4 RPU delivers the ideal balance of cost-efficiency and performance.

To get started with cost-effective analytics, visit the AWS Management Console to create your Redshift Serverless workgroup with 4 RPUs. For more information, refer to the Amazon Redshift Serverless Management Guide or Amazon Redshift best practices. Plan your analytics budget effectively using the AWS Pricing Calculator to estimate costs based on your specific workload patterns, or contact your AWS account team to discuss your particular use case.


About the authors

Ricardo Serafim

Ricardo Serafim

Ricardo is a Senior Analytics Specialist Solutions Architect at AWS. He has been helping companies with Data Warehouse solutions since 2007.

Ashish Agrawal

Ashish Agrawal

Ashish is a Principal Product Manager with Amazon Redshift, building cloud-based data warehouses and analytics cloud services. Ashish has over 25 years of experience in IT. Ashish has expertise in data warehouses, data lakes, and platform as a service. Ashish has been a speaker at worldwide technical conferences.

Andre Hass

Andre Hass

Andre is a Senior Technical Account Manager at AWS, specialized in AWS Data Analytics workloads. With more than 20 years of experience in databases and data analytics, he helps customers optimize their data solutions and navigate complex technical challenges. When not immersed in the world of data, Andre can be found pursuing his passion for outdoor adventures. He enjoys camping, hiking, and exploring new destinations with his family on weekends or whenever an opportunity arises.

I’m Spending the Year at the Munk School

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2025/08/im-spending-the-year-at-the-munk-school.html

This academic year, I am taking a sabbatical from the Kennedy School and Harvard University. (It’s not a real sabbatical—I’m just an adjunct—but it’s the same idea.) I will be spending the Fall 2025 and Spring 2026 semesters at the Munk School at the University of Toronto.

I will be organizing a reading group on AI security in the fall. I will be teaching my cybersecurity policy class in the Spring. I will be working with Citizen Lab, the Law School, and the Schwartz Reisman Institute. And I will be enjoying all the multicultural offerings of Toronto.

It’s all pretty exciting.

Cloud Storage Myths Debunked, Part Three: Onboarding Specialized Providers Is Too Hard

Post Syndicated from David Johnson original https://www.backblaze.com/blog/cloud-storage-myths-debunked-part-three-onboarding-specialized-providers-is-too-hard/

An illustration of computer screens.

Here’s a myth we hear again and again:

Integrating a new storage provider is too complicated. Migrating data, retraining teams, and reconfiguring tools will take too long and create too much risk.

It’s understandable. Data migrations have a reputation for being messy and disruptive. And let’s be honest, nobody wants to babysit infrastructure when there are products to build. For many teams, just the thought of switching cloud providers feels like a detour they don’t have time for.

But in reality, that fear is often bigger than the actual lift. If your workflows already use standard tooling like S3-compatible APIs, switching to a specialized provider is more like a well-marked exit than a hard left turn.

This is the third post in our series debunking persistent myths about cloud storage—check out the first and second posts—and why a best-of-breed, interoperable approach is actually less disruptive than sticking with legacy hyperscaler models.

New Cloud Native Times Call for New Cloud Storage Approaches

Learn more about how the open cloud supports faster development, improved workflows, and reduced cost complexity in our free ebook, “New Cloud Native Times Call for New Cloud Storage Approaches.”

Get the Ebook

Migration anxiety vs. reality

“Storage migration” can sound like it requires weeks of planning and an army of engineers. But if your apps are already using S3-compatible workflows, most of the heavy lifting is already done.

If you know S3, you’re already ready

Many specialized storage providers now support S3-compatible APIs, allowing teams to keep the tools, scripts, and services they already know, such as Terraform, Kubernetes, ArgoCD, boto3, and MinIO.

And because your teams are already familiar with the S3 API and related tools, retraining isn’t a hurdle. The same skills, scripts, and automation frameworks carry forward, keeping onboarding time minimal. In fact, most teams are surprised by how little they need to change to get started.

That means:

  • No need to learn a new SDK or storage interface
  • No retraining your DevOps team
  • No rewriting automation pipelines or batch jobs

In most cases, all it takes is updating your endpoint URL and refreshing credentials. The mental model stays the same, the tools stay the same, and your workflows continue as-is.

You don’t need to rip and replace

Downtime concerns are one of the biggest sources of hesitation when switching providers. But in practice, migrations to S3-compatible cloud storage providers rarely require full cutovers or risky, all-or-nothing switchovers. With a bit of planning, most teams handle migrations incrementally:

  • Start by migrating lower-risk datasets, such as backups or archives.
  • Validate configurations and permissions as data lands in the new system.
  • Slowly expand to production datasets as confidence grows.

Better yet, you don’t have to move everything at once. Many teams adopt phased transitions, running some buckets side-by-side or writing to both systems during the handoff to minimize risk. With a bit of planning and the right migration tools and guidance, you can keep operations stable while gradually shifting workloads at a comfortable pace.

Metadata isn’t a blocker

Migrating files without metadata continuity can break downstream systems, especially if your applications rely on timestamps or version tracking.

Fear not. S3-compatible cloud storage providers can preserve metadata during migration, including timestamps. That means your historical data stays intact and compliant with internal policies or regulatory needs, and you won’t need to reset or alter your data management policies.

Moving isn’t the risk. Staying locked in is.

Let’s flip the narrative. The real risk isn’t switching; it’s staying stuck.

Major cloud provider ecosystems are designed for lock-in. The deeper you go, the harder it becomes to leave. Features that look like conveniences, such as integrated IAM policies, tiered storage, and custom APIs, often become entanglements over time.

Each of these layers is built to reinforce reliance:

  • IAM rules tie access tightly to the provider’s own tooling.
  • Tiered storage creates dependencies on lifecycle rules and retrieval thresholds. 
  • Custom APIs mean even basic storage functions can require provider-specific logic.

And as you expand your usage—adding compute, networking, and security services—everything becomes interdependent. What starts as convenience evolves into constraint. Even small changes to your stack can trigger cascading reviews, system audits, or full reconfigurations.

The result? Innovation slows. Costs creep up. Flexibility disappears.

With a specialized provider, you break that cycle.

Specialized Doesn’t Mean Complicated

Specialized storage doesn’t complicate onboarding. It streamlines it. Solutions like Backblaze B2 are purpose-built to make this shift smooth and sustainable, without the trade-offs or surprises you might expect from switching providers.

  • S3 compatibility allows for seamless integration with the tools and workflows your team already uses.
  • Granular control means you can choose the tools and providers that work best for your architecture, not the ones bundled into a vendor’s ecosystem.
  • Metadata continuity is supported through features like custom upload timestamps, preserving file context during migration.
  • Transparent pricing ensures there are no hidden egress fees, transaction charges, or retention penalties to catch you off guard.
  • Hands-on support helps you plan, validate, and scale your migration with confidence and minimal disruption.

Breaking out of a single-vendor ecosystem may feel intimidating, but it’s often the fastest way to simplify operations, improve performance, and regain control over your cloud strategy.

The best part? Once you’ve made the move, you’re free to experiment. Multi-cloud strategies become more accessible. Your architecture becomes more modular. And your team can focus on building, not babysitting infrastructure.

Next Up: In the final post in this series, we’ll tackle Myth #4: Managing multiple clouds is complicated. (Spoiler: It doesn’t have to be.)

Want to dig even deeper? Download the full whitepaper New Cloud-Native Times Call for New Cloud Storage Approaches.

The post Cloud Storage Myths Debunked, Part Three: Onboarding Specialized Providers Is Too Hard appeared first on Backblaze Blog | Cloud Storage & Cloud Backup

[$] The “impossibly small” Microdot web framework

Post Syndicated from jake original https://lwn.net/Articles/1034121/

The Microdot
web framework is quite small, as its name would imply; it supports both
standard CPython and MicroPython,
so it can be used on systems ranging from internet-of-things (IoT) devices
all the way up to large, cloudy servers. It was developed by Miguel
Grinberg, who gave a presentation about it at EuroPython 2025. His name
may sound familiar from his well-known Flask
Mega-Tutorial
, which has introduced many to the Flask lightweight Python-based
web framework. It should come as no surprise, then, that Microdot is
inspired by its rather larger cousin, so Flask enthusiasts will find much
to like in Microdot—and will come up to speed quickly should their needs turn
toward smaller systems.

Динко, неговите камиони и една спяща държавна машина две години по-късно

Post Syndicated from Боян Юруков original https://yurukov.net/blog/2025/dinko/

Преди две години – на 26 юли 2023-та в 14 часа на магистрала Тракия се обръща камион пълен със скрап. По чудо няма пострадали други коли. Бързо се разбира, че камионът е на Господин Вълев Вълев по-известен като Динко. В първите 24 часа след случая медиите основно тикаха микрофони пред лицето му и чакаха някоя институция да каже нещо. Тъй като никой не се зае, аз изрових всичко налично от публични източници и обобщих нарушенията във Facebook. Отне ми 20 мин. на телефона. След това ги описах подробно в статия, която беше прекопирана и цитирана от доста медии. Оказа се, че не само катастрофиралия, но и много от камионите му нямат технически преглед, гражданска и/или са пътували без винетка, което обикновено е сигнал за по-сериозни нарушения.

17 часа след поста ми във Facebook и два часа след статията в блога Министерството на транспорта и съобщенията излезе с новина, че започват проверка на фирмите на Динко и тази, към която е регистриран камиона. Последваха реакции и информация, че ще му отнемат лиценза заради множество нарушения.

В следващите две години не се чу нищо повече за случая. Динко излезе няколко пъти в медиите за други неща, но не стана ясно нито какво се случва с проверката, нито дали има наистина последствия. С оглед на постоянно изникващите проблеми по пътя, данните за жертви, популистки недомислени изказвания при поредния инцидент и призивите за реални действия реших да проверя къде е лицето Вълев и какво се случва с бизнеса му. Търся отговор на въпроса какво са правили институциите, взели ли са мерки, тези мерки имали ли са ефект и дали поведението на лицето се е променило.

Мисля, че знаете отговора, но нека започнем поред.

Камионите на Динко

В статии от 2023-та се говори, че Вълев има 12 камиона към фирмата си „Бозав“ ЕООД. Обърналият се камион с номер A1256HH е на „Логтранс БГ“ ЕООД. към днешна дата виждаме 10 камиона с данни, че на фирмата на Вълев и още 6, които най-малкото използва, а вероятно също са негова собственост. Така стават 16 камиона с различни конфигурации.

9 от тях виждаме в разрешителни за извършване на пътна помощ актуално към настоящия момент още два се намират в разрешителното от 2022. От първите 9 само 3 са с актуален технически преглед. Един от другите 6 му изтече техническия преглед вчера, а други три не са с ГТП от поне три месеца. Въпреки това, Вълев продължава да им плаща гражданска отговорност и присъстват в регистъра за извършване на пътна помощ.

Два от камионите извършващи пътна дейност заедно с още 7 се намират в регистрациите на „Бозав“ ЕООД за превоз на отпадъци от РИОСВ Стара Загора за цялата страна. Беше получил такова три месеца преди инцидента, след което около седем месеца нямаше разрешение. Издали са му обаче ново през февруари 2024-та. В него се споменават шест камиона, включително катастрофиралия през 2023-та – A1256HH. Към онзи момент той няма нито гражданска отговорност, нито ГТП. Това не пречи да бъде включен във всички разрешения след това – през май и октомври 2024-та и последно януари 2025-та. Други камиони се сменят в тези разрешения, но последното от началото на тази година е интересно, защото от 7 камиона споменати в него, три са без валиден технически преглед и продължават да са независимо, че им се плаща гражданска.

Фирмата на Вълев има и разрешения за третиране на отпадъци. Обновяват му го месец преди инцидента през 2023-та. Въпреки заявките на институциите, че ще му отнемат лицензите за нарушения, получава ново само два месеца след катастрофата – на 20-ти септември 2023-та. Следващото е издадено на 15-ти януари 2024-та. Тук интересното е, че на 25-ти април излиза решение на РИОСВ Стара Загора за отнемане на издаденото през януари и от тогава няма издадено следващо.

Може да ги пренася обаче да пренася отпадъци из цялата страна, макар към днешна дата да е законно да го прави само с два от седемте в разрешението му. Останалите, аналогично на катастрофиралия през 2023-та камион, нямат сведения да са технически изправни.

Пътните такси на Динко

Доколкото задълженията към хазната да не влияят пряко на пътните инциденти, сведения са за отношението на човек и бизнес към закона и ефектът на някакви наложени санкции. В статията ми преди две години писах, че откривам над 11 хиляди лева натрупани глоби към тол агенцията от 2021-ва насам. Към днешна дата тази сума надхвърля 24 хиляди лева. Повечето от натрупаните от тогава са по магистралата за Бургас при Ветрен от камион Y4443AH през септември 2024-та.

Горе виждате обновена карта къде са глобявани 13 камиона използвани от Вълев. В червено са 17-те глоби от 2023-та насам. В тъмно червено е мястото с най-много глоби – 13. В зелено е обновен списък от тези преди инцидента преди две години с други негови камиони, които открих.

Ще се фокусирам върху конкретен случай. Споменатият по-горе камион Y4443AH има наложена санкция за каране без маршрутна карта или валидна тол декларация на 24-ти октомври 2024-та. Глобата е 540 лв., които според сайта на тол агенцията не са платени. Притеснителното обаче е, че техническият преглед на този камион е изтекъл пет дни по-рано – на 19-ти октомври. Глобата е сведение, че се е движил по магистралата незаконно подобно на катасофиралия камион през 2023-та. Нещо повече – вероятно е превозвал пак по същия начин отпадъци, тъй като е включен в разрешителното за това три седмици по-рано – на 3-ти октомври.

Делата на Динко

Открих 34 дела, в които фирмата на Вълев е страна. В 24 от тях той обжалва решения на институции или е страна. 16 от делата са заведени след инцидента през 2023-та.

6 от тях се отнасят до негово заведение в с. Прилеп, Сингуляре. Четири от тях са нарушения в трудовото законодателство – работа без договор или неизплащане на двойна заплата през празниците. Две са за изхвърляне на отпадни води от заведението, включително незаконно изграждане на канал за целта към близката река. В едно документите към едно от делата в Районен съд Карнобат се споменава и незаконно строителство на помещение за хазарт към заведението. Не може да знаем дали е имало слот машини или друг хазарт, но фирмата на Вълев, която е собственик на земята и заведението, не е в никой от регистрите за хазартна дейност.

Останалите 10 дела са заведени от Вълев като обжалва актове. Някои от края на 2023-та са за безопасност на труда и трудовото законодателство. Има няколко за нарушения на ЗУО – липсваща ограда, обезопасяване, незаконни площадки и горене на кабели. Всъщност от последните са и потвърдени най-големите глоби – общо 24 хиляди лв.

5 от делата падат в съда – например за незаконна пътна връзка. Според съда е бил глобен вече през 2019-та за това и в новия акт от края на 2023-та не е упоменато дали пътят е бил направен отново или просто не са го накарали да махне стария. Други актове падат, защото според съда в тях се говори за множество различни нарушения, за които трябвало да се съставят отделни актове. При някои обжалвания намаляват глобите.

Общо след делата заведени от Вълев между октомври 2023 и януари 2025-та са потвърдени глоби за 30000 лв. Осъден е да плати адвокатски разходи за 1270 лв., но и различни институции са осъдени да му платят 6780 лв. по хонорари. Така нето за над година трябва да плати на държавата около 24490 лв. Толкова дължи и в глоби за липса на тол такси и винетки. Не знаем дали е имал и други актове или с тези дела се изчерпва всичко около него.

Институциите и Динко

Докато горе ставаше дума за съдебните дела, има обществена информация и за държанието на Вълев. Преди само три седмици излезе запис за опасно неправилно изпреварване с джипа му, при което за малко да смаже моторист. Цената на пикапа е сериозна и дава да се разбере колко смехотворни са глобите, които му се налагат за постоянни нарушения на трудовото и екологичното законодателство, както и безопасността на труда и движението по пътищата. За същия номер, впрочем има глоби за липса на винетка през 2021 и 2022-ра.

През 2024-та пък излезе информация, че го разследват за участие в престъпна група за изнудвания. Не стана ясно дали е имало резултат от това обвинение или и това разследване е потънало или оставено на трупчета за удобен момент, например при следващ инцидент подобен на онзи през 2023-та. Скоро след инцидента през 2023-та, впрочем, го бяха арестували за агресия спрямо просещ човек. Не стана ясно дали е бил съден или дори глобен – само, че излиза срещу гаранция.

Макар да нямаме информация за всички проверки и актове към бизнеса и личността на Вълев, от заведените дела и описаните факти в протоколи и решения е видно, че институциите са се активизирали сериозно след инцидента. Част от тези актове падат, тъй като не са оформени правилно, санкциите са били прекалени или нарушенията са били отколешни. Забелязва се обаче един вид кампанийно стоварване на регулаторния чук върху Динко.

Какво от това?

От останалата публична информация обаче виждаме, че това не му е повлияло по никакъв начин. Надали има изненадани, но в крайна сметка това е идеята на глобите и проверките. Не помага фактът, че изглежда проверки преди инцидента, както и година след него явно няма. Нарушения като незаконно строителство, горене на отпадъци и проблеми с безопасността и трудовите правоотношения са масови и отколешни и това изглежда не е изключение за бизнеса на Вълев. Достатъчно е да се сниши за известно време и да продължава по старо му. Поне до следващия обърнат тир и навярно размазана жертва по пътя.

Направих си труда да извлека цялата тази информация не за да говоря за една банално патологична личност, а за изводите, които може да си направим за институциите ни и как реагираме на тази патология, нарушения и откровени престъпления. След като прахта улегне, катастрофата се разчисти, дъжда измие кръвта, следват няколко проверки – бланкетни по думите на един съдия, няколко акта, няколко дела, няколко шумни заглавия, няколко нескопосани обвинения връщани на прокурорите години наред за фрапантни грешки и забравяме до следващия път.

Това стана с Динко и оная катастрофа. Това продължава да прави той до ден днешен като лакмус за институциите ни. Това се случва с парашутите, катастрофите, болниците ни, корупцията, обществените поръчки и прочие. Това с тира му беше просто по-очевидно. Всичко това е неизбежен ефект от скритото кадруване в регулаторите, прокуратурата и съдебната система, овладяването им до превръщане в бухалка, а не нещо, което помага на хората. Затова Пеевски и Борисов не искаха конкурси за шефове на регулаторите, спирането на командироването на прокурори и промени във ВСС. Затова беше блокиран одита и създаване на публична, проследима и истински случайно разпределение на делата. Затова Сарафов е все още главен прокурор, а Кошлуков е в БНТ.

Защото не е важно дали регулаторът пише неграмотен акт, прокурорът – нескопосано и често противозаконно обвинение и съдията – решение противоречащо на европейски закони, конституция и десетилетия практика. Важното им е тези да се пишат при команда когато някой трябва да се удари. Това се случва масово в местната власт из страната. Случва се отдавна в бизнеси особено в малките населени места. Затова хора като Динко просъществува и се държи така – защото той е наш проблем и утре той или поредния като него ще убие нашите деца. Овладените регулатори и прокуратура са заети са обслужват други.

The post Динко, неговите камиони и една спяща държавна машина две години по-късно first appeared on Блогът на Юруков.

MoQ: Refactoring the Internet’s real-time media stack

Post Syndicated from Mike English original https://blog.cloudflare.com/moq/

For over two decades, we’ve built real-time communication on the Internet using a patchwork of specialized tools. RTMP gave us ingest. HLS and DASH gave us scale. WebRTC gave us interactivity. Each solved a specific problem for its time, and together they power the global streaming ecosystem we rely on today.

But using them together in 2025 feels like building a modern application with tools from different eras. The seams are starting to show—in complexity, in latency, and in the flexibility needed for the next generation of applications, from sub-second live auctions to massive interactive events. We’re often forced to make painful trade-offs between latency, scale, and operational complexity.

Today Cloudflare is launching the first Media over QUIC (MoQ) relay network, running on every Cloudflare server in datacenters in 330+ cities. MoQ is an open protocol being developed at the IETF by engineers from across the industry—not a proprietary Cloudflare technology. MoQ combines the low-latency interactivity of WebRTC, the scalability of HLS/DASH, and the simplicity of a single architecture, all built on a modern transport layer. We’re joining Meta, Google, Cisco, and others in building implementations that work seamlessly together, creating a shared foundation for the next generation of real-time applications on the Internet.

An evolutionary ladder of compromise

To understand the promise of MoQ, we first have to appreciate the history that led us here—a journey defined by a series of architectural compromises where solving one problem inevitably created another.

The RTMP era: Conquering latency, compromising on scale

In the early 2000s, RTMP (Real-Time Messaging Protocol) was a breakthrough. It solved the frustrating “download and wait” experience of early video playback on the web by creating a persistent, stateful TCP connection between a Flash client and a server. This enabled low-latency streaming (2-5 seconds), powering the first wave of live platforms like Justin.tv (which later became Twitch).

But its strength was its weakness. That stateful connection, which had to be maintained for every viewer, was architecturally hostile to scale. It required expensive, specialized media servers and couldn’t use the commodity HTTP-based Content Delivery Networks (CDNs) that were beginning to power the rest of the web. Its reliance on TCP also meant that a single lost packet could freeze the entire stream—a phenomenon known as head-of-line blocking—creating jarring latency spikes. The industry retained RTMP for the “first mile” from the camera to servers (ingest), but a new solution was needed for the “last mile” from servers to your screen (delivery).

The HLS & DASH era: Solving for scale, compromising on latency

The catalyst for the next era was the iPhone’s rejection of Flash. In response, Apple created HLS (HTTP Live Streaming). HLS, and its open-standard counterpart MPEG-DASH abandoned stateful connections and treated video as a sequence of small, static files delivered over standard HTTP.

This enabled much greater scalability. By moving to the interoperable open standard of HTTP for the underlying transport, video could now be distributed by any web server and cached by global CDNs, allowing platforms to reach millions of viewers reliably and relatively inexpensively. The compromise? A significant trade-off in latency. To ensure smooth playback, players needed to buffer at least three video segments before starting. With segment durations of 6-10 seconds, this baked 15-30 seconds of latency directly into the architecture.

While extensions like Low-Latency HLS (LL-HLS) have more recently emerged to achieve latencies in the 3-second range, they remain complex patches fighting against the protocol’s fundamental design. These extensions introduce a layer of stateful, real-time communication—using clever workarounds like holding playlist requests open—that ultimately strain the stateless request-response model central to HTTP’s scalability and composability.

The WebRTC Era: Conquering conversational latency, compromising on architecture

In parallel, WebRTC (Web Real-Time Communication) emerged to solve a different problem: plugin-free, two-way conversational video with sub-500ms latency within a browser. It worked by creating direct peer-to-peer (P2P) media paths, removing central servers from the equation.

But this P2P model is fundamentally at odds with broadcast scale. In a mesh network, the number of connections grows quadratically with each new participant (the “N-squared problem”). For more than a handful of users, the model collapses under the weight of its own complexity. To work around this, the industry developed server-based topologies like the Selective Forwarding Unit (SFU) and Multipoint Control Unit (MCU). These are effective but require building what is essentially a private, stateful, real-time CDN—a complex and expensive undertaking that is not standardized across infrastructure providers.

This journey has left us with a fragmented landscape of specialized, non-interoperable silos, forcing developers to stitch together multiple protocols and accept a painful three-way tension between latency, scale, and complexity.

Introducing MoQ

This is the context into which Media over QUIC (MoQ) emerges. It’s not just another protocol; it’s a new design philosophy built from the ground up to resolve this historical trilemma. Born out of an open, community-driven effort at the IETF, MoQ aims to be a foundational Internet technology, not a proprietary product.

Its promise is to unify the disparate worlds of streaming by delivering:

  1. Sub-second latency at broadcast scale: Combining the latency of WebRTC with the scale of HLS/DASH and the simplicity of RTMP.

  2. Architectural simplicity: Creating a single, flexible protocol for ingest, distribution, and interactive use cases, eliminating the need to transcode between different technologies.

  3. Transport efficiency: Building on QUIC, a UDP based protocol to eliminate bottlenecks like TCP head-of-line blocking.

The initial focus was “Media” over QUIC, but the core concepts—named tracks of timed, ordered, but independent data—are so flexible that the working group is now simply calling the protocol “MoQ.” The name reflects the power of the abstraction: it’s a generic transport for any real-time data that needs to be delivered efficiently and at scale.

MoQ is now generic enough that it’s a data fanout or pub/sub system, for everything from audio/video (high bandwidth data) to sports score updates (low bandwidth data).

A deep dive into the MoQ protocol stack

MoQ’s elegance comes from solving the right problem at the right layer. Let’s build up from the foundation to see how it achieves sub-second latency at scale.

The choice of QUIC as MoQ’s foundation isn’t arbitrary—it addresses issues that have plagued streaming protocols for decades.

By building on QUIC (the transport protocol that also powers HTTP/3), MoQ solves some key streaming problems:

  • No head-of-line blocking: Unlike TCP where one lost packet blocks everything behind it, QUIC streams are independent. A lost packet on one stream (e.g., an audio track) doesn’t block another (e.g., the main video track). This alone eliminates the stuttering that plagued RTMP.

  • Connection migration: When your device switches from Wi-Fi to cellular mid-stream, the connection seamlessly migrates without interruption—no rebuffering, no reconnection.

  • Fast connection establishment: QUIC’s 0-RTT resumption means returning viewers can start playing instantly.

  • Baked-in, mandatory encryption: All QUIC connections are encrypted by default with TLS 1.3.

The core innovation: Publish/subscribe for media

With QUIC solving transport issues, MoQ introduces its key innovation: treating media as subscribable tracks in a publish/subscribe system. But unlike traditional pub/sub, this is designed specifically for real-time media at CDN scale.

Instead of complex session management (WebRTC) or file-based chunking (HLS), MoQ lets publishers announce named tracks of media that subscribers can request. A relay network handles the distribution without needing to understand the media itself.

How MoQ organizes media: The data model

Before we see how media flows through the network, let’s understand how MoQ structures it. MoQ organizes data in a hierarchy:

  • Tracks: Named streams of media, like “video-1080p” or “audio-english”. Subscribers request specific tracks by name.

  • Groups: Independently decodable chunks of a track. For video, this typically means a GOP (Group of Pictures) starting with a keyframe. New subscribers can join at any Group boundary.

  • Objects: The actual packets sent on the wire. Each Object belongs to a Track and has a position within a Group.

This simple hierarchy enables two capabilities:

  1. Subscribers can start playback at Group boundaries without waiting for the next keyframe

  2. Relays can forward Objects without parsing or understanding the media format

The network architecture: From publisher to subscriber

MoQ’s network components are also simple:

  • Publishers: Announce track namespaces and send Objects

  • Subscribers: Request specific tracks by name

  • Relays: Connect publishers to subscribers by forwarding immutable Objects without parsing or transcoding the media

A Relay acts as a subscriber to receive tracks from upstream (like the original publisher) and simultaneously acts as a publisher to forward those same tracks downstream. This model is the key to MoQ’s scalability: one upstream subscription can fan out to serve thousands of downstream viewers.

The MoQ Stack


MoQ’s architecture can be understood as three distinct layers, each with a clear job:

  1. The Transport Foundation (QUIC or WebTransport): This is the modern foundation upon which everything is built. MoQT can run directly over raw QUIC, which is ideal for native applications, or over WebTransport, which is required for use in a web browser. Crucially, the WebTransport protocol and its corresponding W3C browser API make QUIC’s multiplexed reliable streams and unreliable datagrams directly accessible to browser applications. This is a game-changer. Protocols like SRT may be efficient, but their lack of native browser support relegates them to ingest-only roles. WebTransport gives MoQ first-class citizenship on the web, making it suitable for both ingest and massive-scale distribution directly to clients.

  2. The MoQT Layer: Sitting on top of QUIC (or WebTransport), the MoQT layer provides the signaling and structure for a publish-subscribe system. This is the primary focus of the IETF working group. It defines the core control messages—like ANNOUNCE, and SUBSCRIBE—and the basic data model we just covered. MoQT itself is intentionally spartan; it doesn’t know or care if the data it’s moving is H.264 video, Opus audio, or game state updates.

  3. The Streaming Format Layer: This is where media-specific logic lives. A streaming format defines things like manifests, codec metadata, and packaging rules.
    WARP is one such format being developed alongside MoQT at the IETF, but it isn’t the only one. Another standards body, like DASH-IF, could define a CMAF-based streaming format over MoQT. A company that controls both original publisher and end subscriber can develop its own proprietary streaming format to experiment with new codecs or delivery mechanisms without being constrained by the transport protocol.

This separation of layers is why different organizations can build interoperable implementations while still innovating at the streaming format layer.

End-to-End Data Flow

Now that we understand the architecture and the data model, let’s walk through how these pieces come together to deliver a stream. The protocol is flexible, but a typical broadcast flow relies on the ANNOUNCE and SUBSCRIBE messages to establish a data path from a publisher to a subscriber through the relay network.


Here is a step-by-step breakdown of what happens in this flow:

  1. Initiating Connections: The process begins when the endpoints, acting as clients, connect to the relay network. The Original Publisher initiates a connection with its nearest relay (we’ll call it Relay A). Separately, an End Subscriber initiates a connection with its own local relay (Relay B). These endpoints perform a SETUP handshake with their respective relays to establish a MoQ session and declare supported parameters.

  2. Announcing a Namespace: To make its content discoverable, the Publisher sends an ANNOUNCE message to Relay A. This message declares that the publisher is the authoritative source for a given track namespace. Relay A receives this and registers in a shared control plane (a conceptual database) that it is now a source for this namespace within the network.

  3. Subscribing to a Track: When the End Subscriber wants to receive media, it sends a SUBSCRIBE message to its relay, Relay B. This message is a request for a specific track name within a specific track namespace.

  4. Connecting the Relays: Relay B receives the SUBSCRIBE request and queries the control plane. It looks up the requested namespace and discovers that Relay A is the source. Relay B then initiates a session with Relay A (if it doesn’t already have one) and forwards the SUBSCRIBE request upstream.

  5. Completing the Path and Forwarding Objects: Relay A, having received the subscription request from Relay B, forwards it to the Original Publisher. With the full path now established, the Publisher begins sending the Objects for the requested track. The Objects flow from the Publisher to Relay A, which forwards them to Relay B, which in turn forwards them to the End Subscriber. If another subscriber connects to Relay B and requests the same track, Relay B can immediately start sending them the Objects without needing to create a new upstream subscription.

An Alternative Flow: The PUBLISH Model


More recent drafts of the MoQ specification have introduced an alternative, push-based model using a PUBLISH message. In this flow, a publisher can effectively ask for permission to send a track’s objects to a relay without waiting for a SUBSCRIBE request. The publisher sends a PUBLISH message, and the relay’s PUBLISH_OK response indicates whether it will accept the objects. This is particularly useful for ingest scenarios, where a publisher wants to send its stream to an entry point in the network immediately, ensuring the media is available the instant the first subscriber connects.

Advanced capabilities: Prioritization and congestion control

MoQ’s benefits really shine when networks get congested. MoQ includes mechanisms for handling the reality of network traffic. One such mechanism is Subgroups.

Subgroups are subdivisions within a Group that effectively map directly to the underlying QUIC streams. All Objects within the same Subgroup are generally sent on the same QUIC stream, guaranteeing their delivery order. Subgroup numbering also presents an opportunity to encode prioritization: within a Group, lower-numbered Subgroups are considered higher priority. 

This enables intelligent quality degradation, especially with layered codecs (e.g. SVC):

  • Subgroup 0: Base video layer (360p) – must deliver

  • Subgroup 1: Enhancement to 720p – deliver if bandwidth allows

  • Subgroup 2: Enhancement to 1080p – first to drop under congestion

When a relay detects congestion, it can drop Objects from higher-numbered Subgroups, preserving the base layer. Viewers see reduced quality instead of buffering.

The MoQ specification defines a scheduling algorithm that determines the order for all objects that are “ready to send.” When a relay has multiple objects ready, it prioritizes them first by group order (ascending or descending) and then, within a group, by subgroup id. Our implementation supports the group order preference, which can be useful for low-latency broadcasts. If a viewer falls behind and its subscription uses descending group order, the relay prioritizes sending Objects from the newest “live” Group, potentially canceling unsent Objects from older Groups. This can help viewers catch up to the live edge quickly, a highly desirable feature for many interactive streaming use cases. The optimal strategies for using these features to improve QoE for specific use cases are still an open research question. We invite developers and researchers to use our network to experiment and help find the answers.

Implementation: building the Cloudflare MoQ relay

Theory is one thing; implementation is another. To validate the protocol and understand its real-world challenges, we’ve been building one of the first global MoQ relay networks. Cloudflare’s network, which places compute and logic at the edge, is very well suited for this.

Our architecture connects the abstract concepts of MoQ to the Cloudflare stack. In our deep dive, we mentioned that when a publisher ANNOUNCEs a namespace, relays need to register this availability in a “shared control plane” so that SUBSCRIBE requests can be routed correctly. For this critical piece of state management, we use Durable Objects.

When a publisher announces a new namespace to a relay in, say, London, that relay uses a Durable Object—our strongly consistent, single-threaded storage solution—to record that this namespace is now available at that specific location. When a subscriber in Paris wants a track from that namespace, the network can query this distributed state to find the nearest source and route the SUBSCRIBE request accordingly. This architecture builds upon the technology we developed for Cloudflare’s real-time services and provides a solution to the challenge of state management at a global scale.

An Evolving Specification

Building on a new protocol in the open means implementing against a moving target. To get MoQ into the hands of the community, we made a deliberate trade-off: our current relay implementation is based on a subset of the features defined in draft-ietf-moq-transport-07. This version became a de facto target for interoperability among several open-source projects and pausing there allowed us to put effort towards other aspects of deploying our relay network.

This draft of the protocol makes a distinction between accessing “past” and “future” content. SUBSCRIBE is used to receive future objects for a track as they arrive—like tuning into a live broadcast to get everything from that moment forward. In contrast, FETCH provides a mechanism for accessing past content that a relay may already have in its cache—like asking for a recording of a song that just played.

Both are part of the same specification, but for the most pressing low-latency use cases, a performant implementation of SUBSCRIBE is what matters most. For that reason, we have focused our initial efforts there and have not yet implemented FETCH.

This is where our roadmap is flexible and where the community can have a direct impact. Do you need FETCH to build on-demand or catch-up functionality? Or is more complete support for the prioritization features within SUBSCRIBE more critical for your use case? The feedback we receive from early developers will help us decide what to build next.

As always, we will announce our updates and changes to our implementation as we continue with development on our developer docs pages.

Kick the tires on the future

We believe in building in the open and interoperability in the community. MoQ is not a Cloudflare technology but a foundational Internet technology. To that end, the first demo client we’re presenting is an open source, community example.

You can access the demo here: https://moq.dev/publish/

Even though this is a preview release, we are running MoQ relays at Cloudflare’s full scale, like we do every production service. This means every server that is part of the Cloudflare network in more than 330 cities is now a MoQ relay.

We invite you to experience the “wow” moment of near-instant, sub-second streaming latency that MoQ enables. How would you use a protocol that offers the speed of a video call with the scale of a global broadcast?

Interoperability

We’ve been working with others in the IETF WG community and beyond on interoperability of publishers, players and other parts of the MoQ ecosystem. So far, we’ve tested with:

The Road Ahead

The Internet’s media stack is being refactored. For two decades, we’ve been forced to choose between latency, scale, and complexity. The compromises we made solved some problems, but also led to a fragmented ecosystem.

MoQ represents a promising new foundation—a chance to unify the silos and build the next generation of real-time applications on a scalable protocol. We’re committed to helping build this foundation in the open, and we’re just getting started.

MoQ is a realistic way forward, built on QUIC for future proofing, easier to understand than WebRTC, compatible with browsers unlike RTMP.

The protocol is evolving, the implementations are maturing, and the community is growing. Whether you’re building the next generation of live streaming, exploring real-time collaboration, or pushing the boundaries of interactive media, consider whether MoQ may provide the foundation you need.

Availability and pricing

We want developers to start building with MoQ today. To make that possible MoQ at Cloudflare is in tech preview – this means it’s available free of charge for testing (at any scale). Visit our developer homepage for updates and potential breaking changes.

Indie developers and large enterprises alike ask about pricing early in their adoption of new technologies. We will be transparent and clear about MoQ pricing. In general availability, self-serve customers should expect to pay 5 cents/GB outbound with no cost for traffic sent towards Cloudflare. 

Enterprise customers can expect usual pricing in line with regular media delivery pricing, competitive with incumbent protocols. This means if you’re already using Cloudflare for media delivery, you should not be wary of adopting new technologies because of cost. We will support you.

If you’re interested in partnering with Cloudflare in adopting the protocol early or contributing to its development, please reach out to us at [email protected]! Engineers excited about the future of the Internet are standing by.

Get involved:

Security updates for Friday

Post Syndicated from daroc original https://lwn.net/Articles/1034755/

Security updates have been issued by AlmaLinux (tomcat), Debian (squid), Fedora (matrix-synapse, rust-slab, socat, and webkitgtk), SUSE (firefox-esr, gdk-pixbuf, gdk-pixbuf-devel, govulncheck-vulndb, rust-keylime, and wicked2nm), and Ubuntu (linux-nvidia, linux-oracle, linux-oracle-6.8, php7.0, php7.2, php7.4, python3.13, python3.12, python3.11, python3.10, python3.9, python3.8, python3.7, python3.6, python3.5, python3.4, and ruby-webrick).

AI Agents Need Data Integrity

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2025/08/ai-agents-need-data-integrity.html

Think of the Web as a digital territory with its own social contract. In 2014, Tim Berners-Lee called for a “Magna Carta for the Web” to restore the balance of power between individuals and institutions. This mirrors the original charter’s purpose: ensuring that those who occupy a territory have a meaningful stake in its governance.

Web 3.0—the distributed, decentralized Web of tomorrow—is finally poised to change the Internet’s dynamic by returning ownership to data creators. This will change many things about what’s often described as the “CIA triad” of digital security: confidentiality, integrity, and availability. Of those three features, data integrity will become of paramount importance.

When we have agency in digital spaces, we naturally maintain their integrity—protecting them from deterioration and shaping them with intention. But in territories controlled by distant platforms, where we’re merely temporary visitors, that connection frays. A disconnect emerges between those who benefit from data and those who bear the consequences of compromised integrity. Like homeowners who care deeply about maintaining the property they own, users in the Web 3.0 paradigm will become stewards of their personal digital spaces.

This will be critical in a world where AI agents don’t just answer our questions but act on our behalf. These agents may execute financial transactions, coordinate complex workflows, and autonomously operate critical infrastructure, making decisions that ripple through entire industries. As digital agents become more autonomous and interconnected, the question is no longer whether we will trust AI but what that trust is built upon. In the new age we’re entering, the foundation isn’t intelligence or efficiency—it’s integrity.

What Is Data Integrity?

In information systems, integrity is the guarantee that data will not be modified without authorization, and that all transformations are verifiable throughout the data’s life cycle. While availability ensures that systems are running and confidentiality prevents unauthorized access, integrity focuses on whether information is accurate, unaltered, and consistent across systems and over time.

It’s a new idea. The undo button, which prevents accidental data loss, is an integrity feature. So is the reboot process, which returns a computer to a known good state. Checksums are an integrity feature; so are verifications of network transmission. Without integrity, security measures can backfire. Encrypting corrupted data just locks in errors. Systems that score high marks for availability but spread misinformation just become amplifiers of risk.

All IT systems require some form of data integrity, but the need for it is especially pronounced in two areas today. First: Internet of Things devices interact directly with the physical world, so corrupted input or output can result in real-world harm. Second: AI systems are only as good as the integrity of the data they’re trained on, and the integrity of their decision-making processes. If that foundation is shaky, the results will be too.

Integrity manifests in four key areas. The first, input integrity, concerns the quality and authenticity of data entering a system. When this fails, consequences can be severe. In 2021, Facebook’s global outage was triggered by a single mistaken command—an input error missed by automated systems. Protecting input integrity requires robust authentication of data sources, cryptographic signing of sensor data, and diversity in input channels for cross-validation.

The second issue is processing integrity, which ensures that systems transform inputs into outputs correctly. In 2003, the U.S.-Canada blackout affected 55 million people when a control-room process failed to refresh properly, resulting in damages exceeding US $6 billion. Safeguarding processing integrity means formally verifying algorithms, cryptographically protecting models, and monitoring systems for anomalous behavior.

Storage integrity covers the correctness of information as it’s stored and communicated. In 2023, the Federal Aviation Administration was forced to halt all U.S. departing flights because of a corrupted database file. Addressing this risk requires cryptographic approaches that make any modification computationally infeasible without detection, distributed storage systems to prevent single points of failure, and rigorous backup procedures.

Finally, contextual integrity addresses the appropriate flow of information according to the norms of its larger context. It’s not enough for data to be accurate; it must also be used in ways that respect expectations and boundaries. For example, if a smart speaker listens in on casual family conversations and uses the data to build advertising profiles, that action would violate the expected boundaries of data collection. Preserving contextual integrity requires clear data-governance policies, principles that limit the use of data to its intended purposes, and mechanisms for enforcing information-flow constraints.

As AI systems increasingly make critical decisions with reduced human oversight, all these dimensions of integrity become critical.

The Need for Integrity in Web 3.0

As the digital landscape has shifted from Web 1.0 to Web 2.0 and now evolves toward Web 3.0, we’ve seen each era bring a different emphasis in the CIA triad of confidentiality, integrity, and availability.

Returning to our home metaphor: When simply having shelter is what matters most, availability takes priority—the house must exist and be functional. Once that foundation is secure, confidentiality becomes important—you need locks on your doors to keep others out. Only after these basics are established do you begin to consider integrity, to ensure that what’s inside the house remains trustworthy, unaltered, and consistent over time.

Web 1.0 of the 1990s prioritized making information available. Organizations digitized their content, putting it out there for anyone to access. In Web 2.0, the Web of today, platforms for e-commerce, social media, and cloud computing prioritize confidentiality, as personal data has become the Internet’s currency.

Somehow, integrity was largely lost along the way. In our current Web architecture, where control is centralized and removed from individual users, the concern for integrity has diminished. The massive social media platforms have created environments where no one feels responsible for the truthfulness or quality of what circulates.

Web 3.0 is poised to change this dynamic by returning ownership to the data owners. This is not speculative; it’s already emerging. For example, ActivityPub, the protocol behind decentralized social networks like Mastodon, combines content sharing with built-in attribution. Tim Berners-Lee’s Solid protocol restructures the Web around personal data pods with granular access controls.

These technologies prioritize integrity through cryptographic verification that proves authorship, decentralized architectures that eliminate vulnerable central authorities, machine-readable semantics that make meaning explicit—structured data formats that allow computers to understand participants and actions, such as “Alice performed surgery on Bob”—and transparent governance where rules are visible to all. As AI systems become more autonomous, communicating directly with one another via standardized protocols, these integrity controls will be essential for maintaining trust.

Why Data Integrity Matters in AI

For AI systems, integrity is crucial in four domains. The first is decision quality. With AI increasingly contributing to decision-making in health care, justice, and finance, the integrity of both data and models’ actions directly impact human welfare. Accountability is the second domain. Understanding the causes of failures requires reliable logging, audit trails, and system records.

The third domain is the security relationships between components. Many authentication systems rely on the integrity of identity information and cryptographic keys. If these elements are compromised, malicious agents could impersonate trusted systems, potentially creating cascading failures as AI agents interact and make decisions based on corrupted credentials.

Finally, integrity matters in our public definitions of safety. Governments worldwide are introducing rules for AI that focus on data accuracy, transparent algorithms, and verifiable claims about system behavior. Integrity provides the basis for meeting these legal obligations.

The importance of integrity only grows as AI systems are entrusted with more critical applications and operate with less human oversight. While people can sometimes detect integrity lapses, autonomous systems may not only miss warning signs—they may exponentially increase the severity of breaches. Without assurances of integrity, organizations will not trust AI systems for important tasks, and we won’t realize the full potential of AI.

How to Build AI Systems With Integrity

Imagine an AI system as a home we’re building together. The integrity of this home doesn’t rest on a single security feature but on the thoughtful integration of many elements: solid foundations, well-constructed walls, clear pathways between rooms, and shared agreements about how spaces will be used.

We begin by laying the cornerstone: cryptographic verification. Digital signatures ensure that data lineage is traceable, much like a title deed proves ownership. Decentralized identifiers act as digital passports, allowing components to prove identity independently. When the front door of our AI home recognizes visitors through their own keys rather than through a vulnerable central doorman, we create resilience in the architecture of trust.

Formal verification methods enable us to mathematically prove the structural integrity of critical components, ensuring that systems can withstand pressures placed upon them—especially in high-stakes domains where lives may depend on an AI’s decision.

Just as a well-designed home creates separate spaces, trustworthy AI systems are built with thoughtful compartmentalization. We don’t rely on a single barrier but rather layer them to limit how problems in one area might affect others. Just as a kitchen fire is contained by fire doors and independent smoke alarms, training data is separated from the AI’s inferences and output to limit the impact of any single failure or breach.

Throughout this AI home, we build transparency into the design: The equivalent of large windows that allow light into every corner is clear pathways from input to output. We install monitoring systems that continuously check for weaknesses, alerting us before small issues become catastrophic failures.

But a home isn’t just a physical structure, it’s also the agreements we make about how to live within it. Our governance frameworks act as these shared understandings. Before welcoming new residents, we provide them with certification standards. Just as landlords conduct credit checks, we conduct integrity assessments to evaluate newcomers. And we strive to be good neighbors, aligning our community agreements with broader societal expectations. Perhaps most important, we recognize that our AI home will shelter diverse individuals with varying needs. Our governance structures must reflect this diversity, bringing many stakeholders to the table. A truly trustworthy system cannot be designed only for its builders but must serve anyone authorized to eventually call it home.

That’s how we’ll create AI systems worthy of trust: not by blindly believing in their perfection but because we’ve intentionally designed them with integrity controls at every level.

A Challenge of Language

Unlike other properties of security, like “available” or “private,” we don’t have a common adjective form for “integrity.” This makes it hard to talk about it. It turns out that there is a word in English: “integrous.” The Oxford English Dictionary recorded the word used in the mid-1600s but now declares it obsolete.

We believe that the word needs to be revived. We need the ability to describe a system with integrity. We must be able to talk about integrous systems design.

The Road Ahead

Ensuring integrity in AI presents formidable challenges. As models grow larger and more complex, maintaining integrity without sacrificing performance becomes difficult. Integrity controls often require computational resources that can slow systems down—particularly challenging for real-time applications. Another concern is that emerging technologies like quantum computing threaten current cryptographic protections. Additionally, the distributed nature of modern AI—which relies on vast ecosystems of libraries, frameworks, and services—presents a large attack surface.

Beyond technology, integrity depends heavily on social factors. Companies often prioritize speed to market over robust integrity controls. Development teams may lack specialized knowledge for implementing these controls, and may find it particularly difficult to integrate them into legacy systems. And while some governments have begun establishing regulations for aspects of AI, we need worldwide alignment on governance for AI integrity.

Addressing these challenges requires sustained research into verifying and enforcing integrity, as well as recovering from breaches. Priority areas include fault-tolerant algorithms for distributed learning, verifiable computation on encrypted data, techniques that maintain integrity despite adversarial attacks, and standardized metrics for certification. We also need interfaces that clearly communicate integrity status to human overseers.

As AI systems become more powerful and pervasive, the stakes for integrity have never been higher. We are entering an era where machine-to-machine interactions and autonomous agents will operate with reduced human oversight and make decisions with profound impacts.

The good news is that the tools for building systems with integrity already exist. What’s needed is a shift in mind-set: from treating integrity as an afterthought to accepting that it’s the core organizing principle of AI security.

The next era of technology will be defined not by what AI can do, but by whether we can trust it to know or especially to do what’s right. Integrity—in all its dimensions—will determine the answer.

Sidebar: Examples of Integrity Failures

Ariane 5 Rocket (1996)
Processing integrity failure
A 64-bit velocity calculation was converted to a 16-bit output, causing an error called overflow. The corrupted data triggered catastrophic course corrections that forced the US $370 million rocket to self-destruct.

NASA Mars Climate Orbiter (1999)
Processing integrity failure
Lockheed Martin’s software calculated thrust in pound-seconds, while NASA’s navigation software expected newton-seconds. The failure caused the $328 million spacecraft to burn up in the Mars atmosphere.

Microsoft’s Tay Chatbot (2016)
Processing integrity failure
Released on Twitter, Microsoft‘s AI chatbot was vulnerable to a “repeat after me” command, which meant it would echo any offensive content fed to it.

Boeing 737 MAX (2018)
Input integrity failure
Faulty sensor data caused an automated flight-control system to repeatedly push the airplane’s nose down, leading to a fatal crash.

SolarWinds Supply-Chain Attack (2020)
Storage integrity failure
Russian hackers compromised the process that SolarWinds used to package its software, injecting malicious code that was distributed to 18,000 customers, including nine federal agencies. The hack remained undetected for 14 months.

ChatGPT Data Leak (2023)
Storage integrity failure
A bug in OpenAI’s ChatGPT mixed different users’ conversation histories. Users suddenly had other people’s chats appear in their interfaces with no way to prove the conversations weren’t theirs.

Midjourney Bias (2023)
Contextual integrity failure
Users discovered that the AI image generator often produced biased images of people, such as showing white men as CEOs regardless of the prompt. The AI tool didn’t accurately reflect the context requested by the users.

Prompt Injection Attacks (2023–)
Input integrity failure
Attackers embedded hidden prompts in emails, documents, and websites that hijacked AI assistants, causing them to treat malicious instructions as legitimate commands.

CrowdStrike  Outage (2024)
Processing integrity failure
A faulty software update from CrowdStrike caused 8.5 million Windows computers worldwide to crash—grounding flights, shutting down hospitals, and disrupting banks. The update, which contained a software logic error, hadn’t gone through full testing protocols.

Voice-Clone Scams (2024)
Input and processing integrity failure
Scammers used AI-powered voice-cloning tools to mimic the voices of victims’ family members, tricking people into sending money. These scams succeeded because neither phone systems nor victims identified the AI-generated voice as fake.

This essay was written with Davi Ottenheimer, and originally appeared in IEEE Spectrum.

The collective thoughts of the interwebz