Announcing a new integration between Backblaze B2 and Suite Studios’ S3 Native File Streaming that brings together scalable object storage and the familiar experience of working from a local drive.
Available today with Backblaze B2, the integration allows teams to mount a B2 bucket through Suite Studios and work with files as if they were stored on a high-performance NAS. Suite streams only the parts of each file an application needs, rather than requiring users to download entire datasets first.
Behind that experience, the data remains in Backblaze B2 as standard S3-compatible objects.
Ready to get started now?
To put Suite S3 Native to work on B2, contact Backblaze Sales or your Backblaze reseller partner. We can help validate your architecture, plan a proof of concept, and evaluate migration support for your existing data.
Your B2 bucket becomes the working file space
An existing Backblaze B2 bucket can now be used directly as a Suite file space. There is no separate storage environment hidden inside Suite and no proprietary segmentation happening in the background.
The relationship between the file system and the bucket remains straightforward:
One file in Suite maps to one object in B2.
The folder structure in Suite matches the structure in the bucket.
Files remain standard, whole objects.
B2 remains the system of record.
Suite provides the high-performance file access layer.
For users, cloud storage behaves more like a shared NAS experience. For IT, the underlying data remains in a storage platform the organization owns and controls.
If your files are already stored in B2 as standard objects, Suite can connect to the bucket and make them available through its mounted drive. There is no need to create another copy, convert the data, or run a separate ingest into Suite.
One dataset, multiple ways to use it
The immediate opportunity is media production, where teams work with large files across editing, media management, transcoding, automation, archive, and distribution.
An editor can open high-resolution media in Adobe Premiere Pro through the Suite drive. At the same time, iconik or another MAM can index the library, Telestream Vantage can process the source files, and automation can respond to new objects written into B2.
Each application works with the same underlying media.
But this is not exclusively a media workflow. The same model has practical applications anywhere teams need to work with large cloud datasets through file-based desktop tools:
Life sciences and biotech: Make genomic sequencing, microscopy, medical imaging, and instrument-generated data accessible to researchers without maintaining full copies at every location.
Geospatial and earth observation: Work with satellite imagery, drone captures, LiDAR scans, and GIS datasets while keeping source files centrally stored.
Engineering and design: Give teams file-based access to CAD assemblies, simulations, 3D models, digital twins, and high-resolution scans.
Research and higher education: Create shared access to large scientific datasets for distributed teams while maintaining a common storage foundation.
These are emerging applications for the architecture rather than packaged Suite solutions today, but the underlying opportunity is clear: Large datasets become much more useful when people and applications can access them without moving them into another storage silo.
Built for active data
Media, scientific data, and geospatial imagery rarely sit still. Files are opened, reviewed, processed, moved, analyzed, and shared throughout their lifecycle.
Backblaze B2 provides always-hot, S3-compatible object storage with straightforward pricing and free egress up to three times average monthly storage, plus unlimited free egress through select CDN and compute partners.
That makes B2 a strong foundation for data that needs to remain active and accessible. Suite adds another way to put that data to work by making a B2 bucket accessible through a high-performance file interface.
Available today, with migration support
Backblaze B2 is available today for Suite S3 Native workflows.
If your data is already in B2, you can use an existing bucket as your Suite file space. If it currently resides in another cloud, on a NAS or SAN, or on tape, Backblaze can help plan and execute the move.
Organizations making a qualifying commitment can receive assisted migration into B2 through our Universal Data Migration program.
Bring the data into the storage platform you want to keep. Let cloud applications access it through S3. Let people access it through a high-performance drive.
One dataset. One storage foundation. Multiple ways to put it to work.
AI agents are getting very good at making things: reports, code builds, images, videos, datasets, customer exports, and backups. But generating a file is only half the job. The agent still needs somewhere durable to put it. That’s where B2 comes in.
Today, we’re introducing the Backblaze B2 MCP Server: an open source server that gives AI agents a safe, standard way to operate cloud object storage.
Introducing the Backblaze B2 MCP Server
Model Context Protocol (MCP) provides a common way for AI agents to discover and use external tools. The Backblaze B2 MCP Server applies that model to Backblaze B2 Cloud Storage, connecting MCP-compatible clients to a focused set of storage operations.
The server is open source and designed so the tools presented to the agent reflect the permissions of the connected B2 application key.
And that last point is important. The MCP server does not invent a new permission model or grant an agent blanket access to your account. If the key can only read from one bucket, the agent’s usable tool set and reach are constrained accordingly. If the key does not permit an operation, the agent cannot use the MCP server to bypass that boundary.
Why launch an MCP server?
Storage has always met developers in two places: the API and the console. Agent tools are becoming a third, and increasingly the one where work starts. Our view is that infrastructure should be available inside the tool someone already has open, at the moment they need it.
That spans a wide range of work: everyday tasks that produce outputs which need to be stored, AI workflows that read and write data as they run, and applications that require storage to handle files. A report or an export has to land somewhere durable. An application built around agents needs somewhere to put user uploads, generated assets, and datasets. Customers have also asked us how their own agents can administer their storage.
MCP is what lets one implementation serve all of it. An MCP server provides these storage tools directly to the AI agents people use for everyday tasks and for coding, so the same credentials, scoping, transfer path, and destructive operation policy apply everywhere your team works. B2 is available inside the tools your teams already have open.
See it in action
Suppose an agent has prepared a customer export. Instead of handing it a general-purpose credential and writing the surrounding storage logic yourself, you could ask:
“Prepare a private destination for this customer export. Keep it for 30 days and give me a secure way to upload it.”
Depending on the application key’s permissions and the server’s policy settings, the agent can:
Create a private bucket or select an existing one the key can access.
Apply lifecycle or retention settings, depending on whether the export should expire automatically or be protected from deletion.
Generate a short-lived upload URL.
Verify that the export arrived.
Return a short-lived download URL to the intended recipient.
For a large export, the bytes move directly to B2 using the short-lived URL; they do not need to pass through the model conversation or the MCP server. That keeps bulk data on the right path and leaves the agent responsible for orchestrating the work, not hauling the payload.
Now imagine a follow-up request: “Delete that export.” The B2 MCP Server can require confirmation before the action proceeds or block destructive operations entirely, depending on how the server is configured. The same principle applies to changes that weaken protection, such as clearing retention or introducing lifecycle rules that schedule deletion.
The result is a useful division of labor: The agent can set up the storage, move the file, and verify the outcome, while the B2 MCP server enforces the connected key’s limits and configured policy.
Built for real storage work
The B2 MCP Server includes up to 40 tools across the B2 Native API, B2’s S3 Compatible API, and purpose-built storage analytics. You do not need to expose all of them to every agent. The available surface adjusts to the connected key and the server’s policy settings.
Manage storage. Create and configure buckets, list and manage objects, define lifecycle rules, work with retention and legal holds, and use Object Lock where the workflow calls for stronger protection.
Move large files directly. Generate short-lived upload and download URLs, or use multipart transfers for large objects, so bulk data stays out of the model and MCP server.
Limit access. Connect a scoped B2 application key so the agent can see and use only the buckets and capabilities it needs. For normal storage work, a non-master key is the right starting point.
Gate destructive actions. Set the destructive-action policy to require confirmation, block the operation, or allow it in a trusted environment. Local use defaults to confirmation; hosted HTTP deployments default to blocking these calls.
Inspect usage. Find storage growth, identify egress leaders, locate the largest files in a bucket, and surface unfinished multipart uploads that may be consuming storage.
Safety starts with the key
An MCP server makes tools available to an agent; it should not make good access controls optional. With B2 MCP, the connected application key remains the first boundary. B2 application keys can be limited by bucket, capability, file prefix, and duration, giving developers a practical way to apply least privilege to agentic workflows.
The server adds another layer around operations that deserve friction. Destructive and protection-weakening actions can require an explicit confirmation or be refused before execution. For hosted deployments, the project includes additional controls and deployment guidance for credential handling, authentication, allowed hosts and origins, rate limits, and logging.
The goal is not to pretend an agent can never make a bad request. It is to give developers clear, enforceable boundaries around what the agent is allowed to do.
Already use B2? Your agents can use it too
If your application data, backups, media, or archives already live in B2, you do not need a separate storage product for agents. The MCP server works with standard B2 application keys and existing buckets.
You can connect an existing key, but in most agentic workflows it is worth creating a narrower key for the job. A reporting agent might need read access to a specific prefix. An export workflow might need permission to write to one private bucket and generate a short-lived download link. A cleanup agent might need list access but no delete capability until a human approves the action.
Try the B2 MCP Server
One-click install (Claude Desktop)
Download b2-mcp.mcpb from the latest release and double-click it. Claude Desktop opens, offers to install the extension, and prompts for your Application Key ID and Application Key (Region and master key optional) — no config file to edit, nothing pasted into the conversation. (If your system hasn’t associated .mcpb files with Claude Desktop yet, open Settings → Extensions and install it from there, or drag the file onto that window.)
Run with npx (any MCP client)
You need three things to get started:
A supported Node.js runtime.
A B2 application key. Use a non-master key and scope it as narrowly as the workflow allows.
An MCP-compatible client.
The fastest way to run the server is with npx:
`npx -y @backblaze-labs/b2-mcp`
For Claude Desktop, add the following block to claude_desktop_config.json, substitute your application key values, and restart the client:
Do not paste the application key into the model conversation. Keep it in the client configuration or your deployment’s secret-management layer.
Then try a deliberately simple first prompt: “List the buckets this key can access.”
For copy-and-paste setup instructions for Cursor, VS Code, Cline, Windsurf, Zed, Continue, Goose, Claude.ai, and hosted deployments, see the client setup guide.
See the B2 MCP Server in action. We’ll set it up, run a complete storage workflow, explain the safety boundaries, and answer your questions.
A safe way for agents to operate storage
Agents need more than somewhere to put files. They need a safe way to operate storage: to prepare the right destination, move data on the right path, verify the outcome, and stop when a request exceeds their permissions or policy.
The Backblaze B2 MCP Server brings those capabilities to the MCP clients developers are already using, while keeping B2 application keys and server-side safeguards in control. Try it with your existing B2 storage, or join the webinar to see the complete workflow.
Broadcast production has spent decades organizing media around files, servers, and the physical limits of the facility. That model is familiar, and it works until a live event creates hours of footage across multiple cameras, editors are distributed, the archive is growing by petabytes, and every copy of every clip becomes another operational and financial problem.
Time Addressable Media Store, or TAMS, offers a different model: Organize media by time, store it in object storage, and let interoperable tools access the same content through an open API. Pair that with affordable, high-performance storage, and the result is a practical path toward cloud-native production without forcing media teams into a single vendor’s ecosystem.
That is where a stack built on Backblaze B2, together with TAMS ecosystem partners LiveWyer, CuttingRoom, and Drastic Technologies, gets interesting.
What is TAMS?
When a live event is producing hours of footage across multiple cameras, editors should not have to wait for files to close before they can start working. TAMS, or Time Addressable Media Store, addresses that problem by organizing media around the timeline rather than individual files.
With TAMS, media is stored as timestamped segments that applications can access by time. A news editor can pull a quote from a press conference while it is still underway. A sports producer can create a highlight from a live game feed. A content team can reuse the same moment for social clips, a longer edit, distribution, and archive without creating a separate proprietary copy for every stage.
TAMS gives ingest systems, browser-based editors, desktop NLEs, MAMs, and archive platforms a common way to work with the same media. The store can run on one cloud, ingest can come from another provider, and editors can use the tools they already know.
The specification grew out of BBC Research & Development’s work on cloud-native media production and was published in 2023. Its purpose was practical: Give different companies a shared interface for building interoperable media workflows instead of recreating isolated storage silos in the cloud.
TAMS is not a replacement for an NLE, MAM, ingest system, or archive. It is the layer that helps those systems work together. In this article, we are focusing on three Backblaze ecosystem partners that demonstrate that approach: LiveWyer, CuttingRoom, and Drastic Technologies.
The solution stack
LiveWyer provides TAMOSS, a Kubernetes-native implementation of the TAMS API. TAMOSS is the store management and API layer. It handles the TAMS structure while orchestrating the underlying media segments stored in Backblaze B2.
CuttingRoom brings an editorial-grade video editing experience into the browser. In a TAMS workflow, editors can begin working with media while a live event is still being captured. They can find a moment on the timeline, create a clip, and assemble a story without waiting for an entire file to close or moving the media into a separate editing silo. The CuttingRoom solution allows real-time collaboration, import and exports from and to market leading MAMs and with the optional ShortCut AI assistant editing, editors can do more in less time.
Drastic Technologies’ NetX platform provides the TAMS-compliant readers and writers that handle both ends of the media flow: ingesting and writing media into TAMS, then reading it back for playback and use in professional applications such as Adobe Premiere and Avid. NetX is a key part of making TAMS flows usable across live ingest, playback, and established desktop production workflows including Adobe Premiere.
With Backblaze B2 providing the S3-compatible storage foundation beneath the workflow.
TAMOSS supplies the TAMS interface. CuttingRoom provides the browser-based editorial experience. Drastic NetX handles the critical ingest, writing, reading, and playback paths. B2 stores the media segments at a price that makes large-scale access practical.
These companies are part of a growing ecosystem of roughly 14 technology providers with publicly identifiable TAMS implementations, products, readers, writers, or integrations. That broader ecosystem includes AWS, Konstrukt, LOGIC media solutions, LucidLink, Mimir, Reuters Imagen, Matrox Video, Mavis, Norsk, Techex, and Zixi, alongside LiveWyer, CuttingRoom, and Drastic Technologies.
This is the broader idea behind an interoperable standard. No single company has to provide every part of the workflow, and customers do not have to replace every system they already own.
Where TAMS fits
The use cases are not limited to national broadcasters.
A news organization could ingest a live press conference, let an editor find a quote while the event is still happening, and publish a short clip before the full recording is finished.
A sports production team could create proxy media for immediate clipping while retaining higher-resolution source material for finishing. This is particularly useful because some high-resolution MOV and MXF workflows are difficult to segment and read while they are still growing.
A regional sports league or smaller event producer could avoid building a large on-premises storage environment and give remote editors access to the same media from wherever they are working.
A corporate video team could record a town hall, create short internal clips, publish social content, and retain the full event in the archive using the same underlying media.
The same approach could support houses of worship, education, government, live entertainment, and creator workflows. The common requirement is quick access to a growing body of media without multiplying copies and handoffs.
What if the editor reviewing a live feed did not need to ask where the file was? What if the answer were simply a time range in a shared store?
Why Backblaze B2 matters
Live video is a storage problem before it is almost anything else. A multi-camera sports event, a continuous news operation, or a large livestream can generate terabytes of content quickly.
Backblaze B2 starts at $6.95 per terabyte per month, with free egress up to three times average monthly storage and no egress fees through many compute and CDN partners. That combination makes it easier to keep large live datasets available for editing, review, distribution, and archive.
A TAMS workflow backed by B2 can look like this:
– Use Drastic NetX to ingest and write live feeds into a TAMS store.
– Segment and index the media as it is captured.
– Let editors begin clipping and assembling stories in CuttingRoom.
– Use NetX readers to play and access the same media from Adobe Premiere or Avid.
– Keep the objects available for near-live production, distribution, and archive.
– Add a MAM where richer search, rights, governance, and catalog workflows are required.
Our deployment test offered one final lesson. Getting media into the store was straightforward. Making the service usable by the rest of the team required attention to the operational details. Open standards reduce lock-in, but production still depends on good deployment practices.
That is a healthy place for the industry to be. The pieces are becoming more modular, the interfaces are becoming more open, and media teams have more freedom to assemble the workflow that fits them.
TAMS gives those tools a common language. TAMOSS makes the store deployable. Drastic NetX handles how media is written into and read from the workflow. CuttingRoom makes the content useful to editors and production teams. And Backblaze B2 provides a scalable storage layer that makes the economics work.
The question is no longer whether live media can be stored in the cloud. It is whether we can make that media available to the right person, in the right tool, at the right moment.
Always-on SSE-B2 brings AES-256 encryption at rest to every new upload and destination copy—with no application changes, additional cost, or performance impact.
Security works best when it doesn’t depend on one more checkbox. Starting September 14, 2026, we’re making server-side encryption the automatic baseline for Backblaze B2 Cloud Storage.
Backblaze B2 will automatically encrypt all newly uploaded and copied object data at rest using Server-Side Encryption with Backblaze-managed keys (SSE-B2) and AES-256. The default applies immediately to new buckets. Existing buckets will receive the default gradually; once enabled for a bucket, newly uploaded and copied objects use SSE-B2 automatically. If an application doesn’t specify an encryption method, B2 handles it automatically.
Here’s the TL;DR: You don’t need to update your application, add an encryption header, or turn on a bucket setting. There is no additional charge for SSE-B2 and no impact on upload or download performance. New uploads are encrypted by default, existing objects keep their current encryption state, and SSE-C remains available when you want to provide your own key for an individual object.
Secure by Default, Without Extra Work
SSE-B2 encrypts object data at rest with AES-256 while Backblaze manages the encryption keys. Until now, customers could choose to enable SSE-B2 for a bucket or request it for an individual upload. With always-on encryption, SSE-B2 becomes the baseline for every new upload and destination copy when SSE-C is not explicitly requested.
That means fewer settings to manage, fewer opportunities for configuration drift, and a stronger security baseline across your storage environment. Teams can meet encryption-at-rest requirements without building another check into every application or deployment process. Server-side encryption cannot be disabled for new writes.
For new buckets, the default applies immediately. Existing buckets receive it gradually; once enabled for a bucket, new uploads and destination copies receive SSE-B2 automatically. Buckets already configured for SSE-B2 remain configured as they are, and requests that explicitly use SSE-C continue to use SSE-C.
What This Means for Your Existing Workflows
The best kind of security improvement is one that doesn’t force you to rebuild what already works. Always-on encryption is a behavioral update, not a new API contract. Existing integrations can keep using the same upload, copy, multipart-upload, and download operations they use today. B2 automatically decrypts SSE-B2 data for authorized reads, so the way you access objects does not change either.
See the Default in Action
The important part of these examples is what is missing: Neither upload explicitly requests SSE-B2. After this update, Backblaze applies it automatically.
S3-Compatible API: Upload Without an Encryption Flag
There is no --server-side-encryption option in the command. B2 still encrypts the new object with SSE-B2. In S3-compatible responses, that effective encryption is represented as AES256.
# Response includes "serverSideEncryption": {"mode": "SSE-B2", "algorithm": "AES256"}
The request does not include X-Bz-Server-Side-Encryption. The response still reports SSE-B2 with AES256 because it is now the effective default.
Using the S3-Compatible API?
Requests that omit encryption headers receive SSE-B2 automatically, reported through the S3-compatible AES256 value. PutObject, CopyObject, and multipart uploads all use the new default for the destination object. Valid SSE-C headers still take precedence.
Using the B2 Native API?
You can continue to omit SSE-B2 fields and headers. New uploads, large-file uploads, and destination copies use SSE-B2 by default. Applications can still explicitly request SSE-B2, but doing so is no longer necessary to receive encryption at rest.
Using an SDK, CLI, Integration, or the Web Console?
Because the protection is applied by B2, tools that already upload to Backblaze B2 benefit automatically. No special encryption flag, SDK upgrade, CLI update, or integration change is required. The Backblaze web consoles will show SSE-B2 or AES256 as the effective bucket default and continue to display the encryption actually used for each object.
SSE-C Is Still Available
Some organizations need direct control over the key used for a particular object. SSE-C continues to support that workflow. When a request includes valid SSE-C headers, Backblaze uses the customer-provided AES-256 key instead of SSE-B2 for that object.
As before, Backblaze does not retain the customer key. Customers using SSE-C are responsible for protecting and retaining their keys; a lost key cannot be recovered by Backblaze. You can also continue to encrypt data on the client side before uploading it. SSE-KMS is not part of this update.
What About Objects Already Stored in B2?
Always-on encryption is not retroactive. Objects already stored in a bucket keep the encryption state they had when they were written. We are not rewriting customer data in the background or changing the encryption metadata of historical objects.
If you want an older unencrypted object to use SSE-B2, upload it again or create a new destination copy. The new object is encrypted using the always-on default. Object-information and download responses continue to describe the encryption actually used for each object.
Why Always-On Encryption Matters
Encryption at rest is a foundational part of modern data protection. Making it automatic helps teams establish a consistent security baseline without adding another deployment step or relying on every application to make the same configuration choice.
It also keeps the developer experience simple. Teams can focus on moving, protecting, and using their data while Backblaze applies the default protection behind the scenes—with no added encryption charge or performance trade-off. That’s the kind of cloud storage experience we want to deliver: secure by design, straightforward to operate, and compatible with the tools customers already use.
Frequently Asked Questions
Do I need to change my application?
No. Applications that omit encryption settings automatically receive SSE-B2 for new uploads and destination copies. Existing request formats remain valid.
Can I disable server-side encryption for new uploads?
No. SSE-B2 is the effective default when SSE-C is not requested. Clearing or deleting an explicit bucket encryption configuration does not create an unencrypted default.
Does this encrypt objects that are already stored?
No. Existing objects retain their original encryption state. Uploading or copying an object again creates a new object that uses the always-on default.
Can I still use my own encryption key?
Yes. Supply the required SSE-C headers when you upload, copy, or access an SSE-C object. SSE-C takes precedence over the SSE-B2 default for that object.
Does always-on encryption cost more or affect performance?
No. SSE-B2 is applied at no additional charge and has no impact on upload or download performance. Normal Backblaze B2 storage and API charges still apply.
Available Starting September 14, 2026
Always-on SSE-B2 is available on September 14, 2026 for new buckets in every Backblaze B2 region. Existing buckets will be enabled gradually. For most customers, there’s nothing to turn on and nothing to migrate: once the default is enabled for a bucket, every new upload and destination copy receives SSE-B2 automatically unless you explicitly request SSE-C.
At NVIDIA’s GTC conference, Adobe’s CTO Ely Greenfield walked the audience through the company’s three-year journey building frontier generative AI models from scratch. And the AI training data pipeline required to keep thousands of GPUs productive.
Along the way, he showed a profiler readout from their early training runs—a visualization of exactly how much time each GPU was spending on actual computation versus sitting idle. It revealed that roughly two-thirds of GPU time was spent simply waiting for data. And that idle time had a price tag.
“If we were putting a million dollars into training,” he told the room, “that was $600,000 we were burning away on GPUs sitting and doing nothing.” In other words, roughly sixty cents of every dollar spent on GPU compute was being wasted.
Greenfield’s team traced the waste to two culprits, neither of which is unique to Adobe: whether data can reach your GPUs fast enough, and whether the work it represents is distributed evenly once it arrives.
One culprit is a storage problem. The other isn’t, but you won’t be able to fix it until you’ve solved the first one.
Culprit #1: Storage and retrieval speed
Adobe’s training data lived in petabytes of distributed cloud storage and had to be shipped out to thousands of GPUs constantly, over standard Ethernet. The dataset itself was enormous and varied—images and video, low-res and high-res, simple formats and expensive codecs, all moving at once. Standard networking, which was built for retrieving individual files on request, wasn’t designed for that kind of sustained, parallel, petabyte-scale movement. It became a massive bottleneck.
Checkpointing compounded the problem. As a safeguard, the training run would periodically write a complete copy of the model back to storage. Most of those checkpoints were never needed again. But writing and reading them still consumed real GPU time, which cost a lot of money whether they were used or not.
How Adobe fixed it
Adobe’s solution had two parts.
First, they replaced standard Ethernet with a high-performance networking fabric designed for the petabyte-scale traffic distributed AI training generates. Now, data could finally move at the pace their GPUs needed.
Second, they changed how they saved checkpoints. Instead of writing one giant file containing the entire model, they began breaking the model into smaller pieces and saving pipeline fragments to many places at once. Saving and loading checkpoints now takes significantly less time than it used to. If a checkpoint ever needs to be reassembled, that’s slightly slower, but it’s a rare event. The savings on every other write happen continuously, across thousands of GPUs, around the clock.
The underlying lesson is that for model training, parallel access and sustained high throughput are baseline requirements. Training data has to be immediately accessible at the pace your GPUs consume it, not tucked away in a storage tier that takes minutes or hours to retrieve.
Culprit #2: The data loader problem
A balanced data loader can only do its job if the data it’s balancing is actually available the moment it’s needed. That makes fast storage a precondition for everything else in the pipeline. But even after fixing the storage and retrieval speed culprit, Adobe still had a problem: the way training data was divided across the cluster meant some GPUs were doing far more work than others.
Their pipeline used a straightforward data-parallelism approach: slice the training data into equal-sized chunks and assign one chunk to each GPU in the cluster. But equal-sized chunks weren’t equal work. Some GPUs got simple, low-resolution assets that processed in seconds; others got large, complex files that took minutes. The fast GPUs finished early and sat idle waiting for the slow ones to catch up. And then all of them waited again while their results were merged into one updated model before the next round could begin.
How Adobe fixed it
Adobe stopped treating all data as equivalent. They custom-built a balanced data loader that understood the processing cost of each asset and distributed work so every GPU finished at roughly the same time. Then they restructured how computation was divided across the cluster to make the merging step dramatically cheaper.
A perfectly balanced data loader still idles if the data it’s waiting on hasn’t arrived yet. This is why storage can’t be an afterthought. The loader optimizes what happens once data is there, but storage determines whether it’s there at all.
And building something like Adobe’s balanced data loader takes real engineering investment, including time spent profiling workloads, testing distribution strategies, and tuning until every GPU finishes at roughly the same pace. Teams still fighting storage bottlenecks rarely get to that work. Their engineers are busy figuring out why GPUs are idle in the first place, not optimizing how work gets distributed once data arrives. Removing the storage bottleneck frees up the engineering time needed to tackle the data loader problem properly.
From 40 to 80 cents
After addressing both culprits, Adobe’s GPUs ran at roughly 80% utilization. After accounting for the coordination overhead inherent to running thousands of machines together, 80% is close to the practical ceiling. That’s the difference between a GPU cluster that’s mostly waiting and one that’s mostly working.
The Storage Side of the Solution
Backblaze B2 Overdrive addresses the storage side of exactly the problem Adobe ran into. B2 is always-hot object storage, with no tiering, no retrieval delays, and no waiting for data to be promoted from a cold tier before training can begin. Training data stays immediately accessible whether it was written an hour ago or six months ago.
But availability alone isn’t enough if data can’t move fast enough to keep up with the cluster. B2 Overdrive adds the throughput layer: at up to 1Tbps, it’s designed to keep petabyte-scale GPU clusters fed continuously, not just handle occasional bursts of traffic. Both layers are S3-compatible, so they drop into existing PyTorch or TensorFlow pipelines without a rewrite. And because egress is free, moving training data between storage and GPU compute—across regions or providers—doesn’t add a cost penalty on top of a performance one.
Checkpointing benefits from the same foundation. Adobe’s solution of breaking the model into fragments and saving them to many places at once only works if the underlying storage is fast and parallel enough to make it pay off. Always-hot, high-throughput storage makes checkpoint writes and recoveries faster across the board, so they cost less GPU time whether they happen rarely or often.
Backblaze can’t write your data loader for you, but it can make sure that once you’ve built one, it isn’t waiting on storage to do its job.
Building generative media applications on object storage: What the strongest projects have in common
Between June 22 and August 3, 2026, 1,314 people entered hundreds of projects into a single brief: build a generative media app on Backblaze B2 object storage, orchestrated through the open-source Genblaze SDK, for a share of a $10,000 prize pool.
Almost every entry could generate an image, a video, or a soundtrack, and generate it well. What separated the strongest was the layer underneath: what gets written down, what can be proved later, and what happens when a file has to be checked, corrected, or locked against deletion.
Here’s a closer look at the five that went furthest, and the parts worth reusing. Every entry is browsable in the hackathon project gallery.
The brief, and why it was narrow on purpose
The brief was narrower than most hackathons get. Submissions couldn’t just be a working demo and a repo. Devpost required teams to explain, in writing, how their app actually used both Backblaze B2 and Genblaze, and to list every provider and model behind it. Judging ran against four criteria: real-world utility, production readiness, meaningful use of B2, and meaningful use of Genblaze.
That last pair is the constraint that did the work. “Meaningful use” rules out the default architecture, where a team generates an asset, drops the bytes in a bucket, and calls that storage. It forces a decision about what the storage layer is actually for. What follows is written the way a technical and creative director would walk five finished pipelines against exactly that rubric: who each app is actually for, whether it holds up past the demo, and how deep the storage and orchestration choices go.
The two primitives
Backblaze B2 is the ground every one of these apps had to build on: S3-compatible object storage, sized for the sheer volume of generated assets, thumbnails, metadata, and provenance records a media pipeline throws off, free to start with 10GB included.
Genblaze, Backblaze’s open-source orchestration SDK, is what feeds it: a unified Pipeline API spanning providers like OpenAI, Google, Runway, Luma, ElevenLabs, and Stability Audio, plus models served through platforms such as GMI Cloud and NVIDIA NIM, so a team can swap providers without rewriting its orchestration. Every run produces a canonical provenance manifest that can be embedded directly into the media file itself (an .mp4, a .png, an .mp3) and persisted to B2 or any S3-compatible store.
The hackathon also partnered with GMI Cloud, giving teams easy access to open-source generative models for image, video, audio, chat, reasoning, and multimodal work, which is why it turns up as a provider more than once below.
Two primitives, six weeks, hundreds of teams, and five very different examples of rigor.
A review room for AI video ads that doesn’t make you wait for the whole render.
firstframe builds a review room for the marketing and creative teams who commission AI-generated video ads: instead of waiting on a full multi-scene render before anyone can react, it streams the first finished scene as a live HLS playlist the moment it’s ready, appending segments as later scenes land. A reviewer starts giving notes while the ad is still being made, not after.
Every generated scene is scored by an actual vision model before a human ever sees it, so obviously broken output gets caught and retried automatically rather than shipped to a reviewer’s inbox. A failover step swaps in a backup model only on a genuine provider error, never on an ordinary slow response, and once a scene clears review its master file and manifest are locked against deletion for thirty days: a guarantee the code proves by trying to delete a locked file and catching the rejection. Automated QA paired with a tamper-evident record of what was approved is what turns a generation pipeline into something a brand could actually sign off on.
On B2, the bucket’s folder structure doubles as a workflow: a scene moves through incoming, running, provenance, approved, and rejected prefixes as it clears review. Once a scene is approved, its master file and manifest get a real thirty-day write-once hold using B2’s Object Lock in Governance mode, and the code proves that isn’t just decorative by trying to delete a locked object by its version ID and catching the rejection B2 throws back. A reviewer’s application key is scoped to the readFiles capability with a name prefix restricting it to the approved folder alone. Four separate lifecycle rules cover the bucket’s different prefixes: stalled uploads in incoming have their multipart parts cancelled after 24 hours, while rejected, in-progress, and approved objects each age out on their own separate timers. The video segments themselves land in B2 as ffmpeg finishes each one, with the playlist rewritten after every segment, which makes B2 a live broadcast target rather than an archive that fills up after the fact. Reads go out as path-style presigned URLs, working around a known issue where virtual-host-style presigning fails on a private B2 bucket. B2’s own Event Notifications, five signed webhook rules, keep the review room in sync in real time, with a fallback to plain polling if an account’s Event Notifications API isn’t enabled. The app also watches B2’s own transaction cap and backs off to local disk instead of crashing when a call gets rejected for exceeding it.
On Genblaze, generation runs through an AgentLoop scored by a ThresholdEvaluator. The judge is a real vision model, an NVIDIA NIM llama-3.2-90b-vision-instruct instance grading the actual rendered keyframes, not a fixed retry count. Two pipeline branches, audio and video, fan into a single compositor node instead of running as a straight chain, a fallback_models failover is wired in and confirmed to trigger only on a genuine model error rather than a timeout, and every run carries two layers of lineage: a shared run id across scenes and loop iterations, plus a second, custom chain id layered on top of that. The team even embedded the manifest directly inside the delivered MP4, so a separate verification command can re-download and re-hash every asset the file claims exists. Along the way they filed three pull requests and an issue against Genblaze itself.
firstframe puts B2’s feature set to work in front of the reviewer. Object Lock in Governance mode, scoped application keys, lifecycle rules, Event Notifications, and presigned URLs are all load-bearing parts of the review workflow, doing visible work on every scene that moves through it. That’s architecture built to be trusted, not just to work.
A campaign generator built to double-check its own storage, re-verifying every asset the moment it’s read back.
beavous is built for the marketers and small commerce teams who need a full paid-social campaign out of a single product photo: not one hero image, but four creative concepts, sixteen cropped aspect ratios for every placement, on-label ad copy, and a portrait video reel, packaged as a verified ZIP pulled straight from B2.
A public API hands off to a private worker behind a task queue, and every campaign is namespaced to an organization so tenants stay isolated from each other. When a generation gets rejected, the app doesn’t start over. It chains a correction onto the original attempt, which is closer to how a real creative review actually works than a one-shot retry.
On B2, keys are organized hierarchically by organization and campaign rather than by content hash, because the product is multi-tenant by design. Every upload and download goes through a presigned URL, and the database never stores a raw link, only an object key and a hash. Every time an asset is read back, beavous re-downloads it and re-hashes the bytes independently, rather than trusting a manifest check alone. B2 is treated as the single, sole system of record for every generated asset: a clean, one-source-of-truth design with nothing else to keep in sync.
On Genblaze, three custom providers handle Gemini image generation, Gemini video, and Veo image-to-video, each with its own tiered pricing registered on the model. The more interesting move is a correction chain: when a generation gets rejected, its manifest becomes the parent of the next attempt, an explicit correction lineage most one-shot generation pipelines skip entirely. Prompts are marked private so the text never lands in the public manifest, only a hashed reference to it does, and before any manifest is trusted as a correction parent it gets independently re-verified, not assumed correct just because it was the app’s own write.
beavous’s answer to “do you trust your own storage” is simple: no, never. Check it again, every time. That’s a slower design than trusting your own write, and a more honest one.
A build system for generative media that can prove its own reuse, recovery, and release integrity, live, against B2.
takegraph is built for teams running a production, not a single generation: the kind of project where a script tweak halfway through shouldn’t mean re-rendering everything from scratch. It treats the whole thing like a software build: a content-addressed dependency graph that, when a spec changes, recomputes fingerprints, rebuilds only what’s actually invalidated, and reuses everything else.
A team can also re-download and re-hash the actual bytes behind any reuse, recovery, or release straight from B2, live, checking integrity themselves instead of trusting a log. That kind of self-auditing separates a pipeline meant to run unattended for months from one built to survive a single demo.
On B2, content-addressed keys use a two-level hash split so directory listings stay fast at scale, B2’s own Event Notifications (HMAC-SHA256-signed webhooks) feed a background process, and a separate reconciler periodically re-checks everything by hand in case a webhook is ever missed, coordinated across workers with a database lock so only one reconciler runs at a time. Unvalidated uploads land in a quarantine prefix backed by a real lifecycle rule that expires it automatically, and a bad key gets rejected outright rather than silently rewritten. Two least-privilege application keys, one for day-to-day work and one for releases, are each scoped to a single bucket, and CORS rules on the work bucket exist specifically to support presigned browser uploads. Verification (re-download, re-hash, prove it) is a feature of the product, not an internal tool.
On Genblaze, takegraph builds a real pipeline around the idea: a dedicated run builder, a content-addressable storage sink, manifests, and observability events tied to every step. The team kept its media-generation side lean, calling straight through the GMI Cloud connector for image and video (one of the hackathon’s partner platforms), and pointed all of its custom engineering at the layer that makes the whole pitch work: the storage and consistency system underneath.
takegraph turns storage verification into the product itself: reuse, recovery, and release integrity are things a user can ask the system to prove, live, rather than take on faith. That’s B2 treated as a system of record in the fullest sense: self-healing, event-driven, and built to survive a missed webhook without anyone noticing.
A comic studio with a provider for every model it needed, and a manifest for every frame it drew.
Ninth is built for indie comic authors and motion-comic creators working on a small budget, who revise a scene a dozen times before it’s right and can’t afford to pay for a fresh generation on every pass. Describe a story, and it writes, draws, casts, and stages it panel by panel on an editable timeline, then bakes the result to an MP4.
An edit doesn’t force a re-generation: assets are pulled from a semantic library whenever something close enough already exists, and everything the AI produces (timing, layer position, camera, bubble placement, even the artwork itself) stays editable in place afterward. That reuse-first design is backed by 126 regression tests and a render path that shares its engine with the live editor, so what a creator previews is exactly what gets exported.
Its B2 layer favors verified durability: flat content-addressed keys and a manifest per run, plus real, working endpoints for audit, restore, and backfill that report exactly what’s on B2 versus what’s only on local disk, and rebuild the difference with a SHA-256 check on every recovered file.
On Genblaze, Ninth wrote seven or eight custom providers, covering nearly every model it reached for: Azure’s image model, Gemini’s aspect ratios, Veo keyframes, Gemini’s video interpolation model, ElevenLabs music, and two separate structured-text providers. Each one exists for the same reason: to keep that artifact inside a pipeline and give it a manifest, so a generated frame always carries a record of the run and the prompt that made it.
Ninth’s mention is for discipline: the project that took “every generated frame should be reproducible and recoverable” most literally, and built the tooling to prove it on demand.
A spatial twin that gives a person’s spoken correction the same provenance as a model’s output.
Spatialize is built for venue and accessibility teams who need to rehearse a step-free route through a space before anyone sets foot in it: it turns a flat floor plan into a voice-navigable spatial twin, extracting validated 3D geometry from the plan image, then answering spoken questions with route guidance grounded in that geometry.
Because the underlying use case is safety-adjacent, nothing the model proposes is trusted outright. Every extracted floor plan, and every voice-driven edit to it, has to pass the same deterministic geometry validator before it’s accepted, so a plausible-looking but wrong route never quietly makes it into a rehearsal. When that validator rejects an attempt, it hands back the exact error, and the next attempt corrects specifically that, not a blind retry.
On B2, Spatialize favors resilience: every read goes through a short-lived, presigned link, and if B2 isn’t configured at all, the app quietly falls back to a local, path-safe store instead of failing to boot. It runs two storage paths side by side (its own hierarchical object store for run and scene state, and a separate Genblaze storage sink dedicated to generated-media provenance), keeping application state and generation history cleanly apart.
Under the hood, an agentic loop drives the floor-plan extraction, evaluated at each attempt by that same deterministic validator rather than an LLM’s opinion. Three custom text-to-speech providers form a genuine fallback ladder: a cloud model first, a lightweight third-party API next, and a self-hosted, zero-credential model as the last resort, so a narration request degrades gracefully rather than failing outright. The standout idea: a voice-driven edit to the scene creates a new version tagged as a human change, with the actual transcript kept as evidence, so every point in the final scene traces back to either a model’s manifest or a person’s own words.
Spatialize’s real achievement is conceptual: it gives a spoken correction the same seriousness as a model’s output, tracing every point in the final scene back to either a manifest or a transcript. That’s provenance thinking applied somewhere most generative pipelines never point it.
Same brief, different rigor
Every winning project found its own discipline. What they share is how many different, equally valid kinds of rigor a small team can bring to the same two primitives in six weeks. firstframe made B2’s write-once guarantee visible to a reviewer in real time. beavous made re-verification a habit instead of an afterthought. takegraph made its storage layer heal itself. Ninth made every one of its seven custom providers answerable to the same manifest. Spatialize made a human correction as provable as a model’s.
None of that shows up until a team stops treating storage and orchestration as plumbing and starts treating them as part of the design. firstframe’s win sits at the intersection of both: the same rigor that scores a generated frame with a vision model also locks the finished one against deletion.
What you can build on
Strip the five projects back and the same handful of moves keep appearing, none of which need a hackathon, a large team, or a novel model:
Object Lock in Governance mode turns “approved” from a database flag into a storage-level fact, and firstframe proves the hold is real by attempting the delete by version ID and catching the rejection B2 returns.
Bucket prefixes (incoming, running, approved, rejected) give you queue semantics without a queue, with a separate lifecycle rule per prefix so stalled multipart uploads and dead rejects expire on their own timers.
Re-download and re-hash on read, the way beavous does, because a non-deterministic pipeline produces a lot of near-identical artifacts and hashing is how you know which one you’re holding.
Event Notifications as the fast path, a periodic reconciler as the source of truth, so a dropped webhook costs you latency instead of correctness.
Content-addressed keys with a two-level hash split keep directory listings fast at scale and make reuse cheap: same fingerprint, same bytes, no second generation to pay for.
Application keys scoped one per role, like a reviewer key limited to the readFiles capability with a name prefix restricting it to the approved folder.
Correction lineage instead of retries: the rejected run’s manifest becomes the parent of the next attempt and carries the specific error forward. beavous and Spatialize arrived at this independently, which suggests it’s the general shape rather than a niche trick.
All of it is available today. B2 starts free with 10GB, and Object Lock, lifecycle rules, Event Notifications, scoped application keys, and presigned URLs are in the API from day one. Genblaze gives you one Pipeline API across OpenAI, Google, Runway, Luma, ElevenLabs, and Stability Audio, plus models served through GMI Cloud and NVIDIA NIM, and a canonical provenance manifest out of every run that embeds straight into the .mp4, .png, or .mp3 you ship.
A good first project is smaller than any of these five. Take a pipeline you already have, write a manifest for every run, persist it to B2, and add one endpoint that re-downloads and re-hashes an asset to prove the manifest is honest. That’s a weekend of work, and everything above is a variation on it. Genblaze is open source, and three of the pull requests behind firstframe are already in it.
For more ideas, the full project gallery has every entry from the hackathon, and there’s a lot in there beyond the five covered here.
Since April 2013, Backblaze has published the daily health readings of every hard drive running in our data centers. Model, serial number, failure flag, and dozens of Self-Monitoring, Analysis, and Reporting Technology (SMART) attributes, collected into a CSV for each day, released publicly every quarter, for free. We built Drive Stats as a mechanism to understand our own fleet, then published almost on a whim—so much so that the original idea is credited to two Brians. What happened next was an exciting surprise.
More than 227 papers and articles have cited Drive Stats as a primary dataset since 2018. Researchers have trained transformer architectures, long short-term memory (LSTM) networks, survival models, and gradient-boosted ensembles on it. The dataset that started as an internal reliability tool has become one of the most-cited open benchmarks in hard drive failure prediction research.
What makes Drive Stats so valuable?
Drive Stats reflects a live, continuously operating commercial fleet of drives: different manufacturers, different models, different capacities, all spinning in production Storage Pods under real workload conditions. That combination of scale and heterogeneity is rare, and it is exactly what makes the dataset especially useful to researchers.
For storage engineers and data scientists, the dataset checks every important box: real-world origin, a long time horizon, labeled failures, an open license, and an active maintainer that publishes new data each quarter. Here are a few other specifics worth understanding before diving into the research:
Download access: The full quarterly archive is available for free on the Backblaze Drive Stats page under the MIT License.
Open license: Backblaze asks three things of anyone who uses the data: that they cite Backblaze as the source, they accept responsibility for how the data is used, and they do not resell it.
SMART attributes: Each daily snapshot includes raw and normalized values for more than 70 different SMART attributes per drive. We define a drive failure in our dataset based on a few metrics, which we talk about in previousreports, as well as articles dedicated to parsing the program.
Scope and coverage: We began publishing quarterly hard drive data in 2013. The current dataset spans more than a decade of operation and covers hundreds of thousands of individual drives across manufacturers including Seagate, HGST, Western Digital, and Toshiba. Each year, we also compile data related to annualized failure rates (AFR) and lifetime failure rates across all manufacturers. Lifetime failure rates indicate the probability that a drive will fail over the course of its lifetime, while AFR indicates the probability a drive will fail during a year of operation. Rates are expressed as a percentage. Data related to lifetime failure excludes drive manufacturers with less than 500 units deployed, and all drives must have 100,000 active drive days to be included in the lifetime data set. The 2025 annual report recorded an annualized failure rate (AFR) of 1.36% across the fleet, down slightly from 1.57% in 2024, reflecting multiple factors (average drive age, technology improvements, drive size increases, cost per gigabyte, and market demand).
Most importantly: the economics of drives, and the measures people take to keep them spinning within a data center, are fundamentally different than in a consumer use case. In some ways, it’s the perfect test—the drives are always on, and we run them until they give up the ghost. In others, it’s a bit deceptive—most people don’t have alerts set up to monitor drive health in their home environments.
Of course, many do. Data is important, and for those who schedule weekly maintenance for your home networks: we love it, we’re here for it, and us too.
Academic research built on Drive Stats
The papers below represent a cross-section of research that uses Drive Stats as a primary dataset. Each represents a meaningfully different approach to the same core problem: predicting when a hard drive will fail, with enough lead time to act on it.
TFBEST: Dual-Aspect Transformer With Learnable Positional Encoding for Failure Prediction
Experiments on Seagate hard disk drive (HDD) data led the authors to propose a novel transformer architecture for predicting failures in hard drives that significantly outperformed prior state-of-the-art remaining useful life methods on the Drive Stats benchmark.
Their proposed architecture—the Temporal-Fusion Bi-Encoder Self-attention Transformer—is an encoder-decoder model trained on the full 10-year Drive Stats corpus (from 2013 to the time of article). Rather than classifying drives as failed or healthy within a fixed horizon, TFBEST predicts a sequence of days-to-failure, giving operators a window, rather than a binary alarm. The paper also introduces a confidence-margin statistic that manufacturers can use to set replacement thresholds with quantified uncertainty.
Large-Scale End-of-Life Prediction of Hard Disks in Distributed Datacenters
The authors presented a long short-term memory (LSTM) model that used understanding gleaned from Drive Stats to aid in predicting an output sequence of the number of days remaining before the possible failure of a disk. The LSTM posted a root mean square error of 0.83 during training, and 0.86 during testing across the full 10-year corpus, and generalized competitively across multiple Seagate model families.
The core architecture was an encoder-decoder LSTM network: the encoder processed a window of historical SMART readings for a given drive; the decoder produced a multistep output sequence representing the predicted days-to-failure. The model was trained and validated on all Drive Stats data available at the time, roughly 35GB, covering Seagate drive models with significant failure populations.
Leveraging Survival Analysis in Cost-Aware Deepnet for Efficient Hard Drive Failure Prediction
To address the significant imbalance of real-world datasets used for drive-failure detection—the relatively small number of failures when compared to the number of drives operating normally—the authors relied on the SMART attributes found in Drive Stats to uncover new insights into drive health and failure.
They used a dual-track approach: a deep-learning track for failure prediction and a survival-analysis track for identifying which attributes most strongly govern time-to-failure. Together, the two tracks provided both operational predictions and mechanistic insights useful for data-center management strategy.
Examining the Impact of Critical Attributes on Hard Drive Failure Times: Multi-State Models for Left-Truncated and Right-Censored Semi-Competing Risks Data
Authors: Jordan L. Oakley, Matthew Forshaw, Pete Philipson, Kevin J. Wilson
Many hard-drive failure prediction papers ask a binary question: Will this drive fail in the next N days? This paper asked a more nuanced statistical question: How do intermediate critical states defined by deteriorating SMART attributes affect the time distribution of eventual failure?
Oakley and colleagues first defined critical attributes and critical states using Drive Stats SMART readings, and then fit multistate survival models to the resulting semicompeting risks structure. These risks arise because a drive can move from healthy to critical (nonterminal) before failing (terminal), but failure can also occur without a detectable prior critical state. The multistate framework handled both pathways in a single coherent model.
The key contribution was a set of dynamic predictions of conditional survival probability that updated as the observed state of a drive changed – so operators got a live risk estimate, not a static score. Experiments on Drive Stats data confirmed that drives entering critical states are substantially more likely to fail.
AI/ML Models and Projects Built on Drive Stats
Academic papers are one signal that a dataset has earned its place in a field. Practitioners building things with it are another. A growing body of work, including open-source projects on platforms like GitHub, helps translate academic research into practical, runnable code, and provides additional confirmation of Drive Stats as a standard benchmark for the field.
The project aimed to create a binary classification model using machine-learning algorithms to predict the probability of drive failures based on SMART data, along with other features such as brand and storage capacity. It applied three methods: random forest, XGBoost, and a combined ensemble to Drive Stats SMART attribute data.
Large-Scale End-of-Life Prediction of Hard Disks in Distributed Datacenters
For practitioners building their own Drive Stats pipelines, this is one of the few public examples that addresses the full stack data ingestion, feature engineering, class imbalance, and sequence modeling, rather than demonstrating a model on a precleaned subset. Its combination of XGBoost for feature selection and LSTM for sequence prediction serves as a practical template for anyone working with raw quarterly CSV files.
Beyond the model architecture described in the academic paper section above, this project is notable for its engineering approach to handling Drive Stats at scale. The team built a preprocessing pipeline using PostgreSQL to ingest, filter, and join the quarterly files; used XGBoost across the full SMART attribute set; and applied interpolation to fill gaps, before feeding sequences to the encoder-decoder LSTM.
Are There Manufacturer Differences in Hard-Drive Reliability?
Contributor: Christoph Siemroth, Yeomyung Park
Venue: IEEE Transactions on Cloud Computing
Researchers used Backblaze’s large data-center dataset to compare failure rates across four manufacturers (HGST, Seagate, Toshiba, and Western Digital). Duration regression models controlled for drive age, capacity, and form-factor, and the findings concluded that HGST drives fail least often (about 42% of Seagate’s failure rate. However, WD drives outperformed Seagate but fared worse than HGST. Toshiba’s failure rate is similar to Seagate’s.
The study revealed a significant reliability gap between HGST and Seagate, doubling the financial burden for large-scale operators related to replacement-related labor and other costs. Drive failure analytics highlighted in the study can be used by large-scale operators to forecast future costs, informing procurement decisions.
Drive Stats research at a glance
Paper / Project
Authors
Venue / Platform
What It Predicted / Built
TFBEST: Dual-Aspect Transformer With Learnable Positional Encoding for Failure Prediction
Mohapatra, Sengupta
arXiv (2309.02641)
A novel, high-performing transformer architecture for predicting hard-drive failures
Large-Scale End-of-Life Prediction of Hard Disks in Distributed Datacenters
Mohapatra, Coursey, Sengupta
IEEE
An LSTM helps to predict the number of days to a given disk’s failure to a high accuracy level
Leveraging Survival Analysis in Cost-Aware Deepnet for Efficient Hard Drive Failure Prediction
Ahmed, Green II
Neural Computing and Applications, Vol. 37
Operational predictions and mechanistic insights for data-center management strategy
Examining the Impact of Critical Attributes on Hard Drive Failure Times
Oakley, Forshaw, Philipson, Wilson
Applied Stochastic Models in Business and Industry, Vol. 40, Issue 3
Confirmation that drives entering critical states defined by deteriorating SMART attributes are substantially more likely to fail
HDD Failure Prediction Using Machine Learning
warc0s
GitHub
A binary classification model using machine-learning algorithms to predict the probability of drive failures
Large-Scale End-of-Life Prediction of Hard Disks in Distributed Datacenters
rohanmohapatra
GitHub
A practical template for practitioners working with raw quarterly CSV files to build their own prediction pipelines
Are There Manufacturer Differences in Hard-Drive Reliability?
Siemroth, Christoph Park, Yeomyung
IEEE
A comparison of hard-drive reliability across four manufacturers, using data regression models.
The case for open data
The breadth of research is a direct result of open datasets. These citations occur because the data was consistently available every quarter for more than a decade—and it helps that we built a community of similarly interested people, too.
For research communities, open datasets function the way open-source libraries do: they create a shared foundation that everyone can build on and compare against. Drive Stats has earned that role in hard drive failure prediction by showing up reliably for over 13 years. A few things make open data particularly useful:
Ecosystem reach. Drive Stats doesn’t exist in isolation: We publish it on Hugging Face, where it sits alongside hundreds of thousands of open datasets, across domains from natural language processing to genomics. Kaggle hosts it alongside tens of thousands of community notebooks and kernels.
Reproducibility. Drive Stats is public, permanently archived, and available for download. A paper published today cannot recreate historical data. There’s not really a corollary in a field where most real-world fleet data is proprietary and inaccessible—but we’d love people to join us.
Research velocity. When data is freely available, there’s no need to spend months negotiating access agreements. You simply download Drive Stats, read the schema documentation, and start building. The papers mentioned in this article collectively span transformer architectures, survival models, deep neural networks, and gradient boosting, all on the same dataset. You can’t necessarily call it a direct comparison, but it does make one thing clear: hardware is central to the cloud conversation.
Sign up for the Drive Stats newsletter
The same data that powers academic research also powers our own reliability reporting: The annualized failure rates, SMART attribute analysis, and transparency have made Drive Stats a standard in the field for more than a decade. And that data keeps coming, with a new release every quarter, since 2013.
If you’re working on failure prediction, predictive maintenance, or just want a real-world labeled dataset for benchmarking, this is the one researchers keep reaching for.
Backblaze Drive Stats is a publicly available dataset of daily hard drive health snapshots from our data centers. Published quarterly since 2013, it includes Self-Monitoring, Analysis, and Reporting Technology (SMART) attribute readings; failure labels; and model information for hundreds of thousands of drives. The data is free to download from the Backblaze Drive Stats page.
Has Drive Stats been used in academic research?
Yes. More than 227 papers and articles have cited Drive Stats as a source since 2018. Researchers have used it to develop and test hard drive failure prediction models, survival analysis frameworks, and deep learning architectures across venues including IEEE, Springer, Wiley, and arXiv.
Which AI/ML models were trained on the Backblaze dataset?
Researchers have trained a wide range of models on Drive Stats data, including long short-term memory networks, transformer architectures, 1D convolutional neural networks, gradient-boosted trees, survival analysis models, and ensemble methods.
Is the Backblaze Drive Stats dataset on Hugging Face?
We publish Drive Stats on Hugging Face at huggingface.co/datasets/backblaze/Drive_Stats. The repository contains over 388 million records and grows by more than 240,000 records per day. It is append-only, meaning daily snapshots are never updated or deleted once written, making it particularly useful for reproducible research.
Why do researchers use open hard drive datasets?
Open datasets like Drive Stats allow researchers to benchmark models against consistent real-world data, reproduce published results, and build on prior work without access to proprietary fleet data. Because Drive Stats reflects a heterogeneous, production-scale environment rather than a lab setting, it provides a uniquely credible benchmark for evaluating failure prediction methods.
How many times has Drive Stats been cited in research?
More than 227 papers and articles have cited Drive Stats since 2018. The actual number continues to grow as researchers publish new work on hard drive reliability, predictive maintenance, and artificial intelligence/machine learning (AI/ML) model benchmarking.
At GTC this year, Jensen Huang put a number on a problem that most enterprise AI teams already feel but struggle to articulate: about 90% of what gets generated every year is unstructured data. This includes PDFs, emails, video, contracts, support tickets, and voice recordings.
Until recently, that data was, in his words, almost entirely useless. You could store it or search it by keyword, but machines couldn’t read it, understand what it meant, or draw any connections between it and other content.
GenAI has changed that. RAG pipelines, vector databases, and embedding workflows give machines a way to process unstructured content, comprehend its meaning, and synthesize information across the entire enterprise.
The problem is that these tools put a completely different set of demands on storage than anything that came before them. And most enterprise storage wasn’t built for it.
The access pattern no one planned for
Traditional enterprise storage assumes that data gets written and then occasionally retrieved. The architecture for it was built with hierarchical file systems and tiered storage (hot archives for data that gets accessed regularly, cold archives for anything that hasn’t been touched in a while). It worked well for decades, until RAG pipelines broke it.
A RAG pipeline works by reading your documents and converting them into a format the AI can search. But that conversion process is not a one-time event. It has to be run repeatedly as you test, adjust, and update. Every adjustment means re-reading the same content, re-processing it, and re-writing the results back to storage.
For example, the pipeline breaks documents into chunks because the AI can only search effectively when content is divided into manageable pieces. The size of those chunks directly affects how well the AI answers questions. Too large, and the AI retrieves too much irrelevant content. Too small, and it loses context.
Whenever you have to update your underlying documents, the pipeline has to re-read and re-process everything affected by the change. The same is true when upgrading to a newer AI model, because the new model produces different representations of the same content and requires everything to be re-converted.
While the pipeline is reading, it’s also saving converted content, updated search indices, and checkpoints. And what it saves is tiny. Embeddings, vector chunks, and metadata objects are often just a few kilobytes each, but a production RAG pipeline generates millions of them. Legacy storage treats every save operation the same regardless of file size, so saving a million tiny files costs nearly as much as saving a million large ones.
Tiered storage exacerbates this even more. These systems judge which data is still needed and which can be moved to cheaper, slower storage based on how recently something was accessed. But a RAG pipeline might go quiet for weeks between runs and then suddenly need everything at once. Or it might re-index documents on a schedule that the storage system doesn’t recognize as regular access. Either way, the tiered system sees cold data and moves it to a slower tier. When the pipeline needs that data back, it waits and pays a retrieval fee before it can proceed. That cost and delay compounds across millions of objects and dozens of pipeline runs.
The governance problem hiding inside your index
When a RAG pipeline indexes documents into a vector database, it takes a snapshot of your data at that moment in time. The embeddings it generates reflect the content of those documents as they existed when the pipeline ran. When the original documents are updated or changed, the snapshot doesn’t change with them. The vector index keeps pointing to the old version, and the AI keeps retrieving it and presenting it as current fact.
To a user, that looks like hallucination. The model is confidently citing a document that no longer says what the model thinks it says. But the failure isn’t happening inside the model; it’s happening at the storage layer. Specifically, in the gap between when source documents change and when the index gets updated.
Most enterprises running RAG pipelines today have no systematic way to close that gap. Document management systems aren’t built to notify downstream AI pipelines when content changes. Vector databases aren’t built to automatically invalidate and re-index affected embeddings. The result is an AI system that’s architecturally incapable of knowing when its knowledge has gone stale, so it presents outdated information with the same confidence as current information.
What GenAI storage actually requires
Closing the gap between what RAG pipelines demand and what legacy storage delivers requires three things:
Sustained high throughput. RAG pipelines generate continuous, mixed traffic at every stage of the process—ingesting documents, processing them, writing the results to an index, and querying that index—often all running simultaneously. Storage must be able to move data in and out fast enough to match the speed at which the pipeline generates requests, and sustain that speed across the entire duration of a run, not just handle short bursts.
Always-hot architecture with no tiering. Every document needs to be immediately accessible regardless of when it was last touched. This is not just for pipeline performance, but also because slow re-indexing is what produces stale outputs. When source documents change, the pipeline needs to re-index immediately. Storage that puts documents behind retrieval delays or tiering penalties widens the window in which the AI is working from outdated information and presenting it as fact.
Free data movement. RAG workflows span multiple GPU providers, MLOps platforms, and inference environments across multiple regions. Storage that charges egress fees for every transfer turns architecture decisions into cost calculations and discourages the experimentation that gets AI systems from prototype to production.
The storage layer that doesn’t get in the way
Backblaze B2 Overdrive is built around exactly these requirements. It is always-hot object storage with no tiering, no retrieval delays, and no waiting for data to be promoted before a pipeline can run. Every document is always available, regardless of how recently it was written, which means when the pipeline needs to re-index, storage never slows it down.
At up to 1Tbps sustained throughput, it keeps high-volume RAG pipelines fed continuously through the constant mixed traffic of reads and writes they generate. Because egress is free, teams can move data between storage and compute across regions and providers without a cost penalty. And S3 compatibility lets B2 Overdrive drop into existing LangChain, LlamaIndex, and vector database pipelines without rewrites.
Ninety percent of enterprise data is sitting in formats that, until recently, machines couldn’t do anything useful with. The tools to unlock it now exist. How well they perform and what they cost to run depends largely on the storage underneath them.
Organizations rarely struggle with a lack of storage options. More often, they struggle with determining which solution best fits the way their data is created, accessed, and protected: backup versus cloud storage.
That’s especially true when evaluating backup and cloud storage solutions.
The terms are often used interchangeably, but backup and cloud storage are designed to solve different problems. Understanding those differences can help you build a more effective data protection strategy—whether you’re protecting a personal laptop, a growing media archive, employee endpoints, or critical business data.
At Backblaze, Computer Backup and B2 Cloud Storage serve distinct purposes. For some customers, one solution is the clear choice. For others, the strongest approach combines both.
Before comparing features, it’s helpful to start with a few foundational questions.
The answers often reveal whether you’re primarily trying to protect a computer, store data in the cloud, or address both needs at the same time.
When the goal is protecting a computer
For many individuals and businesses, the most important data still lives on laptops, desktops, and attached external drives.
A photographer may keep active projects on a workstation. A consultant may store client files locally. A small business may rely on employee laptops as the primary location where work is created and managed.
In these situations, the primary concern isn’t cloud infrastructure. It’s protecting the device where the work happens.
That’s where Backblaze Computer Backup fits.
Computer Backup is designed to automatically protect data stored on a Mac or Windows computer, including connected external hard drives (but not NAS devices). Once installed, it runs continuously in the background, backing up files without requiring users to manually manage folders, storage allocations, or backup schedules. For organizations looking to protect NAS data, B2 Cloud Storage can serve as a backup destination through a variety of supported third-party backup and sync tools.
The value becomes clear when something goes wrong:
A laptop is stolen.
A hard drive fails.
Files are accidentally deleted.
A ransomware attack impacts local data.
A computer needs to be restored after a hardware issue.
In each case, the goal is recovery.
Computer Backup is often a good fit when:
Your most important data lives on a computer.
You want automatic, continuous protection.
You need to recover from device loss, hardware failure, or accidental deletion.
You want a solution that requires minimal administration.
Your primary concern is protecting endpoints.
For many professionals, families, and small businesses, those requirements align closely with their day-to-day reality.
When the goal is storing and managing data in the cloud
As organizations grow, data often becomes less tied to individual devices.
Files are shared across teams. Backup software protects servers and NAS devices. Applications generate and consume data continuously. Data needs to remain accessible and manageable independent of the original device, whether that’s for long-term retention, team access, application workflows, or infrastructure backups.
At that point, the challenge shifts from protecting a computer to managing data itself.
That’s where Backblaze B2 Cloud Storage comes in.
Unlike endpoint backup, cloud object storage is designed to store data independently of any single device. Data can be uploaded, accessed, managed, shared, and integrated into workflows across users, systems, and applications.
Organizations use B2 Cloud Storage for a wide range of use cases, including:
In these environments, accessibility, scalability, and integration often matter just as much as protection.
B2 Cloud Storage is often a good fit when:
Data needs to exist independently of a specific computer.
Multiple users or systems require access.
You need API-based access and automation.
You use third-party backup software that requires cloud object storage.
You need centralized storage for growing datasets.
You are building applications or data-driven workflows.
The focus isn’t on protecting a device. It’s on providing a durable, accessible home for data.
Understanding the data lifecycle
One reason organizations often use both backup and cloud storage is that data requirements change over time.
Consider a video production team.
While a project is actively being edited, the files may live on a workstation and several external drives. During that phase, protecting the editing environment is critical.
Once the project is complete, however, the priorities often change. The team may need to retain the content for future revisions, client requests, or compliance purposes. The files are no longer active, but they still need to remain available.
The same pattern appears across industries.
Architectural firms retain project files after construction is complete. Marketing teams archive campaign assets. Businesses preserve records for operational or regulatory reasons.
Not all data serves the same purpose throughout its lifecycle.
Active data often benefits from continuous endpoint protection, particularly when it lives on laptops, workstations, or attached drives. As that data ages, becomes shared across teams, or moves into long-term retention, cloud storage often becomes a more appropriate solution.
This is one reason many organizations use both Computer Backup and B2 Cloud Storage. The two solutions address different stages of the data lifecycle rather than competing for the same role.
When your storage requirements change
A common misconception is that organizations eventually “graduate” from backup to cloud storage. In reality, most environments become more complex over time, adding new requirements rather than replacing existing ones. As data volumes grow, teams collaborate across more systems, and retention needs increase, organizations often find themselves adding cloud storage to support those evolving demands. The shift isn’t typically about moving away from backup—it’s about addressing new use cases that emerge as data becomes more distributed, accessible, and valuable to the business. Common signs that additional cloud storage may make sense include:
Your data is no longer centered around one device
When multiple people need access to the same information, storing everything on a single workstation becomes limiting.
You’re building long-term archives
Completed projects, historical records, and large media libraries often benefit from dedicated cloud storage.
You’re adding automation and integrations
Applications, backup platforms, and workflows frequently require API-accessible storage.
You’re managing more than endpoints
As NAS devices, servers, and infrastructure become part of the environment, storage requirements often extend beyond individual computers.
In these scenarios, cloud storage isn’t replacing endpoint backup. It’s addressing new requirements.
The blind spot many cloud storage users discover
The reverse scenario is also common. An organization adopts cloud storage and establishes a centralized repository for important data, only to discover that important risks still exist at the endpoint level. An employee may accidentally delete a local project folder, lose a laptop, or experience a workstation failure before files have been synchronized elsewhere. Cloud storage protects the data stored in cloud storage, but it does not automatically protect every device where work is created. This is one reason endpoint backup remains an important part of many modern data protection strategies. The risks are different, and each solution is designed to address a different recovery scenario.
Why many organizations use both computer backup and cloud storage
One of the most persistent myths in data protection is that a single tool should solve every challenge. In practice, resilient environments are typically built in layers, with different solutions addressing different risks and recovery scenarios. Employee laptops may be protected with Computer Backup, while a NAS backs up to B2 Cloud Storage. Completed projects may be archived in the cloud while active work remains protected on local devices. Together, these layers create a more comprehensive approach to protecting data throughout its lifecycle.
Example: Creative teams
For creative teams, active projects often live on editing workstations and attached storage where they are constantly being updated. Computer Backup helps protect that work in progress, while completed projects can be moved to B2 Cloud Storage for long-term retention, future revisions, or client requests. This approach allows teams to safeguard current work without keeping every finished project on production systems.
Example: Growing businesses
As businesses grow, their data often becomes distributed across employee devices, shared storage, and business applications. Computer Backup can help protect employee endpoints where work is created, while B2 Cloud Storage provides a centralized location for shared assets, backups, and archives. Together, they support both day-to-day operations and longer-term data retention needs.
Example: IT and infrastructure teams
IT teams frequently manage a mix of endpoints, servers, NAS devices, and other business systems. In these environments, B2 Cloud Storage often serves as a destination for infrastructure backups, while Computer Backup protects employee devices that may not be covered by server or storage backup workflows. Rather than competing with one another, the two solutions often work together as part of a broader data protection strategy.
A quick comparison
Question
Computer Backup
B2 Cloud Storage
Is the primary goal protecting a computer?
Yes
No
Is it designed to protect endpoint data automatically?
Yes
No
Is the data primarily tied to a specific device?
Yes
Not necessarily
Is it designed for shared access across users, systems, or applications?
No
Yes
Is API access a core feature?
No
Yes
Can it serve as a destination for third-party backup tools?
No
Yes
Is the primary goal storing and managing cloud-resident data?
No
Yes
Choosing the right solution
The decision ultimately comes down to what you’re trying to protect and how your data is used.
If your primary concern is recovering files from a lost, stolen, damaged, or compromised computer, Computer Backup is likely the right starting point.
If you need scalable cloud storage for archives, applications, infrastructure backups, or shared datasets, B2 Cloud Storage is likely the better fit.
And if your environment includes both endpoints and cloud-resident data—as many organizations do—you may benefit from using both.
The most effective data protection strategies rarely rely on a single layer. They account for where data is created, where it lives, and how it needs to be recovered.
Understanding those requirements is often the first step toward choosing the right solution.
Extensive news coverage and analyst reports on AI missing productivity and ROI targets mean that AI failure is something of a hot topic. There’s no arguing that some AI initiatives are misguided, including replacing entire specialist teams with AI. For others, the issue actually lies with data knowledge and readiness–Gartner predicts that through 2026, organizations will abandon 60% of AI projects unsupported by AI-ready data, and S&P Global recently highlighted the importance of ingesting previously overlooked or unknown data to discover interdependencies in risk management.
The pressure to move at the perceived speed of AI makes it easy to skip or rush important prep work. Now that major AI pilots have been up and running, supplementing or sometimes entirely replacing select business functions with mixed results, this exposes an already known problem among AI experts: implementing AI too quickly and ignoring the importance of keeping human experts in the loop increases your threshold for error. This is especially prevalent for both internal AI tools that are meant to augment key roles, and customer-facing AI applications that are supposed to increase accessibility to an outcome, such as generating a lifelike video based on natural language prompts.
What’s causing this? It’s not just the LLMs–it’s the data. Now that human experts are more aware of what AI can get wrong, we’re going back to basics to help you get it right. This starts at the very beginning: curating, ingesting, and storing data using infrastructure that’s actually designed for moving data quickly, and without financial penalty.
Data ingest is the process of any type of data being added to a designated collection destination, whether that’s a specific file folder, database, or object storage bucket. For a lot of applications, data ingest is frequent or nearly constant–busy e-commerce sites with a constant flow of customer transactions and feedback, live video feeds, and combining real-time data sources like pairing security footage with physical building security sensors. This is called streaming ingestion–and streaming ingestion being the foundation of data collection for various types of AI is one of the key reasons why storage is an AI infrastructure problem that often flies under the radar until there’s a serious problem.
Streaming ingestion requires constant low-latency access to the data storage repository to prevent data upload lags and errors.
Streaming ingestion for video and other large files requires high rate limits and high-throughput networking capabilities to optimize upload times, especially when an application requires a file to be ready for processing in seconds/minutes instead of hours/days.
Running out of storage capacity is not an option for model performance, and for compliance and auditing purposes–streaming ingestion requires constant access to storage that is as close to infinite as possible.
Automation from the start
In addition to the files themselves, setting up a highly effective AI data pipeline involves building automation from the very beginning to immediately allocate files to the right bucket using taxonomy and collect and store file metadata to begin data aggregation critical for future labeling and processing.
Why taxonomy is critical for AI
The file itself is the data source. Navigating your dataset starts with implementing a taxonomy that makes your dataset highly searchable as your data grows from a few thousand for your first round of training to millions for an AI application operating in production.
Taxonomy is essentially your file storage structure, or how files are automatically “nested” and relationships between files are built immediately upon ingest. When you’re just getting started, developing your taxonomy helps you stay organized and ready to go searching for a specific file when your coworker doesn’t believe what the data is saying. When you’re working in established teams, implementing a new taxonomy or showing that you understand the importance of following an established taxonomy builds a contract of understanding between you and your data engineering or ML teams. (AKA, changing taxonomy mid-project is a big undertaking with ripple effects, and should not be taken lightly.)
A simple example taxonomy for ingesting raw files can look like this:
/<source>/<modality>/<status>/<date>/filename
Taxonomy should also reflect what the data is actually doing or will do, not just the file type and data source. The goal is to make every object self-describing at write time, so downstream training pipelines can filter, version, and partition without touching the data itself.
Partition by date/time at the prefix level so you can use time-range queries and lifecycle rules without scanning everything
Include modality (video, audio, image, text) as a top-level segment so cross-modal datasets stay logically separated but co-located in the same bucket
Use a UUID or content hash as the filename — never rely on source filenames, which are inconsistent and collision-prone at scale
Use B2 bucket policies or object tagging rules to reject objects written to non-conforming prefixes
Maintain a human-readable taxonomy manifest (taxonomy.json at bucket root) that documents each top-level prefix and its schema
For video specifically, consider a separate prefix segment for resolution or codec: …/video/4k/h264/… — this pays off quickly when training jobs need to filter by input spec
Retain all the metadata
Write metadata as object tags and custom headers at ingest using per-object user-defined metadata at PUT, such as:
This way, the metadata travels with the object and is always returned on HEAD requests without a separate lookup.
But to build out a rich dataset, you will need even more metadata in the form of sidecar metadata files. Retain annotations, bounding boxes, ground-truth labels, licensing info, and consent flag metadata by creating a sidecar JSON with a .meta.json suffix. For the warehouse camera video example listed above, you would end up with two files that look like this:
Enable object versioning on training buckets. If a labeling pipeline updates an annotation, it should write a new version (or a new sidecar) rather than overwriting — training reproducibility depends on knowing which metadata was present at the time a dataset was compiled.
Expose metadata to training pipelines via a manifest
Rather than having training jobs scan bucket prefixes directly, generate a manifest file (JSONL or Parquet) at the end of each ingest batch that enumerates every object key + its full metadata. Tools like PyTorch’s WebDataset and HuggingFace datasets can load directly from these manifests, and it decouples the training job from needing object storage credentials for discovery.
Deciding on storage while evaluating data
To summarize, this is why choosing your storage destination based on your current (or, if you’re already undergoing a data management transformation, future-state) scenario is critical:
Ingest type: Does your storage provide enough network bandwidth to capture the correct type of data in real time (especially for video?)
Data type: Will large files potentially incur large upfront costs with upload fees?
Data processing workflow: Does your storage have capacity headroom for file multiplication during processing, and does the cost structure allow this to happen without potentially draining infrastructure budgets?
Even with Backblaze B2’s hot storage at cold storage pricing, it may be beneficial for you to tier your storage based on your ingestion type, and how frequently the data will be accessed for training.
Batch ingestion is better suited for mid to lower performance storage, as this is typically used for historical datasets or a set schedule of pre-determined data updates, such as jobs pulling from relational databases or CSV uploads once a day or once per week.
Streaming ingestion is well-suited for hot storage to support a continuous stream of real-time (or near-real-time) data processing, such as from social media feeds and high-volume e-commerce AI helper agents.
Hybrid ingestion uses a combination of batch and streaming ingestion to handle both historical and real-time data requirements for AI models.
Surprise retrieval penalties for model training can happen even while building out your MVP or proof of concept–so avoid a storage headache and start building your pipeline at data ingest with Backblaze B2 $6.95 per TB/month. Create an account to get started with 10GB for free, or contact our storage experts for assistance with migrations and more.
Looking for more info on data ingest? Watch the on-demand webinar that dives into more details about data ingest with Backblaze’s Director of Applied AI Jeronimo De Leon.
Welcome to the Q2 2026 Network Stats report. While we’ve been tracking trends since December 2023, this is the third quarter since we operationalized the dataset and re-launched the series, allowing ourselves to make direct, quarter-over-quarter comparisons. Why? Because AI workloads were changing traffic patterns across Backblaze’s network, and reshaping the internet.
With three quarters of historical data now available, we’re moving beyond measuring traffic volumes. We’ve been able to spot trends and start drawing conclusions—how predictable or unpredictable those workloads really are, and what that means for infrastructure that supports the next generation of AI applications.
Check out past Network Stats reports
If you’re interested in some of the trends we’ve spotted in previous reports, you can review the past reports here: Q1 2026 Q4 2025 Q3 2025
Previous analysis has been based on the amount of network traffic in bits flowing across in or out of our network, the number of bits and participants per TCP session (our coined “magnitude” metric), and regional geographic trends. In this report, you’ll find charts and heatmaps for the metrics that we’ve been reporting on over the past year, but we’re also going to use statistical analysis to answer a practical question: What kinds of traffic patterns do AI workloads create, and how should infrastructure evolve to support them?
Traffic from neocloud and hyperscaler networks are proving to be very dynamic in nature, and that’s what we’re going to explore in this quarterly report: variance.
Join the webinar
Want to hear more? Join Brent Nowak, Manager, Network Engineering, and Stephanie Doyle, Sr. Manager, Market Intelligence and Keeper of Stats, live on Tuesday, July 28, 2026 at 11:30 a.m. PT / 2:30 p.m. ET to walk through the data and spot the latest trends.
Can’t make it live? Register anyway and we’ll send you the recording.
Why look at variance?
Variance is a deep topic to explore, which involves modeling our traffic patterns against a known baseline. To analyze variance, we built a new time-series dataset using 10-minute traffic samples and modeled traffic behavior against statistical baselines. This lets us distinguish stable, predictable traffic from highly volatile workloads that demand different infrastructure planning.
I refreshed my statistics knowledge, created a new database to hold a timeseries dataset, and spent a few nights experimenting with the SciPy Python library in order to not only produce pretty graphs, but to generate a signal for us to interpret.
The types of questions that we’re interested in answering from the variance signals that affect our business include:
How quickly are AI workloads changing capacity requirements?
Which traffic patterns require different network architecture? And does our current architecture support what we’re growth modeling into the future?
Which signals represent lasting trends versus temporary spikes?
These are big questions! And exciting ones as Backblaze looks to support today and tomorrow’s workflows.
With that, let’s refresh our existing charts with this quarter’s data before diving into the new analysis on variance.
Summer heat-up
The stacked area graph below shows total traffic by network type over time updated with the most current data.
Total traffic by network type.
CDN traffic: New baseline of activity with a 66% increase from last year.
Hosting traffic: The hosting category (the light orange layer right above CDN) has remained incredibly rigid. Unlike neocloud or hyperscalers, which expand and contract elastically, hosting traffic has maintained a nearly identical bandwidth footprint for over a year.
Hyperscaler traffic: Hyperscaler traffic also followed the neocloud pattern, with the lowest amount of activity in January and remaining steady into June. Internal data sources show new workloads across all of the major hyperscalers in the past quarter.
Neocloud traffic: After a low point of activity in January, activity increased rapidly into March and has remained high until June. Internal telemetry shows that not only the amount of neocloud traffic increased in Q1 into Q2, but the number of neoclouds that we are interacting with has increased.
ISP-regional traffic: This was the dominant driver of the massive traffic spike in October 2025. While it dropped significantly into January 2026, it has aggressively rebounded through Q2 2026 and is currently the largest single driver of volume alongside neocloud.
Migration traffic: This traffic includes one way migrations into our environment, primarily serviced by partners such as Flexify.IO. We have migrations running all the time, but we can visually see large amounts around November 2025 and March of 2026.
Over the entire one-year graph range, Backblaze’s total platform traffic experienced volatility, peaking in October 2025 before seeing a multi-tier contraction down to a January 2026 winter baseline. Following this, in both Q1 and Q2 of 2026 we’ve observed an increase of activity led by a rebound in ISP-regional and AI-focused neocloud traffic. An additional standout is CDN traffic, which achieved a permanent and substantial new activity baseline, growing roughly 66% year-over-year.
Heatmaps: How and where data moves
To better understand our network activity, we isolated variables like region and types of provider. Here are the standard definitions we use each report:
Total traffic volume: Where did we send and receive the most traffic?
Magnitude: Where were the data transfers with the most bits per unique IP address?
Uniqueness: What does the number of distinct IP addresses look like?
Quick terminology refresher
Regions US-West: Our largest and longest-running region US-East: Region with the most observed proximity to neocloud infrastructure CA-East: Our newest region in Canada. EU-Central: Our EU region.
Network Types CDN: Networks that use Backblaze as an origin store for content delivery. Hosting: Traditional hosting providers that runs workloads like physical or virtual servers for web, database, or application tasks. Hyperscaler: Large, traditional cloud providers. ISP-regional: Local or regional ISPs; think of these as the “last mile” paths as these networks are very close to customer equipment and efficient. ISP Tier1: National or international ISPs that carry our traffic long distances. Neocloud: AI-focused compute networks.
Heatmap #1: Where did we send and receive the most traffic?
ISP-regional traffic is a hotspot for US-West, as expected. This region has the largest internet exchange (IX) and server footprint behind it. Neocloud traffic remains concentrated in the US-East, but for this quarter traffic increased in the US-West and EU-Central regions. Another standout this quarter is more hyperscaler activity in EU-Central than the previous quarter.
Total number of bits transferred across our regions to each network type for Q2 2026.
Heatmap #2: Where were the data transfers with the most magnitude (bits per IP address)?
Another metric we record is bits per IP or what we term “magnitude.” This combination of the amount of traffic transferred with how many actors are involved per network is a good proxy to measure how heavy or impactful individual data flows are. In short:
High volume, many IPs: Easier to distribute and load-balance across infrastructure. And many source and destination pairs means that we can traffic engineer at the WAN layer, sending some traffic over one provider and some over another.
High volume, few IPs: More difficult, but more interesting, from a NetEng perspective.
Traffic magnitude is currently a driver of decisions for capacity and growth plans. Our US-East region continues to have a high concentration of high bandwidth transfers between a small number of hosts. New for this quarter is an uptick in traffic magnitude in our EU-Central region.
This could be an indication of more geographic spread of AI related workflows as for every quarter that we’ve reported on the metric value, we have seen more diversity into US-West and EU-Central outside of the concentration in US-East. We will continue to watch this trend.
Magnitude transferred across our regions to each network type for Q2 2026.
Heatmap #3: How many unique addresses do we interact with?
Not every graph or heatmap has to show something dramatic. Sometimes it’s good to see exactly what you expect quarter over quarter in a data series. This is especially true for our uniqueness metric, measuring the number of distinct IP addresses per network time.
We interact with the most number of parties out of our US-West region. It’s the most mature and serves a large amount of ISP-regional consumers, so the consistency of the uniqueness metric is a good sanity check on our dataset.
US-West shows the highest overall uniqueness, driven by its larger number of data centers and mix of workloads.
Neocloud traffic, by contrast, tends to involve fewer, more persistent endpoints, consistent with AI pipelines that rely on stable, long-standing connections between storage and compute.
Communication uniqueness across our regions to each network type for Q2 2026.
Neocloud and hyperscaler traffic vs predictive patterns
This next set of charts shows a deeper dive into the metrics associated with neocloud and hyperscalers over time. The contrast between a more “traditional” workload (e.g., CDN, hosting, and ISP regional traffic) and emerging trends with neoclouds and hyperscalers is the easiest place to see the shift in network traffic profiles. The latter represents bursty, high magnitude traffic that reshapes conversations around network planning.
Chart #1: What’s the magnitude of neocloud and hyperscaler traffic over time?
Hyperscaler and neocloud network magnitude May 2025 – May 2026.
Following a highly concentrated, low-magnitude baseline for both categories in January and February 2026, Q1 closed with a dramatic March surge where several individual neocloud networks spiked massively.
Moving into Q2 2026 (April through June), while the absolute highest neocloud peaks compressed slightly downward compared to that March anomaly, the overall volume of high-magnitude neocloud workflows multiplied significantly, resulting in a much denser cluster of active endpoints staying consistently high quarter-over-quarter.
Hyperscaler endpoints experienced a steady and noticeable upward move over the course of Q2, with multiple data points breaking out of their typical floor by May and June.
Ultimately, neocloud retained its dominant, high-magnitude presence across both Q1 and Q2 quarters, while hyperscalers saw a distinct and steady escalation in individual workload sizes.
Heatmap #1 and #2: How dynamic are neocloud and hyperscaler traffic patterns?
Neocloud related traffic continues to show strong concentrations in our US-East region with recent growth March into June. Hyperscaler traffic is the most variable when we compare it to last quarter’s heatmap. There is a new, more distributed concentration across all our three largest regions—US-East, US-West, and EU-Central.
Together with the trends we’ve reported over the past year, these results suggest AI workloads on the Backblaze network are becoming geographically more distributed rather than remaining concentrated in a single region. Note the caveat: it’s possible, even probable, that there’s a macro trend about geographical dispersion of AI data, but it’s important also that Backblaze has become increasingly known as a trusted infrastructure provider specifically in this space.
Layer on the fact that AI workloads can be reflective of fewer players with more data (see also: magnitude or elephant workflows), and what you have is difficulty understanding whether this is a macro trend, or Backblaze specific. We’ll keep our eyes on the data as it develops.
Neocloud monthly traffic totals by region for May 2025–June 2026.
Hyperscaler monthly traffic totals by region for May 2025–June 2026.
Heatmap #3, #4, and #5: How dynamic are CDN, hosting, and ISP-regional traffic patterns?
We’re grouping CDN, hosting, and ISP regional types together because they represent a “steady-state” for us as network operators. These patterns are predictable, spread out over time, and generally do not change month-to-month.
For Q2, we saw the concentration of CDN in US-West remain steady with traffic growth in our US-East region. Hosting traffic is showing a new pattern, with more activity in our EU-Central region starting in April into June.
CDN monthly traffic totals by region from May 2025–June 2026.
Hosting monthly traffic totals by region for March 2025–June 2026.
ISP-regional monthly traffic totals by region for March 2025–June 2026.
Variance study methodology
For our new variance study we needed more granular traffic sampling data than aggregated weekly or monthly totals. Ten minute sample data gave us a balance between sampling fidelity, data warehousing storage, and query time when iterating on the project idea.
Here’s a sample of anonymized data in one region, for one hour, for one ASN (network), with ingress and egress 95th bitrate percentage values:
Anonymized Timeseries Sample Example
datetime
region
asn
ingress
egress
2026-05-01T00:00:00
us-east
asn-number
4408643576.86
72810025561.08
2026-05-01T00:10:00
us-east
asn-number
4202884722.26
72153643081.09
2026-05-01T00:20:00
us-east
asn-number
4282470297.97
72840197796.70
2026-05-01T00:30:00
us-east
asn-number
4462602109.34
74194149854.89
2026-05-01T00:40:00
us-east
asn-number
4011477298.04
73431072317.13
2026-05-01T00:50:00
us-east
asn-number
3919542094.52
71051545108.21
Understanding the use of variance
Raw traffic metrics (total gigabits per second) tell us how much data is moving. Variance tells us how consistently it moves.
Stable traffic is easier to plan for. Highly variable traffic requires more flexible network design and additional capacity planning—it reflects the bursty nature of AI training and inference workflows, where compute clusters can scale rapidly and move enormous datasets over short periods.
Here’s how to read the analysis:
The shape of the bell curves (right column): A very tall, narrow peak indicates low variance. This means the traffic behaves predictably and stays clustered close to its baseline average. A short, wide, flattened curve indicates high variance, meaning the traffic is highly volatile, subject to massive sudden swings, and much harder to provision for.
The interplay of ingress vs. egress (left column): By overlaying both metrics, we can immediately spot structural imbalances. For instance, if one direction has a sharp spike (low variance) while the other is flat and wide (high variance), it signals that asymmetric network events are dominating that infrastructure type.
Below is a sampling of network data in one point in our network over the month of May 2026, with the traffic pattern graphed on the left side and variance on the right side. Immediately we can see different groupings of patterns. For readability and grouping, we’ve separated the types of networks into two categories: the dramatic and the reliable.
Bringing the drama: Hyperscaler and neoclouds
AI infrastructure behaves differently than traditional internet infrastructure. The following comparisons illustrate why.
Hyperscaler and neocloud traffic and distribution variability.
So, what can we learn from this? Let’s examine it by network type.
Hyperscaler: High egress volatility with balanced ingress
Ingress traffic remains tightly controlled around the baseline (sharp dashed peak). However, egress traffic (solid line) shows a flattened, high-variance spread.
The time-series chart reveals constant, jagged fluctuations between 50 Gbps and 150 Gbps, indicating highly bursty customer data retrieval patterns throughout the month.
Neocloud: Synchronized, moderate volatility
Both ingress and egress display structurally similar, moderately wide bell curves. This reflects a well-proportioned network footprint where data-in and data-out scale together.
The time series demonstrates sustained high baseline volumes (Total traffic consistently tracking between 200 Gbps and 350 Gbps) with continuous business-hour cyclical wave patterns.
As network operators we’re using this type of real-world data to help drive our connectivity footprint decisions. Large, bursty traffic patterns are best served by PNI network connections. PNIs allow us to isolate workflows to a distinct physical egress/ingress path in our network, which enables us to be able to more easily route, load-balance, and support these higher performance profiles. That translates into more predictable performance for customers running bandwidth-intensive AI workloads.
We have a high interest in connectivity to partners in our US-East location, as it is located in the Ashburn-Reston datacenter corridor near a lot of existing datacenter campuses. This one again reinforces the notion that geography plays an important role in where entities are placing their data and compute engines rather than the nondescript “cloud”.
If you’re interested in learning more about the geography of neocloud traffic, visit the Q1 2026 report and review the “Where in the world is the neocloud?” section.
Let’s switch over to our three other major network types that we also want to profile for capacity, performance, and scalability considerations.
Bringing the predictability: CDN, hosting, and ISPs
CDN, Hosting, and ISP-Regional traffic and distribution variability.
CDN: Extreme ingress stability vs. massive egress spread
The ingress curve is a razor-thin needle at 0 Gbps variance, proving inbound management traffic is perfectly flat. Conversely, the egress curve is completely flattened across the entire -50 to +50 Gbps spectrum.
This is textbook CDN behavior: steady, quiet ingest lines paired with massive, erratic client-side distribution demands peaking near 600Gbps.
Hosting: Highly predictable footprint with asymmetric egress stability
Inbound traffic displays a slightly wider variance profile, while outbound traffic (egress) forms a remarkably sharp, low-variance peak.
The time series shows a tight, rhythmic diurnal cycle for egress down near 25Gbps, while ingress experiences a steady climb over the course of May, rising from a 75Gbps baseline up past 125Gbps.
Regional consumer traffic demonstrates an ultra-low variance spike on egress, maintaining a very steady floor near 75Gbps. Ingress traffic carries slightly higher variance but remains highly constrained to predictable diurnal rhythms.
This represents localized residential/commercial end-user ingress cycles, peaking consistently between 400 and 500 Gbps every single day.
Signals in the noise
This far into Network Stats, the biggest takeaway isn’t simply that there is more traffic because of AI. It’s that AI traffic has different—and still emerging—patterns compared with traditional cloud workloads. It’s more bursty, more geographically concentrated, and less predictable. Understanding those patterns helps us decide where to add capacity, when to upgrade interconnects, and how to design a network that can support tomorrow’s AI applications—not just today’s.
As our dataset continues to grow, we’ll keep refining these models and sharing what we learn. Each quarter gives us a clearer picture of how AI infrastructure is evolving, and how cloud storage networks must evolve alongside it. Let us know what resonates, what questions you have, and what patterns you’re seeing in the comments section.
And, if you want to stay connected to this and other kinds of technical reporting from Backblaze, check out our Developer Newsletter.
You’ve done the storage evaluation. The per-terabyte price is right. The durability numbers check out. Compliance boxes are ticked. And still, the cloud migration project hasn’t been approved.
That’s not a coincidence.
The cloud storage industry has spent years competing on what happens after you’re already locked in: performance, redundancy, features. Almost nobody competes on what it costs to get there—or what it costs to leave.
Migration friction isn’t an oversight. For most hyperscalers, it’s a business model.
Why cloud migration projects stall before they start
The business case for cloud storage usually looks solid on paper. Lower per-terabyte costs. Less hardware to maintain. A path off aging tape libraries and overloaded NAS environments.
Then someone runs the actual migration math.
Egress fees from the current provider. Data transfer charges. Migration software licenses. Professional services. Tape digitization. Internal engineering hours. Project coordination overhead. For a large dataset, those costs can erase years of projected storage savings before a single byte moves.
Teams spend months building an approval-ready business case, only to find the upfront migration cost makes the model unworkable. The project stalls. Infrastructure the organization already knows is unsustainable stays in place. Modernization gets pushed to next quarter.
This is where most cloud vendors win. The storage decision becomes moot if the organization can never afford to move.
How egress fees trap organizations with their current provider
By the time most IT teams discover what cloud egress fees actually cost, they’re already mid-negotiation with a new provider.
The pricing model is deliberately asymmetric: getting data in is cheap, often free. Moving it out is where providers charge—and for multi-petabyte environments, those charges can run to hundreds of thousands of dollars before a migration has even started. Technically, the organization owns its data. Financially, moving it is a different question.
This reframes the evaluation in a way that favors incumbents. The question stops being which platform best fits long-term needs and becomes whether the organization can afford to leave at all. Once you’re in a major cloud platform with a large archive, exit costs are a structural retention mechanism.
Ask any prospective provider, early: what does it cost to leave? If they’re vague, that’s the answer.
How long does a cloud data migration actually take?
Cost gets scrutinized. Time usually doesn’t—until a migration is already underway and slipping.
A large-scale migration means inventorying data and metadata, evaluating and procuring tooling, coordinating across vendors, monitoring transfer jobs, validating integrity at the destination, and troubleshooting the inevitable edge cases. For multi-petabyte environments, self-managed projects routinely stretch from months into years.
Every quarter that drags on is a quarter the organization is paying to maintain infrastructure it’s already committed to replacing, while its engineering team runs a file-moving operation instead of working on anything strategic. The total cost of a slow migration almost always exceeds the initial estimate—and almost nobody builds that into the business case upfront.
What actually goes wrong during cloud data migration
Migration risk tends to be underestimated until something breaks.
The core questions—will files transfer without corruption, will metadata survive intact, will dependent applications keep working—are harder to answer than they look for LTO tape archives that haven’t been accessed in years, NAS and SAN environments with proprietary metadata structures, media archives with irreplaceable assets, regulated content with chain-of-custody requirements, and datasets large enough that verification at scale is its own engineering problem.
The cost of getting this wrong isn’t just the migration itself. Data loss, integrity gaps, or application failures discovered post-migration can be significantly more expensive than any egress fee. Validation and verification need to be designed into the plan before transfers start, not bolted on after something fails.
Why most cloud providers leave migration to you
Infrastructure teams evaluating cloud storage aren’t looking for a transfer tool. They want infrastructure modernized without burning their engineering team on a multi-year internal project. Predictable costs. A path to cloud that doesn’t require standing up a program management office just to move data.
The standard provider response is documentation and an onboarding checklist. After that, you’re largely on your own.
This isn’t an accident. Selling storage is straightforward. Owning migration means taking on cost, risk, and operational complexity that most providers would rather leave with the customer. The economics of the business favor making entry easy and exit expensive, with as little friction to growth as possible in between.
Providers that treat migration as their problem to solve are a different category. They’re betting that making it genuinely easier to get to their platform is worth more than one-time migration revenue—because a customer who gets there successfully tends to stay.
How Backblaze Universal Data Migration works
We built Universal Data Migration because we kept seeing the same thing: organizations that had already decided to move to Backblaze B2 getting stuck on the migration itself. The technology decision was made. The budget was approved. The project just couldn’t get started.
The program moves data from virtually any source—AWS S3, Microsoft Azure Blob, Google Cloud Storage, Wasabi, NAS and SAN, file servers, LTO tape across all generations, physical hard drives, legacy archives—with Backblaze managing the process rather than handing the customer a tool and a runbook.
The migration cost doesn’t have to be a reason the project stalls. That’s the point.
Before you sign a cloud storage contract, ask these two questions
The storage evaluation isn’t complete until you know what it costs to get there and what it costs to leave.
Most providers make the second number hard to find. If you have to dig for egress pricing, or if a sales rep answers the exit question with “we’d work with you on that,” build the worst-case number into your model before signing anything.
The right provider won’t make you ask. They’ll make migration part of the conversation from the start—because they’re confident enough in their platform to compete on the full picture, not just the monthly storage line.
Have a migration project that keeps getting pushed?Talk to our team about what it would actually take to move your environment to B2.
Long time readers: I must start with a mea culpa that this article was not released on our traditional May the Fourth (or thereabouts) report date. Please know I missed making the many and sundry Star Wars puns as much as (I’m sure) you missed hearing them—so, I’ll probably slip a few in, just because.
That said, there was good reason for the delay. We were diligently investigating a disturbance in the force. (See? We’re back!) We’ll get into the details after we dig into the numbers, but here’s the teaser: this quarter, we observed something unusual in the data that wasn’t entirely a failure trend. It did, however, prompt us to revisit and validate the assumptions behind our Drive Stats analysis before publishing.
Sign up for the Drive Stats webinar
Ready to dive deeper into the data? Tune in Wednesday, July 15, 2026 at 12:00 noon PT, as Sr. Market Intelligence Manager, Stephanie Doyle, and Sr. Product Marketing Manager, Laquie Campbell review the trends.
As of the end of Q1 2026, Backblaze was monitoring 345,662 drives used to store data. For our evaluation, we removed from consideration 3,907 boot drives and 492 hard drives, as they did not meet the criteria to be included. We’ll discuss the criteria we used in the next section of this report. Removing these drives leaves us with 341,263 hard drives to analyze. The table below shows the annualized failure rates for Q1 2026 for this collection of drives.
Backblaze Hard Drive Failure Rates for Q1 2026
Reporting period January 1, 2026–March 31, 2026 inclusive Drive models with drive count > 100 and drive days > 10,000 as of March 31, 2026 in Q1 2026.
MFG
Model
Size (TB)
Drive Count
Avg. Age (Months)
Drive Days
Drive Failures
AFR (%)
HGST
HUH728080ALE600
8
1,026
87.8
3,252,052
125
1.4
HGST
HUH721212ALE600
12
2,608
74.5
6,115,413
104
0.62
HGST
HUH721212ALE604
12
13,277
55
24,018,962
1,114
1.69
HGST
HUH721212ALN604
12
9,780
77.4
26,914,795
1,678
2.28
Seagate
ST8000DM002
8
8,910
105.5
33,115,836
1,380
1.52
Seagate
ST8000NM0055
8
13,232
94.6
45,131,414
2,404
1.94
Seagate
ST10000NM0086
10
981
89.9
3,566,716
306
3.13
Seagate
ST12000NM0007
12
983
31.6
37,325,724
2,297
2.25
Seagate
ST12000NM0008
12
18,650
67.2
43,012,820
2,466
2.09
Seagate
ST12000NM000J
12
1,087
18.6
632,857
22
1.27
Seagate
ST12000NM001G
12
13,216
59.5
25,106,715
679
0.99
Seagate
ST14000NM001G
14
10,519
58.4
20,025,909
756
1.38
Seagate
ST14000NM0138
14
1,251
53.8
2,767,654
438
5.78
Seagate
ST16000NM001G
16
34,729
40.7
44,065,062
820
0.68
Seagate
ST24000NM002H
24
9,605
9.3
2,779,100
220
2.89
Toshiba
MG07ACA14TA
14
37,263
62.6
75,018,084
2,114
1.03
Toshiba
MG07ACA14TEY
14
996
38.4
1,235,801
58
1.71
Toshiba
MG08ACA16TA
16
40,036
30.4
37,906,446
1,113
1.07
Toshiba
MG08ACA16TE
16
6,264
47.6
9,544,691
325
1.24
Toshiba
MG08ACA16TEY
16
4,885
49.4
8,035,780
519
2.36
Toshiba
MG09ACA16TE
16
506
6.5
101,383
4
1.44
Toshiba
MG10ACA20TE
20
20,451
11.5
7,206,903
137
0.69
Toshiba
MG11ACA24TE
24
7,203
4
869,474
10
0.42
WDC
WUH721414ALE6L4
14
8,640
60.6
16,344,031
231
0.52
WDC
WUH721816ALE6L0
16
2,994
50
4,665,417
133
1.04
WDC
WUH721816ALE6L4
16
26,801
34.4
28,463,214
394
0.51
WDC
WUH722222ALE6L4
22
45,638
16.1
22,472,050
360
0.58
WDC
WUH722626ALE6L4
26
3,604
2.5
274,161
5
0.67
Totals
529,968,464
20,212
1.39
Note and observations
The quarterly AFR is 1.24%. This is up from last quarter, but down from the previous year.
No new drive models this quarter. That’s not unheard of, but it’s pretty rare. In the last eight quarters, we’ve had new drives six times.
The investment in higher capacity drives continues. We deployed 10,220 drives last quarter, and 9,404 of those drives had a capacity of more than 20TB. While these drives are still fairly young, their AFR is an impressive 0.85% as a pool.
Impressive, most impressive. These drives had zero failures:
HGST HMS5C4040BLE640 (4TB)
HGST HUH728080ALE600 (8TB)
Seagate ST16000NM002J (16TB)
And these drives had one:
Seagate ST8000NM000A (8TB)
Seagate ST12000NM000J (12TB)
Seagate ST14000NM000J (14TB)
Seagate ST16000NM000J (16TB)
Toshiba MG09ACA16TE (16TB)
That list proves a point we’ve made in a few different quarters: net failure does not necessarily mean a low AFR. On this list, the Seagate ST16000NM000J (16TB) came in with a 3.61% AFR—when there are only 129 drives left in the fleet, even one failure spikes the AFR.
These aren’t the drives you’re looking for
As mentioned, we remove drives that don’t meet our criteria. We’ve covered the reasons that we’ve designed these exclusions in past reports, but here’s the quick and dirty:
Period
Drive Count
Drive Days
Quarterly
> 100
> 10,000
Annual
> 250
> 50,000
Lifetime
> 500
>100,000
Regardless of whether or not a given drive model is included in this article’s charts and tables, all of the line item data is included in our Drive Stats dataset which you can download by visiting our Drive Stats page.
Lifetime hard drive failure rates
Now let’s look at the lifetime failure rates. A reminder that
Backblaze Lifetime Hard Drive Failure Rates
Reporting period January 1, 2026–March 31, 2026 inclusive Drive models with drive count > 100 and drive days > 10,000 as of March 31, 2026 in Q1 2026.
MFG
Model
Size (TB)
Drive Count
Avg. Age (Months)
Drive Days
Drive Failures
AFR (%)
HGST
HMS5C4040BLE640
4
186
106.1
15,858
0
0
HGST
HUH728080ALE600
8
1,026
93
86,921
0
0
HGST
HUH721212ALE600
12
2,608
75.4
234,350
8
1.25
HGST
HUH721212ALE604
12
13,277
56.5
1,183,774
86
2.65
HGST
HUH721212ALN604
12
9,780
80.9
870,252
95
3.98
Seagate
ST8000DM002
8
8,910
112.3
639,473
25
1.43
Seagate
ST8000NM000A
8
247
37.9
21,552
1
1.69
Seagate
ST8000NM0055
8
13,232
101.5
1,176,619
39
1.21
Seagate
ST10000NM0086
10
981
98.3
86,667
11
4.63
Seagate
ST12000NM0007
12
983
75.8
87,389
11
4.59
Seagate
ST12000NM0008
12
18,650
70.7
1,664,558
129
2.83
Seagate
ST12000NM000J
12
1,087
19
96,887
1
0.38
Seagate
ST12000NM001G
12
13,216
60.7
1,185,545
33
1.02
Seagate
ST14000NM000J
14
433
16.2
37,258
1
0.98
Seagate
ST14000NM001G
14
10,519
60.3
944,400
21
0.81
Seagate
ST14000NM0138
14
1,251
62.6
111,709
15
4.9
Seagate
ST16000NM000J
16
129
17
10,117
1
3.61
Seagate
ST16000NM001G
16
34,729
41.2
3,098,105
44
0.52
Seagate
ST16000NM002J
16
466
38.8
41,931
0
0
Seagate
ST24000NM002H
24
9,605
9.3
820,090
74
3.29
Toshiba
MG07ACA14TA
14
37,263
64.1
3,343,724
94
1.03
Toshiba
MG07ACA14TEY
14
996
39.8
88,602
5
2.06
Toshiba
MG08ACA16TA
16
40,036
30.7
3,548,250
102
1.05
Toshiba
MG08ACA16TE
16
6,264
48.9
553,878
21
1.38
Toshiba
MG08ACA16TEY
16
4,885
50.5
435,063
26
2.18
Toshiba
MG09ACA16TE
16
506
6.6
36,988
1
0.99
Toshiba
MG10ACA20TE
20
20,451
11.6
1,665,263
42
0.92
Toshiba
MG11ACA24TE
24
7,203
4
495,387
3
0.22
WDC
WUH721414ALE6L4
14
8,640
61.3
776,557
7
0.33
WDC
WUH721816ALE6L0
16
2,994
50.4
267,868
24
3.27
WDC
WUH721816ALE6L4
16
26,801
34.6
2,402,732
64
0.97
WDC
WUH722222ALE6L4
22
45,638
16.1
3,992,942
42
0.38
WDC
WUH722626ALE6L4
26
3,604
2.5
182,471
4
0.8
Totals
30,203,180
1030
1.24
Notes and observations
Stay on target. The lifetime failure rate is 1.39%.
Hello there! Three drives make their appearance on the lifetime table, including our 26TB drive (which was just under on drive days last time around):
Toshiba MG09ACA16TE (16TB)
WDC WUH722222ALE6L4 (22TB)
WDC WUH722626ALE6L4 (26TB)
You served me well. The last of our 4TB drives have finally dropped below the exclusion criteria for the lifetime table. The HGST HMS5C4040ALE640 is completely gone, while the HGST HMS5C4040BLE640 (4TB) has just 186 drives left in the pool.
Do or do not: When failure isn’t the only metric in play
As I teased in the introduction, this quarter, some unexpected behavior brought forth one of the most interesting edge cases we’ve seen in a while, and it spotlights some relevant truths about the Drive Stats project. More interestingly, it showed up with only slightly higher failure rates, partially because of some mitigation steps we were able to take. Stay with me—there are multiple, overlapping issues at play.
We’ve said often that individual drives within a manufacturer pool, and even within a single drive model, can vary quite a bit. On these particular drives, that was the case: there were two different mechanical issues, one which affected writes and one which affected drives when they power cycled. The fact that it was two different types of issues made the initial investigation more difficult, and depending on the age of the drive, either, both, or neither issue could have affected the drive model.
Additionally, in both of those types of errors, it didn’t affect every drive in the pool, and for a long period of time, the failure rate was actually acceptable. Once we saw more of the failures happening, we correlated with the power issue, and we could help mitigate by reducing the frequency of the power cycling systems for Vaults that contained these drives. What that looked like in practice is that we put the Vaults with suspected issues into no-upload mode and kept them in reserve. Once the investigation was complete, we were able to understand the full picture.
The point I want to highlight here is one that I’ve made quite often: our drive failure rates, and what we report on in general, is the function of people actively managing failure and risk. You can see that clearly here—mitigating one of the two root causes reduced overall risk for this drive model; and, while we were doing that, we had an alternative option that let us use the drives, albeit in a different way.
Meanwhile, it also reflects the point we’ve been making from the beginning of time, from way back when we said “why not use consumer hardware to create a storage cloud?” Some level of risk is always acceptable. A system that pushes for absolutely zero failures on the hardware level is imprudent, given that our software layer largely prevents data loss.
Always two there are
And yet, this investigation was particularly impactful because we saw a decent number of drives (not all of them—again, multiple causes to this incident) that were day one failures. Because that’s true, there’s another important piece of the puzzle to consider, one that speaks to the ways we define a failure and the ways the Drive Stats program is written.
The short version is that at the end of each day, a C++ program collects the various SMART stats. If a drive was there the day before and is not there the day of, then it’s logged as a failure. There are other pieces of the puzzle as well, including a lookback period. So if the disappeared drive serial number shows back up on day two (or three or four), it retroactively tells us that it wasn’t a failure. We cut that off at the end of the quarter, which means there’s a handful of drives that might be false positives on that kind of failure, but the probability is quite low (and you have to cut things off somewhere).
But the more important part of things is this: Drive Stats is a function of conditional logic. Because it defines a failure based on a drive’s presence in the pool the day before, that means we can’t define day one failures using only the Drive Stats program.
What does that mean for us going forward? Have we been completely under-reporting day one failures? Short answer: yes, probably. But, back to that whole “managed environment” caveat—once drives enter data centers, they’ve already gone through a period of drive qualification. So, it’s relatively rare we have day one failures to begin with.
But, that does mean that we should caveat things appropriately, particularly if you’re one of the folks who likes to use the full dataset for your projects. The logline data compiles SMART stats, but only for drives that have the chutzpah to make it to the end of day one.
Remember the Drive Stats will be with you, always
Thank you, as always, for following Drive Stats. We had some folks reach out to make sure the report was going live, particularly because we lost an important member of the team recently, the inimitable Pat Patterson. We appreciate all the well wishes, and the patience.
Here’s our standard fine print:
The complete dataset used to create the tables and charts in this report is available on our Hard Drive Test Data page. You can download and use this data for free for your own purpose. All we ask are three things:
You cite Backblaze as the source if you use the data;
You accept that you are solely responsible for how you use the data, and;
You do not sell this data itself to anyone; it is free.
If you’re a new Drive Stats fan, consider signing up for the newsletter. If you’re not ready for that kind of commitment, sound off in the comments section below or reach out directly to us to let us know what you’re working on. May the Stats be with you!
On April 7, 2026, Anthropic announced a model so capable they refused to release it publicly. Claude Mythos, their most advanced frontier AI, was deemed too dangerous for open access because of one thing: it can hack.
Anthropic locked Claude Mythos behind Project Glasswing, a vetted partner program initially restricted to roughly 50 organizations—AWS, Microsoft, Google, Apple, Cisco, CrowdStrike, and others—to use the model for defensive work before adversaries could develop equivalent capability. By June, that program had expanded to more than 200 organizations across 15 countries, including operators of power grids, water systems, hospitals, and telecommunications infrastructure.
Then, on June 9, Anthropic released Fable 5—the first public version of a Mythos-class model—equipped with safeguards that reroute higher-risk queries to less-capable models. The same day, it released Claude Mythos 5 directly to vetted Glasswing partners. Later in June, after a brief US government export review, the Commerce Department confirmed that “appropriate safeguards are in place” and permitted Anthropic to redeploy Mythos 5 to trusted cyber defenders.
But here’s the part that should be on every IT leader’s radar: Anthropic itself now projects that other AI companies will have Mythos-class models within six to 12 months, and those companies may not ship with equivalent safeguards.
GPT-5.5, released three weeks later, didn’t wait. OpenAI shipped it with expanded cybersecurity capabilities and its own controlled-access program—also designed for defense, also eventually available to people with different intentions.
The AI arms race in cybersecurity isn’t coming. It’s here.
Ransomware 5.0 Doesn’t Need a Skilled Operator
For most of its history, ransomware required a human being at the keyboard: someone doing reconnaissance, identifying targets, crafting phishing lures, moving laterally through a network. Skilled attackers commanded significant ransoms. Amateur operators made rookie mistakes.
That dynamic is collapsing.
Ransomware now appears in 48% of all breach chains, according to the Verizon 2026 Data Breach Investigations Report—up from 44% the year prior. Active ransomware groups jumped 49% year over year. Over 250 new operators entered the market in just the last six months, many of them low-skill actors using generative AI to craft personalized phishing campaigns 60% faster than was possible before. AI-assisted lateral movement was present in over 65% of recent cases.
The Verizon 2026 DBIR also marks a shift in how attackers get in the door: for the first time, exploiting unpatched software vulnerabilities has overtaken stolen credentials as the number one initial access vector, now responsible for 31% of breaches. That’s not a coincidence in a world where AI can scan codebases for exploitable flaws at machine speed.
IBM’s 2026 X-Force Threat Index confirmed that “collapsing barriers to entry” are letting even low-volume operators run campaigns that overwhelm defenders. The average cost of a data breach in the US hit $10.22 million—an all-time record.
Trend Micro’s 2026 security predictions describe what they call “Ransomware 5.0”: a model where AI handles reconnaissance, vulnerability scanning, lateral movement, and even ransom negotiation autonomously, without a human operator directing any of it.
If you’re still designing your security posture around slowing down a skilled human attacker, you’re fighting the last war.
The Thing Nobody Wants to Say Out Loud
Here’s where I’m going to say something a little uncomfortable: the cybersecurity industry has been selling you detection for years when what you actually needed was recovery.
Detection is important. Don’t get me wrong. But detection-centric security assumes you catch the attack before it fully executes. In an era where AI compresses the attack timeline, exploit chains run at machine speed, and hundreds of new ransomware groups just showed up with AI-powered toolkits, detection alone isn’t a resilience strategy. It’s a bet.
The UK Government’s AI Security Institute tested Claude Mythos extensively and confirmed it cannot reliably execute attacks against organizations with well-hardened defenses. That’s genuinely good news. But it raises an obvious follow-up question: how many organizations actually have well-hardened defenses? A 2025 report found that over 45% of discovered security vulnerabilities in large organizations go unpatched after 12 months. Many critical infrastructure operators still run end-of-life software.
The honest answer is: most organizations are not that hardened. And even the ones that are will face a more capable threat next year than they face today.
This is why immutable backups aren’t just a box to check; they’re the safeguard that functions even when everything else fails. If an attacker encrypts your production environment before detection fires, the question isn’t “how did that happen?” It’s “how fast can you recover?”
What Claude Mythos Actually Changes (And What It Doesn’t)
It’s worth separating signal from noise here, because the coverage of Claude Mythos has ranged from measured to apocalyptic.
What Mythos changes: the technical barrier for sophisticated attacks. Vulnerabilities that previously required elite researchers to discover and weaponize can now be found and chained faster. Anthropic’s own red team found that Mythos could identify and exploit a previously unknown FreeBSD remote code execution vulnerability—fully autonomously, no human involved after the initial prompt. Across all Project Glasswing partners, Mythos has now surfaced more than 10,000 high- or critical-severity security flaws in production codebases. That means the window between vulnerability disclosure and active exploitation, already dangerously short, gets shorter. It also means less-skilled threat actors get access to capabilities that used to require significant expertise.
What Mythos doesn’t change: the fundamental anatomy of a ransomware attack. Attackers still need initial access. The Verizon 2026 DBIR confirms they’re still relying on unpatched software, stolen credentials, and phishing as entry points just finding and exploiting them faster. Once inside, they still need to move laterally, identify high-value data, and execute the encryption sequence. The Centre for Emerging Technology and Security at the Alan Turing Institute made this point clearly: more sophisticated ransomware attacks that rely on stolen credentials, social engineering, or already-compromised accounts are “far less likely to be affected” by Mythos-class models on either side.
That matters for how you defend. Hardening access controls, enforcing MFA, patching aggressively, segmenting your environment, and maintaining clean, immutable backups are not glamorous. They are not AI-powered. But they address the attack anatomy that AI tools, offensive or defensive, haven’t fundamentally changed.
The Recovery Imperative
Strengthening cyber fundamentals, in practice, means one thing above all else: knowing that when something gets through, you can recover without paying a ransom.
Immutability. Backups that can’t be encrypted or deleted by ransomware, even by a compromised admin credential. This isn’t optional anymore. If your backups live in the same environment as your production data and share the same access credentials, they aren’t backups; they’re part of your blast radius. Backblaze B2 Object Lock is S3-compatible, so if your team is already running Veeam, Commvault, MSP360, or Nutanix, you’re not replacing your backup stack. You’re giving it an immutable target that ransomware can’t touch.
Air-gap or off-site isolation. Object Lock, WORM storage, and geographically separate backup targets all put meaningful distance between your recovery point and an active attack. When AI tools can chain dozens of steps in a corporate network attack simulation autonomously, “isolated backups” means genuinely isolated, not just a separate folder. Version history matters here too: the ability to roll back to a known pre-attack state, not just the most recent snapshot, is what separates a clean recovery from discovering your restore point was already compromised.
Recovery time that matches the threat. AI-accelerated attacks mean recovery has to be fast. A backup strategy built around 72-hour RTOs made sense in a different threat environment. In 2026, breach costs approaching $10.22 million in the US, the question your leadership should be asking is: how long does it actually take us to restore from a clean state? Cold storage tiers that require hours of retrieval before a restore can even begin are a liability when the clock is running. Backblaze B2 is hot storage: your data is available immediately after detection, with no retrieval queue to wait on.
A Practical Checklist for IT Leaders Right Now
The Claude Mythos announcement, the Fable 5 public release, and GPT-5.5’s expanded cybersecurity capabilities are a forcing function. Not because Mythos-class capability is in attackers’ hands today, but because the direction of travel is confirmed, the timeline is compressed, and the question is no longer whether equivalent offensive tools will proliferate, only when.
A few things worth doing before that happens:
Audit your backup environment’s blast radius. Can ransomware that has compromised your production environment also reach your backups? If yes, fix that first.
Test your recovery time. Not just that backups exist, but how long an actual restore takes from your most recent clean snapshot. If you don’t know the number, you don’t have a recovery plan. You have a filing system. Backblaze gives you 3x your stored data in free egress each month, which removes the cost barrier that causes most teams to skip DR testing entirely. Run the restore. Know the number.
Pressure-test your identity controls. Credential abuse and phishing remain the dominant entry vectors. MFA, compromised credential monitoring, and least-privilege access aren’t new ideas, but they’re still the fastest path to closing the doors AI-powered attacks walk through.
Patch faster. The Verizon 2026 DBIR found exploited vulnerabilities are now the leading breach entry point. The median time organizations take to fix a known flaw is 55 days. AI-assisted attackers don’t wait 55 days.
Layer your defenses, but anchor to recovery. Perimeter protection, endpoint detection, vulnerability scanning: these all matter. But they’re all designed to catch something before it executes. Immutable backups are what you rely on when something executes anyway.
Revisit your RTO and RPO against today’s breach costs. The math has changed. A $10.22 million average US breach cost changes the calculus on what it’s worth spending on faster, more resilient recovery infrastructure.
The Last Thing
Anthropic made a decision that deserves credit: they looked at what Claude Mythos could do and chose not to hand it to the world on day one. Project Glasswing is a serious attempt to use the model’s capabilities on the right side of this fight, and the coordinated disclosure of thousands of vulnerabilities to the organizations responsible for patching them is meaningful defensive work.
But the history of powerful technology is not “we invented it and kept it safe.” It’s “we invented it, others reproduced it, and everyone had to adapt.” The 6-to-12-month window for equivalent capability to reach adversarial hands isn’t fearmongering; it’s Anthropic’s own forecast. Other AI companies are building toward the same capability threshold right now, and not all of them will ship with the same safeguards.
The organizations that come through this transition will be the ones that took recovery seriously before they needed it. Not because detection failed, but because recovery is the one safeguard that works regardless of what the attacker is running.
Backblaze B2 with Object Lock puts immutable, air-gapped backup storage within reach of organizations that can’t afford hyperscaler pricing (which, as it turns out, is most of them). Start a free trial or talk to our team about building a ransomware-resilient backup architecture before the threat landscape shifts again.
Backblaze, Inc. (Nasdaq: BLZE), the cloud storage platform for the AI era, today announced an agreement with CoreWeave, Inc. (Nasdaq: CRWV), The Essential Cloud for AI.
Under the multi-exabyte, $335 million agreement, Backblaze will provide cost-efficient storage capacity that supports portions of CoreWeave’s managed storage infrastructure, helping optimize placement of data across performance tiers while preserving high-performance storage resources for the demands of AI workloads. The Backblaze technology supports HDD-based storage tiers in CoreWeave AI Object Storage. Customers already utilizing CoreWeave AI Object Storage with its patented LOTA distributed cache will immediately have access to new service tiers without any code modifications.
Every stage of the AI lifecycle depends on the ability to store and move massive volumes of data efficiently. Training, inference, checkpointing, data preparation, model outputs, and retrieval-augmented generation (RAG) all require storage that performs at the speed and scale modern AI demands.
“Storage is the foundation every AI workflow is built on — without it, even the world’s most powerful compute sits idle,” said Gleb Budman, co-founder and CEO, Backblaze. “We’re pleased to work with CoreWeave on elements of their storage environment. This collaboration demonstrates how our platform can help organizations meet growing infrastructure demands.”
Backblaze serves more than 100,000 customers worldwide and has extensive experience operating large-scale storage infrastructure. Its cloud platform is designed to deliver reliable, cost-efficient storage services across a range of enterprise and data-intensive use cases.
“Backblaze has built a reputation for making complex, HDD-based storage infrastructure reliable and easy-to-consume at scale. We’re pleased to work with them as we continue expanding our platform and managed service offerings to support AI workloads at scale,” said Nick Hoover, Vice President at CoreWeave.
CoreWeave’s AI cloud platform spans infrastructure, technology, tools, and services. The company serves leading AI model developers, enterprises, and research organizations, including 9 of the top 10 AI model providers.
The generative AI wave is moving from text into media. ChatGPT taught a generation of developers how to wire up an LLM. The next chapter is harder and more interesting: video, image, audio, and multimodal workflows that combine them. New models for each of these are landing every few weeks, from established labs and from teams nobody had heard of a quarter ago. The quality keeps climbing, and the catalog keeps expanding.
That pace is good news, but it asks more of the apps that depend on it. A pipeline that hard-codes today’s best video model is going to feel dated in a month. The apps that come out ahead will be the ones built on reactive architectures: pipelines that stream progress as events, fan out concurrent work, fall back when a provider stalls, and let you swap in a new model with a single line of code.
That’s where the moat is now. If you’re building an AI media product, the differentiation is in the pipeline: which models you call, in what order, how you fall back when one is slow, how you keep latency tolerable, where the assets land, and how you prove what was generated. The teams winning this space treat the orchestration layer as their core IP. The frontend on top is the easy part.
We’re looking for builds that solve real-world use cases. Show us what becomes possible when the pipeline stops being a bottleneck.
What we’re running
The Backblaze Generative Media Hackathon is an online competition for developers building generative media apps. $10,000 in cash prizes, 10 GB of free Backblaze B2 storage to start with, and GMI Cloud credits for the first 270 eligible participants.
Grand Prize: $7,000
Second Place: $2,000
Third Place: $1,000
Registration and submissions run June 22 through August 3, 2026 (5 pm ET). Winners are announced August 12.
The stack
Backblaze B2, Genblaze, and GMI Cloud are the three pieces of a media pipeline that’s both easy to vibe code and serious enough to take to production.
Backblaze B2 is S3-compatible object storage for the generated assets, the metadata, the provenance manifests, the thumbnails, and the logs. Durable URLs that never expire. Event Notifications fire downstream work without polling. Object Lock makes your provenance records tamper-evident. Lifecycle Rules handle cleanup so intermediate artifacts don’t pile up.
Genblaze is our open-source Python SDK for orchestrating generative media workflows. One Pipeline API spans OpenAI, Google, Runway, Luma, Decart, ElevenLabs, Stability Audio, NVIDIA NIM, GMI Cloud, and others. Every run produces a SHA-256-verified provenance manifest. Swap providers with a one-line change. New models drop every couple of months, and your app doesn’t need a rewrite each time.
GMI Cloud is a unified inference platform for open source AI. Image, video, audio, chat, reasoning, and multimodal models behind a single API key. If you want to try five video models in a weekend, this is the easiest way.
What to build
Anything that turns a generative AI capability into something a real person would actually use. Some directions:
AI video generation or editing apps
Image generation, remixing, or transformation tools
Audio, music, or voice generation workflows
Multimodal apps that combine text, image, video, and audio
AI media libraries for storing, organizing, and searching generated assets
Provenance-aware workflows that track how each piece of media was generated
Agentic media pipelines that generate, evaluate, retry, and store outputs
Tools for creators, marketers, educators, entertainers, or developers
Judges will weigh real-world utility, production readiness, and how meaningfully your app uses B2 and Genblaze. We’re looking for projects where the pipeline is doing real work. Apps that handle multiple providers, recover from errors, track provenance, and store assets reliably.
How to enter
Visit the hackathon page on Devpost for registration details, eligibility requirements, prizes, submission guidelines, and important dates:
Build your generative AI media application using Backblaze B2 and Genblaze, then submit your project before the deadline.
If you want a head start on what a Genblaze + B2 app looks like in code, two reference repos already exist. genblaze-gen-media-multi-provider-sample chains five providers into one prompt-to-MP4 pipeline. genblaze-gmicloud-pipeline is a deep multi-model composition inside one provider.
Customer records live in the database. Payment activity is safely stored in your payment processor. Call recordings and transcripts live in Zoom, Teams, Webex, or another video conferencing application–or are shared to Gong for customer insights. Telemetry resides in an observability tool like Grafana or DataDog. Your own day-to-day work is in Google Drive or OneDrive. It takes hundreds of human hours to figure out what customer behavior and business continuity patterns can be extracted from all of this data.
Extracting insights from your data starts with knowing what you have. The first step is centralizing it — pulling multimodal data from across your systems into a single storage repository where your engineering team and AI agents can actually access it. From there, you can assess what’s useful, what’s usable, and what still needs to be labeled or anonymized before it’s ready to work with.
Assessing your data is like cleaning out the garage: first, you have to do a full inventory to know what you actually have before deciding on new data destinations and purposes.
The hidden data silos most organizations overlook
One of the less-discussed barriers to AI readiness is that many organizations lack a complete picture of their own data assets.
Financial assets are documented. Physical assets are tracked. But images, audio recordings, video files, email archives, documents, logs, and customer interaction histories often sit across systems with inconsistent labeling, unclear ownership, disparate tooling, and no centralized catalog.
Customer calls, support chat transcripts, QA screen captures, surveillance footage, and product images all contain operational insight that can inform AI applications, assuming they’re stored in a way that makes them accessible and usable. Most organizations haven’t done that inventory and don’t know what data they’re sitting on.
In our experience, organizations that broaden their definition of data — and build infrastructure to collect and manage it centrally — consistently find that their AI potential is larger than they initially estimated. The inverse is also true. Organizations that skip this step tend to hit the data silo problem mid-project, when data they assumed was available turns out to be fragmented, unlabeled, or simply missing.
The term “multimodal” describes this in practice: datasets that span formats—images, audio, video, text, and structured records—within the same pipeline. Managing multimodal data at a meaningful scale requires infrastructure decisions made well before an AI project kicks off.
Where the infrastructure question meets the strategy question
Here’s what aligning AI strategy with data strategy actually requires:
Inventory what you have. Before sourcing anything new, take stock of what exists. Support call recordings, usage footage, survey data, transaction histories—these are continuously generated across most organizations and rarely treated as AI assets. A governance committee (described below) is the natural owner of this inventory.
Establish governance before you deploy. Who can use which data, under what conditions, and for what purposes. When data governance is established early, teams get answers in days rather than weeks. When it’s deferred, it becomes a bottleneck mid-project.
Plan storage infrastructure for what you will have, not just what you have. A storage decision made today carries a different cost profile 18 months from now. Hyperscaler egress fees that look manageable on a pilot-scale workload become structural constraints at training scale. Archive tiers that appear to reduce costs carry retrieval latencies incompatible with active AI pipelines. Modeling these costs before committing to a provider architecture prevents the predictable trade-offs: smaller datasets, shorter retention windows, fewer training cycles.
Make the C-suite part of the conversation. IBM’s 2025 CEO Study found that 68% of AI-first organizations have mature, well-established data and governance frameworks. When the CEO is involved in AI governance decisions, the conversation stays connected to business strategy instead of fragmenting into siloed technical decisions.
The competitive advantage lives in the data (silos)
Foundation models are increasingly commoditized. The leading model today will be superseded within months, and capable alternatives are widely available from multiple providers. The latest generation from any major provider is capable, widely available, and will be superseded by something better within months. What cannot be licensed, replicated, or accessed by a competitor is the proprietary data your organization has built up over years of operation: customer patterns, process histories, institutional knowledge.
Getting that data foundation right is what separates AI programs that scale from those that stall.
Organizations that align their AI strategy with their data strategy from the start make fundamentally different infrastructure decisions. They choose storage providers that support active data movement without penalizing it. They build governance structures that give the right people access without creating bottlenecks. And they treat data growth as a business opportunity, not a cost to manage.
For most organizations, that shift in thinking starts with a simple question: who owns AI strategy? If the answer is “it’s fragmented across different teams,” then the second question is: what would it take to bring those conversations into one room?
Everything that follows—the data readiness, the governance, the infrastructure that actually works at scale—flows from that first alignment.
A year ago, a good video model was a novelty. Today there are at least six worth using, and most of the teams we talk to are wiring up two or three of them into the same product, alongside image models, voice synthesis, and music generation. The hard question isn’t whether you can generate this kind of media. It’s how to build a pipeline that handles five providers without falling over.
That’s why we built Genblaze, an open-source Python SDK from Backblaze for building generative media pipelines: one API across video, image, and audio providers, swappable models, durable object storage, and a SHA-256-verified provenance manifest on every run.
The pipeline is becoming the moat
Models are commoditizing. New video, image, and audio releases drop every couple of months, and each one tends to be the best at one specific thing and middling at the rest. Nobody we work with is betting on a single provider anymore. They build a portfolio and configure fallbacks.
The pipeline is what stays. It’s where you’ve figured out which model handles which shot type and which voice fits which brand. It’s where retry logic and output guards live, and where your audit trail comes from. That work survives the next model release. The prompts you tuned for last quarter’s hero model don’t.
For a pipeline to actually be durable, though, it has to be reactive. Hard-coding one provider, blocking on every step, and returning a single synchronous result is fine for a demo. In production it ages out in weeks. The pipelines that hold up stream progress as events, fan out concurrent work, handle backpressure from slow providers, and let you add a new model with a one-line change.
That’s what Genblaze is designed to be. One pipeline object, every provider behind the same surface, and a new model is one more .step().
A workflow that uses five providers
Here’s a concrete example: producing a short brand film from a one-paragraph brief.
1. Storyboard frames. Lock the visual direction with Seedream 5.0 Lite or FLUX via GMI Cloud, or Imagen on Google.
2. Animate the approved frame. Kling image-to-video on GMI Cloud, Veo on Google, Runway Gen-4 Turbo, or Luma Ray-2. They’re good at different shot types, so we usually try two and pick. Setting chain=True on the pipeline passes the image from step one into the video step automatically.
3. Score and sound design. Music from Stability AI’s Stable Audio or GMI Cloud’s MiniMax. Ambient effects and voiceover from ElevenLabs. LMNT for low-latency text to speech (TTS) when responsiveness matters.
4. Upscale. There’s an upscale step type built in. Route the rendered video through a Replicate upscaler like Real-ESRGAN to hit delivery resolution.
5. Classify and tag. Use a vision-capable chat() call to tag scenes, run brand safety checks, or generate accessibility metadata. Gemini 2.5, GPT-4o, or Llama 3.2 Vision on GMI Cloud all handle this.
That’s five providers across five different model types, defined in one pipeline. The same retry behavior, fallback chains, and provenance manifest apply to every step.
from genblaze_core import Pipeline, Modality from genblaze_gmicloud import ( GMICloudImageProvider, GMICloudVideoProvider, GMICloudAudioProvider, ) from genblaze_replicate import ReplicateProvider from genblaze_google import GeminiChatProvider
Swap any step for a different provider and nothing else in the pipeline has to change.
Provenance
Every run produces a canonical, hash-bound manifest that records the provider, model, prompt, parameters, timestamps, and the URI of every asset it produced. You can embed it directly into the output file (.mp4, .png, .jpg, .webp, .mp3, .wav are all supported by the matching media handler), or persist it as a sidecar JSON.
The hash is deterministic, so anyone downstream can verify the file by calling manifest.verify(). The same manifest is replayable: genblaze replay manifest.json reconstructs the run with the same parameters. And because every manifest carries a parent_run_id, you can trace a v3 video back through v2 and v1, including the fork where you tried Runway instead of Kling.
If you’re building customer-facing pipelines, this is what gets you from “we generated this” to “here’s the proof.”
Storage
Assets and manifests land wherever you want. We default to Backblaze B2, which the SDK wires up with ObjectStorageSink(S3StorageBackend.for_backblaze("my-bucket")) and which gives you durable URLs that don’t expire and don’t need credentials to fetch. The same sink works against any S3-compatible store: AWS S3, Cloudflare R2, MinIO.
A few B2 features pair particularly well with this kind of pipeline.
Event Notifications fire to a webhook, queue, or function endpoint when an asset or manifest lands. That gives you a clean way to kick off downstream encoding, indexing, or moderation without polling.
Object Lock lets you write manifests under a retention policy that nobody (not even the account root) can overwrite until the window expires. Combined with the SHA-256 hash inside the manifest, you’ve got cryptographic integrity and storage-layer immutability.
Lifecycle rules handle the cleanup. Final assets and manifests stay around as long as you want them to. Storyboard iterations, rejected takes, and pre-upscale renders prune themselves on whatever schedule you set.
Partnering with GMI Cloud: a unified AI inference platform for open source
GMI Cloud is a unified AI inference platform for open source. It supports LLM, image, video, and multimodal inference through one consistent API. The catalog covers Seedance, Kling, Veo, and Wan for video; Seedream and FLUX for image; MiniMax for music; ElevenLabs voices; and Llama, DeepSeek, and Qwen for chat and multimodal. One API key reaches all of it.
The genblaze-gmicloud adapter maps GMI’s image, video, audio, and chat endpoints onto the pipeline surface and tracks their catalog as new models ship. The first sample app below uses it heavily.
Two sample apps
genblaze-gmicloud-pipeline goes deep on a single provider. A prompt becomes an anchor image via seedream-5.0-lite. You iterate by passing the current image to flux-kontext-pro for reference-based refinement. Once you approve a frame, the app fans out concurrently to three video models (Kling-Image2Video-V2.1-Master, wan2.6-i2v, pixverse-v5.6-i2v). Manifests get written to B2 next to the assets, and the Genblaze integration sits in a single ~100-line file.
genblaze-gen-media-multi-provider-sample is the workflow above end to end. One sentence becomes a narrated, scored, captioned MP4. gpt-4.1-nano writes the storyboard, Imagen 4 produces the keyframes, Decart Lucy or GMI Cloud Kling animates them, NVIDIA Magpie TTS narrates, GMI Cloud MiniMax scores, and ffmpeg composes the final video. Five providers, one pipeline, every artifact ending up in B2 with a verifiable manifest.
That umbrella installs genblaze-core plus the B2/S3 storage backend, which is enough for a working provenance pipeline. Add genblaze[gmicloud], genblaze, or genblaze[all] to pull in providers.
Where this goes
Most of the interesting work in generative media is happening above the models now, in the pipelines that string them together. Whatever model you’re using today probably won’t be your favorite in six months. The orchestration around it is what lasts.
AI infrastructure is difficult to observe directly. That’s because AI systems are highly distributed. They span storage platforms, GPU clusters, training pipelines, and multiple cloud providers—and those looking to optimize their stacks have to build out a visibility layer that takes all of those things into account. Data is constantly moving between different tools and infrastructure layers, and it’s imperative to understand how and why.
Learn how to structure your data for AI
Your organization is ready to prioritize AI-first projects. But is your data ready? Whether you’re developing AI-powered products to bring in new revenue or building internal AI tools to increase productivity, your AI initiatives start with the data you already have and how it’s stored.
One critically important aspect: understanding the network layer. Network telemetry makes it possible to observe how data actually moves between environments: which endpoints exchange data, how long those connections persist, and how much information flows between them over time.
Since re-launching the Network Stats series in November 2025, we have had direct visibility into what that movement looks like in practice. Backblaze has always been an advocate for the benefits of a multi-cloud design, and our position as a pure storage provider offers a unique perspective on the movement of storage to the neoclouds where AI processing, inference, and model training take place. Our network telemetry captures data flows across those connections in real time.
The Q1 2026 Network Stats report covers a full quarter of observed network traffic across those connections. This makes it possible to see patterns in how AI workloads move data between storage and compute environments.
However, while these patterns are real and worth studying, they need to be interpreted carefully. A single quarter of data may highlight emerging behaviors, but those behaviors may not reflect how AI infrastructure will ultimately be designed.
And, as we often say—the dataset is young, and network landscape is going through a fundamental change. It will be exciting to see what happens over time, but we, as a community, should be discerning.
What network telemetry actually reveals
Network-level metrics expose behaviors that application or storage metrics often don’t reveal. Application logs can show what a system is trying to do, and storage metrics can show how much capacity is being used. But, it’s the network layer that reveals how data actually moves between systems.
Several network patterns help make that movement visible:
Traffic volume shows how much data is flowing through the system.
Connection persistence indicates whether that data moves through many short-lived exchanges or a smaller number of sustained transfers.
Endpoint concentration and magnitude (that is: how much data is transferred per IP address) show how many systems are involved in those exchanges and how much data each endpoint moves.
Ingress and egress data shed light on different kinds of workloads.
When we examine the network data, it begins to reveal how modern workloads operate.
One of the clearest examples appears in the Q4 2025 Network Stats data. In that quarter, neocloud traffic was concentrated around a relatively small number of endpoints, and those connections carry sustained, high-magnitude data transfers over time. Compared with traditional internet traffic patterns, fewer connections are responsible for moving a much larger share of the data. We call those elephant flows.
In Q1 2026, we dug into the ways that the data also show a geographic pattern. Neocloud traffic clusters in specific regions with dense compute and interconnect infrastructure, rather than being evenly distributed across the network.
Caption: Bits moved on the Backblaze network by country. Caption: Bits moved on the Backblaze network by country, but excluding the United States. Caption: Bits moved on the Backblaze network, limited to U.S. states.
These signals in the network data are meaningful. They reveal important characteristics of how AI workloads behave across infrastructure. But, they do not by themselves explain what those patterns imply for infrastructure design.
What a single quarter of network data cannot tell us
A single quarter of telemetry can reveal emerging patterns, but it cannot distinguish durable infrastructure trends from short-term experimentation.
AI infrastructure is still evolving quickly. New training frameworks are emerging, GPU availability continues to shift, and companies are experimenting with different compute providers and network connections. Because of this, infrastructure decisions today are still temporary. Teams are building pipelines with the resources they can access now, and many are finding that multi-cloud architectures allow them the ability to optimize their tech stack—traditionally only considered cost play, this is far more important from a performance perspective.
That means some of the patterns visible in current network data aren’t constraints for anyone attempting to deal with a changing landscape. For example, these are some of the patterns we can’t attribute causality to in a vacuum:
Regional clustering may simply result from where GPU capacity happens to be available today.
Changes in traffic mix may depend on how early training pipelines are assembled across providers.
Differences in connection patterns may be influenced by how mature different platforms’ network interconnects are.
As these conditions change, the same workloads may produce different network signatures. And, more importantly, a flexible, interoperable tech stack means that enterprises can be responsive.
This also is why network telemetry becomes much more informative when observed quarter by quarter and over time. Multiple quarters of data make it easier to distinguish temporary conditions from infrastructure patterns that persist.
But even before those longer-term patterns emerge, network telemetry still provides valuable insight into how AI infrastructure behaves under current conditions.
How to interpret network telemetry data
The patterns visible in network telemetry become more useful when we consider what they imply for infrastructure planning. Rather than describing the traffic itself, they help engineers understand the operational demands AI workloads place on real systems.
Sustained high-volume transfers point to the level of throughput infrastructure must support. If large datasets move repeatedly across the network, drive IOPS, API servers, load balancers, network switch pairs, and other interconnect layers must be capable of sustaining those flows without becoming bottlenecks.
Traffic concentrated among a relatively small number of systems suggests that storage and compute environments may become tightly coupled in real deployments. AI pipelines often depend on repeated exchanges between specific components rather than distributing activity evenly across many endpoints. For example, it may make sense to co-locate storage on flash hardware when you’re immediately feeding GPUs; however, your warm or cold data may be better served in different environments.
Regional clustering highlights where compute capacity and high-bandwidth interconnect infrastructure are currently concentrated. For operators, that can indicate where AI workloads are most likely to run and where supporting infrastructure must perform reliably at scale.
For engineers and infrastructure planners, data like this helps answer practical questions such as:
Where sustained high-throughput networking will be required
How tightly storage and compute systems may become coupled
Which regions currently concentrate the infrastructure supporting AI workloads
Taken together, this information provides a useful view into the operational demands AI workloads place on infrastructure today. They do not, however, define how AI infrastructure will ultimately be designed.
Infrastructure decisions shouldn’t be locked to early signals
The real problem is that, in traditional environments, infrastructure decisions tend to stick. When a team chooses where data lives, how pipelines move it, and which providers are connected, those choices quickly become part of the system.
Over time, early infrastructure decisions can quietly lock an organization into a specific architecture, even if better options appear later.
Once large datasets are stored in a particular environment, changing course becomes slow and expensive. Moving the data can take days or weeks. Pipelines may need to be rewritten. Teams may have to rebuild integrations between storage and compute systems entirely.
This doesn’t mean early signals should be ignored. Network telemetry still shows how workloads behave today, and can indicate what happens tomorrow. That visibility helps teams understand the operational demands AI workloads place on infrastructure.
These observations should be treated as a snapshot of a system that is still evolving. Infrastructure that allows data to move easily between platforms and compute environments gives teams the freedom to adapt as tools, providers, and architectures change. In fast-moving AI ecosystems, flexibility can matter just as much as raw performance.
Using network telemetry responsibly means learning from the pressures visible today while recognizing that the systems producing these patterns are still taking shape.
Two-thirds of enterprise leaders see significant potential in integrating AI models with their proprietary data. Yet only 22% feel confident their current IT infrastructure could actually support new AI applications. That gap—between ambition and capability—is where most AI investments stall, budgets balloon, and promising projects quietly get shelved.
The problem isn’t the models, the change management, or even the cost. It’s the underlying data foundation.
The conversation that never happens
The problem is not that either conversation is wrong. It is that they happen separately.
Most organizations run two parallel conversations about AI, and they rarely meet in the same room until a deployment runs into trouble.
The first happens at the strategy level: executives identify use cases, project ROI, approve budgets, and set timelines.
AI is treated as a technology question with business outcomes.
The second happens further down the stack: infrastructure teams make decisions about where data lives, how it moves, and who can access it.
Storage is treated as a cost question, divorced from strategy.
This separation is intuitive. Different people work on it. Different timelines apply. Different success metrics matter. But the separation is a liability disguised as organizational structure.
Two-thirds of executives leading infrastructure efforts say they are excluded from key AI decision-making conversations. Every model selected, application built, and workflow redesigned depends on the same thing: a data foundation that either supports the strategy or constrains it.
That foundation has to be built alongside the strategy, not bolted on as an afterthought.
The cost of misalignment
There are consequences to this misalignment. When data infrastructure decisions get made separately from AI strategy, the result is predictable: AI teams discover too late that the data they need is fragmented, inaccessible, or poorly governed. Infrastructure teams optimize for cost without knowing what future AI workloads will require. Business leaders fund use cases without validating whether the data foundation can support them.
And more specifically, a company building an internal tool to surface insights from customer support transcripts needs audio and text data organized, labeled, and retrievable before the tool can work. A company developing an AI-powered product for external customers needs guardrails in place for data provenance, consent, and version control before model selection matters. Both depend on data that is governed and accessible before any model enters the picture.
Yet according to Gartner’s survey of data management leaders, 63% of organizations either don’t have or aren’t sure they have the right data management practices for AI. Gartner’s projection is stark: through 2026, organizations will abandon 60% of AI projects unsupported by AI-ready data.
Each abandoned project represents sunk cost, such as engineering hours and vendor contracts.
And yet, the pattern repeats because the problem is structural. Strategy teams set direction without validating that infrastructure can support it. Infrastructure teams make decisions without understanding what the strategy actually requires.
Neither group is wrong; they’re simply operating from different information, different incentives, and different success criteria.
What changes when AI strategy becomes business strategy
When organizations treat AI strategy and data strategy as the same conversation, the outcomes shift. Three things happen:
Infrastructure decisions get made with strategic context. Where data lives, how it moves, what it costs—these have become capability questions rather than cost optimization questions. AI development depends on iteration: moving data between tools, environments, teams, and models. A storage provider that charges $90 per TB for egress can become architecturally limiting, penalizing the high-frequency data movement that accelerates AI development. When CFOs understand that the storage decision determines whether the organization can iterate quickly or iterate slowly, the conversation changes.
AI initiatives get funded with data readiness built in.BCG’s research on future-built companies found that top performers define AI programs with ambitious cost and revenue targets set at the executive level and hold teams accountable to near-term results. What they also have in common: they’ve established governance structures, inventoried their data assets, and planned infrastructure before deployment pressure arrives.
Accountability shifts. According to McKinsey’s 2025 State of AI survey, 28% of organizations using AI report that their CEO is responsible for overseeing AI governance—the policies, processes, and controls that determine how AI is developed and deployed. McKinsey’s analysis found that CEO oversight of AI governance is one of the factors most correlated with meaningful bottom-line impact from AI use. When executive leadership owns both strategy and governance, the two stay aligned.
A companion resource to help you make data decisions
AI strategy cannot succeed as a standalone initiative. It depends on the organization’s ability to make data available, portable, governed, and cost-effective at the exact moments teams need it. For leaders, that means treating infrastructure not as a downstream implementation detail, but as part of the strategy itself.
Read the ebook, Navigating Multimodal Dataset Economics, to get the guide on making decisions about the AI datasets at your organization, and how interoperable-by-design object storage is critical for multimodal AI datasets.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
. 
