Post Syndicated from David Johnson original https://www.backblaze.com/blog/neoclouds-your-storage-layer-just-got-a-public-grade/
SemiAnalysis published ClusterMAX 3.0, and storage helps decide whether a neocloud lands in Platinum, Gold, Silver, Bronze, or the new Participation Ribbon tier. Your customers can read that grade before they take your call.
Why it matters: Only 19 of the 77 neoclouds tested in depth earned a Medallion rating, and plenty that held Silver or Gold last round lost it. For most of them, the storage grade doesn’t reflect a decision anyone made. It reflects a default.
Storage decides whether you get rated at all
Storage isn’t something the evaluation gets to eventually. You can’t be rated without it.
- Every provider has to supply at least 10TB of high-performance file storage and 10TB of S3-compliant object storage before testing begins.
- Then storage is tested twice: once for speed, once for whether that speed holds through 15 minutes of sustained mixed traffic.
- Some providers lost nearly 40% of their bandwidth in the second test.
Every neocloud is in one of three states
Size isn’t what matters here. AWS and Azure both had storage problems this round. TensorWave, far smaller than either, had storage SemiAnalysis called exceptional. The difference is whether anyone deliberately decided what storage is for, what it runs on and who owns it.
- Not decided yet. You rent GPUs by the hour. Customers bring data in, run a job, and take the results home. Storage is scratch space, and there’s nothing to migrate if you change your mind.
- Decided by default. Someone stood something up once, and nobody’s job includes going back to check it. It was fine at the time, and it has stayed “fine at the time” for two years.
- Decided deliberately. A person or a contract is responsible for whether it still works, and you pay for that one way or another.
Most neoclouds are in the second state and think they’re in the first. The first state is fine. The second is the one that gets graded.
Why the second state is so common
Decision by default isn’t negligence. If you sell compute by the hour, customers manage their own data, and nobody has to decide what your storage offering is. GPUs are what customers evaluate, so that’s where the attention goes.
But that changes when customers start leaving their data on your platform. Now you have to decide where it lives, who can reach it, what happens when a machine dies and what to charge for it. That decision usually arrives while the business is growing too fast to stop and make it.
What “decision by default” looks like from the outside
SemiAnalysis found four recurring storage failures:
- Storage servers not enforcing RBAC correctly on volumes
- Storage misconfigured on the overlay network, so customers could reach the underlay
- Backend storage sharing a network with no isolation
- A BlueField data processing unit (DPU) left in its default host-trusted mode
The DPU case shows the pattern most clearly.
A DPU is a small computer built into the network card. In NVIDIA’s reference design, it handles networking, storage, and security work, including the rules that keep tenants apart. Out of the box, it trusts anyone with root access on the server. That’s fine in NVIDIA’s model, where the server admin is trusted. But in a GPU cloud, the customer often has root. That means the customer can reach the DPU itself, which is the component running the rules that separate them from everyone else on that hardware. Anyone who gets that far is in a position to weaken or turn off the boundary they’re supposed to be sitting behind.
SemiAnalysis found at least one host where that path was open. Nobody chose that setting. It’s how the card ships, and nobody went back to change it.
Read the reviews in tier order and the pattern holds: the further down the tiers, the less anyone has looked at storage.
- At the top, the complaints are about ergonomics. CoreWeave’s only storage note was a wish for one-click local GPUDirect Storage, which SemiAnalysis itself called nitpicky.
- In the middle, there are configuration gaps. Azure’s default StorageClass couldn’t attach, and a model load from NFS timed out because no parallel filesystem was attached to the Slurm cluster.
- At Bronze and below, storage often isn’t set up at all. One Seattle cluster had no shared filesystem to test. Another had no RWX StorageClass, so there was nowhere to put
/shared. At a third, NVMe existed but nothing exposed it, and PVCs hung in Pending.
One case sums it up best. A provider’s default StorageClass provisioned and bound, then left the pod stuck in ContainerCreating. Meanwhile, the Lustre tier on the same worker nodes hit 91.7 GB/s aggregate read across 32 clients. The hardware was fast, but anyone who used the default would have concluded the storage didn’t work.
Customers will ask these questions at any size
The tenant-isolation failures SemiAnalysis found hit providers big enough to be testing GB300 racks:
- A shared Kubernetes control plane exposing other tenants’ namespaces
- A default InfiniBand partition key surfacing 532 hostnames
- A Grafana dashboard that separated tenants on screen while the key behind it read every tenant’s logs
That reads like someone else’s problem, but it isn’t. It’s a preview of the diligence review you’ll get the first time a serious customer evaluates you.
The exposed customers in the report included banks, telcos, universities, research institutions, AI labs, and a national intelligence agency. Buyers like these ask:
- What isolates my workloads from other tenants: containers, VMs, or dedicated nodes?
- Is the control plane shared?
- Who can read my volumes, and what enforces that: the storage layer or the interface in front of it?
- Is backend storage on its own isolated network?
- Can anyone at your company reach my logs and monitoring data without my permission?
- Are machines reprovisioned from scratch between customers, or recycled?
It’s important to have answers to these questions before a customer asks, not while they’re waiting.
Three ways out of the second state
Getting out of the second state means picking one of three paths, and committing to it:
- Build on Ceph. You get full control over the design and no dependence on anyone else’s release schedule. The cost is a permanent, dedicated headcount line: Ceph specialists aren’t interchangeable with your compute engineers.
- Build your own hardware. You design, rack, and run your own storage servers, shaped to your customers’ workloads, and the revenue stays in-house. The cost is procurement, refresh cycles, and capital committed years ahead of demand you can’t forecast.
- Partner. Storage works and stays working without anyone on your team owning it, and your engineers stay on compute. What you’re trading is some control: a partner’s pricing, roadmap and deprecation timelines are decisions you’re part of rather than decisions you make alone, and when a customer’s job stalls on storage, they still call you. The right partner expects those questions early and answers them plainly. It’s worth asking before you sign.
All three are legitimate. The option most neoclouds are actually running is none of them.
Decide before the grade, not after
Every provider SemiAnalysis tested had storage. The ones who had decided what it was for, and kept checking that it worked, had an answer when customers asked. The rest had whatever the default gave them, and now that’s published.
If you rent GPUs by the hour today, your customers will eventually stop taking their data home. Decide on your storage before that happens, while there’s less data to migrate, no customers waiting on an answer, and no grade already in front of your prospects.
Read the full report here. And visit us to explore how Backblaze partners with neoclouds to ensure more open, AI-ready storage architectures.
The post Neoclouds: Your Storage Layer Just Got a Public Grade appeared first on Backblaze Blog | Cloud Storage & Cloud Backup