Why 1% Block partitioning, and why now

A deep look at the per-tenant economics of GPU sharing — why MIG isn't fine-grained enough for course-scale or dev-scale workloads, and how CNLab's software slicer fills the gap without sacrificing isolation.

Three years ago we measured the average GPU utilization in a cluster of fifteen H100s shared across a dozen graduate-student projects. Mean utilization, weighted by 1-second samples, was 4.7%. The 95th percentile was 31%. Nine of fifteen GPUs spent more than half of every wall-clock day at zero. We knew the situation was bad. We didn't realize it was that bad.

The standard answer at the time was MIG: NVIDIA's Multi-Instance GPU. Take a single H100 and split it into seven hardware-isolated slices. Give each slice to a different student. Problem solved.

Except — for a graduate-level deep-learning course of 90 students, seven slices is still 13× too coarse. Each MIG slice has 10–20 GB of dedicated VRAM. A student loading a base BERT for a homework assignment is using 700 MB. The other 95% of that slice is dark silicon while the next student waits for office hours.

The economics of slicing

If your workload is dev-scale — a notebook, a debugging session, a sweep of small models — your effective GPU footprint is something like 0.5–10% of a card. We measured this on three real customer clusters: 11%, 6%, and 2.4% median per-tenant footprint. The minimum MIG slice (1g) is 14% of an H100's compute. So MIG, as the only sharing primitive, leaves between 60% and 95% of the slice unused per tenant.

Either you fix that, or you accept it as the cost of doing business. We didn't want to accept it.

Why software slicing

CNLab's 1% Block partitioning is software-level. We intercept CUDA memory allocations and kernel launches via a thin shim, enforce per-tenant quotas in a per-node daemon, and isolate at the cgroup + namespace level. The minimum allocation is 1% of the GPU's CUDA cores and 1% of its VRAM. On an H100, that's 1.32 SMs and 800 MB.

Three things made this possible now that weren't possible three years ago:

1. Driver stability. Driver 535+ exposes per-process memory metrics with sub-millisecond resolution. We can detect over-allocation immediately, not after a kernel crash.
2. cgroup v2 maturity. The unified hierarchy and memory.high accounting let us enforce VRAM quotas the same way we enforce CPU and host RAM. The kernel does the bookkeeping; our daemon just sets the bounds.
3. CUDA stream isolation. Recent CUDA versions made cross-stream interference much smaller. Two tenants on the same GPU don't see each other's latency in any meaningful way for our test workloads.

Isolation: the question that decided everything

The first review meeting I took with a customer security team was about isolation. "Software slicing? So tenant A can OOM tenant B?" The honest answer was: not in our model, but the question deserved a serious test.

We built a 200-tenant chaos suite. Each tenant runs a workload designed to stress one resource: huge memory allocations, kernel launches at the highest possible rate, illegal CUDA calls, intentional infinite loops. We ran the suite continuously for six weeks against an H100 with 100 active blocks.

Result: zero cross-tenant impact in the latency or throughput of well-behaved tenants. Two driver crashes, both isolated to a single tenant's process; the rest of the GPU recovered without downtime. Three OOM kills of misbehaving tenants by our daemon, exactly as designed.

What we don't claim

1% Block partitioning is software-level. It's strong enough to host 90 students on the same H100. It's not strong enough to host two competing companies on the same H100 — that's what MIG is for. The two compose: we use MIG when you need hardware isolation, and we use blocks underneath where you don't.

A typical CNLab deployment uses both. The shared university cluster I mentioned at the top of this post now runs four MIG slices per H100 — one per faculty group — with 25 1% blocks inside each. 100 students fit. Mean utilization is 91%. The professor still has time for office hours.

Where to next

We're working on dynamic block resize without job restart, and on AMD MI300 support for the same primitives. Both are in the 2026 H1 roadmap. If you have a deployment story you'd like us to write up, email us.