Orchestration for the Heterogeneous AI Data Center

Three technologies. Three architectures. One scheduler.

Sharing, scheduling, and migration — and how the same primitives compose into single-cluster, multi-cluster, or hybrid-cloud topologies. Built on Kubernetes, the NVIDIA Device Plugin, and InfiniBand/RDMA.

🧠

Intelligent Scheduler

Real-time DL execution detection. Priority + deadline queues. Proactive memory control prevents OOM under contention.

🦾

Resource Manager

Quota and partition assignment. RBAC-aware. Per-project, per-user limits. Audit-logged.

Migration Engine

Zero-checkpoint live migration. RDMA-direct on-prem; encrypted TCP for cloud. Sub-second handoff.

Per-Environment Operation

Auto-detect deep-learning GPU allocation timing

Whatever environment your researchers use, CNLab routes GPU allocations transparently. Your code calls .cuda(); we handle the rest.

📓

Jupyter Notebook

Notebook server runs on the closest scheduled slice. CUDA calls intercepted; allocations routed to MIG or 1% blocks based on profile.

💻

VS Code & IDEs

Remote runtime mounts your workspace, attaches GPU, forwards SSH and secrets. Compatible with Cursor, Windsurf, JetBrains.

⌨️

CLI

cnlab run train.py --gpu h100-25. Streams stdout, captures artifacts, releases the GPU on exit.

The Infrastructure Lifecycle

From access request to job completion, every workload flows through six stages. Each stage is observable, audited, and policy-bound.

01

Access Controls

JWT + RBAC. Per-user quota.

02

Node Agents

Daemons on every GPU node.

03

Intelligent Scheduler

Priority + deadline queue.

04

Job Submission

Idempotent, audited.

05

Global Triage

Best-fit cluster selection.

06

Dynamic Slicing

MIG + 1% blocks at run.

GPU Sharing & Partitioning

Eliminate GPU Idle Waste with Granular Slicing

The average enterprise GPU sits at 5% utilization. CNLab's two-tier slicing pushes it to 95%+ — without rewriting any user code.

Hardware MIG (NVIDIA H100/A100)

Up to 7 hardware-isolated instances per H100. Full memory bandwidth within each. Use for tenant boundaries that must survive driver bugs.

Software 1% Block (all NVIDIA + AMD)

100 simultaneous tenants per GPU. Resize without restart. Combines with MIG: a 1g MIG slice can host 14 1% blocks.

Memory pinning & IPC isolation

Per-tenant VRAM quotas. cgroup + namespace isolation on the host. Cross-tenant CUDA IPC denied.

GPU utilization: 5% → 95%+

Core Technology 02

Intelligent Multitier Scheduling

A single scheduler with policies you can layer: deadline, priority, fair-share, and bin-packing. Pre-emptive when needed; backfill-friendly always.

Deadline-aware

Submit jobs with target completion. Scheduler reserves slices forward in time and pre-empts lower-priority work to meet the deadline.

Fair-share

Long-term per-team usage normalised. A team that ran heavy yesterday yields ground to a starved team today.

Bin-packing

Where deadlines and shares allow, scheduler packs work onto the fewest nodes. Idle nodes power down for energy savings.

Pre-emption

Production inference pre-empts development training when capacity tightens. Pre-empted work resumes from the latest auto-checkpoint.

Proactive memory control

Watches per-tenant working set; resizes 1% block VRAM allocation in flight to prevent OOM crashes.

Per-cluster policy

Different clusters can run different scheduling policies. Federation respects each cluster's local rules.

Core Technology 03

Transparent Cloud Bursting

When local capacity tightens, CNLab provisions cloud GPUs in seconds, migrates the workload, and returns it when local frees up — without the user noticing.

Cloud cost: 95% → 28% of cloud-only baseline

Predictive provisioning

Scheduler forecasts saturation 90s ahead based on queue depth and historical patterns. Cloud provisioning starts before the local queue stalls.

Live migration to cloud

Same migration engine as cluster-to-cluster. Memory pages travel over TLS-encrypted, accelerated TCP. Sub-second pause; resume from exact instruction.

Automatic return

When local utilization drops, scheduler reclaims by migrating the cloud workload back. Cloud node terminates within 30 seconds.

Architecture Comparison

Quantifiable Infrastructure Impact

DimensionOn-Premise SingleMulti On-PremiseHybrid Cloud
GPU pool size4–3232–256128+ + cloud
Initial costLowestMediumMedium + opex
TCO vs cloud-only−40%−45%−50%+
FailoverSingle clusterCross-clusterCross-cluster + cloud
Data residency100% on-prem100% on-premOn-prem first
Setup timeDaysWeeksWeeks
Supported Stack

Vendor-Neutral & Native Integration

Multi-Cloud

AWS · GCP · Azure · Naver Cloud · KT Cloud — first-class. OCI & on-prem-only via API.

Kubernetes

CNLab installs as a K8s operator (CRD-driven). Compatible with K8s 1.27+, OpenShift 4.13+, Rancher.

Frameworks

PyTorch · TensorFlow · JAX · Mojo · NVIDIA NeMo · Megatron — all run unchanged.

GPU Vendors

NVIDIA H200/H100/A100/L40S/L4/RTX 6000 · AMD MI300/MI250 · roadmap: Intel Gaudi 3.

Networking

InfiniBand HDR/NDR · RoCEv2 · standard 100/400G Ethernet · WireGuard for cloud egress.

Observability

Prometheus + Grafana out of the box · OpenTelemetry traces · Loki logs · DCGM exporter for GPU metrics.

Get the Full Architecture Guide

68-page whitepaper. Details on slicing, scheduling, migration, and integration. Available immediately — no email gate.