# GPU Orchestration on Kubernetes in 2026: DRA, MIG, and Time-Slicing for LLM Serving

> Comparative analysis of GPU allocation mechanisms on Kubernetes and OpenShift AI: Dynamic Resource Allocation (DRA), Multi-Instance GPU, and Time-Slicing.

- **Date de publication** : 2026-09-09T08:00:00.000Z

---

In production generative AI architectures, GPUs represent both the highest infrastructure cost and the primary scalability bottleneck. Allocating hardware accelerators (NVIDIA H100, L40S, A100) to LLM ServingRuntimes (**vLLM**, **KServe**, **TGI**) or auxiliary microservices (embedding models, rerankers, RAG pipelines) requires orchestration granularity far beyond the legacy model of static, whole-node reservations.

Scaling on **Kubernetes** and **Red Hat OpenShift AI** relies on three complementary allocation mechanisms: **Time-Slicing**, **Multi-Instance GPU (MIG)**, and the adoption of **Dynamic Resource Allocation (DRA)** introduced in modern Kubernetes releases.

---

### 1. The Limitations of Legacy NVIDIA Device Plugins (1 Pod = 1 GPU)
Historically, the `k8s-device-plugin` exposed GPUs as discrete integer resources (`nvidia.com/gpu: 1`). While this approach suits heavy distributed training workloads (PyTorch, Ray Train), it proves inefficient for inference:
* **Severe VRAM Underutilization:** Running a lightweight embedding model (0.5 GB VRAM) or a reranker on an 80 GB GPU locks the entire accelerator away from other workloads.
* **Lack of Hardware Isolation:** Unconstrained application-level sharing leads to memory bus contention and unpredictable spikes in Time to First Token (**TTFT**).
* **Topology Blindness:** Inability to express NVLink affinity or inter-GPU bandwidth requirements without cumbersome manual scheduling setups.

---

### 2. Comparative Matrix: GPU Allocation Strategies

| Strategy | Memory & Compute Isolation | Recommended Use Case | Key Advantages | Trade-offs |
| :--- | :--- | :--- | :--- | :--- |
| **Time-Slicing** | None (Temporal Sharing) | Dev, testing, low-traffic auxiliary embeddings | Zero hardware configuration, high oversubscription | Risk of cross-tenant OOMs, no latency guarantees |
| **NVIDIA MIG** | Strict Hardware Slicing (VRAM + SMs) | Multi-tenant inference, collocated ServingRuntimes | Deterministic QoS, full failure domain isolation | Rigid profiles (e.g., 1g.10gb, 3g.40gb), high-end GPUs only (A100/H100) |
| **DRA (Dynamic Resource Allocation)** | Dynamic via CDI / Structured Parameters | Heterogeneous clusters, advanced LLMOps pipelines | Declarative hardware matching, decoupled scheduler | Requires modern Kubernetes (1.30+) and compatible driver stack |

---

### 3. The Power of Dynamic Resource Allocation (DRA)
Dynamic Resource Allocation (DRA) and the **Container Device Interface (CDI)** fundamentally modernize accelerator management on Kubernetes:
* **Structured Parameters:** Instead of a simple integer request, workloads define *ResourceClaims* based on required VRAM capacity, NVLink topology, or compute capability.
* **On-Demand Allocation:** The Kubernetes scheduler matches claims dynamically during scheduling passes without requiring cluster operators to statically carve node profiles.
* **Optimized Co-scheduling:** Native coordination between GPU allocations, NUMA affinity, and high-throughput network fabric for vLLM Tensor Parallelism.

---

### 4. Implementation on Red Hat OpenShift AI (RHOAI)
Within **Red Hat OpenShift AI**, these capabilities are delivered as an enterprise-grade platform:
* **NVIDIA GPU Operator on OpenShift:** Automated deployment of drivers, CDI, MIG profiles, and DCGM telemetry export via **Prometheus**.
* **KNative & KServe Scale-to-Zero:** Idling inactive ServingRuntimes to immediately release GPU slices for high-priority training or batch jobs.
* **Elastic Scheduling with Ray / KubeRay:** Coordinated distributed compute across heterogeneous infrastructure.

---

### 5. Advance Your AI Infrastructure Engineering with Aperta Scientia
To design, secure, and operate scalable AI infrastructure:
* **AS300 Curriculum — AI Platform Engineer (3 months):** The intensive live program to master OpenShift AI, GPU partitioning, vLLM, KServe, and prepare for the official **Red Hat Certified Specialist in AI/ML (EX267)** certification.
* **AS200 Curriculum — DevOps & Platform Engineering (3 months):** The fundamental platform engineering foundation covering hardened OpenShift clusters, Ansible automation, and GitOps.

👉 **[Explore the AS300 AI Platform Engineer Curriculum](/curriculums/as300/)** | **[Explore the AS200 DevOps Curriculum](/curriculums/as200/)** | **[Contact our Engineering Advisors](/contact/)**

---

### 6. Technical References & Documentation
* **Kubernetes Dynamic Resource Allocation (DRA):** [Official Kubernetes Documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic-resource-allocation/)
* **NVIDIA Multi-Instance GPU (MIG) Architecture:** [NVIDIA MIG User Guide](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/)
* **Red Hat OpenShift AI Hardware Acceleration:** [OpenShift AI Acceleration Guide](https://docs.redhat.com/en/documentation/red_hat_openshift_ai/)

---

*Source : https://aperta.io/radar/2026-09-09-gpu-orchestration-kubernetes-dra-mig-time-slicing-llmops/ — Aperta Scientia Tech Radar*