# Kubernetes LLM Serving: Chunked Prefill and PagedAttention in Production

> How to stabilize TTFT latency and eliminate VRAM fragmentation when scaling LLM serving on Kubernetes and OpenShift clusters.

- **Date de publication** : 2026-09-02T09:00:00.000Z

---

Deploying Large Language Models at scale on Kubernetes is not merely about spinning up a Pod with a declarative GPU resource request.

---

### 1. Concurrency Breakdown in Standard Deployments
Under concurrent enterprise load, the compute-bound prompt prefill phase chokes ongoing memory-bound token decode iterations, destroying Time-To-First-Token (TTFT) predictability and causing severe tail latencies.

---

### 2. Decoupled Execution with vLLM & KServe
Pairing **Chunked Prefill** with **PagedAttention** inside modern inference runtimes fundamentally stabilizes serving performance:
* **Near-Zero VRAM Fragmentation:** KV-Cache blocks are dynamically allocated into non-contiguous memory pages on demand.
* **Stable P99 Latency:** Interleaving sliced prompt chunks across engine iterations prevents decode starvation.
* **Deterministic GPU Memory Utilization:** Eliminates Out-Of-Memory (OOM) failures under heavy request batching.

---

### 3. Production Engineering in AS300
These core architectural primitives are built hands-on in the **AS300 (AI Platform Engineer)** curriculum at Aperta Scientia:
* Fine-tuning KServe ServingRuntimes on Red Hat OpenShift.
* Hardware acceleration profiling across NVIDIA GPU clusters.
* End-to-end Prometheus SLO metrics for inference monitoring.
---

### 4. Technical Sources & References
* **Kubernetes GPU Scheduling & Topology:** [Kubernetes Official Docs](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/)
* **vLLM KV Cache & Memory Management:** [vLLM Core Architecture](https://blog.vllm.ai)
* **KServe vLLM Runtime on OpenShift:** [KServe GitHub](https://github.com/kserve/kserve)

---

*Source : https://aperta.io/radar/2026-09-02-kubernetes-llm-serving-chunked-prefill-pagedattention/ — Aperta Scientia Tech Radar*