# ODF & Metro-DR: Multi-Cluster RPO=0 Resilience and Synchronous Storage on OpenShift

> OpenShift Data Foundation Metro-DR architecture: synchronous Rook-Ceph RBD mirroring, RamenDR and RHACM declarative orchestration for RPO=0 resilience on stateful and AI workloads.

- **Date de publication** : 2026-10-08T08:30:00.000Z

---

In production cloud-native environments, high availability for stateless architectures is well-mastered through multi-zone pod spreading. Conversely, for **mission-critical stateful services**—transactional databases, Kafka brokers, **KubeVirt** virtual machine disks, or generative AI vector databases (**Milvus, Qdrant**)—the sudden loss of an entire datacenter represents the ultimate platform engineering challenge: state corruption, irreversible data loss (*RPO > 0*), or extended downtime (*high RTO*).

While asynchronous periodic snapshot backups (CSI / OADP / Velero) are ideal for regional, long-distance disaster recovery, they fall short of meeting zero-data-loss business continuity mandates across metropolitan distances. This is precisely where **OpenShift Data Foundation (ODF)** in a **Metro-DR** topology delivers: combining the **Rook-Ceph** distributed storage engine, block-level synchronous replication (**Ceph RBD Mirroring**), and the **RamenDR** multi-cluster orchestration operator governed by **Red Hat Advanced Cluster Management (RHACM)** to guarantee **RPO = 0** and automated failover within minutes.

---

### 1. Two Disaster Recovery Architectures in ODF: Metro-DR vs Regional-DR

Building a resilient enterprise storage architecture requires distinguishing between the two primary DR modes available in OpenShift Data Foundation:

1. **Regional-DR (Asynchronous — RPO > 0):** Deployed across two OpenShift clusters separated by hundreds of kilometers, where wide-area network latency makes synchronous replication impractical. RBD mirroring operates via differential snapshots scheduled at fixed intervals (e.g., every 5 to 15 minutes). In the event of a datacenter disaster, writes committed after the latest snapshot are lost.
2. **Metro-DR (Synchronous — RPO = 0):** Deployed across a metropolitan area connecting two data centers via dark fiber with round-trip network latency (**RTT < 10 ms**). Every block write operation is acknowledged simultaneously across both sites before returning success to the application layer. In a complete site outage, **zero data is lost (RPO = 0)**.

In enterprise two-cluster Metro-DR setups, two independent OpenShift clusters each host a dedicated local ODF/Ceph storage instance, synchronously mirrored over encrypted inter-cluster tunnels managed by **Submariner**. A lightweight third site hosts the RHACM hub cluster to act as an independent arbiter and prevent split-brain scenarios.

---

### 2. Under the Hood: Rook-Ceph RBD Mirroring and Submariner

At the storage layer, ODF relies on the open-source **Rook-Ceph** operator:

* **Synchronous rbd-mirror Daemon:** Ceph RADOS Block Device (RBD) pools backing Persistent Volume Claims (PVCs) operate with journaled synchronous replication. Every I/O write request is recorded in Ceph's journal and committed to the peer cluster prior to completing the POSIX call.
* **Secure Networking with Submariner:** Storage replication traffic between OpenShift clusters flows across dedicated IPSec/WireGuard tunnels established at the CNI layer (OVN-Kubernetes) by **Submariner**, eliminating any requirement to expose Ceph MON or OSD daemons to external networks.
* **Fault Domain Isolation:** Unlike a single stretch cluster, a two-cluster Metro-DR model fully decouples the Kubernetes control planes and etcd instances. Network instability or API Server failure on Site A has zero blast radius on Cluster B.

---

### 3. Declarative Orchestration with RamenDR and RHACM

The operational power of this architecture lies in declarative orchestration driven by Kubernetes Custom Resource Definitions (CRDs) from the open-source **RamenDR** project, integrated natively into **Red Hat Advanced Cluster Management (RHACM)**:

* **`DRPolicy` CRD:** Specifies the protection policy, replication scheduling interval (0m for strict synchronous Metro-DR), and participating cluster pairs:
```yaml
apiVersion: ramendr.openshift.io/v1alpha1
kind: DRPolicy
metadata:
  name: metro-dr-sync-policy
spec:
  drClusters:
    - ocp-cluster-primary
    - ocp-cluster-secondary
  schedulingInterval: 0m # Strict synchronous mode (RPO=0)
```
* **`DRPlacementControl` (DRPC) CRD:** Binds application workloads (managed via RHACM `Placement` or ArgoCD `ApplicationSet`) to the `DRPolicy`. RamenDR continuously reconciles replication health across all PVCs bound to the target namespace.
* **Failover vs Relocate:**
  * **Relocate (Planned Switchover):** Used for maintenance windows. RamenDR cleanly pauses the application on the primary cluster, drains pods, flushes I/O buffers, unmounts Ceph PVCs, demotes the primary RBD mirror, promotes the peer volume to primary, and boots workloads on the target cluster with zero data loss.
  * **Failover (Unplanned Outage):** When the primary cluster suffers an unrecoverable failure, RHACM triggers emergency failover. RamenDR promotes the secondary RBD volumes to read/write mode and immediately deploys pods on the surviving cluster.

---

### 4. Enterprise Workloads: KubeVirt and AI Vector Databases

This resilient storage architecture addresses two cornerstone challenges in platform engineering:

1. **KubeVirt Virtual Machines (OpenShift Virtualization):** VM disks run on raw Block PVCs backed by RBD StorageClasses. With Metro-DR, critical legacy virtual machines running inside KubeVirt can fail over between datacenters without reprovisioning, preserving total disk state and filesystem integrity.
2. **AI Pipelines & RAG Vector Databases (Milvus / Qdrant):** Production vector stores supporting enterprise LLM applications demand high-throughput, low-latency persistence for dense vector indexes. Metro-DR protects persistent vector indexes and metadata from split-brain corruption and datacenter loss, ensuring uninterrupted sovereign AI operations.

---

### 5. Curriculum Alignment at Aperta Scientia

* **AS200 Track (DevOps Platform Engineer — 399h):** Advanced modules DO380 (OpenShift Automation and Integration) and DO370 (Enterprise Storage with OpenShift Data Foundation)—deploying and tuning **Rook-Ceph** pools, configuring RBD/CephFS StorageClasses, establishing multi-cluster tunnels with **Submariner**, and automating declarative DR failover workflows with **RHACM** and **RamenDR**.
* **AS300 Track (AI Platform Engineer — 399h):** High-availability storage infrastructure for AI data pipelines and LLM inference—vector database persistence (**Milvus**, **Qdrant**), managing stateful cache volumes and model weights with zero-data-loss guarantees across multi-zone sovereign platforms.

---

### Technical Sources & References

* **Red Hat Developer Blog** — *Disaster recovery for stateful applications on OpenShift*: [developers.redhat.com/articles/2025/08/14/disaster-recovery-stateful-applications-openshift](https://developers.redhat.com/articles/2025/08/14/disaster-recovery-stateful-applications-openshift)
* **Red Hat Documentation** — *Configuring OpenShift Data Foundation Disaster Recovery for OpenShift Workloads (Metro-DR)*: [docs.redhat.com/en/documentation/red_hat_openshift_data_foundation/4.16/html/configuring_openshift_data_foundation_disaster_recovery_for_openshift_workloads/](https://docs.redhat.com/en/documentation/red_hat_openshift_data_foundation/4.16/html/configuring_openshift_data_foundation_disaster_recovery_for_openshift_workloads/)
* **Rook-Ceph Documentation** — *Ceph Cluster CRD & Synchronous Mirroring*: [rook.io/docs/rook/latest-release/CRDs/Cluster/ceph-cluster-crd/](https://rook.io/docs/rook/latest-release/CRDs/Cluster/ceph-cluster-crd/)
* **RamenDR Project (CNCF Sandbox)** — *Disaster Recovery for Kubernetes Stateful Applications*: [ramendr.github.io/ramen/](https://ramendr.github.io/ramen/)
* **Red Hat Advanced Cluster Management (RHACM)** — *Multi-Cluster Application Lifecycle & DRPolicy Automation*: [access.redhat.com/documentation/en-us/red_hat_advanced_cluster_management_for_kubernetes/](https://access.redhat.com/documentation/en-us/red_hat_advanced_cluster_management_for_kubernetes/)

---

*Source : https://aperta.io/radar/2026-10-08-odf-rook-ceph-metro-dr-multicluster-resilience-openshift/ — Aperta Scientia Tech Radar*