Ceph, ZFS replication and an external SAN or NAS can help Proxmox VMs survive a host failure. They differ in data freshness, recovery behavior and failure domains. Simply ticking “high availability” can hide the actual requirement: how many minutes of data loss can the application tolerate (RPO), and when must it be usable again (RTO)?
Here we compare three storage approaches for the same hypothetical three-host cluster. How Ceph rebuilds copies after a host fails is covered separately. This decision starts with applications and their verifiable recovery objectives. Separate backup with a restore test is additionally required for all three approaches.
Three architectures, three different failure paths
Ceph on local SSDs provides VM disks as distributed block storage accessible throughout the cluster. A replicated pool holds copies on multiple hosts; quorum, pool rule, network and free capacity must match the chosen host failure domain. After a VM host fails, Proxmox HA can restart the VM on another host if quorum and resources are available. An in-progress write, the guest application and its transactions still need assessment in the failure test. Ceph redundancy alone promises neither an uninterrupted VM nor guaranteed zero RPO for every application.
Local ZFS with Proxmox storage replication transfers VM volumes periodically to another host using snapshots. The target holds the state of the last successfully completed replication. Proxmox allows HA with this replication, but explicitly warns of possible data loss since the last synchronisation. A configured five-minute interval is therefore no blanket RPO commitment: delays, failed jobs and application state matter. Without a synchronous shared disk, the restarted VM must cope with exactly that older state.
External shared storage such as NFS or iSCSI makes VM disks accessible to several hosts and typically allows live migration without copying the VM disk. For host failure, data risk depends on the actual storage design. Check dual controllers, separate data and power paths, iSCSI multipathing or redundant NFS network paths, support and tested controller failover; a single NAS remains a central failure domain. Proxmox documentation explicitly warns against making a ZFS-over-iSCSI appliance a single point of failure. “Shared” in configuration does not prove a redundant appliance.
Which workload can tolerate which RPO and RTO?
RPO measures the maximum acceptable data gap; RTO measures the maximum acceptable time until the application is usable. The following are decision questions, not product SLAs. For each service, test the database, VM restart, DNS, user login and dependent systems together.
| Application and objective | Plausible approach | Decisive check |
|---|---|---|
| Transactional ERP database; little data loss tolerable | Ceph or demonstrably redundant shared storage; possibly application-native replication too | Write latency, transaction consistency, quorum and actual restart under host/storage failure |
| Internal test VM or recoverable web service; a data gap of minutes accepted | Local ZFS with replication may suffice | Age of last successful snapshot, delay alert, startup with older data |
| Existing supported SAN/NAS with adequate headroom | Continuing to use it as shared storage may be economical | Test controllers, paths, power, firmware, maintenance contract and live failover together |
For a production database with a very tight RPO, asynchronous ZFS replication may be unsuitable even if HA restart works. Conversely, a recoverable test service does not automatically justify a Ceph cluster. Three Proxmox hosts do not make a single storage server redundant. The technical choice follows accepted data loss and measured interruption, beyond the product name.
Three- and five-year calculation without invented market prices
A hypothetical comparison assumes three identical single-socket hosts with purchase price h per host and a separate backup server including media at price b . According to the current vendor list, Proxmox VE Standard costs €550 per occupied CPU socket/year; Proxmox Backup Server Standard costs €2,240 per server/year. Common support is therefore 3 × 550 + 2,240 = €3,890 annually. The model assumes €90 net per working hour. It does not assume equal IOPS or usable capacity without checking: real quotes first need normalization to the same workload.
d stands for additional Ceph data SSDs per host, z for local ZFS SSD equipment per host, a for a redundantly designed external appliance, nc, nz and ns for each required network or fabric investment. ec, ez and es are annual additional power/colocation costs; s is the external appliance’s annual vendor service contract. Every letter is a net quoted amount in euros, not an asserted market price.
| Approach | Setup / operations per year | 3 years | 5 years |
|---|---|---|---|
| Ceph | 80 h / 50 h | 3h + b + 3d + nc + 32,370 + 3ec € | 3h + b + 3d + nc + 49,150 + 5ec € |
| ZFS replication | 50 h / 30 h | 3h + b + 3z + nz + 24,270 + 3ez € | 3h + b + 3z + nz + 37,450 + 5ez € |
| External storage | 60 h / 35 h | 3h + b + a + ns + 26,520 + 3s + 3es € | 3h + b + a + ns + 40,600 + 5s + 5es € |
Working hours are model assumptions, not a DAXS package rate. Example: a €1,000 higher annual appliance service charge increases the external option by €3,000 over three years and €5,000 over five. An existing appliance may reduce a within the period, but remaining lifetime, support, spares and expansion belong in s or a separate investment field. Hardware prices, storage use, performance profile, network, tax and operating location need evidence from quotes. The lowest formula value is no purchase decision without equal usable capacity, IOPS, RPO and RTO.
Acceptance: actually test one failure in each failure domain
DAXS would require four proofs for each candidate architecture: (1) record the time of the last consistent application state, (2) measure host failure and VM restart under workload, (3) also deliberately fail the critical storage or network path, and (4) restore a VM and individual application data from separate backup. For ZFS, include failed or delayed replication jobs; for Ceph, the degraded pool and its rebuilding; for SAN/NAS, controller and path failover. Measurements determine whether RPO and RTO are met.
Ceph copies, replication snapshots and array mirroring also propagate deletions and bad data. They do not replace versioning, an external copy or a restore test. The DAXS article Proxmox Backup Server: verification, retention and restore goes deeper into backup acceptance. With an application inventory and RPO/RTO objectives, you can work with DAXS via Contact to measure the three approaches against real load and existing contracts.
Sources: Proxmox VE Administration Guide: storage, storage replication and HA · Proxmox VE: current support prices · Proxmox Backup Server: current support prices.
