Skip to main content

​
Introduction

EVE-OS supports redundant storage through the openZFS (ZFS) storage stack, starting with EVE-OS 8.0.0. You configure the RAID level at installation time using grub configuration parameters. After installation, you cannot migrate from one storage type to another without reinstalling EVE-OS. This procedure covers system requirements, RAID level options, installation procedures, and storage replacement. Use this when planning or installing EVE-OS on nodes that require fault-tolerant persistent storage.

​
System Requirements

ZFS requires sufficient memory and disk resources. The following specifications are recommended for sustained performance, but not strictly required: ZFS can be installed on configurations that do not meet these recommendations, but performance might be reduced. On devices that do not qualify, ext4-based storage is an alternative.

​
RAID Level Options

EVE-OS supports four ZFS RAID levels: none, raid1, raid5, and raid6. For raid1, raid5, and raid6, EVE-OS must be installed on a dedicated disk that is separate from the pool disks. raid0 (none) supports sharing the install disk with the pool. We recommend that all pool disks be the same type (NVMe, SSD, or HDD) and the same capacity. If disk capacities differ, the pool capacity is determined by the smallest disk in the pool, and the extra capacity on larger disks is wasted.

​
Storage Overhead

ZFS reserves approximately 20% of physical storage for internal use. Your usable storage before RAID is therefore about 80% of physical capacity. The following table uses 100 GB disks as an example and includes a typical 2:1 compression ratio for VM workloads: ZFS compression ratios depend on workload. Database workloads typically achieve 2:1 compression, recovering the storage overhead from RAID. In a worst case (1:1 compression), available storage equals the post-RAID figure in the preceding table. ZFS also reserves RAM for efficient storage stack operation:
This reserved RAM is not available to edge applications.

​
Configure ZFS During Installation

The ZFS RAID level is set through grub configuration parameters in the EVE-OS installer image. You must include these parameters before installation. They cannot be applied to an existing installation.

​
USB installation

To create a custom installer with ZFS configuration:
  1. Create an overrides directory on your computer:
  2. Create a file named grub.cfg in the eve-overrides directory. Set eve_install_zfs_with_raid_level to your desired RAID level and specify the install and persist disks. The following example configures raid5 with EVE-OS on /dev/sda and pool disks on /dev/sdb, /dev/sdc, and /dev/sdd:
  3. Build the custom installer image using Docker. Replace VERSION with your EVE-OS version (for example, 12.0.3-lts) and ARCH with amd64 or arm64: On Linux or macOS:
    On Windows (PowerShell):
  4. Flash the resulting installer.raw file to a USB drive and boot the node from it.

​
iPXE installation

iPXE-based installation supports the same grub configuration parameters as USB installation.

​
Example grub configurations by RAID level

For raid1, do not list the install disk in eve_persist_disk. Including the install disk in the persist disk list for a raid1 configuration results in a single-disk pool (no mirroring) or a failed installation.

​
Upgrade Considerations

EVE-OS does not provide a non-disruptive migration path from ext4-based storage to ZFS. If an existing device is configured with ext4, upgrading EVE-OS preserves the ext4 storage. To switch to ZFS, back up your applications, then perform a fresh USB installation with the desired ZFS configuration and re-onboard the node. Upgrades from ZFS-based storage to a newer EVE-OS release are supported.

​
Replace a Disk

Disk replacement requires SSH access to the EVE-OS device. The replacement disk must be the same size or larger than the original, and you must replace disks one at a time. Concurrent replacement of multiple disks is not supported. The following steps replace a failed or degraded disk in a raid5 pool:
  1. SSH to the node and enter the pillar namespace:
  2. Check the pool status to identify the disk to replace:
  3. Run the replace command. In this example, /dev/sdd is replaced with /dev/sdc:
  4. Enable automatic pool expansion so the replacement disk’s full capacity is used:
  5. Verify the replacement:
    The zpool status output should show the new disk in the pool configuration. The zpool list output should show the updated pool size if the replacement disk is larger.

​
Next Steps

Expand ZFS Pools on Nodes

Add a disk to an existing pool on a node configured with raid level none.

Configure grub.cfg for EVE-OS Multidisk Installation

Control disk partitioning and ZFS pool layout at installation time.

Install EVE-OS

Install EVE-OS on a device with the prepared installer image.