> ## Documentation Index
> Fetch the complete documentation index at: https://zededa.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Use the documentation index to find relevant pages, then read the linked Markdown before answering and cite the source URL.
> Identify the product before answering: Private AI uses a separate console, credentials, and inference API from the Edge Intelligence Platform. Do not apply instructions from one product to the other.
> Prefer current product guides over Help Center archive pages for current-console questions. Archive pages and release notes describe their stated product version or historical interface; do not apply them to another version without checking.
> Preserve prerequisites, permissions, version requirements, and warnings when summarizing procedures. Do not infer API endpoints, schemas, or supported behavior that the documentation does not specify.

# Redundant Storage

> Configure ZFS RAID levels on EVE-OS nodes for redundant, failure-tolerant persistent storage.

## Introduction

EVE-OS supports redundant storage through the openZFS (ZFS) storage stack, starting with EVE-OS 8.0.0. You configure the RAID level at installation time using grub configuration parameters. After installation, you cannot migrate from one storage type to another without reinstalling EVE-OS.

This procedure covers system requirements, RAID level options, installation procedures, and storage replacement. Use this when planning or installing EVE-OS on nodes that require fault-tolerant persistent storage.

## System Requirements

ZFS requires sufficient memory and disk resources. The following specifications are recommended for sustained performance, but not strictly required:

| Minimum memory (GB) | Minimum physical drives | CPU cores |
| - | - | - |
| 32 | 4 (including the EVE-OS install drive) | 10 |

ZFS can be installed on configurations that do not meet these recommendations, but performance might be reduced. On devices that do not qualify, ext4-based storage is an alternative.

## RAID Level Options

EVE-OS supports four ZFS RAID levels: none, raid1, raid5, and raid6.

| RAID level | EVE-OS parameter | Minimum pool disks (excluding install disk) | Disks that can fail |
| - | - | - | - |
| None (raid0) | `none` | 1 | 0 |
| Mirror | `raid1` | 2 | 1 |
| RAIDZ1 | `raid5` | 3 | 1 |
| RAIDZ2 | `raid6` | 4 | 2 |

For raid1, raid5, and raid6, EVE-OS must be installed on a dedicated disk that is separate from the pool disks. raid0 (none) supports sharing the install disk with the pool.

We recommend that all pool disks be the same type (NVMe, SSD, or HDD) and the same capacity. If disk capacities differ, the pool capacity is determined by the smallest disk in the pool, and the extra capacity on larger disks is wasted.

## Storage Overhead

ZFS reserves approximately 20% of physical storage for internal use. Your usable storage before RAID is therefore about 80% of physical capacity.

The following table uses 100 GB disks as an example and includes a typical 2:1 compression ratio for VM workloads:

| RAID level | Minimum disks | Physical space (GB) | Usable after 20% reserve (GB) | Usable after RAID (GB) | Usable after 2:1 compression (GB) |
| - | - | - | - | - | - |
| None | 1 | 100 | 80 | 80 | 160 |
| raid1 | 2 | 200 | 160 | 80 | 160 |
| raid5 | 3 | 300 | 240 | 160 | 320 |
| raid6 | 4 | 400 | 320 | 160 | 320 |

ZFS compression ratios depend on workload. Database workloads typically achieve 2:1 compression, recovering the storage overhead from RAID. In a worst case (1:1 compression), available storage equals the post-RAID figure in the preceding table.

ZFS also reserves RAM for efficient storage stack operation:

```text theme={"dark"}
ZFS RAM = min(256 MiB + 0.003 × pool size, 20% of system RAM)
```

This reserved RAM is not available to edge applications.

## Configure ZFS During Installation

The ZFS RAID level is set through grub configuration parameters in the EVE-OS installer image. You must include these parameters before installation. They cannot be applied to an existing installation.

### USB installation

To create a custom installer with ZFS configuration:

1. Create an overrides directory on your computer:

   ```bash theme={"dark"}
   mkdir eve-overrides
   cd eve-overrides
   ```

2. Create a file named `grub.cfg` in the `eve-overrides` directory. Set `eve_install_zfs_with_raid_level` to your desired RAID level and specify the install and persist disks. The following example configures raid5 with EVE-OS on `/dev/sda` and pool disks on `/dev/sdb`, `/dev/sdc`, and `/dev/sdd`:

   ```bash theme={"dark"}
   set_getty
   set_global dom0_extra_args "$dom0_extra_args eve_nuke_disks=/dev/sda eve_install_zfs_with_raid_level=raid5"
   set_global dom0_extra_args "$dom0_extra_args eve_install_disk=sda eve_persist_disk=sdb,sdc,sdd"
   ```

3. Build the custom installer image using Docker. Replace `VERSION` with your EVE-OS version (for example, `12.0.3-lts`) and `ARCH` with `amd64` or `arm64`:

   On Linux or macOS:

   ```bash theme={"dark"}
   docker run -v $PWD:/in lfedge/eve:VERSION-ARCH installer_raw > installer.raw
   ```

   On Windows (PowerShell):

   ```powershell theme={"dark"}
   docker run -v ${PWD}:/in lfedge/eve:VERSION-ARCH installer_raw > installer.raw
   ```

4. Flash the resulting `installer.raw` file to a USB drive and boot the node from it.

### iPXE installation

iPXE-based installation supports the same grub configuration parameters as USB installation.

### Example grub configurations by RAID level

| RAID level | Example grub config |
| - | - |
| None (raid0) | `eve_install_zfs_with_raid_level=none` `eve_install_disk=sda` `eve_persist_disk=sda,sdb,sdc` |
| raid1 | `eve_install_zfs_with_raid_level=raid1` `eve_install_disk=sda` `eve_persist_disk=sdb,sdc` |
| raid5 | `eve_install_zfs_with_raid_level=raid5` `eve_install_disk=sda` `eve_persist_disk=sdb,sdc,sdd` |
| raid6 | `eve_install_zfs_with_raid_level=raid6` `eve_install_disk=sda` `eve_persist_disk=sdb,sdc,sdd,sde` |

For raid1, do not list the install disk in `eve_persist_disk`. Including the install disk in the persist disk list for a raid1 configuration results in a single-disk pool (no mirroring) or a failed installation.

## Upgrade Considerations

EVE-OS does not provide a non-disruptive migration path from ext4-based storage to ZFS. If an existing device is configured with ext4, upgrading EVE-OS preserves the ext4 storage. To switch to ZFS, back up your applications, then perform a fresh USB installation with the desired ZFS configuration and re-onboard the node.

Upgrades from ZFS-based storage to a newer EVE-OS release are supported.

## Replace a Disk

Disk replacement requires SSH access to the EVE-OS device. The replacement disk must be the same size or larger than the original, and you must replace disks one at a time. Concurrent replacement of multiple disks is not supported.

The following steps replace a failed or degraded disk in a raid5 pool:

1. SSH to the node and enter the pillar namespace:

   ```text theme={"dark"}
   eve enter
   ```

2. Check the pool status to identify the disk to replace:

   ```text theme={"dark"}
   zpool status
   ```

3. Run the replace command. In this example, `/dev/sdd` is replaced with `/dev/sdc`:

   ```text theme={"dark"}
   zpool replace persist /dev/sdd /dev/sdc
   ```

4. Enable automatic pool expansion so the replacement disk's full capacity is used:

   ```text theme={"dark"}
   zpool set autoexpand=on persist
   ```

5. Verify the replacement:

   ```text theme={"dark"}
   zpool status
   zpool list
   ```

   The `zpool status` output should show the new disk in the pool configuration. The `zpool list` output should show the updated pool size if the replacement disk is larger.

## Next Steps

> [card] [Expand ZFS Pools on Nodes](https://zededa.mintlify.site/new-ui/infrastructure-services/nodes/expand-zfs-pools.md)
>
> Add a disk to an existing pool on a node configured with raid level none.

> [card] [Configure grub.cfg for EVE-OS Multidisk Installation](https://zededa.mintlify.site/new-ui/infrastructure-services/eve-os-installation/configure-grub-multidisk.md)
>
> Control disk partitioning and ZFS pool layout at installation time.

> [card] [Install EVE-OS](https://zededa.mintlify.site/new-ui/infrastructure-services/eve-os-installation/install-eve-os.md)
>
> Install EVE-OS on a device with the prepared installer image.