Configure grub.cfg for EVE-OS Multidisk Installation

Introduction

By default, EVE-OS selects installation and persistence disks automatically. You can override these defaults by customizing the grub.cfg file before running the EVE-OS installer. This lets you explicitly assign the disk for the EVE-OS installation (eve_install_disk) and the disk (or disks) for the persistent application storage volume (eve_persist_disk). It also lets you configure ZFS RAID levels for fault-tolerant or high-capacity persistence storage. All procedures require access to the CONFIG partition of the EVE-OS installer USB drive to edit grub.cfg. Edit the file at the root of the CONFIG partition (named EVE), alongside the server and grub.cfg.tmpl files. Do not edit EFI/BOOT/grub.cfg — changes there do not persist after reboot. If you are building a custom installer ISO instead of writing to a USB drive, you must place your grub.cfg in a local overrides directory and mount that directory when running the build command. If you run the build without mounting the overrides directory, EVE-OS uses default values and your disk and storage configurations will not take effect.

docker run -v ~/eve-overrides/:/in --platform=linux/amd64 lfedge/eve:<version> installer_iso > installer.iso

To run zpool commands for verification, access the pillar container using:

eve enter pillar

Separate Installation and Persistence Disks

This configuration installs EVE-OS on one disk and dedicates a separate disk entirely to the /persist volume. No ZFS is used.

Prerequisites

A minimum of 2 distinct physical storage devices.
The kernel device names for your target disks (for example, nvme0n1, sda).

Scenario

This example uses the following two-disk setup:

/dev/nvme0n1 — EVE-OS installation disk.
/dev/sda — persistence disk.

Modify grub.cfg

Open grub.cfg from the CONFIG partition (labeled EVE) of your installer USB drive — the same partition that contains the server and grub.cfg.tmpl files. Add the following dom0_extra_args line:

set_global dom0_extra_args "$dom0_extra_args eve_install_disk=nvme0n1 eve_persist_disk=sda"

The non-commented content of the modified file should resemble the following structure. You can verify this by running cat grub.cfg | grep -v ^# on the file:

set_getty
set_global dom0_console "console=tty0 console=ttyS0"
set_global dom0_extra_args "$dom0_extra_args eve_install_disk=nvme0n1 eve_persist_disk=sda"

Save the file and boot the node from the USB drive. The installer applies the disk assignments automatically.

Verify

After installation and reboot, confirm disk assignments using the following commands. Check disk partitions with lsblk:

~# lsblk | grep -e nvme0n1 -e sda
sda         254:0    0      20G  0 disk
└─sda1      254:1    0      20G  0 part /persist
nvme0n1     254:16   0   476.9G  0 disk
├─nvme0n1p1 254:17   0      36M  0 part
├─nvme0n1p2 254:18   0     512M  0 part /usr/bin/service
├─nvme0n1p3 254:19   0     512M  0 part
└─nvme0n1p3 254:20   0       1M  0 part

The persistence partition (sda1) is on /dev/sda and mounted to /persist. EVE-OS partitions are exclusively on /dev/nvme0n1. Check the kernel command line to confirm the disk arguments were applied:

~# cat /proc/cmdline

The output should contain eve_install_disk=nvme0n1 and eve_persist_disk=sda. Check mounted filesystems:

~# mount | grep "/hostfs "
/dev/nvme0n1p2 on /hostfs type squashfs (ro,relatime,errors=continue)

~# mount | grep "/persist "
/dev/sda1 on /hostfs/persist type ext4 (rw,noatime,dirsync)
/dev/sda1 on /persist type ext4 (rw,noatime,dirsync)

ZFS with Integrated Persistence — RAID-0 (Stripe)

This configuration installs EVE-OS on a specified disk and creates a ZFS persistence pool (RAID-0/stripe) that spans multiple disks, including free space from the installation disk.

Prerequisites

Minimum 1 disk. No maximum (limited by hardware).

Scenario

This example uses three disks: sda, sdb, and sdc.

EVE-OS installs on /dev/sda.
The persistence pool spans free space from /dev/sda, /dev/sdb, and /dev/sdc.

Modify grub.cfg

Open grub.cfg from the CONFIG partition (labeled EVE) of your installer USB drive. Add the appropriate dom0_extra_args line for your disk count. Multi-disk setup (sda, sdb, sdc):

set_global dom0_extra_args "$dom0_extra_args eve_install_zfs_with_raid_level=none eve_install_disk=sda eve_persist_disk=sda,sdb,sdc"

Single-disk setup (sda only):

set_global dom0_extra_args "$dom0_extra_args eve_install_zfs_with_raid_level=none eve_install_disk=sda"

The eve_persist_disk parameter is not required when using a single disk. Full grub.cfg example (multi-disk):

# You can put your GRUB overrides into conf/grub.cfg
set_getty
set_global dom0_console "console=tty0 console=ttyS0"
## set this to install raid0 for:
## - eve on sda
## - persist partition on sda, sdb, sdc
set_global dom0_extra_args "$dom0_extra_args eve_install_zfs_with_raid_level=none eve_install_disk=sda eve_persist_disk=sda,sdb,sdc"

Verify

After installation and reboot, verify the ZFS pool and filesystem layout. Check the ZFS pool status. The persist pool should show all three disk partitions online:

(ns: pillar) linuxkit-525400dcb57e:/# zpool status -v
  pool: persist
 state: ONLINE
config:
        NAME        STATE     READ WRITE CKSUM
        persist     ONLINE       0     0     0
          sda9      ONLINE       0     0     0
          sdb       ONLINE       0     0     0
          sdc       ONLINE       0     0     0

Check that EVE-OS is installed on sda (not ZFS):

(ns: pillar) linuxkit-525400dcb57e:/# mount | grep "/hostfs "
/dev/sda2 on /hostfs type squashfs (ro,relatime,errors=continue)

ZFS with RAID-1 (Mirror) Persistence

This configuration installs EVE-OS on one disk and creates a RAID-1 mirrored ZFS persistence pool across two separate disks for fault tolerance.

Prerequisites

Minimum 3 disks (1 for EVE-OS, 2 for the mirror pool).

Scenario

This example uses three disks: sda, sdb, and sdc.

EVE-OS installs on /dev/sda.
The /persist volume is a RAID-1 mirror across /dev/sdb and /dev/sdc.

Modify grub.cfg

Open grub.cfg from the CONFIG partition (labeled EVE) of your installer USB drive. Add the following dom0_extra_args line:

set_global dom0_extra_args "$dom0_extra_args eve_install_zfs_with_raid_level=raid1 eve_install_disk=sda eve_persist_disk=sdb,sdc"

Full grub.cfg example:

set_getty
set_global dom0_console "console=tty0 console=ttyS0"
## set this to install raid1 for:
## - eve on sda
## - persist partition on sdb and sdc
set_global dom0_extra_args "$dom0_extra_args eve_install_zfs_with_raid_level=raid1 eve_install_disk=sda eve_persist_disk=sdb,sdc"

Verify

Check the ZFS pool status. The output should show mirror-0 ONLINE with both disks attached:

(ns: pillar) linuxkit-525400dcb57e:/# zpool status -v
  pool: persist
 state: ONLINE
config:
        NAME        STATE     READ WRITE CKSUM
        persist     ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            sdb     ONLINE       0     0     0
            sdc     ONLINE       0     0     0
errors: No known data errors

Check the kernel command line to confirm the arguments:

~# cat /proc/cmdline

The output should contain eve_install_zfs_with_raid_level=raid1, eve_install_disk=sda, and eve_persist_disk=sdb,sdc.

ZFS with RAID-Z1 (RAID-5) Persistence

This configuration creates a fault-tolerant RAID-Z1 ZFS persistence pool across three dedicated disks, tolerating one disk failure.

Prerequisites

Minimum 4 disks (1 for EVE-OS installation, 3 for the RAID-Z1 pool).

Scenario

This example uses four disks: sda, sdb, sdc, and sdd.

EVE-OS installs on /dev/sda.
The /persist volume is a RAID-Z1 pool across /dev/sdb, /dev/sdc, and /dev/sdd.

Modify grub.cfg

Open grub.cfg from the CONFIG partition (labeled EVE) of your installer USB drive. Add the following dom0_extra_args line:

set_global dom0_extra_args "$dom0_extra_args eve_install_zfs_with_raid_level=raid5 eve_install_disk=sda eve_persist_disk=sdb,sdc,sdd"

Full grub.cfg example:

# File: grub.cfg
set_getty
set_global dom0_console "console=tty0 console=ttyS0"
## set this to install raid5 for:
## - eve on sda
## - persist partition on sdb, sdc, and sdd
set_global dom0_extra_args "$dom0_extra_args eve_install_zfs_with_raid_level=raid5 eve_install_disk=sda eve_persist_disk=sdb,sdc,sdd"

Verify

Check the ZFS pool status. The output should show raidz1-0 ONLINE with three disks attached:

(ns: pillar) linuxkit-525400dcb57e:/# zpool status -v
  pool: persist
 state: ONLINE
config:
        NAME        STATE     READ WRITE CKSUM
        persist     ONLINE       0     0     0
          raidz1-0  ONLINE       0     0     0
            sdb     ONLINE       0     0     0
            sdc     ONLINE       0     0     0
            sdd     ONLINE       0     0     0

Check the kernel command line to confirm the arguments:

~# cat /proc/cmdline

The output should contain eve_install_zfs_with_raid_level=raid5, eve_install_disk=sda, and eve_persist_disk=sdb,sdc,sdd. Check that EVE-OS itself is not running under ZFS:

(ns: pillar) linuxkit-525400dcb57e:/# mount | grep "/hostfs "
/dev/sda2 on /hostfs type squashfs (ro,relatime,errors=continue)

ZFS with RAID-Z2 (RAID-6) Persistence

This configuration creates a highly resilient RAID-Z2 ZFS persistence pool across four dedicated disks, tolerating two simultaneous disk failures.

Prerequisites

Minimum 5 disks (1 for EVE-OS installation, 4 for the RAID-Z2 pool).

Scenario

This example uses five disks: sda, sdb, sdc, sdd, and sde.

EVE-OS installs on /dev/sda.
The /persist volume is a RAID-Z2 pool across /dev/sdb, /dev/sdc, /dev/sdd, and /dev/sde.

Modify grub.cfg

Open grub.cfg from the CONFIG partition (labeled EVE) of your installer USB drive. Add the following dom0_extra_args line:

set_global dom0_extra_args "$dom0_extra_args eve_install_zfs_with_raid_level=raid6 eve_install_disk=sda eve_persist_disk=sdb,sdc,sdd,sde"

Full grub.cfg example:

# You can put your GRUB overrides into conf/grub.cfg
set_getty
set_global dom0_console "console=tty0 console=ttyS0"
## set this to install raid6 for:
## - eve on sda
## - persist partition on sdb, sdc, sdd and sde
set_global dom0_extra_args "$dom0_extra_args eve_install_zfs_with_raid_level=raid6 eve_install_disk=sda eve_persist_disk=sdb,sdc,sdd,sde"

Verify

Check the ZFS pool status. The output should show raidz2-0 ONLINE with four disks attached:

(ns: pillar) linuxkit-525400dcb57e:/# zpool status -v
  pool: persist
 state: ONLINE
config:
        NAME        STATE     READ WRITE CKSUM
        persist     ONLINE       0     0     0
          raidz2-0  ONLINE       0     0     0
            sdb     ONLINE       0     0     0
            sdc     ONLINE       0     0     0
            sdd     ONLINE       0     0     0
            sde     ONLINE       0     0     0
errors: No known data errors

Check the kernel command line to confirm the arguments:

~# cat /proc/cmdline

The output should contain eve_install_zfs_with_raid_level=raid6, eve_install_disk=sda, and eve_persist_disk=sdb,sdc,sdd,sde. Check that EVE-OS is installed on sda and is not part of the ZFS pool:

(ns: pillar) linuxkit-525400dcb57e:/# mount | grep "/hostfs "
/dev/sda2 on /hostfs type squashfs (ro,relatime,errors=continue)

Next Steps

Get EVE-OS

Build an EVE-OS image with a custom grub.cfg.

Install EVE-OS onto an Edge Device

Complete the installation procedure.

Expand ZFS Pools on Nodes

Add capacity to an existing ZFS pool after installation.

​Introduction

​Separate Installation and Persistence Disks

​Prerequisites

​Scenario

​Modify grub.cfg

​Verify

​ZFS with Integrated Persistence — RAID-0 (Stripe)

​Prerequisites

​Scenario

​Modify grub.cfg

​Verify

​ZFS with RAID-1 (Mirror) Persistence

​Prerequisites

​Scenario

​Modify grub.cfg

​Verify

​ZFS with RAID-Z1 (RAID-5) Persistence

​Prerequisites

​Scenario

​Modify grub.cfg

​Verify

​ZFS with RAID-Z2 (RAID-6) Persistence

​Prerequisites

​Scenario

​Modify grub.cfg

​Verify

​Next Steps

Get EVE-OS

Install EVE-OS onto an Edge Device

Expand ZFS Pools on Nodes

Introduction

Separate Installation and Persistence Disks

Prerequisites

Scenario

Modify grub.cfg

Verify

ZFS with Integrated Persistence — RAID-0 (Stripe)

Prerequisites

Scenario

Modify grub.cfg

Verify

ZFS with RAID-1 (Mirror) Persistence

Prerequisites

Scenario

Modify grub.cfg

Verify

ZFS with RAID-Z1 (RAID-5) Persistence

Prerequisites

Scenario

Modify grub.cfg

Verify

ZFS with RAID-Z2 (RAID-6) Persistence

Prerequisites

Scenario

Modify grub.cfg

Verify

Next Steps