Attach an Existing ZFS Pool to the Compose Runtime

Introduction

Some nodes already hold application data in a ZFS pool that was created by an earlier application instance, for example a virtual machine that ran Kubernetes with OpenEBS storage. When you move those workloads to Docker Compose, the data needs to come with them. The ZFS variant of the Compose Runtime solves this. It attaches the existing ZFS pool volume as a second drive on the runtime, imports the pool when the runtime boots, discovers the datasets inside it, and mounts each one at a stable path inside the runtime. Docker Compose applications then bind those paths as ordinary volumes. The Compose file never has to name the dataset’s real identifier, which on Kubernetes-managed pools is often an opaque string such as pvc- followed by a UUID. This page covers the three pieces of configuration the ZFS variant adds on top of the standard Compose Runtime: the second drive, the two configuration files in the runtime’s configuration template, and the volume binding in each Compose application. For everything else, the ZFS variant deploys exactly like the standard one. This capability is designed for migrating existing ZFS-backed workloads to Docker Compose on a node where the data already lives. If you are starting a new deployment and do not have an existing pool, use the standard Compose Runtime variant. Contact your ZEDEDA account team before adopting the ZFS variant for a new deployment.

How It Works

When the ZFS Compose Runtime boots, a service named zrun-zvol-mount runs before the Docker Engine starts. It reads two files that are written by the runtime’s configuration template:

/etc/zededa/zvol-mounts.sh names the pool to import and declares a list of logical dataset names. For each logical name, it defines a small detection function that inspects a dataset’s contents and returns success when the dataset matches. This lets the runtime identify “the database dataset” or “the configuration dataset” by what is inside it rather than by its identifier, so the same template works across nodes where every dataset has a different identifier.
/etc/zededa/zvol-mounts.conf sets the permission mode and numeric owner for each logical name’s mount point.

The service imports the pool, mounts it under /imported_zpool_mount_root, runs the detection functions against a read-only staged copy of each dataset, and bind-mounts each matched dataset at /imported_zfs_vols/<logical name>. Compose applications reference that path. As shipped, the pool name in zvol-mounts.sh is empty. With an empty pool name the service does nothing and exits successfully, so a ZFS runtime with an unedited template behaves like a standard runtime. You enable the feature by filling in the pool name and dataset configuration. You configure the template once per edge app. Every runtime instance deployed from that edge app, on any node, uses the same detection logic.

Prerequisites

A Compose Runtime edge app of the ZFS variant (zed-compose-runtime_<size>-zfs) in your library. See Add a Compose Runtime Image.
EVE-OS (Edge Virtualization Engine) 14.5.0 LTS or greater on the target node. If the node uses ZFS persistent storage, a 16.0.x LTS release of EVE-OS or greater is required.
The ZFS pool volume instance already exists on the target node. It was created by an earlier application instance and survived that instance’s removal. You know the volume’s Name, which you use as its label.
The earlier application instance that owned the pool is stopped or deleted. A ZFS pool can be imported by only one consumer at a time.
You know the ZFS pool name as it appears inside the pool itself (for example, zfspoolname), and you know at least one file or directory that uniquely identifies each dataset you want to mount.
You have the SysManager or SysAdmin role in your enterprise. See Manage Roles.

Add the ZFS Pool Volume as a Second Drive

The runtime edge app needs a second drive that points at the existing volume by label. If you imported the ZFS variant from the Marketplace, open it in Marketplace > My Library, select Duplicate, and then select Duplicate & Configure to reach the wizard.

In the edge app wizard, advance to the Drives step.
Confirm that the first drive references the Compose Runtime image.
Select Add Drive.
In the second drive, keep the source as From Image but do not select an image. Instead, set Volume Label to the Name of the existing ZFS pool volume instance. The label is how EVE-OS maps this drive to the volume instance that already exists on the node. A drive that has a volume label and no image attaches the node’s existing volume instead of creating a new one.
Set Max Size to the size of the existing volume.
Leave Purge on Update off. A labeled drive is never purged, so the volume continues to belong to the node and is not deleted if the runtime instance is removed.
Select Next and continue through the wizard to the configuration template.

The volume must exist on the node before the runtime instance starts. If the label does not match a volume instance on the target node, the runtime instance fails to start. Verify the volume in Infrastructure > Workloads > Volumes before deploying.

Configure Pool Import and Dataset Discovery

The configuration template on the ZFS variant already contains both files. You edit them in place in the Configurations step of the edge app wizard. Do not remove or reorder other parts of the template.

Edit zvol-mounts.sh

Locate the write_files entry whose path is /etc/zededa/zvol-mounts.sh. Its content defines four variables and a set of detection functions.

Variable	Description
`ZPOOL_IMPORT_NAME`	The name of the ZFS pool to import. Leave empty to disable pool import.
`ZPOOL_MOUNT_ROOT`	Where the imported pool is mounted inside the runtime. The default is `/imported_zpool_mount_root`. You do not normally change this.
`ZFS_VOL_ROOT`	The parent path for the friendly bind mounts. The default is `/imported_zfs_vols`. Compose applications bind paths under this root. You do not normally change this.
`ZFS_LOGICAL_DATASET_NAMES`	A comma-separated list of logical names, one per dataset you want to mount. Each name must have a matching detection function.

For every name in ZFS_LOGICAL_DATASET_NAMES, define a shell function named zfs_dataset_detect_<logical name>. The runtime calls each function once per dataset in the pool, passing three arguments: the logical name, the dataset name, and the path to a read-only staged copy of the dataset. The function must return 0 when the dataset matches and 1 otherwise. Use only simple file system tests against the staged path. Each function runs under a timeout. The following example declares two logical names and identifies each dataset by a file that exists only in that dataset.

ZPOOL_IMPORT_NAME="zfspoolname"
ZPOOL_MOUNT_ROOT="/imported_zpool_mount_root"
ZFS_VOL_ROOT="/imported_zfs_vols"
ZFS_LOGICAL_DATASET_NAMES="example_ds1,example_ds2"

zfs_dataset_detect_example_ds1() {
  [ -f "$3/root/example_ds1" ] && return 0
  return 1
}

zfs_dataset_detect_example_ds2() {
  [ -f "$3/data/example_ds2" ] && return 0
  return 1
}

Replace the pool name, logical names, and detection tests with values that match your data. Be specific in each test. A test that matches more than one dataset causes the wrong dataset to be mounted under that logical name. A logical name with no matching function is a configuration error, and the service reports it rather than skipping it.

Edit zvol-mounts.conf

Locate the write_files entry whose path is /etc/zededa/zvol-mounts.conf. This file sets permissions on each mount point. Permissions apply to the mount point only, not recursively to the files inside. Each line has the following format:

<logical name> [mode] [uid:gid] [options]

Field	Description
`<logical name>`	One of the names from `ZFS_LOGICAL_DATASET_NAMES`.
`mode`	Octal permission mode for the mount point. The default is `0770`.
`uid:gid`	Numeric user and group ID that own the mount point. The default is `0:0`. The IDs must be numeric and must match the user the container runs as, because there is no user namespace remapping between the runtime and its containers. Group names from the host mean nothing inside a container.
`options`	Optional. To set a world-writable mode such as `0777`, you must also add `allow_world_writable=1` here. Without it, a world-writable mode is rejected.

The following example matches the two logical names from the preceding section. The first mount is owned by root. The second is owned by UID and GID 999, which is a common default for database containers.

example_ds1 0770 0:0
example_ds2 0770 999:999

Blank lines and lines beginning with # are ignored.

Save the edge app

Select Next through the remaining wizard steps.
On the Review step, confirm that two drives are listed and that the configuration template contains your edited zvol-mounts.sh and zvol-mounts.conf entries.
Select Create Edge App, or Save Changes if you are editing an existing edge app.

Deploy the runtime as described in Deploy the Compose Runtime Application.

Bind an Imported Dataset in a Compose Application

After the runtime is online, any Docker Compose application on that node can mount an imported dataset by binding its friendly path. Add a volumes entry to the service in your compose.yaml:

services:
  app:
    image: your-registry.example.com/your-app:latest
    volumes:
      - /imported_zfs_vols/example_ds1:/data

The left side is the path inside the runtime, formed from ZFS_VOL_ROOT and the logical name. The right side is the path inside the container. The container reads and writes the dataset directly. If the container runs as a non-root user, make sure the uid:gid you set in zvol-mounts.conf for that logical name matches the container’s user. Otherwise the container can see the mount but cannot write to it. For the rest of the Compose application definition, see Add a Docker Compose Image.

Verify the Configuration

In the sidebar, select Infrastructure > Workloads > Applications.
Confirm that the Compose Runtime instance shows Run State as Online.
Connect to the runtime over SSH as described in Troubleshoot Docker Compose.
List the mounted datasets. Each logical name you declared appears as a directory.

ls /imported_zfs_vols/

Check one of the mounts. The size and usage reflect the dataset, not the runtime’s own disk.

df -h /imported_zfs_vols/<logical name>

Deploy a Compose application that binds one of the paths, write a test file from inside the container, and confirm that the file appears under /imported_zfs_vols/<logical name> on the runtime.

If a logical name is missing from /imported_zfs_vols/, check the runtime logs for messages from zrun-zvol-mount. The most common causes are a pool name that does not match, a detection function that matched no dataset, or a logical name with no detection function.

Next Steps

This article is part of the Docker Compose setup series.

Docker Compose Overview: background on the architecture and deployment model.
Add a Compose Runtime Image: choose the ZFS variant and import it from the Marketplace.
Deploy the Compose Runtime Application: deploy the configured runtime to the node that holds the pool.
Attach an Existing ZFS Pool to the Compose Runtime (you are here).
Compose App Instance Control Overview: how the Compose Runtime manages Docker Compose application lifecycles.
Add a Docker Compose Image: create the Compose application that binds the imported datasets.
Deploy a Docker Compose Application: deploy the Docker Compose app as an application instance on the node that runs the Compose Runtime.
Troubleshoot Docker Compose: diagnose issues with the Compose Runtime or Docker Compose application instances.

Start Here

Infrastructure Services Overview

Core infrastructure

Build

Applications and delivery

Edge Kubernetes

Operations

Monitor

Administration and Security

Troubleshooting

Partnerships

Attach an Existing ZFS Pool to the Compose Runtime

Introduction

How It Works

Prerequisites

Add the ZFS Pool Volume as a Second Drive

Configure Pool Import and Dataset Discovery

Edit zvol-mounts.sh

Edit zvol-mounts.conf

Save the edge app

Bind an Imported Dataset in a Compose Application

Verify the Configuration

Next Steps

​Introduction

​How It Works

​Prerequisites

​Add the ZFS Pool Volume as a Second Drive

​Configure Pool Import and Dataset Discovery

​Edit zvol-mounts.sh

​Edit zvol-mounts.conf

​Save the edge app

​Bind an Imported Dataset in a Compose Application

​Verify the Configuration

​Next Steps

Introduction

How It Works

Prerequisites

Add the ZFS Pool Volume as a Second Drive

Configure Pool Import and Dataset Discovery

Edit zvol-mounts.sh

Edit zvol-mounts.conf

Save the edge app

Bind an Imported Dataset in a Compose Application

Verify the Configuration

Next Steps