> ## Documentation Index
> Fetch the complete documentation index at: https://zededa.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Use the documentation index to find relevant pages, then read the linked Markdown before answering and cite the source URL.
> Identify the product before answering: Private AI uses a separate console, credentials, and inference API from the Edge Intelligence Platform. Do not apply instructions from one product to the other.
> Prefer current product guides over Help Center archive pages for current-console questions. Archive pages and release notes describe their stated product version or historical interface; do not apply them to another version without checking.
> Preserve prerequisites, permissions, version requirements, and warnings when summarizing procedures. Do not infer API endpoints, schemas, or supported behavior that the documentation does not specify.

# Manage Clusters

> Monitor cluster status, update cluster identity, replace a failed node, upgrade EVE-OS across member nodes, and delete clusters.

After creating a cluster, you can monitor its status, update its identity, replace a node that needs to be swapped out, and delete the cluster when it is no longer needed.

## Introduction

This article covers day-to-day management of clusters through the console. For ZCLI-based cluster management, see the ZCLI Procedures page.

## Prerequisites

* You have the SysManager or SysAdmin role in your enterprise. See [Manage Roles](https://zededa.mintlify.site/new-ui/infrastructure-services/administration/manage-roles.md).
* You have created at least one cluster. See [Create Clusters](https://zededa.mintlify.site/new-ui/infrastructure-services/node-clusters/create-node-clusters.md).

## View Clusters

1. Open **Infrastructure > Fleet > Clusters**.
2. Review the list of clusters. Select any cluster name to open its detail page.

If your enterprise has at least one cluster running in ZEDEDA Edge Kubernetes Service mode, the cluster list adds a **Mode** column and a matching **Mode** filter. The column shows one of the available values for each cluster:

* **Edge Node Cluster**: a cluster of ZEDEDA-managed edge nodes.
* **K8s Cluster**: a cluster running in ZEDEDA Edge Kubernetes Service mode.
* **Imported K8s Cluster**: a third-party Kubernetes cluster brought under ZEDEDA management.

A cluster that uses two-node HA with a witness also shows a **Two-node HA** badge in the **Mode** column. Its **Status** column shows the HA state (for example, **Healthy** or **Degraded**) instead of the run state. See [Cluster Status and Health](https://zededa.mintlify.site/new-ui/infrastructure-services/node-clusters/node-cluster-status-and-health.md#interpret-two-node-ha-state).

The same list adds an **Import Cluster** button next to the create (**+**) button. Selecting it opens the dialog for importing a third-party Kubernetes cluster. Both buttons require create permissions, and **Import Cluster** additionally requires the Kubernetes Orchestration entitlement. When the list has no clusters to show, the same two actions appear in the empty state as **Create Cluster** and **Import Cluster** buttons, under the same permission and entitlement rules. For the import procedure, see [Create and Manage a Kubernetes Cluster](https://zededa.mintlify.site/new-ui/infrastructure-services/edge-kubernetes/create-and-manage-a-kubernetes-cluster.md).

> [!NOTE]
>
> Without the Kubernetes Orchestration entitlement, the cluster list omits K8s and imported clusters entirely, and the **Mode** column, the **Mode** filter, and the **Import Cluster** button do not appear. The list counts and filters reflect only the clusters you can see. Entitlements are set per enterprise and must also be enabled at the root enterprise. See [Admin Dashboard](https://zededa.mintlify.site/new-ui/infrastructure-services/administration/admin-dashboard.md#entitlements).

An imported Kubernetes cluster that has not completed registration shows an inline registration card on its detail page. The card provides the registration command for your cluster and a `cluster-admin` role-binding command, each with a copy control, and it stops appearing after the cluster finishes registering. A **Registration Commands** action in the upper right of the detail page opens the same commands. That action is available on any imported Kubernetes cluster and remains available after registration completes, so you can retrieve the commands again to re-register the cluster or add another node. For the full registration procedure, see [Create and Manage a Kubernetes Cluster](https://zededa.mintlify.site/new-ui/infrastructure-services/edge-kubernetes/create-and-manage-a-kubernetes-cluster.md#register-the-cluster).

The detail page is organized into up to six tabs: **Overview**, **Details**, **Nodes**, **Applications**, **Virtual Interfaces**, and **Storage**. Clusters running in ZEDEDA Edge Kubernetes Service mode and imported clusters show a different set of tabs. See [Kubernetes dashboard tab](#kubernetes-dashboard-tab) for which tabs appear on those clusters.

On a cluster with Kubernetes access, the header also offers **Download Kubeconfig**, which saves a kubeconfig file for the cluster, and **Cluster Shell**, which opens a `kubectl` session in the console drawer. Treat the kubeconfig file with the same care as an API key: it grants access to the cluster. On an imported cluster that has not finished registering, the header offers **Registration Commands**. A **View Events** button in the upper right of the detail page opens the Events page filtered to this cluster. A **Notifications** bell icon shows a color-coded badge when EVE-OS has reported errors, warnings, or notices. Select the bell to view notifications. For badge severity meanings and troubleshooting guidance, see [Cluster Status and Health](https://zededa.mintlify.site/new-ui/infrastructure-services/node-clusters/node-cluster-status-and-health.md).

### Overview tab

The **Overview** tab provides a high-level summary of the cluster. It contains the following sections.

**Cluster ribbon**: a row of quick-reference fields at the top of the page:

| Field | Description |
| - | - |
| Status | Operational run state of the cluster. For a description of each status value and admin state, see [Cluster Status and Health](https://zededa.mintlify.site/new-ui/infrastructure-services/node-clusters/node-cluster-status-and-health.md). |
| Members | Total number of nodes in the cluster, with a ready count (for example, 2/3 ready). |
| Updated | How recently the cluster configuration was last updated. |
| Type | Cluster type as configured at creation time. |
| Project | The project the cluster belongs to. Select to navigate to the project. |
| Applications | Number of application instances running on the cluster. Select to navigate to the Applications tab. |

**Two-node HA**: appears at the top of the tab only on a cluster that uses two-node HA with a witness. See [Manage Two-Node HA](#manage-two-node-ha).

**Cluster topology**: shows the health breakdown of member nodes (Ready, Not Ready, Unspecified counts) and a role breakdown (for example, Server × 2, Tie Breaker × 1).

**Identity**: shows the cluster's ID, name, title, project, type, cluster prefix, admin state, run state, member count, and timestamps at a glance.

**Workloads**: a sub-tabbed section summarizing applications, virtual interfaces, and nodes associated with the cluster. Select any item to navigate to its detail page.

**Recent activity**: lists recent recorded events for the cluster, followed by the cluster lifecycle entries. Each event shows its timestamp, its severity (for example INFO, WARNING, or ERROR), and a short description of what happened. Up to 20 events are listed, and the section scrolls when the list is longer than the panel. Beneath the events, the lifecycle entries always appear: the last reported state, the last configuration update, and the project the cluster was created in. A cluster with no listed events is expected and does not indicate a problem.

### Details tab

The **Details** tab presents the full cluster configuration in the following cards.

**Identity & Ownership**: ID, Name, Title, Description, Project, and Tags.

A cluster with Kubernetes access has two identifiers. The **ID** on this tab is the cluster UUID, which the cluster lifecycle and application instance APIs use. The orchestrator assigns a second, shorter ID of the form `c-xxxxx` (the `zksClusterId` field in the API), which Helm deployments, the Kubernetes dashboard proxy, and snapshot APIs use. When a GitOps target refers to a cluster, use the cluster **Name**, not either ID. If an API call that takes a cluster ID fails with a not-found error, check that you are passing the right one.

**Configuration**: cluster Type and Cluster prefix (the private IP range for inter-node communication).

**State & Timeline**: Admin State, Run State, Created timestamp, and Last Updated timestamp.

### Nodes tab

The **Nodes** tab lists all member nodes with the following columns:

| Column | Description |
| - | - |
| Node | Name of the node. Select to open the node's detail page. |
| Type | Node role in the cluster: **Server**, **Agent**, **Master**, **Worker**, or **Tie Breaker**. |
| Status | Readiness of the node within the cluster. For a description of each readiness state and secondary conditions, see [Cluster Status and Health](https://zededa.mintlify.site/new-ui/infrastructure-services/node-clusters/node-cluster-status-and-health.md#interpret-member-node-readiness). |
| Interface | The cluster interface (network adapter) assigned to this node. |
| Age | How long the node has been part of the cluster. |
| Internal IP | The internal IP address assigned to the node within the cluster. |
| Actions | A **Replace** button for swapping out the node. Available to users with update permissions. |

On a two-node HA cluster, each node also shows a **Seed** or **Non-seed** label. The seed node hosts the etcd-only witness. The non-seed node is the preferred application host and is the only node you can replace.

Above the table, an **Add node** button opens the **Add Node** dialog. The button is available to users with update permissions and requires the cluster to be online. It does not appear on imported Kubernetes clusters.

### Applications tab

The **Applications** tab lists all application instances currently running on the cluster.

| Column | Description |
| - | - |
| Status | Current operational state of the application instance. |
| Name | Name of the application instance. Select to open the instance's detail page. |
| Edge App | Name of the edge app the instance is based on. |
| Version | User-defined version label. |
| Project | The project the application instance belongs to. |
| Created | When the application instance was created. |

### Virtual Interfaces tab

The **Virtual Interfaces** tab lists all network instances deployed on the cluster.

| Column | Description |
| - | - |
| Virtual Interface | Name of the network instance. Select to open its detail page. |
| Status | Network instance state: **Online**, **Provisioning**, **Error**, or **Offline**. |
| Kind | Network type: local or switch. |
| Uplink | The uplink interface used by the network instance. |
| Subnet | The subnet or bridge IP address. |
| Applications | Number of application interfaces (VIFs) attached to this network instance. |
| Updated | How recently the network instance was last updated. |

### Storage tab

The **Storage** tab lists all cluster volume instances. This tab does not appear while a cluster running in ZEDEDA Edge Kubernetes Service mode is online, because the **Kubernetes Dashboard** tab carries its own storage view for that cluster. If that cluster goes offline, the console withdraws the **Kubernetes Dashboard** tab and the **Storage** tab returns.

| Column | Description |
| - | - |
| State | Volume lifecycle state: **Creating**, **Attached**, **Detached**, **Attaching**, **Detaching**, or **Deleting**. |
| Name | Name of the volume instance. Select to open its detail page. |
| Size | Configured maximum size. |
| Actual Size | Current consumed size. Volume instances are thin-provisioned: they grow as data is written but do not shrink when data is deleted. |
| Created | When the volume instance was created. |
| Attachment State | Storage binding state: **Bound**, **Pending**, **Available**, **Released**, or **Failed**. |
| Sync Status | Number of online replicas out of total replicas, for example **2/3**. |

### Kubernetes dashboard tab

A cluster running in ZEDEDA Edge Kubernetes Service mode adds a **Kubernetes Dashboard** tab. The tab appears only when the cluster reports Kubernetes mode, is online, and has been assigned a Kubernetes cluster ID. If any of those conditions is not met, the console omits the tab rather than showing it as unavailable.

The clusters list offers a shortcut to this tab. Hover over the row of a cluster that meets the same conditions. If your enterprise has the Kubernetes Orchestration entitlement, a **Dashboard** icon appears on the right side of the row. Selecting the icon opens that cluster's detail page directly on the **Kubernetes Dashboard** tab. The icon is hidden for clusters that do not meet the conditions, so it never links to a cluster where the tab is unavailable.

The tab opens a second-level set of tabs for the live Kubernetes resources of the cluster. The first tab, **Overview**, summarizes those resources. The remaining tabs group them by kind: **Cluster**, **Workloads**, **Service Discovery**, **Storage**, **Policy**, and **Configuration**. Because this set includes its own **Storage** group, the page-level **Storage** tab is hidden on these clusters.

On an imported Kubernetes cluster, the **Applications** and **Virtual Interfaces** tabs are also hidden. An imported cluster is managed outside ZEDEDA, so ZEDEDA application instances and network instances are never deployed onto it and both tabs would always be empty.

#### Switch between the Edge and Kubernetes views

The cluster detail page header carries an **Edge** / **Kubernetes** switch. **Edge** shows the cluster as ZEDEDA manages it: member nodes, deployed applications, virtual interfaces, and volumes. **Kubernetes** opens the **Kubernetes Dashboard** tab. On a cluster that does not run in ZEDEDA Edge Kubernetes Service mode yet, the Kubernetes side of the switch is an **Enable Kubernetes Access** action instead of a view.

The console hides the switch entirely while the cluster is not online. Every Kubernetes panel reads live from the cluster's own API, and Kubernetes cannot be installed onto a cluster that is down, so neither side of the switch is actionable in that state. A link that points directly at one of the Kubernetes tabs returns to the **Overview** tab while the cluster is offline. If an enable succeeds but leaves the cluster Unprovisioned, the switch stays hidden until the cluster comes online.

#### Enable Kubernetes access on an existing cluster

Selecting **Enable Kubernetes Access** opens a confirmation dialog that explains what the action does and states that enabling is permanent. Confirm it only if you intend to keep Kubernetes access on this cluster.

> [!WARNING]
>
> Enabling Kubernetes access cannot be undone. There is no disable action in the console or in the API. To return to a cluster without Kubernetes access, you must delete the cluster and create a new one.

Enabling is non-destructive. The cluster's storage and virtualization stay in place and running application instances keep running while Kubernetes access is installed. This differs from the previous behavior, where joining a cluster to ZEDEDA Edge Kubernetes Service tore those components down. The cluster must be online, and its nodes must run an EVE-OS build that supports Kubernetes access; on older builds the action stays unavailable.

#### Track a Kubernetes install

Selecting **Enable Kubernetes Access** starts an install that continues on the cluster after the console accepts the request. While the install runs, the cluster detail header shows an **Enabling Kubernetes** indicator with a spinner. The indicator sits in the header rather than beside the switch, so it stays visible on every tab.

The Kubernetes side of the switch is disabled for the duration of the install and explains why in a tooltip: "Kubernetes is being installed on this cluster. The view opens once the install finishes." An install can take several minutes. The page polls cluster status, so the indicator clears itself and the **Kubernetes Dashboard** tab becomes available when the install finishes. You do not need to reload the page.

## Update a Cluster

You can update the cluster's title, description, and tags after creation. You cannot change the cluster name.

The **Configure** (gear) action is not currently available on the **Infrastructure > Fleet > Clusters** list or the cluster detail page. Use the ZCLI `edge-node-cluster update` command instead:

```bash theme={"dark"}
zcli edge-node-cluster update <name> [--title=<title>] [--description=<description>] [--tags=<key:value> ...]
```

The following example updates the title of a cluster and adds a tag:

```bash theme={"dark"}
zcli edge-node-cluster update MY_CLUSTER --title="Store 42 Cluster" --tags=env:production
```

## Replace a Node

If a node fails or you otherwise need to swap it out, you can replace it with another node from the same project.

In a two-node high availability (HA) configuration, you can replace a tie-breaker node only with another tie-breaker node. You cannot replace a tie-breaker node with a regular node, or a regular node with a tie-breaker node.

On a two-node HA cluster with a witness, you can replace only the non-seed node. You can replace it even while the cluster reports an unhealthy HA state, which is the usual repair when the non-seed node fails. The **Replace** button is disabled on the seed node, with the reason `The seed node cannot be replaced. Use Reset quorum to remove it and promote the other node.` After a quorum reset leaves one node, **Replace** is disabled with the reason `This is the only configured node. Add a node to restore redundancy.` See [Reset the quorum](#reset-the-quorum).

1. Open **Infrastructure > Fleet > Clusters** and select the target cluster.
2. Select the **Nodes** tab.
3. Select **Replace** in the **Actions** column of the node you want to replace.
4. In the **Replace Node** dialog, select the replacement node from the **Replacement Node** list. Only nodes in the same project that are not already cluster members are available.
5. Select the **Cluster Interface** for the replacement node from the interface list.
6. Select **Replace Node**.

After replacing the node, verify the cluster status on the **Overview** tab.

You cannot remove a node without replacing it.

## Add a Node

1. Open **Infrastructure > Fleet > Clusters** and select the target cluster.
2. Select the **Nodes** tab, then select **Add node**.
3. In the **Add Node** dialog, select a **Node** and its **Cluster Interface** to stage the node. Repeat to stage more nodes.
4. Select **Add Node**.

The dialog shows the resulting node count, for example `1 current + 1 staged = 2 nodes.` A cluster must end up with 1 node, or 3 or more nodes. A total of two nodes is valid only on a two-node HA cluster, or when you convert a single-node cluster to two-node HA.

Adding nodes is available on any cluster that has ZEDEDA edge nodes, with or without Kubernetes access. It is not available on imported clusters. The console and the `/cluster` API only add nodes; neither removes a node from a cluster. To take a node out of service, replace it as described in [Replace a Node](#replace-a-node), or delete the cluster.

### Convert a single-node cluster to two-node HA

When the cluster has one node and you stage exactly one node, the **Add Node** dialog shows an **Enable two-node HA** switch. Turn it on to keep the cluster at two nodes, with an etcd-only witness as the third quorum vote instead of a third node. After you select **Add Node**, the console shows `Node added and two-node HA enabled`.

> [!WARNING]
>
> You cannot turn two-node HA off after you enable it. The only way to leave the scheme is to delete the cluster.

## Manage Two-Node HA

A cluster that uses two-node HA with a witness shows a **Two-node HA** panel at the top of the **Overview** tab. The cluster detail page refreshes the status of these clusters every five seconds so that you can follow recovery as it happens.

The panel shows the following fields:

| Field | Description |
| - | - |
| HA state | Overall health of the two-node HA cluster, as reported by the controller, with a description of what the state means. See [Interpret Two-Node HA State](https://zededa.mintlify.site/new-ui/infrastructure-services/node-clusters/node-cluster-status-and-health.md#interpret-two-node-ha-state). |
| Witness | State of the etcd-only witness: **Pending**, **Idle**, **Joining**, **Joined**, or **Error**. |
| Witness IP | The address the witness is configured to join. The witness shows **Pending** until the node reports that it joined this exact address. |
| Witness error | The error the witness reported, if any. |
| Recovery generation | A counter that increases on every quorum reset. A value of `0` means the cluster has never been reset. Each node reports the generation it applied, which is how you track recovery progress. |
| Application placement | `Two-node HA uses Preferred affinity to allow application failover.` |

After a quorum reset, a **Node recovery** section appears below the fields. It lists each node that reports recovery progress, with its **Applied generation**, **Last transition** time, and any **Error**. The section stays hidden on a cluster that has never been reset, because there is no recovery to report. Each row shows one of the following states:

| State | Meaning |
| - | - |
| Recovery applied | The node applied the current recovery generation. |
| Applying | The node reports an older generation than the current one. |
| Converging | The node is still converging after a reset. |
| Recovery error | The node reported a recovery error. |
| Awaiting report | The node has not reported the generation it applied. A missing report does not mean recovery succeeded. |

HA state and node recovery describe the cluster quorum only. They do not confirm that application, storage, and network follow-up finished after a reset.

When a quorum reset leaves the cluster with one configured node, the panel shows the banner `This cluster has one physical node and no physical redundancy. Add a node to restore redundancy.` Select **Add node** in the banner to open the **Add Node** dialog and stage exactly one replacement node. The cluster keeps its **Two-node HA** badge while it runs on one node.

### Reset the quorum

Reset the quorum when the seed node fails permanently and you need the other node to take over. A reset removes the seed node from the cluster and promotes the surviving node to seed. The survivor force-resets etcd, takes ownership of applications and storage, and runs the cluster without redundancy until you add a node.

1. Open **Infrastructure > Fleet > Clusters** and select the two-node HA cluster.
2. In the **Two-node HA** panel on the **Overview** tab, select **Reset quorum**. The button is available to users with update permissions.
3. Read the confirmation, then select **Reset quorum** to confirm.

The confirmation depends on how many configured nodes the cluster has:

* **Two configured nodes**: the dialog asks `Reset quorum on <cluster>?` and states that the seed node is permanently removed and the other node is promoted to seed. This operation cannot be undone. The dialog also recommends that you power off the removed seed node so that it does not keep operating with stale cluster state. Before you reuse that node, follow the clean join procedure.
* **One configured node**: a reset was already saved. The dialog asks `Re-run recovery on <cluster>?` and the confirm button reads **Re-run recovery**. Continuing re-runs recovery follow-up work on the surviving node. No additional node is removed.

After you confirm, the console shows `Quorum reset requested. Review the cluster's HA state as recovery progresses.` If the request fails, the console shows the controller's error message and refreshes the cluster. Review the refreshed state before you start a new reset.

After the reset completes, add a replacement node to restore redundancy. See [Add a Node](#add-a-node).

## Upgrade EVE-OS on a Cluster

Upgrading EVE-OS on a cluster applies the new version to all member nodes. All nodes reboot during the upgrade, which briefly interrupts running workloads. Cluster nodes require the EVE-k flavor of EVE-OS.

The **Upgrade EVE-OS** action is not currently available on the cluster detail page or in Release Manager. Use the ZCLI `edge-node-cluster upgrade` command instead:

```bash theme={"dark"}
zcli edge-node-cluster upgrade <name> --image=<eve-os-image>
```

To check the progress of the upgrade on each member node, run:

```bash theme={"dark"}
zcli edge-node-cluster show <name> --upgrade
```

The upgrade status on the cluster detail page updates automatically as the rollout progresses.

## Delete a Cluster

When you delete a cluster, every node in the cluster reboots and reconfigures itself to operate as a standalone node. The member nodes are not deleted.

Application instances, volume instances, and virtual networks that were deployed to the cluster itself are deleted along with the cluster. The confirmation dialog lists them before you confirm. Workloads deployed directly to an individual member node are not affected.

> [!WARNING]
>
> There is no way to reclaim the data of a deleted cluster-deployed application instance or volume instance after the delete operation is performed.

1. Open **Infrastructure > Fleet > Clusters** and select the target cluster.
2. Select the **trash** icon in the upper-right area of the detail page.
3. Review the list of resources affected by the deletion. Member nodes are listed as affected because they become standalone nodes. Cluster-deployed application instances, volume instances, and virtual networks are listed because they are deleted.
4. Confirm the deletion.

Deletion is asynchronous. The cluster shows a deleting state while Edge Infrastructure Services removes its dependents, tears down Kubernetes access if it was enabled, and reconfigures the member nodes. The cluster disappears from the list when that work finishes, which can take several minutes. Imported clusters are deleted through the same action. If a Helm deployment still targets a cluster with Kubernetes access, the delete fails with a conflict; remove the deployment first.

## Edge View Kubernetes Commands

[Edge View](https://zededa.mintlify.site/new-ui/infrastructure-services/edge-view-and-remote-access.md) supports read-only `kubectl` commands for troubleshooting clusters running the EVE-k image.

Available operations include `get` and `list` commands. Management, modification, and deletion operations are not available through Edge View.

This feature is available in the standalone Edge View client only. Web-based Edge View sessions do not support Kubernetes commands.

## Next Steps

> [card] [Cluster Status and Health](https://zededa.mintlify.site/new-ui/infrastructure-services/node-clusters/node-cluster-status-and-health.md)
>
> Interpret cluster status, admin state, and member node readiness.

> [card] [App Instance Failover in Clusters](https://zededa.mintlify.site/new-ui/infrastructure-services/node-clusters/node-cluster-failover.md)
>
> Understand how applications behave during node failures and maintenance.

> [card] [Clusters Overview](https://zededa.mintlify.site/new-ui/infrastructure-services/node-clusters.md)
>
> Background on cluster types and automatic application migration.

> [card] [Manage ZEDEDA Edge Kubernetes Service Clusters With ZCLI](https://zededa.mintlify.site/zcli-reference/kubernetes-clusters.md)
>
> Create, upgrade, and inspect clusters from the command line.