Manage Kubernetes Cluster Instances

Use the ZEDEDA CLI (ZCLI) to manage cluster instances from the command line. This article covers zcli cluster-instance commands, with syntax and examples for each operation. A cluster instance is a Kubernetes cluster that runs inside edge applications. You choose an edge app that packages the Kubernetes distribution and a pool of edge nodes, and Edge Infrastructure Services deploys the edge app on those nodes as Kubernetes server and agent nodes. ZCLI supports two cluster types:

k3s, a K3s cluster.
tkg-attach, a cluster that is attached to VMware Tanzu Kubernetes Grid (TKG). The edge app is a virtual machine edge app with the TKG Attach deployment type.

The cluster can be onboarded to an external Kubernetes manager. With manual onboarding, you give a call-home URL. With automatic onboarding, Edge Infrastructure Services onboards the cluster through an orchestrator integration plugin, for example for Rancher. Cluster instances are separate from ZEDEDA Edge Kubernetes Service clusters, which run on node clusters of edge nodes running EVE-k, and which you manage with zcli edge-kubernetes-cluster. See Manage ZEDEDA Edge Kubernetes Service Clusters and Manage Node Clusters. The zcli edge-kubernetes-cluster and zcli edge-kubernetes-cluster-group commands do not act on cluster instances.

Before You Start

ZCLI is installed and authenticated against your enterprise. See Get Started.
You have a role that allows you to deploy edge apps in the project. [TBD: confirm the minimum role required to create and delete cluster instances with ZCLI.]
The project, the edge app, and the edge nodes exist. See Manage Edge Apps and Manage Nodes.
For automatic onboarding, an orchestrator integration plugin exists in your enterprise. To list plugins, run zcli plugin show.

Create a Cluster Instance

zcli cluster-instance create <name> (--k3s-edge-app=<edge-app-name> | --edge-app=<edge-app-name> --cluster-type=<type>) --project=<project-name> --cluster-config=<cluster-config> (--edge-node-pool-tag=<key:value>... | --edge-node=<edge-node-name>) --network-assignment=<interfaceName:assignmentType> --onboard-type=<type> [--orchestrator-plugin=<plugin>] [--network-config=<config-file-path>] [--network-tag=<key:value>...] [--title=<title>] [--description=<description>] [--tags=<key:value>...]

Flag	Description
`<name>`	Name of the cluster instance.
`--edge-app`	Name of the edge app that runs the cluster. Use with `--cluster-type`.
`--cluster-type`	Required with `--edge-app`. Use `k3s` or `tkg-attach`, in lowercase.
`--k3s-edge-app`	Deprecated. Same as `--edge-app=<edge-app-name> --cluster-type=k3s`.
`--project`	Required. Name of the project.
`--cluster-config`	Required. Path to the cluster configuration file. See Cluster configuration file.
`--edge-node-pool-tag`	Selects every edge node that has this tag, in `key:value` format. Repeat the flag for more tags. Use this or `--edge-node`.
`--edge-node`	Selects one edge node by name. Use this or `--edge-node-pool-tag`.
`--network-assignment`	Required. How the edge app interface connects to a virtual network, in `<interface>:<assignment>` format. See Network assignment.
`--network-tag`	Required when the assignment is `tag`. Selects the virtual network on each edge node that has this tag, in `key:value` format. Repeat the flag for more tags.
`--onboard-type`	Required. Use `manual` or `automatic`, in lowercase. See Onboarding.
`--orchestrator-plugin`	Required when `--onboard-type=automatic`. Name of the orchestrator integration plugin.
`--network-config`	Path to a JSON file of virtual networks that Edge Infrastructure Services creates on the selected edge nodes. See Virtual network configuration file.
`--title`	Title of the cluster instance. If you leave it out, ZCLI uses the name.
`--description`	Description of the cluster instance.
`--tags`	A tag for the cluster instance, in `key:value` format. Repeat the flag for more tags.

ZCLI creates the cluster instance in the active state. create prints nothing when it succeeds. To check the result, see View Cluster Instances. Example, a K3s cluster on every edge node tagged site:factory-1, with manual onboarding:

zcli cluster-instance create MY_CLUSTER_INSTANCE --edge-app=MY_K3S_APP --cluster-type=k3s --project=MY_PROJECT --cluster-config=cluster-inst.conf --edge-node-pool-tag=site:factory-1 --network-assignment=eth0:default --onboard-type=manual

Example, a TKG Attach cluster on one edge node, with automatic onboarding and the interface connected to a tagged virtual network:

zcli cluster-instance create MY_TKG_INSTANCE --edge-app=MY_TKG_APP --cluster-type=tkg-attach --project=MY_PROJECT --cluster-config=cluster-inst.conf --edge-node=MY_NODE --network-assignment=eth0:tag --network-tag=net:cluster --onboard-type=automatic --orchestrator-plugin=MY_PLUGIN

The examples in zcli cluster-instance --help leave out --onboard-type. That flag is required, so those examples fail with a usage error.

ZCLI checks very few values before it sends the request:

ZCLI does not check the project, edge app, edge node, or orchestrator plugin names. If a name does not exist, ZCLI sends an empty ID for it.
A --cluster-type or --onboard-type value other than the listed lowercase values is sent as an unspecified type, with no error from ZCLI.
If an --edge-node-pool-tag value is not in key:value format, ZCLI prints Please provide tag values as key-value pairs. Example: region:India but still sends the request, with no edge nodes selected. A bad --tags value is dropped in the same way.

Check each name and value before you run create.

Cluster configuration file

The cluster configuration file sets the size of the cluster. Each setting is on its own line, in key=value format. The following example is the sample file that ships with ZCLI:

cluster-conf = {
  minEdgeDevices=1
  maxEdgeDevices=1
  minServerNodes=1
  maxServerNodes=1
  minAgentNodes=1
  maxAgentNodes=1
  callHomeURL=http://sampleCallHome.url
}

Setting	Description
`minEdgeDevices`, `maxEdgeDevices`	Minimum and maximum number of edge nodes in the cluster.
`minServerNodes`, `maxServerNodes`	Minimum and maximum number of Kubernetes server nodes.
`minAgentNodes`, `maxAgentNodes`	Minimum and maximum number of Kubernetes agent nodes.
`callHomeURL`	The URL of the cluster manifest that the cluster calls home to for manual onboarding. Required unless you use automatic onboarding with an orchestrator plugin. With automatic onboarding and a plugin that ZCLI finds, ZCLI ignores it.

All six size settings are required. If one is missing, ZCLI prints Fields missing in cluster config. Refer to example supplicant file at /zcli/config_templates/cluster-inst.conf and does not create the cluster instance. In the ZCLI container, the template is actually at /zcli/zcli/config-templates/cluster-inst.conf, and the sample is at /zcli/zcli/resources/cluster-inst.conf.sample. To copy the sample to your working directory, run:

cp /zcli/zcli/resources/cluster-inst.conf.sample cluster-inst.conf

ZCLI reads each value up to the next = sign. If your callHomeURL contains an =, for example in a query string, ZCLI sends only the part before it. If the file does not exist, ZCLI stops with a Python error instead of a message.

Network assignment

--network-assignment connects the network interface of the edge app to a virtual network on each edge node. The interface name is the one defined in the edge app. ZCLI applies the same interface setting to both the server and the agent nodes.

Assignment	Description
`default`	Connect the interface to the default virtual network of each edge node, for example `--network-assignment=eth0:default`.
`tag`	Connect the interface to the virtual network that has the `--network-tag` tags, for example `--network-assignment=eth0:tag --network-tag=net:cluster`.

If the value is not in <interface>:<assignment> format, or the assignment is tag and you do not pass --network-tag, ZCLI prints an error and creates nothing. These error messages refer to the flags as --network_assignment and --network_tags. The flags that you type are --network-assignment and --network-tag.

Onboarding

Onboard type	Description
`manual`	Edge Infrastructure Services uses the `callHomeURL` from the cluster configuration file.
`automatic`	Edge Infrastructure Services onboards the cluster through the orchestrator integration plugin that you name with `--orchestrator-plugin`. If you leave out `--orchestrator-plugin`, ZCLI prints `Enter Orchestrator Integration Plugin Name` and creates nothing.

Virtual network configuration file

With --network-config, Edge Infrastructure Services creates virtual networks on the edge nodes that the cluster instance selects. The file is a JSON array with one object for each virtual network.

Key	Description
`name_prefix`	Required. Prefix for the virtual network name.
`title_prefix`	Required. Prefix for the virtual network title.
`kind`	Required. `local`, `switch`, `cloud`, `transparent`, or `honeypot`.
`type`	Required. `v4`, `v6`, `crypto-v4`, or `crypto-v6`.
`port`	Required. `uplink`, `logical port name`, or `port tag name`.
`logical_port_name`	Required when `port` is `logical port name`. The port to use.
`port_match_tags`	Required when `port` is `port tag name`. A list of `{"key": "<key>", "value": "<value>"}` objects that select the port.
`set_as_default`	Required when `port` is `port tag name`. `true` to make this the default virtual network of the edge node.
`subnet`, `gateway`, `domain`, `dns`, `ntp`	IP settings. ZCLI sends these only when you also set `dhcp_range`.
`dhcp_range`	`{"start": "<ip>", "end": "<ip>"}`. When you set it, also set `subnet`.
`static_dns`	A list of `{"host_name": "<host>", "ip_set": ["<ip>"]}` objects.
`tags`	A list of `{"key": "<key>", "value": "<value>"}` objects that tag the virtual network.

Example:

[
  {
    "name_prefix": "cluster-net",
    "title_prefix": "Cluster network",
    "kind": "local",
    "type": "v4",
    "port": "uplink",
    "subnet": "10.10.0.0/24",
    "gateway": "10.10.0.1",
    "dhcp_range": {"start": "10.10.0.10", "end": "10.10.0.100"},
    "tags": [{"key": "net", "value": "cluster"}]
  }
]

The template that ships with ZCLI (/zcli/zcli/config-templates/netinst-config.tmpl) uses some key names that ZCLI does not read:

ZCLI reads set_as_default, not set_default. With port set to port tag name and only set_default in the file, ZCLI prints 'set_as_default' missing in netinst config and creates nothing.
ZCLI reads dns and ntp, not name_server and ntp_server. Values under the template names are ignored.

If the file is not valid JSON, or if you set gateway or dhcp_range without subnet, ZCLI stops with a Python error instead of a message.

View Cluster Instances

zcli cluster-instance show [<name> | --uuid=<uuid>] [--name-pattern=<name-pattern>] [--detail] [--summary] [--page-size=<page-size>] [--project=<project-name>]

View a single cluster instance

Pass the name or --uuid. Add --detail to see the edge node pool, the cluster size settings, the call-home URL, the version, and who created and last updated the cluster instance.

zcli cluster-instance show MY_CLUSTER_INSTANCE --detail

The output shows the cluster type, title, description, tags, the edge app (as Cluster Name), the admin state, and the run state. With --detail, it also lists each edge node in the pool with its run state.

List cluster instances

Run show without a name to list cluster instances.

Flag	Description
`--name-pattern`	Show only cluster instances whose names contain this text.
`--project`	Show only cluster instances in this project.
`--summary`	Show a count of cluster instances by state instead of the list.
`--page-size`	Number of records that ZCLI requests in each API call. ZCLI still retrieves the full list.

Example:

zcli cluster-instance show --project=MY_PROJECT --name-pattern=factory

The list has the columns Name, Admin Status, Type, Edge Node Pool Tags, and Project. The Edge Node Pool Tags column shows the number of edge nodes in the pool, not the tags. When you pass a name or --uuid, ZCLI ignores --name-pattern, --project, and --page-size. When you pass --summary, ZCLI ignores every other option.

Update a Cluster Instance

You can change only the title, the description, and the tags.

zcli cluster-instance update <name> [--title=<title>] [--description=<description>] [--tags=<key:value>...]

Example:

zcli cluster-instance update MY_CLUSTER_INSTANCE --description="Line 2 cluster" --tags=site:factory-1 --tags=line:2

--tags replaces all of the existing tags. To keep a tag, pass it again.
If a --tags value is not in key:value format, ZCLI prints Please provide tag values as key-value pairs. Example: region:India, keeps the existing tags, and still applies the other changes.
If you pass no options, ZCLI prints Nothing to update.

Activate or Deactivate a Cluster Instance

To activate a cluster instance:

zcli cluster-instance activate <name>

To deactivate a cluster instance:

zcli cluster-instance deactivate <name>

deactivate stops the cluster instance, and activate starts it again. If the cluster instance is already in the requested state, nothing changes. Both commands print nothing when they succeed.

Restart or Refresh a Cluster Instance

To restart a cluster instance:

zcli cluster-instance restart <name>

To refresh a cluster instance:

zcli cluster-instance refresh <name> [--purge]

restart restarts the cluster instance. refresh refreshes it. Add --purge to purge the cluster instance during the refresh.

refresh --purge deletes the data of the cluster instance. There is no confirmation prompt for restart or refresh, with or without --purge, and the commands print nothing when they succeed.

These two commands are not described in the Commands section of zcli cluster-instance --help, but they appear in its usage lines.

Delete a Cluster Instance

zcli cluster-instance delete <name> [-f]

ZCLI asks Delete cluster-instance <name>? [Y/N] before it deletes the cluster instance. If you answer no, ZCLI prints Cluster-Instance <name> not deleted. Add -f to delete without the prompt. Example:

zcli cluster-instance delete MY_CLUSTER_INSTANCE -f

Next Steps

Manage Edge Apps

Create the edge app that runs the cluster.

Manage Virtual Networks

Create and tag the virtual networks that cluster instances connect to.

Manage ZEDEDA Edge Kubernetes Service Clusters

Manage Kubernetes clusters that run on node clusters.

Get Started

Install ZCLI, configure access to Edge Infrastructure Services, and sign in.

Get started

Infrastructure

Applications

Monitoring and Usage

Administration and Marketplace

Manage Kubernetes Cluster Instances

Before You Start

Create a Cluster Instance

Cluster configuration file

Network assignment

Onboarding

Virtual network configuration file

View Cluster Instances

View a single cluster instance

List cluster instances

Update a Cluster Instance

Activate or Deactivate a Cluster Instance

Restart or Refresh a Cluster Instance

Delete a Cluster Instance

Next Steps

Manage Edge Apps

Manage Virtual Networks

Manage ZEDEDA Edge Kubernetes Service Clusters

Get Started

​Before You Start

​Create a Cluster Instance

​Cluster configuration file

​Network assignment

​Onboarding

​Virtual network configuration file

​View Cluster Instances

​View a single cluster instance

​List cluster instances

​Update a Cluster Instance

​Activate or Deactivate a Cluster Instance

​Restart or Refresh a Cluster Instance

​Delete a Cluster Instance

​Next Steps

Manage Edge Apps

Manage Virtual Networks

Manage ZEDEDA Edge Kubernetes Service Clusters

Get Started

Before You Start

Create a Cluster Instance

Cluster configuration file

Network assignment

Onboarding

Virtual network configuration file

View Cluster Instances

View a single cluster instance

List cluster instances

Update a Cluster Instance

Activate or Deactivate a Cluster Instance

Restart or Refresh a Cluster Instance

Delete a Cluster Instance

Next Steps