> ## Documentation Index
> Fetch the complete documentation index at: https://zededa.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Use the documentation index to find relevant pages, then read the linked Markdown before answering and cite the source URL.
> Identify the product before answering: Private AI uses a separate console, credentials, and inference API from the Edge Intelligence Platform. Do not apply instructions from one product to the other.
> Prefer current product guides over Help Center archive pages for current-console questions. Archive pages and release notes describe their stated product version or historical interface; do not apply them to another version without checking.
> Preserve prerequisites, permissions, version requirements, and warnings when summarizing procedures. Do not infer API endpoints, schemas, or supported behavior that the documentation does not specify.

# Benchmark Models and Manage Device Pools

> Benchmark and evaluate AI model performance across target device types using the Evaluation page (formerly Benchmarks) in ZEDEDA Edge Inference Services.

## Introduction

The Evaluation page in ZEDEDA Edge Inference Services lets you measure how AI models perform on specific edge hardware before committing to a full deployment. You run benchmarks to collect metrics such as inference latency, throughput, and resource utilization across one or more device types. Evaluation was previously called Benchmarks.

The Evaluation page has two tabs:

* **Benchmarks**: create and monitor benchmark runs.
* **Device Pools**: define the target hardware types available for benchmarking.

### Prerequisites

* To create, cancel, or delete benchmarks, you have the Organization Admin, Data Scientist, MLOps Engineer, MLOps Admin, or Super Admin role. To add, edit, or delete device pools, you have the Super Admin role. See [Organizations, Users, and Roles](https://zededa.mintlify.site/new-ui/edge-inference-services/organizations-users-and-roles.md#roles-and-permissions).
* At least one model is available in your model library.
* At least one device pool is configured with clusters that have the target hardware.

## Navigate to the Evaluation Page

1. In the bottom navigation bar, select **Inference**.
2. In the sidebar, select **Inference > Deploy > Evaluation**.

The page opens on the **Benchmarks** tab by default. Select the **Device Pools** tab to manage device pool definitions.

## Manage Device Pools

A device pool defines a hardware type that benchmarks run on. Each pool specifies the device architecture, GPU availability, and the ZEDEDA Edge Kubernetes Service clusters that provide that hardware. Device pools are global: they are shared by all organizations, and only a Super Admin can add, edit, or delete them.

### Add a device pool

1. On the Evaluation page, select the **Device Pools** tab.

2. Select the contextual **+** create action to open the Add New Device dialog.

3. Complete the following fields:

   * **Device Type ID**: a stable identifier for this hardware type, for example `jetson_agx_orin`. Benchmarks use this ID to target specific hardware. The Device Type ID cannot be changed after the device pool is created.
   * **Description**: an optional description of the hardware.
   * **Architecture**: the CPU architecture. Select **AMD64** for x86 hardware or **ARM64** for ARM-based hardware.
   * **Helm Chart**: the name of the Helm chart used to deploy the benchmark workload. The default value is `edgeai-benchmark`.
   * **Cluster Names**: a comma-separated list of ZEDEDA Edge Kubernetes Service cluster names that provide this hardware. Each cluster must exist in Edge Infrastructure Services and can belong to only one device pool. Each benchmark run uses one available cluster from the pool; if every cluster is busy, the run waits in the Queued status.
   * **GPU Available**: select this checkbox if the device has a GPU accelerator.
   * **GPU Type**: the GPU type identifier, for example `cuda` or `tegra`. This field appears when **GPU Available** is selected.

4. Select **Add Device** to save the device pool.

### View device pools

The **Device Pools** tab lists all device pools with the following columns: Device Type, Architecture, GPU, Clusters, Status, and Description.

The **Status** column shows:

* **Available**: at least one cluster in the device pool is free to run a benchmark.
* **Unavailable**: every cluster in the device pool is busy with an active benchmark. New benchmarks that target this pool wait in the Queued status.

### Edit a device pool

1. On the **Device Pools** tab, select the row actions menu (three-dot icon) for the device pool you want to update and select **Edit**.
2. Modify the description, architecture, Helm chart, cluster names, or GPU settings as needed. Note that the Device Type ID cannot be changed.
3. Select **Update Device** to save the changes.

### Delete a device pool

You cannot delete a device pool while an active benchmark uses it.

1. On the **Device Pools** tab, select **Delete** from the row actions menu of the device pool you want to remove.
2. Confirm the deletion in the dialog.

## Run Benchmarks

### Create a benchmark

1. On the Evaluation page, select the **Benchmarks** tab.
2. Select the contextual **+** create action to open the Create Benchmark wizard.

#### Model Selection

Enter a **Name** and optional **Description** for the benchmark run. Then add one or more models to benchmark:

1. Select a model from the **Model** dropdown.
2. Select one or more versions of that model.
3. Select **Add** to add the model and versions to the benchmark.
4. Repeat for additional models.

#### Target Devices

Select the device pools that the benchmark should run on. The benchmark runs each selected model on each selected device type.

#### Benchmark Type

Select the benchmark type. The **Inference Speed** benchmark measures how quickly the model processes requests under load.

#### Parameters

Configure the benchmark run parameters:

| Parameter | Description |
| - | - |
| Duration | Total benchmark duration in seconds, from 30 to 3600. |
| Warmup Iterations | Number of initial requests discarded before measurement begins, from 0 to 100. Default: 10. |
| Batch Size | Number of requests processed simultaneously per inference call, from 1 to 32. Default: 1. |
| Concurrency | Number of parallel request streams, from 1 to 16. Default: 1. |

#### Summary

Review the benchmark configuration and select **Start Benchmarking** to launch the run.

### View benchmarks

The **Benchmarks** tab lists all benchmark runs in your organization with the following columns: Name, Model (Version), Type, Target Devices, Status, Progress, and Created.

The **Status** column reflects the current state of the benchmark run:

| Status | Description |
| - | - |
| Pending | The run is preparing. |
| Queued | The run is waiting for a free cluster in the device pool. |
| Scheduled | Deployment requests are created for the assigned clusters. |
| Deploying | The benchmark workload is being deployed to the target clusters. |
| Running | The benchmark is actively collecting metrics. |
| Completed | The run finished successfully. |
| Failed | The run encountered an error and did not complete. |
| Cancelled | The run was manually cancelled. |

### View benchmark results

1. On the **Benchmarks** tab, select a benchmark run name to open the detail page.

   The detail page shows the benchmark configuration and, for completed runs, the collected performance metrics organized by model version and device type:

   * **Latency**: P50, P95, P99, and average response times in milliseconds.
   * **Throughput**: frames per second (FPS) or tokens per second (TPS) depending on the model type.
   * **Resource utilization**: CPU usage, memory usage, GPU usage, and power consumption.
   * **Reliability**: total requests, failed requests, and error rate.

2. Use these metrics to compare model versions across device types and select the configuration that meets your latency and throughput requirements before deployment.

## In This Section

* [Edge Inference Services Overview](https://zededa.mintlify.site/new-ui/edge-inference-services/overview.md)
* [Inference Dashboard](https://zededa.mintlify.site/new-ui/edge-inference-services/inference-dashboard.md)
* [Inference Quick Start](https://zededa.mintlify.site/new-ui/edge-inference-services/inference-quick-start.md)
* [Access and Navigate Edge Inference Services](https://zededa.mintlify.site/new-ui/edge-inference-services/access-and-navigation.md)
* [Organizations, Users, and Roles](https://zededa.mintlify.site/new-ui/edge-inference-services/organizations-users-and-roles.md)
* [Manage Providers](https://zededa.mintlify.site/new-ui/edge-inference-services/inference-model-providers.md)
  * [Manage Providers Using Jupyter Notebook Integration](https://zededa.mintlify.site/new-ui/edge-inference-services/providers-jupyter.md)
* [Manage Models](https://zededa.mintlify.site/new-ui/edge-inference-services/manage-inference-models.md)
  * [Manage Models Using Jupyter Notebook Integration](https://zededa.mintlify.site/new-ui/edge-inference-services/models-jupyter.md)
  * [Benchmark Models and Manage Device Pools](https://zededa.mintlify.site/new-ui/edge-inference-services/inference-model-benchmarking.md)
  * [Monitor Benchmark Runs and Interpret Results](https://zededa.mintlify.site/new-ui/edge-inference-services/use-ondevice-eval-agent.md)
* [Submit Model Deployment Requests](https://zededa.mintlify.site/new-ui/edge-inference-services/submit-model-deployment-requests.md)
* [Approve and Validate Models](https://zededa.mintlify.site/new-ui/edge-inference-services/deployments-and-validation.md)
* [Monitor Inference Jobs](https://zededa.mintlify.site/new-ui/edge-inference-services/jobs-and-monitoring.md)
* [Use ZEIA to Query and Operate Your Fleet](https://zededa.mintlify.site/new-ui/edge-inference-services/zeia.md)
* [Use Prevalidated Agents](https://zededa.mintlify.site/new-ui/edge-inference-services/prevalidated-agents.md)
  * [Use the Faulty Printed Circuit Board Detection Agent](https://zededa.mintlify.site/new-ui/edge-inference-services/faulty-pcb-detection-agent.md)
  * [Use the Autonomous Retail Billing Agent](https://zededa.mintlify.site/new-ui/edge-inference-services/autonomous-retail-billing-agent.md)
* [Inference Troubleshooting](https://zededa.mintlify.site/new-ui/edge-inference-services/inference-troubleshooting.md)