Skip to main content

​
Introduction

The Evaluation page in ZEDEDA Edge Inference Services lets you measure how AI models perform on specific edge hardware before committing to a full deployment. You run benchmarks to collect metrics such as inference latency, throughput, and resource utilization across one or more device types. Evaluation was previously called Benchmarks. The Evaluation page has two tabs:
  • Benchmarks: create and monitor benchmark runs.
  • Device Pools: define the target hardware types available for benchmarking.

​
Prerequisites

  • To create, cancel, or delete benchmarks, you have the Organization Admin, Data Scientist, MLOps Engineer, MLOps Admin, or Super Admin role. To add, edit, or delete device pools, you have the Super Admin role. See Organizations, Users, and Roles.
  • At least one model is available in your model library.
  • At least one device pool is configured with clusters that have the target hardware.

​
Navigate to the Evaluation Page

  1. In the bottom navigation bar, select Inference.
  2. In the sidebar, select Inference > Deploy > Evaluation.
The page opens on the Benchmarks tab by default. Select the Device Pools tab to manage device pool definitions.

​
Manage Device Pools

A device pool defines a hardware type that benchmarks run on. Each pool specifies the device architecture, GPU availability, and the ZEDEDA Edge Kubernetes Service clusters that provide that hardware. Device pools are global: they are shared by all organizations, and only a Super Admin can add, edit, or delete them.

​
Add a device pool

  1. On the Evaluation page, select the Device Pools tab.
  2. Select the contextual + create action to open the Add New Device dialog.
  3. Complete the following fields:
    • Device Type ID: a stable identifier for this hardware type, for example jetson_agx_orin. Benchmarks use this ID to target specific hardware. The Device Type ID cannot be changed after the device pool is created.
    • Description: an optional description of the hardware.
    • Architecture: the CPU architecture. Select AMD64 for x86 hardware or ARM64 for ARM-based hardware.
    • Helm Chart: the name of the Helm chart used to deploy the benchmark workload. The default value is edgeai-benchmark.
    • Cluster Names: a comma-separated list of ZEDEDA Edge Kubernetes Service cluster names that provide this hardware. Each cluster must exist in Edge Infrastructure Services and can belong to only one device pool. Each benchmark run uses one available cluster from the pool; if every cluster is busy, the run waits in the Queued status.
    • GPU Available: select this checkbox if the device has a GPU accelerator.
    • GPU Type: the GPU type identifier, for example cuda or tegra. This field appears when GPU Available is selected.
  4. Select Add Device to save the device pool.

​
View device pools

The Device Pools tab lists all device pools with the following columns: Device Type, Architecture, GPU, Clusters, Status, and Description. The Status column shows:
  • Available: at least one cluster in the device pool is free to run a benchmark.
  • Unavailable: every cluster in the device pool is busy with an active benchmark. New benchmarks that target this pool wait in the Queued status.

​
Edit a device pool

  1. On the Device Pools tab, select the row actions menu (three-dot icon) for the device pool you want to update and select Edit.
  2. Modify the description, architecture, Helm chart, cluster names, or GPU settings as needed. Note that the Device Type ID cannot be changed.
  3. Select Update Device to save the changes.

​
Delete a device pool

You cannot delete a device pool while an active benchmark uses it.
  1. On the Device Pools tab, select Delete from the row actions menu of the device pool you want to remove.
  2. Confirm the deletion in the dialog.

​
Run Benchmarks

​
Create a benchmark

  1. On the Evaluation page, select the Benchmarks tab.
  2. Select the contextual + create action to open the Create Benchmark wizard.

​
Model Selection

Enter a Name and optional Description for the benchmark run. Then add one or more models to benchmark:
  1. Select a model from the Model dropdown.
  2. Select one or more versions of that model.
  3. Select Add to add the model and versions to the benchmark.
  4. Repeat for additional models.

​
Target Devices

Select the device pools that the benchmark should run on. The benchmark runs each selected model on each selected device type.

​
Benchmark Type

Select the benchmark type. The Inference Speed benchmark measures how quickly the model processes requests under load.

​
Parameters

Configure the benchmark run parameters:

​
Summary

Review the benchmark configuration and select Start Benchmarking to launch the run.

​
View benchmarks

The Benchmarks tab lists all benchmark runs in your organization with the following columns: Name, Model (Version), Type, Target Devices, Status, Progress, and Created. The Status column reflects the current state of the benchmark run:

​
View benchmark results

  1. On the Benchmarks tab, select a benchmark run name to open the detail page. The detail page shows the benchmark configuration and, for completed runs, the collected performance metrics organized by model version and device type:
    • Latency: P50, P95, P99, and average response times in milliseconds.
    • Throughput: frames per second (FPS) or tokens per second (TPS) depending on the model type.
    • Resource utilization: CPU usage, memory usage, GPU usage, and power consumption.
    • Reliability: total requests, failed requests, and error rate.
  2. Use these metrics to compare model versions across device types and select the configuration that meets your latency and throughput requirements before deployment.

​
In This Section