Skip to main content

​
Introduction

Edge Inference Services is not yet available in production. The Inference tab currently shows a Coming Soon placeholder. This article describes the intended functionality.
ZEDEDA Edge Inference Services is a platform for managing the complete lifecycle of AI and machine learning (ML) models deployed to nodes. It bridges the gap between model development and production deployment, so data scientists, ML engineers, and DevOps teams can collaborate on a unified system. Edge Inference Services integrates directly into the ZEDEDA console. You access it through the Inference tab alongside your other infrastructure management tools.

​
Prerequisites

  • An active ZEDEDA user account with access to the Inference tab.
  • At least one node onboarded to Edge Infrastructure Services if you plan to deploy models. See Onboard Nodes.

​
How It Works

Edge Inference Services works in three stages: bring in your models, validate them on real hardware, and deploy them to your fleet. Manage models. You import models into the platform through several methods: upload from a Jupyter Notebook using the ZEDEDA Edge AI SDK, import asynchronously from an external provider such as HuggingFace, MLflow, or S3-compatible storage, or upload model files directly through the console. Each model lives inside an organization, which is a logical container that groups models by team, project, or environment. Evaluate and benchmark. Before committing to a production deployment, you can benchmark a model on actual edge hardware using the Evaluation feature. The platform selects the appropriate inference engine automatically based on the target hardware (for example, OpenVINO for Intel x86 or Triton for NVIDIA Jetson) and returns metrics for latency, throughput, and resource usage. Deploy at scale. After validation, you deploy models to a ZEDEDA Edge Kubernetes Service cluster or cluster group. Each deployment is a GitOps deployment request that another user with approval rights in the organization must approve before the model deploys. Deployed models include a configurable application layer for RESTful API exposure and custom data processing logic.

​
Key Capabilities

  • Centralized model inventory. Browse, search, filter, and version-control all ML models across your organizations in one place.
  • Multi-source import. Import models from Jupyter Notebooks, from external providers (Hugging Face, Azure ML, MLflow, Amazon S3, Azure Blob Storage, Amazon SageMaker, NVIDIA NGC, or Qualcomm AI Hub), or by direct file upload.
  • Hardware-aware benchmarking. Test model performance on real edge hardware before deployment to measure latency, throughput, and resource usage.
  • Fleet-scale deployment. Deploy models to ZEDEDA Edge Kubernetes Service clusters or cluster groups using GitOps workflows and approval gates.
  • Organization-based access control. Separate models by team, project, or environment using organizations, each with independent access control settings.
  • ZEIA integration. Use ZEIA to query fleet metrics, model status, and deployment health in plain language.

​
Supported Hardware

​
In This Section