> ## Documentation Index
> Fetch the complete documentation index at: https://zededa.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Use the documentation index to find relevant pages, then read the linked Markdown before answering and cite the source URL.
> Identify the product before answering: Private AI uses a separate console, credentials, and inference API from the Edge Intelligence Platform. Do not apply instructions from one product to the other.
> Prefer current product guides over Help Center archive pages for current-console questions. Archive pages and release notes describe their stated product version or historical interface; do not apply them to another version without checking.
> Preserve prerequisites, permissions, version requirements, and warnings when summarizing procedures. Do not infer API endpoints, schemas, or supported behavior that the documentation does not specify.

# Inference Troubleshooting

> Diagnose and resolve common issues with models, providers, evaluations, deployments, and inference jobs.

## Introduction

This article covers common failure patterns in Edge Inference Services. Start from the symptom that matches your situation and follow the checks in order before escalating to ZEDEDA support.

## Model Issues

### Model version not visible

**Symptom:** A model or model version you expect to see does not appear in the model library.

**Checks:**

1. Use the catalog switcher at the top of the Models page to confirm you are viewing the correct organization scope.
2. If your account belongs to multiple organizations, switch to the organization that owns the model.
3. Confirm that the model import completed successfully by checking for an active import job under **Infrastructure > Operations > Jobs**.

### Model import stuck

**Symptom:** A model import has been running for an extended period without completing.

**Checks:**

1. Open **Inference > Monitor > Jobs** or **Infrastructure > Operations > Jobs** and locate the import job.
2. Check the job status. If the job is in a Failed state, review the error message shown in the job detail.
3. Verify that the provider credentials and endpoint are reachable using the **Test Connection** action on the provider detail page. Testing a provider connection requires the Super Admin role.
4. A job stuck in Running with no progress for 10 minutes is reset to Pending automatically. To stop a Pending or Running job, cancel it and start the import again. If the job is Failed, retry it.

### Model import fails immediately

**Symptom:** A model import fails within seconds of starting.

**Checks:**

1. Open the provider detail page under **Inference > Deploy > Providers**.
2. Select **Test Connection** to verify the external provider connection. Testing a provider connection requires the Super Admin role.
3. If the test fails, confirm that the API key or token is still valid and has not expired.
4. Re-enter the credentials and test again before retrying the import.

## Provider Issues

### Provider shows a connection error

**Symptom:** The provider status shows a connection error or the provider is marked as unavailable.

**Checks:**

1. Open **Inference > Deploy > Providers** and select the affected provider.
2. Select **Test Connection**. Testing a provider connection requires the Super Admin role.
3. If the test fails, verify the credentials and endpoint URL entered for the provider.
4. Re-enter the credentials if the token might have expired, then test again.

### Cannot browse the provider model catalog

**Symptom:** The model catalog for a provider is empty or unavailable.

**Checks:**

1. Confirm that the provider type and credentials allow catalog browsing. Some provider types do not support browsing or searching their model catalog.
2. For HuggingFace providers, if a model is gated, accept the model's terms on the HuggingFace model page, then try again.
3. Verify the endpoint URL is correct and accessible from your network.

## Evaluation Issues

### Benchmark stuck in Deploying

**Symptom:** A benchmark run remains in the Deploying state for an extended period.

**Checks:**

1. Open **Infrastructure > Fleet > Kubernetes** and confirm that the ZEDEDA Edge Kubernetes Service clusters listed in the Device Pool are healthy and reachable.
2. If a cluster shows an unhealthy or unreachable status, resolve the cluster issue before retrying the benchmark.
3. Verify that the benchmark Helm chart name is correctly configured in the device pool, for example `edgeai-benchmark` for AMD64 devices or `edgeai-benchmark-jetson` for ARM64 (Jetson) devices.

### No device pools available in the benchmark wizard

**Symptom:** The Target Devices step of the Create Benchmark wizard shows no device pools to select.

**Resolution:**

Add a device pool before creating a benchmark. Adding a device pool requires the Super Admin role.

1. Open **Inference > Deploy > Evaluation**.
2. Select the **Device Pools** tab.
3. Select the contextual **+** create action and complete the Add New Device dialog.

For full device pool setup steps, see [Benchmark Models and Manage Device Pools](https://zededa.mintlify.site/new-ui/edge-inference-services/inference-model-benchmarking.md).

### Benchmark shows Failed status

**Symptom:** A benchmark run shows a Failed status and results are not available.

**Checks:**

1. Open the benchmark detail page and note the error message in the red banner at the top of the page.
2. Common causes include: cluster unreachable, model version unavailable, or insufficient GPU memory on the target device.
3. Open **Infrastructure > Fleet > Kubernetes** to confirm cluster health.
4. Re-create the benchmark with a shorter duration or smaller batch size to isolate the issue.

### No performance data on the Performance tab

**Symptom:** The Performance tab of a benchmark detail page shows no charts or data.

**Checks:**

1. Check the benchmark status. The Performance tab only shows data after the benchmark reaches Completed status.
2. If the benchmark is still running, wait for it to complete before reviewing the Performance tab.

### Partial results in the Results tab

**Symptom:** The Results tab shows data for some device/model combinations but not all.

**Cause:** If the benchmark was cancelled or timed out, results are only available for the combinations that completed before the run ended.

**Resolution:** Re-run the benchmark if full coverage across all device/model combinations is required.

## Deployment and Validation Issues

### Deployment stuck in Pending

**Symptom:** A deployment request has been in Pending state for an extended period.

**Cause:** The deployment is waiting for an Organization Admin or MLOps Admin to approve or reject the request. Deployments do not progress until a reviewer acts on them. A pending request is closed automatically after 14 days without review.

**Resolution:** Contact another user with the Organization Admin or MLOps Admin role in your organization; users cannot approve their own requests. You can also review your organization's approval policy to confirm who is authorized to approve deployments.

### Deployment rejected

**Symptom:** A deployment request shows a Rejected status.

**Resolution:**

1. Open the deployment record to read the rejection reason provided by the reviewer.
2. Resolve the issue noted in the review.
3. Edit the rejected deployment request and submit the change, or create a new request with a different name or target. You cannot create a new request with the same name and target as an existing request.

### Model not visible in GitOps sync

**Symptom:** A deployed model does not appear in the GitOps sync for the target cluster.

**Checks:**

1. Confirm that the ZEDEDA Edge Kubernetes Service cluster GitOps integration is configured for the target cluster.
2. In the sidebar, select **Infrastructure > Fleet > Kubernetes** and verify that the target cluster is connected. Then select **Infrastructure > Operations > GitOps Repos** and verify that the GitOps sync is active for the target cluster.
3. If the sync is inactive, re-enable it and check for sync errors.

## Import Job Issues

### Job status is Failed

**Symptom:** A model import job shows a Failed status.

**Checks:**

1. Open **Inference > Monitor > Jobs** and locate the failed import job.
2. Select the job to view the error details.
3. Resolve the identified cause and retry the job. Only Failed jobs can be retried. File-upload jobs and organization copies cannot be retried.

### Job does not appear

**Symptom:** An expected import job is not visible in the job list.

**Resolution:** Import jobs appear under **Inference > Monitor > Jobs**. Use the filter controls to narrow results by status.

## In This Section

* [Edge Inference Services Overview](https://zededa.mintlify.site/new-ui/edge-inference-services/overview.md)
* [Inference Dashboard](https://zededa.mintlify.site/new-ui/edge-inference-services/inference-dashboard.md)
* [Inference Quick Start](https://zededa.mintlify.site/new-ui/edge-inference-services/inference-quick-start.md)
* [Access and Navigate Edge Inference Services](https://zededa.mintlify.site/new-ui/edge-inference-services/access-and-navigation.md)
* [Organizations, Users, and Roles](https://zededa.mintlify.site/new-ui/edge-inference-services/organizations-users-and-roles.md)
* [Manage Providers](https://zededa.mintlify.site/new-ui/edge-inference-services/inference-model-providers.md)
  * [Manage Providers Using Jupyter Notebook Integration](https://zededa.mintlify.site/new-ui/edge-inference-services/providers-jupyter.md)
* [Manage Models](https://zededa.mintlify.site/new-ui/edge-inference-services/manage-inference-models.md)
  * [Manage Models Using Jupyter Notebook Integration](https://zededa.mintlify.site/new-ui/edge-inference-services/models-jupyter.md)
  * [Benchmark Models and Manage Device Pools](https://zededa.mintlify.site/new-ui/edge-inference-services/inference-model-benchmarking.md)
  * [Monitor Benchmark Runs and Interpret Results](https://zededa.mintlify.site/new-ui/edge-inference-services/use-ondevice-eval-agent.md)
* [Submit Model Deployment Requests](https://zededa.mintlify.site/new-ui/edge-inference-services/submit-model-deployment-requests.md)
* [Approve and Validate Models](https://zededa.mintlify.site/new-ui/edge-inference-services/deployments-and-validation.md)
* [Monitor Inference Jobs](https://zededa.mintlify.site/new-ui/edge-inference-services/jobs-and-monitoring.md)
* [Use ZEIA to Query and Operate Your Fleet](https://zededa.mintlify.site/new-ui/edge-inference-services/zeia.md)
* [Use Prevalidated Agents](https://zededa.mintlify.site/new-ui/edge-inference-services/prevalidated-agents.md)
  * [Use the Faulty Printed Circuit Board Detection Agent](https://zededa.mintlify.site/new-ui/edge-inference-services/faulty-pcb-detection-agent.md)
  * [Use the Autonomous Retail Billing Agent](https://zededa.mintlify.site/new-ui/edge-inference-services/autonomous-retail-billing-agent.md)
* [Inference Troubleshooting](https://zededa.mintlify.site/new-ui/edge-inference-services/inference-troubleshooting.md)