Documentation version: main

main

Enable JavaScript to list other versions.

Theme

Search documentation

Documentation

Overview

Getting started

Architecture Prerequisites Walkthrough

Devices Manager

Overview Preflight Discovery Resource requests Scheduling Admission NVIDIA MIG Hygon MIG T-Head MIG

RDMA Manager

Overview Interface discovery Operations

Topology Aware

Overview Scheduling Operations

KV Cache

Overview Walkthrough Backends Pools and bindings Pod injection Leader Local disk tier Disk-heavy nodes

Model Delivery

Overview Artifacts Image sources Prefetch and quotas Node cache Peer sync Operations API reference

Model Deployment

Overview Configuration Engine versions Prefill and decode Routing Metrics Status Shutdown

GPU Instances

Overview

Operations

Installation High availability

Upgrades

From v0.5 Subchart migration Cache data types Troubleshooting

Reference

Commands Settings Resource presets Instance metrics

Docs/ Getting started

Getting started

View page source Edit this page