# GPUStack Operator

> GPUStack Operator discovers hardware, shapes accelerator capacity, and helps Kubernetes place workloads where that capacity is available.

Documentation version: v0.9.2. Source revision: 25c6524fc31e93cd924f83ce4da57c5ae0243103.
These pages come from the repository's docs/ sources. For version-specific behavior, use the
documentation and code for your release. This index routes questions; follow its links for evidence.

## Getting Started

- [Architecture](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/getting-started/architecture/index.md): See the operator’s four-stage path.
- [Vendor Prerequisites](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/getting-started/vendor-prerequisites/index.md): Prepare each manufacturer’s driver.

## Walkthroughs

- [Walkthroughs](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/walkthroughs/index.md): Choose a complete example by module.
- [Heterogeneous Devices Walkthrough](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/walkthroughs/devices/scheduling/index.md): Follow a recorded accelerator scheduling run.
- [RDMA Networking Walkthrough](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/walkthroughs/rdma/network-endpoints/index.md): Request and verify RDMA endpoints and NUMA alignment.
- [Hierarchical Placement Walkthrough](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/walkthroughs/topology/hierarchical-placement/index.md): Place serving replicas across zones and physical racks.
- [Shared Cache Walkthrough](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/walkthroughs/kv-cache/shared-cache/index.md): Create a shared cache from backend to workload.
- [Model Prefetch Walkthrough](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/walkthroughs/model-delivery/prefetch/index.md): Cache weights and verify captured model CR output.
- [Model Deployment Walkthroughs](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/walkthroughs/model-deployment/index.md): Choose a serving configuration and its hardware, model and `InstanceType` setup.
- [Prefill and Decode Walkthrough](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/walkthroughs/model-deployment/prefill-decode/index.md): Deploy H100 prefill and decode roles with tested TCP transfer settings.
- [Encode, Prefill and Decode Walkthrough](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/walkthroughs/model-deployment/encode-prefill-decode/index.md): Deploy encode, prefill and decode roles and verify a multimodal request.
- [Elastic EP Walkthrough](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/walkthroughs/model-deployment/elastic-ep/index.md): Expand an H100 collective from four to eight GPUs and verify effective width and inference.
- [External DP Walkthrough](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/walkthroughs/model-deployment/external-dp/index.md): Deploy four fixed H100 ranks and verify captured endpoint and Router coverage.
- [Multi-Host Serving Walkthrough](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/walkthroughs/model-deployment/multi-host/index.md): Scale models across multiple GPU nodes.

## Heterogeneous Devices

- [Heterogeneous Devices](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/devices/index.md): Choose an accelerator request.
- [Preflight Checks](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/devices/preflight/index.md): Check a node before installation.
- [Device Discovery](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/devices/discovery/index.md): Trace hardware detection and allocation.
- [Accelerator Requests](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/devices/requests/index.md): Look up resource keys and valid requests.
- [Scheduling Chain](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/devices/scheduling/index.md): Follow capacity from labels to queues.
- [Admission](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/devices/admission/index.md): Understand the five request checks.
- [NVIDIA MIG Operations](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/devices/nvidia-mig/index.md): Enable and recover NVIDIA partitions.
- [Hygon MIG Operations](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/devices/hygon-mig/index.md): Enable and recover Hygon partitions.
- [T-Head MIG Operations](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/devices/thead-mig/index.md): Enable and recover T-Head partitions.

## RDMA Networking

- [RDMA Networking](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/rdma/index.md): Start with RDMA requests.
- [Network Topology](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/rdma/network-topology/index.md): See how links and devices are discovered.
- [RDMA Operations](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/rdma/operations/index.md): Request and check RDMA endpoints.

## Topology

- [Topology](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/topology/index.md): Start with placement by domain.
- [Topology-Aware Scheduling](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/topology/scheduling/index.md): Follow topology data into scheduling.
- [Topology-Aware Scheduling Operations](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/topology/operations/index.md): Enable and diagnose topology placement.

## KV Cache

- [KV Cache](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/kv-cache/index.md): Start with a shared inference cache.
- [KV Cache Backend](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/kv-cache/backend/index.md): Understand the store and its capacity.
- [KV Cache Pools and Bindings](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/kv-cache/pool/index.md): Grant and limit cache use.
- [KV Cache Injection Reference](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/kv-cache/injection/index.md): Attach a Pod to a pool.
- [KV Cache Election and Failover](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/kv-cache/leader/index.md): Understand leader election and health.
- [KV Cache Local Disk Tier](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/kv-cache/local-disk-tier/index.md): Configure local disk storage.
- [KV Cache on Disk-Heavy Nodes](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/kv-cache/disk-heavy-nodes/index.md): Size memory on disk-heavy nodes.

## Model Delivery

- [Model Delivery](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-delivery/index.md): Choose how weights reach workloads.
- [Model Artifact](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-delivery/artifact/index.md): Resolve and verify model weights.
- [Model Artifact API](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-delivery/views/index.md): Read artifact and node cache status.
- [Model Image Source](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-delivery/image-source/index.md): Package weights in an image.
- [Model Prefetch](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-delivery/prefetch/index.md): Warm weights before a workload starts.
- [Node Model Store](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-delivery/node-store/index.md): Understand node cache state and collection.
- [Node-to-Node Sync](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-delivery/peer-sync/index.md): Move cached weights between nodes.
- [Model Store Operations](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-delivery/operations/index.md): Operate the node model cache.

## Model Deployment

- [Model Deployment](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/index.md): Start with managed model serving.
- [Model Deployment Configuration](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/deployment/index.md): Configure serving roles and overrides.
- [Engine Compatibility](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/engine-versions/index.md): Check engine releases, client and store requirements, and transport support.
- [Prefill and Decode](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/prefill-decode/index.md): Pair serving roles.
- [Prefill and Decode Benchmark](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/prefill-decode-benchmark/index.md): Compare equal-GPU H100 aggregated and disaggregated serving across fixed shapes and mixed traffic.
- [Encode, Prefill and Decode](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/encode-prefill-decode/index.md): Disaggregate visual encoding, prefill and decode roles.
- [Encode, Prefill and Decode Benchmark](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/encode-prefill-decode-benchmark/index.md): Compare EPD, prefill/decode and aggregated on measured latency and throughput.
- [Elastic EP](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/elastic-ep/index.md): Resize one serving instance's collective in place.
- [Elastic EP Benchmark](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/elastic-ep-benchmark/index.md): Compare four- and eight-GPU H100 collectives and measure requests during live expansion.
- [External DP](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/external-dp/index.md): Run one fixed data-parallel group with an HTTP endpoint on every rank.
- [External DP Benchmark](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/external-dp-benchmark/index.md): Compare External DP ranks with a monolithic wide tensor-parallel group.
- [Model Deployment Routing](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/routing/index.md): Choose a routing policy, confirm a Router serves.
- [Model Deployment Metrics](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/metrics/index.md): Read serving metrics.
- [Model Deployment Status](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/status/index.md): Diagnose deployment conditions, follow a retirement.
- [Model Deployment Shutdown](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/model-deployment/shutdown/index.md): Understand replica retirement and draining.

## GPU Instances

- [GPU Instances](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/modules/instances/index.md): Start an accelerator-backed workspace.

## Operations

- [Installation Modes](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/operate/installation-modes/index.md): Choose chart or image installation.
- [High Availability Operations](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/operate/high-availability/index.md): Set replica counts for control-plane parts.
- [Migrating from v0.5.x](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/operate/migration/from-v0.5/index.md): Upgrade across the queue refactor.
- [Migrating to Bundled Subcharts](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/operate/migration/to-subcharts/index.md): Transfer chart ownership.
- [Upgrading to an Enforced Binding Dtype](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/operate/migration/kv-cache-dtype/index.md): Check dtype before upgrading.
- [Migration Troubleshooting](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/operate/migration/troubleshooting/index.md): Recover a stuck upgrade.

## Reference

- [Command Reference](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/reference/commands/index.md): Look up binary commands and flags.
- [Settings & Environment Variables](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/reference/settings/index.md): Look up operator configuration.
- [Instance Type Unit Resources Reference](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/reference/instance-type-unit-resources/index.md): Look up CPU and memory presets.
- [Instance Metrics Reference](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/reference/instance-metrics/index.md): Read an Instance’s resource use.

## Contribution


## Optional

- [Repository navigation](https://github.com/gpustack/gpustack-operator/blob/25c6524fc31e93cd924f83ce4da57c5ae0243103/docs/README.md): Module relationships, design records, code entry points and applicable skills for contributors.
- [Development](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/contribute/development/index.md): Build, generate and lint the project.
- [Internals](https://docs.gpustack.ai/gpustack-operator/v0.9.2/docs/contribute/internals/index.md): Review startup and naming constraints.