GPUStack Operator

Walkthroughs

Follow a complete example, then compare the resource output with the observed result. Each module has its own directory. Configuration contracts remain in the capability guides.

Contents

Choose a walkthrough

Serving configurations use tensor parallelism (TP), data parallelism (DP) and expert parallelism (EP). The Model Deployment guides explain their configuration contracts.

Module Walkthrough Task
Heterogeneous Devices Heterogeneous Devices Follow detection, queues, admission, and accelerator allocation.
RDMA Networking RDMA Networking Request and verify shared and exclusive RDMA endpoints with NUMA alignment.
Topology Hierarchical Placement Publish zone and rack hierarchies and constrain replicas to a topology domain.
KV Cache Shared Cache Create a backend, pool, namespace grant, and serving workload.
Model Delivery Model Prefetch Cache and verify model weights before inference.
Model Deployment Preparation and choices Choose a serving configuration and follow its hardware, model and AcceleratorType requirements.
Model Deployment Prefill and Decode Disaggregate prefill and decode roles fronted by a router.
Model Deployment Encode, Prefill and Decode Separate multimodal encode, prefill and decode roles.
Model Deployment Elastic EP Grow DP width 2 → 4 with TP = 2.
Model Deployment External DP Deploy four fixed H100 DP ranks and verify captured per-rank inference and Router coverage.
Model Deployment Multi-Host Deploy distributed model instances across multiple GPU hosts.
Model Deployment Scaling Observe and activate scaling for a deployment.

Connect configurations

These walkthroughs connect resources from more than one module. Follow each page’s prerequisites and limitations; its example does not establish that other combinations have been tested.

Configuration Walkthrough Linkage
Model delivery and serving Model Prefetch A store binding grants cache access; a deployment references the artifact.
Shared cache and serving Shared Cache A backend supplies a pool; a namespace binding grants a deployment access.
RDMA and shared cache RDMA Networking The cache backend uses an RDMA fabric and requests interfaces.
Delivery, serving and scaling Scaling A deployment references an artifact; a scaler controls its replica counts.
Topology and serving Hierarchical Placement Topology inventory supplies domains that constrain replica placement.

Read the evidence

Commands sit beside captured output or the checks needed to interpret it. A CR’s declared configuration does not prove runtime parallelism or successful inference. Recorded model-serving walkthroughs pair CR snapshots with completed requests and runtime observations. Multi-Host provides a configuration example and checks without captured runtime results. Confirm each claim against the walkthrough’s stated evidence and the live CR status and Pod output.