Hierarchical Placement Walkthrough

Configure topology levels across cloud zones and physical racks, request domain placement for multi-Pod replicas, and see what a node move and a hierarchy change do.

Contents

• Summary
• Environment
• Step 1: Discover cloud zones
• Step 2: Request zone placement
• Step 3: Publish an inventory snapshot
• Step 4: Request rack placement
• Step 5: Move a node
• Limitations
• Troubleshooting

Summary

Topology-aware scheduling (TAS) places multi-Pod replicas inside one failure domain — a cloud zone or a physical rack. Configure a read-only cloud-zone TopologySource, then narrow placement to a static rack inventory snapshot. Moving a node between racks and changing the hierarchy levels have different effects on existing workloads.

The steps discover infrastructure hierarchies, apply placement constraints to serving workloads, and manage topology updates.

Environment

Step 1: Discover cloud zones

Verify existing node labels and cluster status before configuring topology sources:

kubectl get nodes --show-labels
kubectl get topologysources
kubectl get topologies.kueue.x-k8s.io

When nodes already carry cloud provider labels for regions and zones, define a read-only TopologySource using nodeLabels:

apiVersion: worker.gpustack.ai/v1
kind: TopologySource
metadata:
  name: cloud-zones
spec:
  nodeSelector:
    matchLabels:
      node-role.kubernetes.io/worker: ""
  levels:
    - topology.kubernetes.io/region
    - topology.kubernetes.io/zone
  nodeLabels: {}

The operator validates that labels form a strict tree (each zone belongs to exactly one region) and computes a profile hash. Nodes receive topology.gpustack.ai/profile=fnv64-<hash>, and Kueue materializes the matching Topology resource.

Check source readiness:

kubectl get topologysources cloud-zones -o yaml

Step 2: Request zone placement

When a serving replica consists of multiple Pods (such as tensor-parallel groups spanning multiple nodes), request placement within a single availability zone:

 apiVersion: worker.gpustack.ai/v1
 kind: ModelDeployment
 metadata:
   name: qwen-distributed
   namespace: team-a
 spec:
   model:
     name: Qwen/Qwen2.5-7B-Instruct
   engine:
     name: vLLM
     image: gpustack/runner:cuda13.0-vllm0.29.0
   roles:
     - name: server
       replicas: 1
       size: 2
       instanceType: gpustack--nvidia-a10g-linux-amd64
       resources:
         accelerator: "1"
+      topology:
+        requiredLevel: topology.kubernetes.io/zone

Kueue TAS admits the replica only when a single zone contains enough unallocated capacity to host all 2 Pods. If capacity is scattered across zones without filling any single zone, the workload waits in Pending rather than stranding split Pods.

Step 3: Publish an inventory snapshot

In private data centers without cloud labels, publish rack hierarchies using a versioned snapshot stored in a ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: topology-inventory
  namespace: gpustack-system
data:
  snapshot.yaml: |
    apiVersion: topology.gpustack.ai/v1alpha1
    revision: inventory-42
    nodes:
      worker-a:
        topology.gpustack.ai/region: region-a
        topology.gpustack.ai/zone: zone-a
        topology.gpustack.ai/rack: rack-01
      worker-b:
        topology.gpustack.ai/region: region-a
        topology.gpustack.ai/zone: zone-a
        topology.gpustack.ai/rack: rack-02
---
apiVersion: worker.gpustack.ai/v1
kind: TopologySource
metadata:
  name: data-center
spec:
  nodeSelector: {}
  levels:
    - topology.gpustack.ai/region
    - topology.gpustack.ai/zone
    - topology.gpustack.ai/rack
  configMap:
    configMapRef:
      namespace: gpustack-system
      name: topology-inventory
    key: snapshot.yaml
    maxStaleness: 10m

The operator validates the snapshot atomically, creates NFD NodeFeature resources, and writes the custom labels to each node.

Step 4: Request rack placement

With rack labels published, narrow the workload constraint from zone-level to rack-level:

 apiVersion: worker.gpustack.ai/v1
 kind: ModelDeployment
 metadata:
   name: qwen-distributed
   namespace: team-a
 spec:
   roles:
     - name: server
       topology:
-        requiredLevel: topology.kubernetes.io/zone
+        requiredLevel: topology.gpustack.ai/rack

All Pods within each replica group are placed within the same physical rack, minimizing inter-switch network latency.

Step 5: Move a node

When a node is cabled into a new rack, update the ConfigMap snapshot with an incremented revision:

 apiVersion: v1
 kind: ConfigMap
 metadata:
   name: topology-inventory
   namespace: gpustack-system
 data:
   snapshot.yaml: |
     apiVersion: topology.gpustack.ai/v1alpha1
-    revision: inventory-42
+    revision: inventory-43
     nodes:
       worker-a:
         topology.gpustack.ai/region: region-a
         topology.gpustack.ai/zone: zone-a
         topology.gpustack.ai/rack: rack-01
       worker-b:
         topology.gpustack.ai/region: region-a
         topology.gpustack.ai/zone: zone-a
-        topology.gpustack.ai/rack: rack-02
+        topology.gpustack.ai/rack: rack-03

The profile is a hash of the ordered level keys, and the keys did not change. The node keeps its profile, so the worker creates no flavor and applies no hold. The node carries the new rack value after NFD publishes it.

A change to the ordered level keys is a different operation that can evict and readmit workloads, unlike the revision-only change shown above. See Change a live hierarchy for the HoldAndDrain migration sequence and its interruption impact before editing levels.

Limitations

• Single-source node ownership and the hostname-only profile fallback are covered in Hierarchy profiles .
• Domain capacity packing and multi-role gang holds are covered in Troubleshooting .
• Changing the ordered levels list on a live hierarchy triggers HoldAndDrain migration; see Change a live hierarchy .

Troubleshooting

For general failure surfaces (including AwaitingQueueStatus, HoldAndDrain flavor migration, and domain capacity fragmentation), see Topology-Aware Scheduling Failure Surfaces and Change a live hierarchy .