GPU Instances
An Instance is a container workspace backed by a Kubernetes Pod. It can use a whole, sliced
or partitioned accelerator. With an SSH public key, the operator adds a sidecar that lets
the user enter the workload container.
Contents
Overview
Configuration
Instance configuration
This minimal example requests one accelerator and exposes a service port. For the
InstanceSSHPublicKey manifest and SSH login commands, see Usage
.
apiVersion: worker.gpustack.ai/v1
kind: Instance
metadata:
name: workspace-demo
namespace: team-a
spec:
type: gpustack--nvidia-a10g-linux-amd64 # Select an available InstanceType.
image: ubuntu:24.04
command:
- sleep
- infinity
resources:
accelerator: "1" # one accelerator; see Accelerator Requests for slice percentages
ports:
- name: http
port: 8080
volume:
ephemeral:
capacity: 1Gi- Type: Select an
InstanceTypewith available capacity. - Resources: Request one accelerator family. See Accelerator Requests .
- Ports: Optional; each entry allocates a node port on the instance
Service.
Device access
The workload container owns the accelerator request. The optional SSH sidecar uses a separate visibility resource to enter the same workspace without acquiring a second accelerator. See Device Discovery . Instance Metrics explains the reported utilization.
Service lifecycle
An Instance listing spec.ports is reached through a NodePort Service named after it, and
status.ports reports the node port allocated for each of them.
Stopping the Instance (spec.stop: true) deletes the Pod but retains the Service, so the node ports
a user already connected to stay allocated. Starting it again (spec.stop: false) builds a new Pod
and reports the same node ports. The Service belongs to the Instance rather than to the Pod, and
selects the Pod by the part-of label the two share, so it keeps routing across both the Pod
replacement and a change of the Instance’s type.
spec.ports may be edited while the Instance is stopped, and the changes take effect on the Service
when the Instance starts again: ports left unchanged keep their allocated node port, a port added to
spec.ports is allocated a new one, a port removed from spec.ports is dropped from the Service,
and clearing spec.ports deletes the Service along with its allocations. Deleting the Instance
itself also deletes the Service.
Network isolation
The operator isolates Instances of different tenants at the namespace boundary. The behavior is
opt-in: with the instance-isolation-pod-cidrs Setting blank — the default — no policy is
rendered and Instance networking stays untouched.
When the Setting names the cluster’s pod CIDRs, the operator renders one NetworkPolicy named
gpustack-instance-isolation in every namespace that holds an Instance, selecting the
instance Pods by their resource.gpustack.ai/type: instances label.
The policy allows traffic to and from Pods in the same namespace and addresses outside the
configured pod CIDRs. It permits egress to any destination on TCP or UDP port 53 for DNS.
Other policies selecting the same Pods can add permissions. Do not treat this policy alone as a
complete tenant isolation boundary.
The pod CIDRs cannot be learned reliably for every CNI — a node’s pod CIDR and what the CNI’s own
IPAM allocates routinely disagree — so the Setting is the one source. Find the values with
kubectl get ippools.crd.projectcalico.org on a Calico-IPAM cluster, or
kubectl get nodes -o jsonpath='{.items[*].spec.podCIDRs}' where the nodes carve the pod
addresses.
The chart carries the same value as worker.instanceIsolationPodCIDRs; the Setting can also
be edited at runtime through the Settings API.
The object belongs to the namespace rather than to one Instance: deleting the last Instance of
a namespace does not remove it, and the reconcile rewrites any manual edit back to the rendered
shape. Emptying the Setting collects the policy again — but only the object the operator rendered
itself, never a same-named one written by hand.
Enforcing the policy takes a CNI that implements NetworkPolicy, such as Calico or Cilium; with a
non-enforcing CNI the object renders but filters nothing. The Kubernetes policy contract always
allows traffic from the node hosting a Pod, so a client inside one Instance can still reach
another tenant’s Service through the hosting node’s address and node port.
Operations
Step 1: Create workspace
Replace the example values. Save the manifest as instance.yaml, then apply it:
kubectl apply -f instance.yamlStep 2: Check workspace
kubectl -n team-a wait --for=jsonpath='{.status.phase}'=Ready instance/workspace-demo --timeout=5m
kubectl get instancetypesVerification: The wait completes when the Instance phase is Ready. The InstanceType reports remaining capacity.
See Instance Metrics
for utilization.
Troubleshooting
If an Instance remains pending, inspect its phase message:
kubectl -n team-a get instance workspace-demo -o jsonpath='{.status.phaseMessage}'For Kueue Workload inspection, capacity checks and admission fixes, see the
walkthrough troubleshooting
.