GPUStack Operator

KV Cache

A shared KV cache lets compatible inference workloads reuse cached state. A backend runs the store. Pools and bindings grant capacity to namespaces. Cache quota is separate from accelerator quota.

Contents

Overview

KV cache backend and workload flow

A binding grants capacity only; read the isolation limitations before sharing a backend between tenants.

Configuration

Start with the annotated backend example and its constraints. Configure pools and bindings for quotas and reuse domains. KV Cache Injection defines the Pod labels, annotations and engine settings.

For an end-to-end configuration, follow the Shared Cache Walkthrough .

Operations

Step 1: Configure storage

Select memory and optional disk tiers. See KV Cache Backend , Local Disk Tier and Disk-Heavy Nodes .

Step 2: Check status

Inspect the backend and its conditions:

bash
kubectl get kvcb
kubectl describe kvcb <backend-name>

Verification: Check the phase and conditions against Backend Status . For leader health and metrics, see KV Cache Election and Failover .

Troubleshooting

If a workload cannot use the cache, inspect backend, pool and binding status.

Condition Reason Resolution
MembersMounted=False AmbiguousMemberIdentity Use node selectors that place the groups on different nodes.
Writes fail with RPC_FAIL (-900) Possible store and client minor-line mismatch Check version compatibility .

Use Injection Refusals for webhook messages and fixes. For quota and domain failures, see Pool Operating Notes . For backend failures, check Backend Status .