GPUStack Operator

KV Cache

A backend runs the cache store. A pool and binding decide which workloads may use it and how much capacity they receive.

Contents

Quick start

KV Cache Walkthrough creates the backend, pool, binding and deployment in order. A plain Pod can also use a binding through KV Cache Injection .

Cache configuration

Read KV Cache Backend for the store and KV Cache Pool for grants and quotas. The leader and local disk tier have separate operating details. Disk-Heavy Nodes covers cache placement on storage nodes.