Model Prefetch

Warming is declared, not performed: a tenant names an artifact and a target set, and the operator lands the weights through the node cache’s ordinary delivery. These resources divide the responsibilities:

The node’s effective configuration is still NodeModelStore: the store layer merges above the cluster defaults under Model Store Operations , and the plugin side of every field is in the Node Model Store .

Contents

• Cache resources
• Placement
• The warm-up pod
• Budgets and admission
• Pinning and expiry
• Status and views

Cache resources

A ModelStore selects its pool with spec.nodeSelector and states watermarks and download limits as overrides field by field; the pool layer defines the override semantics, the overlap condition and the tie-break.

A ModelStoreBinding is the provisioning point: creating one in a namespace is what grants that namespace a quota.bytes budget and, with allowPinned, the right to pin. The grant is explicit (there is no default pool a binding falls back to) and frozen: storeRefs cannot move, the budget can only grow, and allowPinned only turns on.

Why — a grant that shrank under consumption would retroactively make an admitted prefetch over budget, and a re-pointed grant would strand the accounting of everything admitted before the edit. Both rules are admission-time refusals, so the namespace sees the refusal at the moment of the edit, not as drift later.

A ModelPrefetch names an artifactRef and a bindingRef in its own namespace. Deleting it revokes the intent: the warm-up pods go with their owner references, the pins leave on the next union pass, and the bytes are reclaimed by the node cache’s own collection once the content is unreferenced and past grace.

Placement

spec.placement picks the target set one of three ways, and admission and delivery share one expansion, so the two can never disagree about what a prefetch would warm:

Setting both kinds is refused; the schema cannot see it because neither field is required.

The warm-up pod

Delivery is one warm-up pod per target node, rendered by the prefetch controller. It mounts the artifact’s own CSI volume (the same volume a consumer mounts, with the same artifact, UID and digest attributes) under a restricted pod security context, reads every file back, and exits. The node cache downloads, verifies and publishes exactly once per node; the pod is the delivery operation, and there is no separate job state machine.

A warm-up pod never carries a kueue.x-k8s.io/queue-name label: a labeled pod is held by Kueue’s admission and topology gates and never scheduled, and this operator’s own pod webhook ignores unlabeled pods.

The image is a Setting : the pod needs only a shell, find and sha256sum, and the one thing the default cannot guarantee is that a given cluster’s registry has it.

Budgets and admission

A grant’s budget counts filesystem usage: each distinct digest at its full size, for every node holding it, whatever the entry’s state. A tree two namespaces share counts fully against each of them; that is conservative, and impossible to game by warming what another tenant already warmed. The measured footprint lands on the binding’s status:

• status.usedBytes: the measured figure, absent while any referenced artifact has not resolved (an absent figure was not measured; zero is a measurement).
• OverQuota: True when the footprint passed the grant, Unknown when it could not be counted.

Admission refuses the shapes that cannot work: a missing artifact or grant, a target set outside every store the grant names, pinning without the grant’s permission, and a projection past the budget.

The projection is the accounting at admission time: per distinct digest, the resolved size times the widest node count asked of it. An artifact that has not resolved yet contributes no size, and the accounting reports the drift that projection could not see.

Pinning and expiry

spec.retention.pinned keeps the content on every target node against collection: a pinned digest is never a collection candidate, though it still counts toward usage, so a cache full of pinned content reports its saturation exactly as a cache full of references does.

Pinning requires the grant. The nodes’ pin lists are written as one union by the prefetch controller, the field’s single writer, so two namespaces pinning one node keep both digests. See Node Model Store for how a pinned digest behaves under collection.

spec.retention.ttlAfterLastUse unpins a node’s copy once nothing mounted it for that long. It is enforced at the hour granularity the node already reports in lastUsedTime.

Why — retention does not need finer precision, and finer recording would multiply the node’s status writes for a decision that cannot tell the difference.

Status and views

The prefetch’s status aggregates the target nodes’ own reports: desiredNodes, readyNodes, downloadingNodes, and Progressing / Available / Degraded / Lapsed, where Available means readyNodes reached minReady (0 = all of them).

Lapsed turns True when a shortfall below that bar is the retention TTL’s doing: the copies that would close the gap lapsed past ttlAfterLastUse, were unpinned and are not re-warmed, and the message says how many. It stays False while the shortfall has any other cause, so a lapse never masks a warm-up still running or a delivery that failed.

The ModelPrefetch API supports resource operations and adds no subresource: the per-node facts already live on the nodes’ reports, and the table prints who warms what and how far.