Model Deployment Status

A ModelDeployment reports its phase, ready replicas and serving endpoint in status. Use these fields and its conditions to diagnose a deployment that is still starting or has lost capacity.

Contents

• Status

Status

status.phase is the field to read first: Starting, Ready, Degraded or Deleting. Degraded means some required serving capacity is ready and some is not: either some role replicas are down, or all role replicas are ready but a declared router has no ready replica. status.roles[] carries name, kind, desired, ready, quotaReserved, unmanaged and assignedFlavors per role.

kubectl get modeldeployments shows status.roleSummary in its ROLES column. It counts current Ready instances by kind: R is managed router Pods, S is ordinary servers, P is prefillers, and D is decoders. For example, 1R2S means one Ready router and two Ready servers; 1R3P4D means one Ready router, three Ready prefillers and four Ready decoders.

Without a router, the R part is absent; a declared kind with nothing Ready shows zero. Several roles of the same kind are summed. A serving instance made of several Pods still counts once; use status.roles[].desired to see the requested count.

Without a router, status.endpoint is the address the deployment-wide Service serves on, in the form <scheme>://<name>.<namespace>.svc:<port>. The scheme is read from the same role the port is (the first one), and is https only where that role passes --ssl-certfile or --ssl-keyfile. With a router it is absent until the router is ready, then uses the router Service’s own http transport.

status.router publishes the contract used to render the router: its implementation and address, the cache pool’s client endpoint, the metrics names and port, and one entry for every role containing the effective kind, selector, direct endpoint and any dialable KV-event endpoints. The event entries are absent when the corresponding rendered Pods do not carry publisher configuration.

This router contract names metrics sources for routing; it is not a live utilization sample. The separate Model Deployment Metrics subresource reads the current router and engine Pod endpoints and reports partial coverage explicitly.

Only --ssl-certfile or --ssl-keyfile turn the scheme to https. The rest of the --ssl-* family does not: both engines enable TLS for those two alone, so any other ssl argument passed by itself leaves an ordinary HTTP server.

ready counts replicas whose engine answered, not replicas whose process started, except on the three shapes listed below, which carry no gates and keep the weaker meaning. One still loading its model counts as not ready for as long as that takes.

Both figures count replicas, which is the same as counting Pods only while a replica is one Pod. A role at size: 2 with replicas: 2 runs four Pods and reports desired: 2. A replica of several Pods is ready only when every member it declares is, so a role showing 1/2 with three of its four Pods running is reporting one whole instance, not three quarters of the role.

A gated replica carries a startup gate, a readiness gate and a liveness gate, all reading the engine’s own GET /health on the port the Service targets, so ready == desired and “the endpoint answers” are one fact rather than two.

The KV cache store’s leader answers a route of the same name that is deliberately not a readiness signal; see KV Cache Leader . They are different programs, and only the engine’s route is read here.

The startup gate allows thirty minutes per attempt, not in total. A replica that has not answered by then is restarted and gets the same budget again, so a model that never finishes loading is a restart loop rather than a replica stuck at not-ready.

The liveness gate never sees that window, because the kubelet suppresses liveness and readiness alike until the startup gate succeeds.

A replica that stops answering loses readiness first, and is restarted only if it stays quiet for longer. The liveness gate’s failure threshold is wider than the readiness gate’s.

Readiness withdraws a replica from the Service and is undone by answering again; a restart throws away a model that took the startup budget to load. Different thresholds keep those two events different, and the liveness gate is what gives a quiet replica a way back, since this operator deletes a replica only to shed one the spec no longer declares or to turn over an outdated one.

The operator tells the engine where to listen. It renders --host 0.0.0.0 and --port <the port the Service targets> into the engine’s own command line, so the address traffic is sent to and the address the engine opens are one decision. The two engines do not agree on a default (vLLM opens every interface on 8000, SGLang opens 127.0.0.1:30000), and an engine left on its own would be unreachable on one of them.

Both flags are filled when missing. A role that passes either through extraArgs keeps its own value, and the operator adds only the one that is absent.

A role that enables TLS is still gated, over HTTPS, on the criterion above. Only the transport moved (the address is still the operator’s own), and the kubelet does not verify the server certificate on a probe, so a self-signed pair is graded like any other.

Three shapes carry no gates at all: a role that replaces the command through roles[].command; a role that moves where its engine listens, with --host or --port, or that demands a client certificate a probe has none to present, with --ssl-cert-reqs set to anything but 0 or 1 alongside a certificate or key; and a role declaring its port as UDP or SCTP.

The --ssl-cert-reqs values: that flag is an integer, and only 0 (CERT_NONE, the default) and 1 (CERT_OPTIONAL) let a certificate-less probe through, so a role passing either keeps all three gates. Everything else withdraws them: 2, a value outside that enum, one that is not a number, and the flag with no value at all. Losing a gate costs a signal, while fitting one to a listener that refuses it restarts a replica that is serving.

The first two fail one way and the third the other. Gating an address the operator does not know would restart a replica answering every request; an HTTP engine speaks TCP whatever the declaration says, so a gate would pass while the published endpoint forwards a protocol nothing answers. Replicas of all three are Ready as soon as their process starts.

quotaReserved is how many of the role’s replicas hold a quota reservation. Each replica is its own Workload and reserves on its own, so a role sits anywhere between zero and desired while capacity arrives; a role that shared one Workload passed all-or-nothing, and this figure could not exist.

The field is always present once written, and its zero is an observed zero: it is counted from Pod and Workload lists that succeeded, and a failed list writes no status at all rather than a zero. The two readings call for opposite actions: a visible 0 means the pass read everything and no replica holds quota, so investigate capacity; a field that is not there means the pass could not see, so wait, or look at the operator rather than the pool.

assignedFlavors answers which accelerator models did this role’s replicas actually get, read from each replica’s own Workload’s per-PodSet assignment. It is a deduplicated, sorted list, because each replica is its own Workload and Kueue assigns a flavor per Workload, so two replicas of one role can land on different flavors.

It is absent rather than empty while no assignment exists, and absent does not only mean “not admitted yet”. The field reports the flavor of a replica’s accelerator credits: a role admitted onto a pool carrying no accelerator at all reports nothing here while being perfectly healthy: its Workloads hold assignments, naming a flavor for cpu.

One entry means every assigned replica of the role names it. Several entries mean the replicas were assigned different flavors, the signal to investigate and not a degraded form of one answer.

Which replica carries which flavor is deliberately not here: the ordinal such an answer would key on is the converger’s internal slotting rather than a promise this API makes, and a reader who needs the mapping reads the replicas’ own Pods. A flavor reported here is read through the same lens the per-accelerator admission gate uses, so the two cannot disagree.

Eight conditions carry the axes a single phase cannot. They are independent: “quota reserved but cache not attached” is a real and actionable state. Seven are described below; WeightsReady is described with the artifact it reports on, in Model Artifact .

DomainRegistered — whether the referenced Binding resolved and its domain was read.

QuotaReserved — whether every one of the deployment’s Workloads holds Kueue quota. Every replica is its own group with its own Workload, so a deployment of N replicas is N Workloads. True is therefore an answer about the whole set and not about whichever Workload sorts first, because half a deployment holding quota is not the deployment holding quota.

A message that names groups names them by role, never by instanceType: two roles can name one type, so a type would point at two roles’ groups at once while a role names exactly the set that waits. The queue an operator has to free follows from the named role’s own instanceType.

It mirrors the Workloads’ own conditions, so it stays current after admission rather than reporting only the moment a Workload was admitted.

PodGroupIncomplete, NoQueueInReservedNamespace and AdmissionInFlight all observe no Workload but mean different things. The first clears when the missing replica exists; the second is permanent because reserved namespaces deliberately have no LocalQueue; the third clears when Kueue composes the Workload. Telling them apart is why the reason exists.

Pending, PreemptedInPart and Parked all say the set does not hold quota, and they are separate because the action differs. Pending resolves itself when capacity appears. Parked is over already: those workloads were deactivated and no longer ask for anything. PreemptedInPart is the one to act on, and the replicas that kept their quota hold accelerators until either the reclaimed replicas are admitted again or the deployment is deleted.

How long the wait is worth making depends on what preempted the deployment, which is outside this object and this operator. The condition reports the state and leaves that decision with you.

The cost of the wait lands on the queue, not on this deployment. The replicas that kept their quota hold accelerators while the set cannot serve in full, and Kueue accounts that quota as used: the next workload drawing from the same ClusterQueue is refused with insufficient unused quota. A deployment preempted in part starves whatever is queued behind it.

What to do about it:

• Add capacity. The reclaimed replicas are admitted again on their own once capacity returns. The state self-heals but is UNBOUNDED: no timeout, no give-up, nothing makes the holder yield, so capacity that never comes back leaves it standing.
• Delete and recreate the deployment. Deleting is the other release for the quota the kept replicas hold.
• Move the roles apart. Each role draws from the ClusterQueue its own instanceType resolves to; giving the roles different types keeps one preemption from reclaiming both halves out of the same queue.

CacheAttached — whether the cache is observed to be in effect, which is a different question from whether it was configured. It is judged downstream of the engine and never on a rendered flag or a log line.

NoObservationAvailable is Unknown rather than False because an attached deployment that is simply idle looks exactly like an unread one: no supported engine signals “the connector initialized” before any traffic, so silence is the most common steady state. A connector that cannot come up keeps its replica from ever becoming Ready, which is reported there.

ReplicasUpToDate — whether the running replicas match what the convergence renders for them now. It is not a statement about spec alone: the comparison includes the rendered KV cache connector, so a replica carrying the current spec can still differ from the render once that connector stops resolving — the state behind RolloutHeldByCache, which the other conditions do not describe.

A pass answers only for the replicas it read or created from the render it just performed; one that could account for none reports Unknown, rather than a zero indistinguishable from “all up to date”. This happens in ordinary operation — a teardown, and the pass between a rollout’s delete and the create that answers it, both write status this way.

ReplacementInProgress and RolloutInProgress answer opposite questions — whether an edit landed, and why capacity moved when nothing was edited. While a replacement waits, watch the departed Pod: the replacement is created once no Pod for that ordinal reads on the API server.

The answer is not left standing between passes. After a steady deployment it would keep reporting True while every replica is on its way out, because a pass that finds every replica departing accounts for none of them.

RolloutHeldByCache is the condition to read when an edit appears to have no effect; it is the only condition on the object that names a held rollout.

The state does not imply an edit is waiting. During an outage the render carries no connector, so every attached replica differs from it whether or not anyone changed the deployment, and an ordinary store blink puts every deployment on the pool into this state. The message names what would be delayed, never that something is.

Not every change waits: a replicas change proceeds during an outage — the ordinals it adds do not exist yet and are created without a connector, and the ordinals it sheds leave on the ordinary path. An edit that changes a running replica’s rendered Pod is what waits. Scaling out does not release it: replicas added during the outage turn over once more when the store returns, and the running replicas still wait. The store’s own recovery bounds the wait.

The stall is deliberate: the alternative — recreating every replica whose render lost its connector — deletes every replica of every deployment on a pool for a few seconds of store unavailability. Measured, a withheld edit lands within seconds of the store returning.

RoleKindsReady — whether every role kind the deployment declares has at least one ready replica. It is deliberately not replica completeness: “every role has all the replicas it asked for” is what phase answers, by summing every role’s counts before judging.

A deployment whose roles all share one kind (the shape you get when no role names a kind, since it defaults to Server) reports True as soon as any replica is ready. That is not a blind spot. The degradation is real and phase reports it as Degraded; this condition answers a different question and is silent on that shape by design.

A role that took over its command line counts toward its kind like any other. Such a role gets no readiness probe, so its Ready says its containers started rather than that anything answered on the serving path. status.roles[].unmanaged is published per role for a reader who needs the stronger reading.

Read this condition alongside phase: it reports readiness by role kind, while phase reports replica completeness. Neither guarantees that an inference request succeeds; that also depends on the engine and Service connectivity.

KVEventsPublishing — whether every role that produces cache blocks has publisher configuration in its rendered Pods. It reports configuration, not live traffic: a configured publisher that later crashes remains True.

RouterReady — whether the managed router rendered and has a ready replica. A render refusal is reported here rather than returned as an error, so the rest of the status (phase, the other conditions, the role counts) keeps computing with the cause named.