Engine Compatibility

Check engine and transport compatibility before choosing images for a deployment. The requirements cover minimum releases, Mooncake client and store versions, supported transports and known failures.

Engines below the listed minimum versions are unsupported. Upgrade before diagnosing failures on an older engine release.

Contents

• Minimum versions
• Reading the table
• Transport compatibility
• Limitations

Minimum versions

The table covers aggregated serving, prefill/decode (PD) and encode/prefill/decode (EPD) configurations.

A version counts as run when the configuration answered requests and, where it moves KV, the engine or the store reported blocks moving: a transfer or a store write, not a Pod reaching Ready. The vLLM-Ascend rows ran on the runner’s cann9.1-910b-vllm0.23.0 tag, named through roles[].image.

The NVIDIA vLLM EPD rows (behind llm-d-router and vllm-router) and the NVIDIA SGLang EPD rows (run on 0.5.18, behind both routers) ran live. The vLLM-Ascend EPD row behind llm-d-router ran live on Ascend 910B2. The AMD EPD rows and the SGLang Ascend rows are admitted but not verified on hardware (#763 tracks them).

Reading the table

The minimum is a property of the image you name. There is no engine.version field to set: the image a role runs resolves to roles[].image when the role names one, and to spec.engine.image otherwise (Role images ). Nothing assembles a tag and no admission check reads a version, so meeting a minimum here means naming an image whose engine build carries it.

A deployment without an Encode role needs --enable-scale-out behind llm-d-router. The router counts prompt tokens through the engine’s render endpoint, which vLLM serves only under that flag, from 0.30.0 on; the gate measured on vllm-ascend is 0.30.0rc1. Name an image at those versions and set the flag on each role (Counting engine tokens ).

The 0.29.0 and 0.23.0 minimums above predate the flag: they were measured as transfer-path runs, before the operator rendered this dependency.

The floor belongs to the engine, so an SGLang EPD row states SGLang 0.5.10 on every platform, while the vLLM Ascend row states a vllm-ascend release.

The client comes with the image. The column above is read off the published runner images, and the vLLM rows are the CUDA ones; a ROCm runner image compiles its own client and has not been run. A role naming its own roles[].image carries whatever client that image embeds: read it off the image, then pick the store from it.

A store runs on the client’s minor line. The store column is a requirement rather than a suggestion; see Version compatibility .

vLLM-Ascend’s direct transfer does not support direct tcp transfer, and one router renders it; see Pair connections .

The configurations above move KV over tcp, or over ascend on vLLM-Ascend. Which other transport each engine can use is the matrix below .

“Not verified” means no run, not a known failure. Nothing refuses such a configuration; nothing here says it works.

An EPD encoder image must meet its row’s floor, and nothing refuses a lower one. The requirement: the image an Encode deployment runs must carry vllm-ascend 0.30.0rc1 or later for Ascend ECExampleConnector, and vLLM 0.30.0 or later for NVIDIA/AMD ECCPUConnector (the admitted matrix ). Admission does not check it — the version field and its gate are gone.

An image below the floor fails at container start with an ImagePullBackOff on a tag that does not exist, or an engine that exits on an unknown argument.

The group-forward probe has a verified version set, and nothing bars an unverified one. The requirement: the version whose probe binding was read is 0.25.1/0.29.0 for vLLM (the response id is cmpl- + the request’s request_id) and 0.5.18/0.5.21 for SGLang (the response meta_info.id is the request’s rid).

On any other version the probe still runs and its outcome still decides: Verified admits a member, Unknown grants nothing new and withdraws nothing already granted. Only a failed health check of the same qualification withdraws an eligible member. Neither probe outcome is an admission refusal.

Transport compatibility

A store cell is the pool’s KVCacheBackend.spec.transport.protocol, which a member group can override and the engine is handed. A direct cell is spec.kvTransfer.protocol on a prefill/decode pair. The first column gives the API and Mooncake spellings. Auto renders tcp on either path; the direct path also renders tcp when the field is unset.

The vLLM and SGLang verdicts are read on their CUDA runner images. What each verdict means:

• Works: run on this operator with blocks moving, by the criterion under the minimum , on the images the cell names.
• Own image: works only on an engine image you build and keep up to date yourself.
• Not supported: admission refuses it, the operator renders nothing for it, or no image the cell could run carries the transport; the cell says which.
• Not verified: no run. Nothing refuses it; nothing here says it works.

“Default images” needs no image work from you: a stock runner image the project publishes, named in engine.image or roles[].image, and the kv-cache-backend-image default for the store members. “Named by hand” is a published image you write into roles[].image or the backend’s spec.image.

A cell is one path. A deployment using both paths is a separate configuration, read from the minimum rather than added up from two cells; both paths over EFA at once have not been run. With a positive resources.interface, an RDMA path beside an EFA one is refused at admission.

On AWS the RDMA rows do not apply; choose EFA or TCP .

Limitations

• SGLang with a store holds a pinned host pool and needs the node’s available memory above a fixed reserve plus that pool; see SGLang’s host-memory tier .

• SGLang prefill/decode over TCP runs out of local ports under sustained load, with or without a store. The prefill half opens a new connection for every transfer, to the decode half and to each store member, all from the same ephemeral ports of its container. Connections left in TIME-WAIT use them up; from then on every request through that pair fails and keeps failing until the engine Pods are restarted. On the measured pair without a store, the gateway then answered 503 with No available prefill workers.

  Turning on model-deployment-tcp-tw-reuse prevented failure in a measured SGLang 0.5.18 pair with a store: it answered 1905 sequential requests without failure, with about 19,500 TIME-WAIT sockets. It needs a kubelet change on every eligible node.

  The pair without a store was not tested with this setting. Widening net.ipv4.ip_local_port_range only delays exhaustion; whether a fabric transport avoids it remains unverified.

  vLLM did not show this behavior in the measured runs: its prefill half keeps its transfer connections open instead of opening one per transfer, and held at most nine sockets in TIME-WAIT over a direct path with no store and at most sixteen with a TCP store beside it.