Inference APIs
OpenAI-Compatible APIs
GPUStack provides OpenAI-compatible APIs at the /v1 endpoint.
You can integrate and use models deployed on GPUStack with any application or framework that supports the OpenAI-compatible API, simply by pointing it to GPUStack's OpenAI-compatible endpoint.
Supported Endpoints
The following API endpoints are supported:
- List Models
- Create Completion
- Create Chat Completion
- Create Response
- Create Embeddings
- Create Image
- Create Image Edit
- Create Speech
- Create Transcription
Usage
cURL Example
Anthropic-Compatible APIs
GPUStack provides the Anthropic-compatible /v1/messages API.
Usage
Jina-Compatible Rerank API
In the context of Retrieval-Augmented Generation (RAG), reranking refers to the process of selecting the most relevant information from retrieved documents or knowledge sources before presenting them to the user or utilizing them for answer generation.
Note that the OpenAI-compatible APIs do not provide a rerank endpoint. To fill this gap, GPUStack provides a Jina-compatible Rerank API at the /v1/rerank path.
Usage
cURL Example
Example output:
System One Decision API
Decision models — deployments in the decision category — answer the System One (Jev) decision contract instead of generating text: given a state and a set of questions, they return typed answers rather than completions. GPUStack exposes the contract at the /v1/systemone path.
The model field is required and takes the model route name, exactly like /v1/chat/completions. GPUStack resolves the model and passes the rest of the request body, and the whole response, through to the decision engine untouched.
Usage
cURL Example
Example output:
A state is the situation the decision is made against — plain text or structured JSON such as chat messages. Each entry under questions maps a question id to a question definition, and three types are conventional:
| Type | Meaning | criteria | Answer field |
|---|---|---|---|
choice | Pick one option | Option name → description | choice — the picked option name |
score | Score the state against ordered levels | Ordered list of level descriptions | score — the level index |
noul | Binary (yes/no) question | Optional descriptions for the true / false sides | noul — a boolean |
The answer for a question comes back under the same id in answers, carrying confidence and probabilities alongside the verdict. The question set is open in the contract — engines may accept further fields — and GPUStack validates only model, forwarding the rest verbatim. For deploying decision models, see Using Decision Models.
Other APIs
For other API types, GPUStack allows you to enable the Generic Proxy feature when deploying a model.
Once enabled, there are two ways to address the target model:
Path-based form (recommended) — encode the model route id directly in the URL as /model/proxy/<model_route_id>/<upstream-path>. No extra header is required:
Header-based form (deprecated) — send to /model/proxy/<upstream-path> and specify the target model via the X-GPUStack-Model header or the model field in the JSON body:
The header-based form is retained for backward compatibility and will be removed in a future release; new integrations should use the path-based form.
For more details, see Enable Generic Proxy.