API Reference
The Radicalbit AI Gateway implements the OpenAI API specification — including Chat Completions, Embeddings, Audio Transcriptions, and the Responses API — making it compatible with existing OpenAI libraries and tools. It also serves the Anthropic Messages API, so Anthropic SDKs and Claude Code work through it, and proxies Typesafe's decision API for decision models.
Base URL
http://localhost:9000
Authentication
API Key Authentication
curl -H "Authorization: Bearer your-api-key" \
-H "Content-Type: application/json" \
http://localhost:9000/v1/chat/completions
The gateway reads the credential from the Authorization: Bearer header, which OpenAI SDKs and compatible libraries send for you. A credential past its expiry date is rejected with 401 and code key_expired_unauthorized. After a rotation, the previous value works only until the grace period ends, and is then rejected with 401 and code key_invalid_unauthorized, like a wrong value.
On the Messages API the gateway also reads the credential from the x-api-key header, which Anthropic SDKs send for you.
Endpoints
Chat Completions
Endpoint: POST /v1/chat/completions
Description: Creates a completion for the provided chat messages.
Request Body:
{
"model": "project-name/route-name",
"messages": [
{
"role": "user",
"content": "Hello, how are you?"
}
],
"temperature": 0.7,
"max_tokens": 100
}
Parameters:
| Parameter | Type | Required | Description |
|---|---|---|---|
model | string | Yes | The route in project-name/route-name format |
messages | array | Yes | Array of message objects |
temperature | number | No | Sampling temperature (0.0 to 2.0) |
max_tokens | number | No | Maximum tokens to generate |
top_p | number | No | Nucleus sampling parameter |
stop | string/array | No | Stop sequences |
presence_penalty | number | No | Presence penalty (-2.0 to 2.0) |
frequency_penalty | number | No | Frequency penalty (-2.0 to 2.0) |
Message Object:
{
"role": "user|assistant|system",
"content": "Message content"
}
Response:
{
"id": "chatcmpl-123",
"object": "chat.completion",
"created": 1677652288,
"model": "project-name/route-name",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! I'm doing well, thank you for asking."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 10,
"completion_tokens": 8,
"total_tokens": 18
}
}
Embeddings
Endpoint: POST /v1/embeddings
Description: Creates an embedding for the provided input.
Request Body:
{
"model": "project-name/route-name",
"input": "The text to embed"
}
Parameters:
| Parameter | Type | Required | Description |
|---|---|---|---|
model | string | Yes | The route in project-name/route-name format |
input | string/array | Yes | Text to embed (string or array of strings) |
Response:
{
"object": "list",
"data": [
{
"object": "embedding",
"index": 0,
"embedding": [0.1, 0.2, 0.3, ...]
}
],
"model": "project-name/route-name",
"usage": {
"prompt_tokens": 5,
"total_tokens": 5
}
}
Audio Transcriptions
Endpoint: POST /v1/audio/transcriptions
Description: Transcribes audio into text, following the OpenAI Audio API format. Unlike the other endpoints, the request body is multipart/form-data, not JSON.
Request Body (multipart/form-data):
curl http://localhost:9000/v1/audio/transcriptions \
-H "Authorization: Bearer your-api-key" \
-F model="project-name/route-name" \
-F file="@audio.mp3" \
-F language="en"
Parameters:
| Parameter | Type | Required | Description |
|---|---|---|---|
model | string | Yes | The route in project-name/route-name format |
file | file | Yes | The audio file to transcribe |
language | string | No | Language of the input audio, in ISO-639-1 format (e.g. en) |
prompt | string | No | Text to guide the model's style, or to continue a previous audio segment |
response_format | string | No | json (default) or verbose_json. verbose_json is only supported by whisper-1 models |
temperature | number | No | Sampling temperature (0.0 to 1.0) |
stream | boolean | No | Stream the transcript as it's generated, via SSE. Only supported by the gpt-4o-transcribe family, not by whisper-1 |
The gateway supports two model families, with different capabilities:
whisper-1: returns a fullverbose_jsonresponse with language, duration and segments, but does not support streaming.gpt-4o-transcribefamily (gpt-4o-transcribe,gpt-4o-mini-transcribe): supports streaming, but only returns a plainjsonresponse.
Response (response_format: "json"):
{
"text": "Hello, how are you today?"
}
Response (response_format: "verbose_json", whisper-1 only):
{
"language": "english",
"duration": 3.2,
"text": "Hello, how are you today?",
"segments": [
{
"id": 0,
"start": 0.0,
"end": 3.2,
"text": "Hello, how are you today?"
}
]
}
Streaming Response:
When stream is true, the gateway returns a Server-Sent Events (SSE) stream of transcript deltas, ending with a [DONE] marker:
data: {"type":"transcript.text.delta","delta":"Hello,"}
data: {"type":"transcript.text.delta","delta":" how are you today?"}
data: {"type":"transcript.text.done","text":"Hello, how are you today?"}
data: [DONE]
stream: true is not supported by whisper-1 and returns an error.
Responses API
Endpoint: POST /v1/responses
Description: Creates a response with the OpenAI Responses API. The request is forwarded to the provider as sent, so every Responses API field is supported.
Request Body:
{
"model": "project-name/route-name",
"input": "What is the capital of France?",
"instructions": "You are a helpful assistant.",
"max_output_tokens": 200
}
Parameters:
| Parameter | Type | Required | Description |
|---|---|---|---|
model | string | Yes | The route in project-name/route-name format |
input | string/array | Yes | A prompt, or the list of conversation items |
instructions | string | No | System instructions |
stream | boolean | No | Stream the response as events |
max_output_tokens | number | No | Maximum tokens to generate |
tools | array | No | Tools the model can call |
store | boolean | No | Whether the provider stores the response |
previous_response_id | string | No | Continue a response stored by the provider |
Any other field of the Responses API is forwarded too.
Response:
{
"id": "resp_123",
"object": "response",
"created_at": 1677652288,
"model": "project-name/route-name",
"output": [
{
"type": "message",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "The capital of France is Paris."
}
]
}
],
"usage": {
"input_tokens": 10,
"output_tokens": 8,
"total_tokens": 18
}
}
Supported providers
Model in config.yaml | Responses API |
|---|---|
openai/... | Yes |
openai/... with a base_url | Yes, if the endpoint serves /v1/responses; otherwise you get the endpoint's own error |
azure/..., azure_openai/... | Yes |
deepseek/... | Yes |
anthropic/..., google-genai/..., mistralai/... | No: 400 ... does not serve the Responses API. |
Fallback
Fallback works as for Chat Completions, except:
- a fallback model that doesn't serve the Responses API is skipped;
- when streaming, the gateway falls back only before the first event reaches the client;
- requests with
previous_response_idnever fall back, because only the provider that stored the response can continue it.
Conversations across models
Most agents send the whole conversation in input every turn. After a fallback, a routing decision or a configuration switch, that conversation can contain items produced by another provider, which the new model would reject. Before each attempt the gateway adapts input for the model about to receive it:
- Messages, tool calls and tool outputs always pass.
- Reasoning passes only to the model that can recognise it as its own; otherwise it is dropped and the model reasons again.
- Item ids pass only where they are accepted; otherwise they are removed and the items stay.
Model in config.yaml | Reasoning it gets | Item ids |
|---|---|---|
openai/... without base_url | only OpenAI's own | ids in another provider's format are removed |
openai/... with a base_url | none | removed |
deepseek/... | its own, and the readable summary of OpenAI's | kept |
| any other model | none | kept |
DeepSeek in thinking mode needs reasoning before each tool call of the current turn; when a tool call came from another model, the gateway adds the note Reasoning not available: this step came from another model. in its place.
Route settings
A route can force store: false and restrict the tools the provider runs: see Responses API settings.
Only POST /v1/responses is served. Stored-response endpoints (GET/DELETE /v1/responses/{id}, cancel, input_items), compact, input_tokens and the Conversations API return 404; background: true returns 400.
Messages API
Endpoint: POST /v1/messages
Description: Creates a message with the Anthropic Messages API. The request is forwarded to the provider as sent, so every Messages API field is supported: tools, images, documents, extended thinking and prompt caching (cache_control). The route's guardrails, routing, limits, budgets, cache and tracing apply to it, as they do to Chat Completions.
Any Anthropic client works through the gateway. Point its base URL at the gateway, use a gateway API key, and set model to project-name/route-name.
Request:
curl http://localhost:9000/v1/messages \
-H "x-api-key: your-api-key" \
-H "anthropic-version: 2023-06-01" \
-H "Content-Type: application/json" \
-d '{
"model": "project-name/route-name",
"max_tokens": 200,
"system": "You are a helpful assistant.",
"messages": [{"role": "user", "content": "What is the capital of France?"}]
}'
The credential goes in x-api-key or in Authorization: Bearer. The anthropic-version and anthropic-beta headers are forwarded to the provider. Your gateway credential never is.
Parameters:
| Parameter | Type | Required | Description |
|---|---|---|---|
model | string | Yes | The route in project-name/route-name format |
messages | array | Yes | The conversation turns |
max_tokens | number | Yes | Maximum tokens to generate |
system | string/array | No | System prompt |
stream | boolean | No | Stream the answer as events |
tools | array | No | Tools the model can call |
thinking | object | No | Extended thinking settings |
Any other field of the Messages API is forwarded too.
Response:
{
"id": "msg_123",
"type": "message",
"role": "assistant",
"model": "project-name/route-name",
"content": [
{
"type": "text",
"text": "The capital of France is Paris."
}
],
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 17,
"output_tokens": 9,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0
}
}
model holds your route, not the provider's model name. The rest is the provider's answer.
Supported providers
Model in config.yaml | Messages API |
|---|---|
anthropic/... | Yes |
| any other model | No: 400 ... does not serve the Messages API |
A request is never translated to another provider's format. Use Chat Completions to reach the other providers.
Guardrails
Guardrails read the text of the request: the system prompt, the text of each message, and the text of tool results. Images, documents, tool calls and thinking are not read and go to the model untouched.
| Guardrail result | What the client gets |
|---|---|
| Redact | The text is replaced in place. The rest of the request is unchanged |
| Block | 400 with error type invalid_request_error |
| Soft block | 200 with a normal message that holds the soft block text, and zero usage. The model is not called |
Output guardrails read the text of the answer. LLM judges and decision judges work as on Chat Completions, and their calls count on the caller's limits.
A system prompt set on the model is added before the request's own system. It changes the start of the prompt, so it also changes what the provider's prompt cache can reuse.
Streaming
With "stream": true the gateway returns the Messages event stream (message_start, content_block_delta, message_stop and so on).
- No output guardrail on the route: events are relayed as the model produces them.
- Output guardrails on the route: the gateway waits for the whole answer, checks it, then sends it as a valid event stream. The client gets the answer in one go, not token by token.
A soft block and a cache hit are also sent as a valid event stream.
Routing and fallback
Routing rules work as for Chat Completions. They run after redact guardrails, so they see the redacted text. If the chosen model does not serve the Messages API, the request fails with 400.
Fallback works as for Chat Completions, except:
- a fallback model that doesn't serve the Messages API ends the chain, and the request fails;
- when streaming, the gateway falls back only before the first event reaches the client.
Limits and cost
Rate, token and budget limits apply on the route, the credential and the project. A full limit returns 429 with error type rate_limit_error.
Tokens are read from the provider's usage. Input tokens include prompt cache reads and prompt cache writes. Cost uses the price of each kind: input, cache read, cache write (5 minutes or 1 hour) and output. After a fallback, cost and limits use the prices of the model that answered.
Cache
Exact and semantic caching work on Messages requests.
- The exact cache matches on the whole request body after guardrails.
streamandmetadataare ignored, so a streamed and a non-streamed request share one entry. - The semantic cache compares the text of the last user message.
- Only complete answers are stored: stop reason
end_turn,stop_sequenceortool_use. An answer cut bymax_tokensis never stored. - A cached answer still passes the output guardrails. It does not count on token and budget limits.
- Entries are not shared with Chat Completions or the Responses API.
Observability
Each request is traced like a chat request: the route, the model that answered, input and output tokens, prompt cache tokens, cost and stop reason. See Monitoring.
Claude Code
Claude Code can use a route as its model:
export ANTHROPIC_BASE_URL=http://localhost:9000
export ANTHROPIC_AUTH_TOKEN=your-api-key
export ANTHROPIC_MODEL=project-name/route-name
export ANTHROPIC_SMALL_FAST_MODEL=project-name/route-name
claude
Errors
Errors on this endpoint use the Anthropic format, so Anthropic SDKs raise them as regular API errors:
{
"type": "error",
"error": {
"type": "invalid_request_error",
"message": "The route name [my-project/my-route] was not found in the config"
}
}
| Status | Error type |
|---|---|
400 | invalid_request_error (also for an invalid request body) |
401 | authentication_error |
404 | not_found_error |
429 | rate_limit_error |
5xx | api_error |
An error returned by the provider is forwarded unchanged. If a stream fails after it started, it ends with an error event.
Only POST /v1/messages is served. Token counting (/v1/messages/count_tokens) and Message Batches return 404.
Decision Models
Endpoint: POST /v1/systemone
Description: Asks a decision model (Typesafe's Jev) structured questions about a piece of input, and returns a typed answer with a confidence for each question. The endpoint mirrors Typesafe's API: the body uses Typesafe's format, except that model is your gateway route. The gateway forwards the request and returns Typesafe's status and body unchanged.
Any Typesafe client works through the gateway. Point its base URL at the gateway, use a gateway API key, and set model to project-name/route-name.
Streaming is not supported. The response is always returned in one piece.
Request Body:
{
"model": "project-name/route-name",
"state": "My card was charged twice and I need the money back today!",
"questions": {
"intent": {
"type": "choice",
"instructions": "What does the customer want?",
"criteria": {
"refund": "Wants money back",
"information": "Wants an explanation",
"complaint": "Wants to complain"
}
},
"is_urgent": {
"type": "noul",
"instructions": "Does this convey urgency?"
},
"frustration": {
"type": "score",
"instructions": "How frustrated is the customer?",
"criteria": ["Calm", "Mildly annoyed", "Clearly frustrated", "Angry"]
}
}
}
Parameters:
| Parameter | Type | Required | Description |
|---|---|---|---|
model | string | Yes | The route in project-name/route-name format. The gateway replaces it with the configured Typesafe model |
state | string/object/array | Yes | The input to evaluate. Route input guardrails check and redact its text values |
questions | object | Yes | Map of question id to question object |
| other fields | any | No | Forwarded to Typesafe unchanged |
Question Object:
| Field | Type | Required | Description |
|---|---|---|---|
type | string | Yes | choice, score or noul |
instructions | string | Yes | The question to answer |
criteria | object/array | Depends on type | choice: map of option to description. score: ordered list of levels. noul: optional description of what yes and no mean |
Question Types:
| Type | Answers | Main answer field |
|---|---|---|
choice | One option out of a set | choice, with probabilities and confidence |
score | A position on an ordered scale | score, with probabilities and confidence |
noul | A yes/no statement | noul, the probability (0 to 1) that the statement is true |
See Typesafe's documentation for the full question and answer format.
Response:
{
"model": "jev-1.13.0",
"answers": {
"intent": {
"type": "choice",
"choice": "refund",
"confidence": 1.0,
"probabilities": { "refund": 1.0, "information": 0.0, "complaint": 0.0 }
}
},
"usage": {
"input_tokens": 337,
"output_tokens": 31
}
}
The response is Typesafe's body, unchanged. model holds the Typesafe model that answered, not your route. Cost is computed on usage.input_tokens only. See Limits and Cost.
Errors:
| Status | Cause |
|---|---|
400 | Unknown route, a route with no decision models, or a guardrail block or soft block (guardrail_violation_bad_request) |
401 | Missing, invalid or expired gateway key, or a key that does not belong to the route |
422 | The request body is invalid, for example model is missing |
429 | A rate, token or budget limit is full |
502 | No decision model answered (model_invoker_bad_gateway) |
Other 4xx | Returned by Typesafe and forwarded unchanged. They are not retried and do not trigger fallback |
Error Responses
Standard Error Format
Errors follow the OpenAI error format, so OpenAI SDKs raise them as regular API errors. The Messages API uses the Anthropic format instead.
{
"error": {
"message": "The route name [my-project/my-route] was not found in the config",
"type": "gateway_error",
"param": null,
"code": "gateway_bad_request"
}
}
typeis the error family, for exampleauthentication_error,rate_limit_errororguardrail_error.codeis the specific error. Use it to tell errors apart.
Common Error Codes
| Status | Code | Description | Solution |
|---|---|---|---|
400 | gateway_bad_request | Unknown route, or a route with no models of the requested kind | Check the model field (project-name/route-name) and the route configuration |
400 | guardrail_violation_bad_request | Content blocked by a guardrail. param.reason.message_indices says which message triggered it | Review the guardrail configuration |
401 | key_missing_unauthorized | No credential in the request | Send the credential in the Authorization: Bearer header |
401 | key_invalid_unauthorized | Unknown credential, a credential whose group is not linked to the route, or a previous value used after its grace period | Check the credential value and its group's routes |
401 | key_expired_unauthorized | "API key has expired.": the credential's expiry date has passed | Ask an Admin for a new credential |
401 | invalid_jwt_token | Invalid identity provider token (Enterprise, see Access Control) | Get a new token from the identity provider |
429 | request_rate_limited | A request limit is full: on the route or the credential. See which limit blocked the request | Retry later, or raise the limit |
429 | token_limit_rate_limited | A token limit is full: on the route or the credential | Retry later, or raise the limit |
429 | budget_limit_rate_limited | A budget is spent: on the route, the credential or the project | Raise the budget, or wait for the next budget window |
502 | model_invoker_bad_gateway | No decision model answered, fallbacks included | Check Typesafe's status and the fallback configuration |
Which Limit Blocked the Request
Route limits, credential limits and the project budget all apply to every request, and the first one that is full blocks it. A 429 from a limit carries the X-RB-AIGATEWAY-LIMIT-SCOPE header, which says where that limit is set:
X-RB-AIGATEWAY-LIMIT-SCOPE | The limit is set on | Where to change it |
|---|---|---|
route | The route, in the project configuration (rate_limiting, token_limiting, duration_limiting, budget_limiting) | The project configuration |
credential | The credential that sent the request | Credentials › Manage limits, or the credential limits API |
project | The project budget | Projects › Budget limits, or the project budget API |
The body and the code are the same as without the header: code says which kind of limit (requests, tokens, budget), the header says which one. Streaming transcriptions blocked by a limit carry the header too.
curl -i https://<gateway>/v1/chat/completions \
-H "Authorization: Bearer $GATEWAY_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "my-project/my-route", "messages": [{"role": "user", "content": "Hi"}]}'
HTTP/1.1 429 Too Many Requests
X-RB-AIGATEWAY-LIMIT-SCOPE: credential
{
"error": {
"message": "Credential rate limit exceeded: 3 requests per 1 minute on credential \"my-key\". Please retry after 49 seconds.",
"type": "rate_limit_error",
"param": "my-route",
"code": "request_rate_limited"
}
}
HTTP header names are case-insensitive: some clients show it as x-rb-aigateway-limit-scope. A browser app calling the gateway from another origin may not be able to read it, like the other X-RB-AIGATEWAY-* headers.
Each block is also recorded: the route's Events tab lists it in the Limits section, with who blocked it.