Skip to main content

API Reference

The Radicalbit AI Gateway implements the OpenAI API specification — including Chat Completions, Embeddings, Audio Transcriptions, and the Responses API — making it compatible with existing OpenAI libraries and tools. It also serves the Anthropic Messages API, so Anthropic SDKs and Claude Code work through it, and proxies Typesafe's decision API for decision models.

Base URL​

http://localhost:9000

Authentication​

API Key Authentication​

curl -H "Authorization: Bearer your-api-key" \
-H "Content-Type: application/json" \
http://localhost:9000/v1/chat/completions

The gateway reads the credential from the Authorization: Bearer header, which OpenAI SDKs and compatible libraries send for you. A credential past its expiry date is rejected with 401 and code key_expired_unauthorized. After a rotation, the previous value works only until the grace period ends, and is then rejected with 401 and code key_invalid_unauthorized, like a wrong value.

On the Messages API the gateway also reads the credential from the x-api-key header, which Anthropic SDKs send for you.

Endpoints​

Chat Completions​

Endpoint: POST /v1/chat/completions

Description: Creates a completion for the provided chat messages.

Request Body:

{
"model": "project-name/route-name",
"messages": [
{
"role": "user",
"content": "Hello, how are you?"
}
],
"temperature": 0.7,
"max_tokens": 100
}

Parameters:

ParameterTypeRequiredDescription
modelstringYesThe route in project-name/route-name format
messagesarrayYesArray of message objects
temperaturenumberNoSampling temperature (0.0 to 2.0)
max_tokensnumberNoMaximum tokens to generate
top_pnumberNoNucleus sampling parameter
stopstring/arrayNoStop sequences
presence_penaltynumberNoPresence penalty (-2.0 to 2.0)
frequency_penaltynumberNoFrequency penalty (-2.0 to 2.0)

Message Object:

{
"role": "user|assistant|system",
"content": "Message content"
}

Response:

{
"id": "chatcmpl-123",
"object": "chat.completion",
"created": 1677652288,
"model": "project-name/route-name",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! I'm doing well, thank you for asking."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 10,
"completion_tokens": 8,
"total_tokens": 18
}
}

Embeddings​

Endpoint: POST /v1/embeddings

Description: Creates an embedding for the provided input.

Request Body:

{
"model": "project-name/route-name",
"input": "The text to embed"
}

Parameters:

ParameterTypeRequiredDescription
modelstringYesThe route in project-name/route-name format
inputstring/arrayYesText to embed (string or array of strings)

Response:

{
"object": "list",
"data": [
{
"object": "embedding",
"index": 0,
"embedding": [0.1, 0.2, 0.3, ...]
}
],
"model": "project-name/route-name",
"usage": {
"prompt_tokens": 5,
"total_tokens": 5
}
}

Audio Transcriptions​

Endpoint: POST /v1/audio/transcriptions

Description: Transcribes audio into text, following the OpenAI Audio API format. Unlike the other endpoints, the request body is multipart/form-data, not JSON.

Request Body (multipart/form-data):

curl http://localhost:9000/v1/audio/transcriptions \
-H "Authorization: Bearer your-api-key" \
-F model="project-name/route-name" \
-F file="@audio.mp3" \
-F language="en"

Parameters:

ParameterTypeRequiredDescription
modelstringYesThe route in project-name/route-name format
filefileYesThe audio file to transcribe
languagestringNoLanguage of the input audio, in ISO-639-1 format (e.g. en)
promptstringNoText to guide the model's style, or to continue a previous audio segment
response_formatstringNojson (default) or verbose_json. verbose_json is only supported by whisper-1 models
temperaturenumberNoSampling temperature (0.0 to 1.0)
streambooleanNoStream the transcript as it's generated, via SSE. Only supported by the gpt-4o-transcribe family, not by whisper-1
note

The gateway supports two model families, with different capabilities:

  • whisper-1: returns a full verbose_json response with language, duration and segments, but does not support streaming.
  • gpt-4o-transcribe family (gpt-4o-transcribe, gpt-4o-mini-transcribe): supports streaming, but only returns a plain json response.

Response (response_format: "json"):

{
"text": "Hello, how are you today?"
}

Response (response_format: "verbose_json", whisper-1 only):

{
"language": "english",
"duration": 3.2,
"text": "Hello, how are you today?",
"segments": [
{
"id": 0,
"start": 0.0,
"end": 3.2,
"text": "Hello, how are you today?"
}
]
}

Streaming Response:

When stream is true, the gateway returns a Server-Sent Events (SSE) stream of transcript deltas, ending with a [DONE] marker:

data: {"type":"transcript.text.delta","delta":"Hello,"}

data: {"type":"transcript.text.delta","delta":" how are you today?"}

data: {"type":"transcript.text.done","text":"Hello, how are you today?"}

data: [DONE]
warning

stream: true is not supported by whisper-1 and returns an error.

Responses API​

Endpoint: POST /v1/responses

Description: Creates a response with the OpenAI Responses API. The request is forwarded to the provider as sent, so every Responses API field is supported.

Request Body:

{
"model": "project-name/route-name",
"input": "What is the capital of France?",
"instructions": "You are a helpful assistant.",
"max_output_tokens": 200
}

Parameters:

ParameterTypeRequiredDescription
modelstringYesThe route in project-name/route-name format
inputstring/arrayYesA prompt, or the list of conversation items
instructionsstringNoSystem instructions
streambooleanNoStream the response as events
max_output_tokensnumberNoMaximum tokens to generate
toolsarrayNoTools the model can call
storebooleanNoWhether the provider stores the response
previous_response_idstringNoContinue a response stored by the provider

Any other field of the Responses API is forwarded too.

Response:

{
"id": "resp_123",
"object": "response",
"created_at": 1677652288,
"model": "project-name/route-name",
"output": [
{
"type": "message",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "The capital of France is Paris."
}
]
}
],
"usage": {
"input_tokens": 10,
"output_tokens": 8,
"total_tokens": 18
}
}

Supported providers​

Model in config.yamlResponses API
openai/...Yes
openai/... with a base_urlYes, if the endpoint serves /v1/responses; otherwise you get the endpoint's own error
azure/..., azure_openai/...Yes
deepseek/...Yes
anthropic/..., google-genai/..., mistralai/...No: 400 ... does not serve the Responses API.

Fallback​

Fallback works as for Chat Completions, except:

  • a fallback model that doesn't serve the Responses API is skipped;
  • when streaming, the gateway falls back only before the first event reaches the client;
  • requests with previous_response_id never fall back, because only the provider that stored the response can continue it.

Conversations across models​

Most agents send the whole conversation in input every turn. After a fallback, a routing decision or a configuration switch, that conversation can contain items produced by another provider, which the new model would reject. Before each attempt the gateway adapts input for the model about to receive it:

  • Messages, tool calls and tool outputs always pass.
  • Reasoning passes only to the model that can recognise it as its own; otherwise it is dropped and the model reasons again.
  • Item ids pass only where they are accepted; otherwise they are removed and the items stay.
Model in config.yamlReasoning it getsItem ids
openai/... without base_urlonly OpenAI's ownids in another provider's format are removed
openai/... with a base_urlnoneremoved
deepseek/...its own, and the readable summary of OpenAI'skept
any other modelnonekept

DeepSeek in thinking mode needs reasoning before each tool call of the current turn; when a tool call came from another model, the gateway adds the note Reasoning not available: this step came from another model. in its place.

Route settings​

A route can force store: false and restrict the tools the provider runs: see Responses API settings.

warning

Only POST /v1/responses is served. Stored-response endpoints (GET/DELETE /v1/responses/{id}, cancel, input_items), compact, input_tokens and the Conversations API return 404; background: true returns 400.

Messages API​

Endpoint: POST /v1/messages

Description: Creates a message with the Anthropic Messages API. The request is forwarded to the provider as sent, so every Messages API field is supported: tools, images, documents, extended thinking and prompt caching (cache_control). The route's guardrails, routing, limits, budgets, cache and tracing apply to it, as they do to Chat Completions.

tip

Any Anthropic client works through the gateway. Point its base URL at the gateway, use a gateway API key, and set model to project-name/route-name.

Request:

curl http://localhost:9000/v1/messages \
-H "x-api-key: your-api-key" \
-H "anthropic-version: 2023-06-01" \
-H "Content-Type: application/json" \
-d '{
"model": "project-name/route-name",
"max_tokens": 200,
"system": "You are a helpful assistant.",
"messages": [{"role": "user", "content": "What is the capital of France?"}]
}'

The credential goes in x-api-key or in Authorization: Bearer. The anthropic-version and anthropic-beta headers are forwarded to the provider. Your gateway credential never is.

Parameters:

ParameterTypeRequiredDescription
modelstringYesThe route in project-name/route-name format
messagesarrayYesThe conversation turns
max_tokensnumberYesMaximum tokens to generate
systemstring/arrayNoSystem prompt
streambooleanNoStream the answer as events
toolsarrayNoTools the model can call
thinkingobjectNoExtended thinking settings

Any other field of the Messages API is forwarded too.

Response:

{
"id": "msg_123",
"type": "message",
"role": "assistant",
"model": "project-name/route-name",
"content": [
{
"type": "text",
"text": "The capital of France is Paris."
}
],
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 17,
"output_tokens": 9,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0
}
}

model holds your route, not the provider's model name. The rest is the provider's answer.

Supported providers​

Model in config.yamlMessages API
anthropic/...Yes
any other modelNo: 400 ... does not serve the Messages API

A request is never translated to another provider's format. Use Chat Completions to reach the other providers.

Guardrails​

Guardrails read the text of the request: the system prompt, the text of each message, and the text of tool results. Images, documents, tool calls and thinking are not read and go to the model untouched.

Guardrail resultWhat the client gets
RedactThe text is replaced in place. The rest of the request is unchanged
Block400 with error type invalid_request_error
Soft block200 with a normal message that holds the soft block text, and zero usage. The model is not called

Output guardrails read the text of the answer. LLM judges and decision judges work as on Chat Completions, and their calls count on the caller's limits.

A system prompt set on the model is added before the request's own system. It changes the start of the prompt, so it also changes what the provider's prompt cache can reuse.

Streaming​

With "stream": true the gateway returns the Messages event stream (message_start, content_block_delta, message_stop and so on).

  • No output guardrail on the route: events are relayed as the model produces them.
  • Output guardrails on the route: the gateway waits for the whole answer, checks it, then sends it as a valid event stream. The client gets the answer in one go, not token by token.

A soft block and a cache hit are also sent as a valid event stream.

Routing and fallback​

Routing rules work as for Chat Completions. They run after redact guardrails, so they see the redacted text. If the chosen model does not serve the Messages API, the request fails with 400.

Fallback works as for Chat Completions, except:

  • a fallback model that doesn't serve the Messages API ends the chain, and the request fails;
  • when streaming, the gateway falls back only before the first event reaches the client.

Limits and cost​

Rate, token and budget limits apply on the route, the credential and the project. A full limit returns 429 with error type rate_limit_error.

Tokens are read from the provider's usage. Input tokens include prompt cache reads and prompt cache writes. Cost uses the price of each kind: input, cache read, cache write (5 minutes or 1 hour) and output. After a fallback, cost and limits use the prices of the model that answered.

Cache​

Exact and semantic caching work on Messages requests.

  • The exact cache matches on the whole request body after guardrails. stream and metadata are ignored, so a streamed and a non-streamed request share one entry.
  • The semantic cache compares the text of the last user message.
  • Only complete answers are stored: stop reason end_turn, stop_sequence or tool_use. An answer cut by max_tokens is never stored.
  • A cached answer still passes the output guardrails. It does not count on token and budget limits.
  • Entries are not shared with Chat Completions or the Responses API.

Observability​

Each request is traced like a chat request: the route, the model that answered, input and output tokens, prompt cache tokens, cost and stop reason. See Monitoring.

Claude Code​

Claude Code can use a route as its model:

export ANTHROPIC_BASE_URL=http://localhost:9000
export ANTHROPIC_AUTH_TOKEN=your-api-key
export ANTHROPIC_MODEL=project-name/route-name
export ANTHROPIC_SMALL_FAST_MODEL=project-name/route-name
claude

Errors​

Errors on this endpoint use the Anthropic format, so Anthropic SDKs raise them as regular API errors:

{
"type": "error",
"error": {
"type": "invalid_request_error",
"message": "The route name [my-project/my-route] was not found in the config"
}
}
StatusError type
400invalid_request_error (also for an invalid request body)
401authentication_error
404not_found_error
429rate_limit_error
5xxapi_error

An error returned by the provider is forwarded unchanged. If a stream fails after it started, it ends with an error event.

warning

Only POST /v1/messages is served. Token counting (/v1/messages/count_tokens) and Message Batches return 404.

Decision Models​

Endpoint: POST /v1/systemone

Description: Asks a decision model (Typesafe's Jev) structured questions about a piece of input, and returns a typed answer with a confidence for each question. The endpoint mirrors Typesafe's API: the body uses Typesafe's format, except that model is your gateway route. The gateway forwards the request and returns Typesafe's status and body unchanged.

tip

Any Typesafe client works through the gateway. Point its base URL at the gateway, use a gateway API key, and set model to project-name/route-name.

warning

Streaming is not supported. The response is always returned in one piece.

Request Body:

{
"model": "project-name/route-name",
"state": "My card was charged twice and I need the money back today!",
"questions": {
"intent": {
"type": "choice",
"instructions": "What does the customer want?",
"criteria": {
"refund": "Wants money back",
"information": "Wants an explanation",
"complaint": "Wants to complain"
}
},
"is_urgent": {
"type": "noul",
"instructions": "Does this convey urgency?"
},
"frustration": {
"type": "score",
"instructions": "How frustrated is the customer?",
"criteria": ["Calm", "Mildly annoyed", "Clearly frustrated", "Angry"]
}
}
}

Parameters:

ParameterTypeRequiredDescription
modelstringYesThe route in project-name/route-name format. The gateway replaces it with the configured Typesafe model
statestring/object/arrayYesThe input to evaluate. Route input guardrails check and redact its text values
questionsobjectYesMap of question id to question object
other fieldsanyNoForwarded to Typesafe unchanged

Question Object:

FieldTypeRequiredDescription
typestringYeschoice, score or noul
instructionsstringYesThe question to answer
criteriaobject/arrayDepends on typechoice: map of option to description. score: ordered list of levels. noul: optional description of what yes and no mean

Question Types:

TypeAnswersMain answer field
choiceOne option out of a setchoice, with probabilities and confidence
scoreA position on an ordered scalescore, with probabilities and confidence
noulA yes/no statementnoul, the probability (0 to 1) that the statement is true

See Typesafe's documentation for the full question and answer format.

Response:

{
"model": "jev-1.13.0",
"answers": {
"intent": {
"type": "choice",
"choice": "refund",
"confidence": 1.0,
"probabilities": { "refund": 1.0, "information": 0.0, "complaint": 0.0 }
}
},
"usage": {
"input_tokens": 337,
"output_tokens": 31
}
}

The response is Typesafe's body, unchanged. model holds the Typesafe model that answered, not your route. Cost is computed on usage.input_tokens only. See Limits and Cost.

Errors:

StatusCause
400Unknown route, a route with no decision models, or a guardrail block or soft block (guardrail_violation_bad_request)
401Missing, invalid or expired gateway key, or a key that does not belong to the route
422The request body is invalid, for example model is missing
429A rate, token or budget limit is full
502No decision model answered (model_invoker_bad_gateway)
Other 4xxReturned by Typesafe and forwarded unchanged. They are not retried and do not trigger fallback

Error Responses​

Standard Error Format​

Errors follow the OpenAI error format, so OpenAI SDKs raise them as regular API errors. The Messages API uses the Anthropic format instead.

{
"error": {
"message": "The route name [my-project/my-route] was not found in the config",
"type": "gateway_error",
"param": null,
"code": "gateway_bad_request"
}
}
  • type is the error family, for example authentication_error, rate_limit_error or guardrail_error.
  • code is the specific error. Use it to tell errors apart.

Common Error Codes​

StatusCodeDescriptionSolution
400gateway_bad_requestUnknown route, or a route with no models of the requested kindCheck the model field (project-name/route-name) and the route configuration
400guardrail_violation_bad_requestContent blocked by a guardrail. param.reason.message_indices says which message triggered itReview the guardrail configuration
401key_missing_unauthorizedNo credential in the requestSend the credential in the Authorization: Bearer header
401key_invalid_unauthorizedUnknown credential, a credential whose group is not linked to the route, or a previous value used after its grace periodCheck the credential value and its group's routes
401key_expired_unauthorized"API key has expired.": the credential's expiry date has passedAsk an Admin for a new credential
401invalid_jwt_tokenInvalid identity provider token (Enterprise, see Access Control)Get a new token from the identity provider
429request_rate_limitedA request limit is full: on the route or the credential. See which limit blocked the requestRetry later, or raise the limit
429token_limit_rate_limitedA token limit is full: on the route or the credentialRetry later, or raise the limit
429budget_limit_rate_limitedA budget is spent: on the route, the credential or the projectRaise the budget, or wait for the next budget window
502model_invoker_bad_gatewayNo decision model answered, fallbacks includedCheck Typesafe's status and the fallback configuration

Which Limit Blocked the Request​

Route limits, credential limits and the project budget all apply to every request, and the first one that is full blocks it. A 429 from a limit carries the X-RB-AIGATEWAY-LIMIT-SCOPE header, which says where that limit is set:

X-RB-AIGATEWAY-LIMIT-SCOPEThe limit is set onWhere to change it
routeThe route, in the project configuration (rate_limiting, token_limiting, duration_limiting, budget_limiting)The project configuration
credentialThe credential that sent the requestCredentials › Manage limits, or the credential limits API
projectThe project budgetProjects › Budget limits, or the project budget API

The body and the code are the same as without the header: code says which kind of limit (requests, tokens, budget), the header says which one. Streaming transcriptions blocked by a limit carry the header too.

curl -i https://<gateway>/v1/chat/completions \
-H "Authorization: Bearer $GATEWAY_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "my-project/my-route", "messages": [{"role": "user", "content": "Hi"}]}'
HTTP/1.1 429 Too Many Requests
X-RB-AIGATEWAY-LIMIT-SCOPE: credential

{
"error": {
"message": "Credential rate limit exceeded: 3 requests per 1 minute on credential \"my-key\". Please retry after 49 seconds.",
"type": "rate_limit_error",
"param": "my-route",
"code": "request_rate_limited"
}
}

HTTP header names are case-insensitive: some clients show it as x-rb-aigateway-limit-scope. A browser app calling the gateway from another origin may not be able to read it, like the other X-RB-AIGATEWAY-* headers.

Each block is also recorded: the route's Events tab lists it in the Limits section, with who blocked it.