LLM Gateway endpoints
The provider APIs each project's LLM Gateway serves, at which URLs, how to authenticate, and the headers kosem adds.
The endpoints of a project's LLM Gateway, which speak Anthropic's and OpenAI's APIs. Point a provider's SDK at their base URLs: Call models from your code shows how.
Endpoints
{project} is the project's ID, such as prj_01m5r9dh9mf0ps98ktkap3562c.
| API | Method and URL |
|---|---|
| Anthropic Messages | POST https://api.kosem.ai/v1/projects/{project}/llm/anthropic/v1/messages |
| Anthropic token counting | POST https://api.kosem.ai/v1/projects/{project}/llm/anthropic/v1/messages/count_tokens |
| Anthropic models | GET https://api.kosem.ai/v1/projects/{project}/llm/anthropic/v1/models |
| OpenAI Chat Completions | POST https://api.kosem.ai/v1/projects/{project}/llm/openai/v1/chat/completions |
| OpenAI Responses | POST https://api.kosem.ai/v1/projects/{project}/llm/openai/v1/responses |
| OpenAI models | GET https://api.kosem.ai/v1/projects/{project}/llm/openai/v1/models |
The SDKs' base URLs are https://api.kosem.ai/v1/projects/{project}/llm/anthropic and https://api.kosem.ai/v1/projects/{project}/llm/openai/v1. The models endpoints list the project's models by their aliases.
Authentication
Send a kosem API key, of a service account with the LLM Gateway User role or another role that can call the project's models, as the SDKs send the provider's key:
Authorization: Bearer kosem_sk_…, as OpenAI's SDKs do;x-api-key: kosem_sk_…, as Anthropic's SDKs do.
Requests
| Field or header | What kosem does |
|---|---|
model | The project's alias for a model, which kosem replaces with the provider's name for it |
Anthropic-Version, Anthropic-Beta | Forwarded to the provider |
| Every other header | Not forwarded. The provider gets your organization's key in their place |
store, background in a Responses request | Refused for a model served through translation: send the whole conversation each time |
Answers
| Header | Meaning |
|---|---|
Kosem-Request-Id | The request's ID in the project's request log, such as req_01m5rc4tq2e9vbt6g0d4x7qz1m. Every answer has it |
Kosem-Translated: true | kosem translated the request to the API that the model's provider speaks, and the answer back |
Kosem-Translation-Warnings | How many settings translation approximated, when there were some |
X-Should-Retry: false | Retrying won't help, as when the provider refused your organization's key |
RateLimit-Policy, RateLimit | The project's gateway requests a minute: the limit, what's left, and the seconds until more fits |
Retry-After | On a 429 for the project's gateway requests a minute, the seconds until the request fits |
| The provider's request ID and rate limit headers | As the provider sent them. They describe your organization's account at the provider |
An answer that kosem doesn't translate is the provider's, with only its status, request ID and rate limit headers passed on: its model is the provider's name for the model. A translated answer names the alias you sent.
Translation
Every model takes all three APIs. When its provider speaks another, kosem translates:
- What it carries: text, files the target reads, function tools and their results, and the sampling and output settings that the target can express.
- What it refuses: stored Responses conversations, a provider's own hosted tools, and fields it doesn't know. It refuses them with
400, rather than drop them. - What it approximates: settings the target has no exact match for, such as a reasoning effort it lacks. Each counts in
Kosem-Translation-Warnings. - Token counting of a translated model answers
501, so that clients fall back to their own estimates.