Skip to Content
kosemdocs
Reference

LLM Gateway endpoints

The provider APIs each project's LLM Gateway serves, at which URLs, how to authenticate, and the headers kosem adds.

The endpoints of a project's LLM Gateway, which speak Anthropic's and OpenAI's APIs. Point a provider's SDK at their base URLs: Call models from your code shows how.

Endpoints

{project} is the project's ID, such as prj_01m5r9dh9mf0ps98ktkap3562c.

APIMethod and URL
Anthropic MessagesPOST https://api.kosem.ai/v1/projects/{project}/llm/anthropic/v1/messages
Anthropic token countingPOST https://api.kosem.ai/v1/projects/{project}/llm/anthropic/v1/messages/count_tokens
Anthropic modelsGET https://api.kosem.ai/v1/projects/{project}/llm/anthropic/v1/models
OpenAI Chat CompletionsPOST https://api.kosem.ai/v1/projects/{project}/llm/openai/v1/chat/completions
OpenAI ResponsesPOST https://api.kosem.ai/v1/projects/{project}/llm/openai/v1/responses
OpenAI modelsGET https://api.kosem.ai/v1/projects/{project}/llm/openai/v1/models

The SDKs' base URLs are https://api.kosem.ai/v1/projects/{project}/llm/anthropic and https://api.kosem.ai/v1/projects/{project}/llm/openai/v1. The models endpoints list the project's models by their aliases.

Authentication

Send a kosem API key, of a service account with the LLM Gateway User role or another role that can call the project's models, as the SDKs send the provider's key:

  • Authorization: Bearer kosem_sk_…, as OpenAI's SDKs do;
  • x-api-key: kosem_sk_…, as Anthropic's SDKs do.

Requests

Field or headerWhat kosem does
modelThe project's alias for a model, which kosem replaces with the provider's name for it
Anthropic-Version, Anthropic-BetaForwarded to the provider
Every other headerNot forwarded. The provider gets your organization's key in their place
store, background in a Responses requestRefused for a model served through translation: send the whole conversation each time

Answers

HeaderMeaning
Kosem-Request-IdThe request's ID in the project's request log, such as req_01m5rc4tq2e9vbt6g0d4x7qz1m. Every answer has it
Kosem-Translated: truekosem translated the request to the API that the model's provider speaks, and the answer back
Kosem-Translation-WarningsHow many settings translation approximated, when there were some
X-Should-Retry: falseRetrying won't help, as when the provider refused your organization's key
RateLimit-Policy, RateLimitThe project's gateway requests a minute: the limit, what's left, and the seconds until more fits
Retry-AfterOn a 429 for the project's gateway requests a minute, the seconds until the request fits
The provider's request ID and rate limit headersAs the provider sent them. They describe your organization's account at the provider

An answer that kosem doesn't translate is the provider's, with only its status, request ID and rate limit headers passed on: its model is the provider's name for the model. A translated answer names the alias you sent.

Translation

Every model takes all three APIs. When its provider speaks another, kosem translates:

  • What it carries: text, files the target reads, function tools and their results, and the sampling and output settings that the target can express.
  • What it refuses: stored Responses conversations, a provider's own hosted tools, and fields it doesn't know. It refuses them with 400, rather than drop them.
  • What it approximates: settings the target has no exact match for, such as a reasoning effort it lacks. Each counts in Kosem-Translation-Warnings.
  • Token counting of a translated model answers 501, so that clients fall back to their own estimates.

On this page