LLM Gateway
How projects get models, shared by your organization or through providers, and how your code calls them with the providers' SDKs and a kosem API key.
The LLM Gateway serves AI models to your projects. Your code, your tools and your agents call them with the providers' own SDKs and a kosem API key, and kosem records what each request used. kosem never provides model access itself: every request goes through a provider, your organization's or a project's own, with your own key at the provider.
How it works
Organization
Provider: Anthropic, your key, shared with no project
Model: fast → claude-haiku-4-5, shared with Web
Provider: OpenAI, your key, shared with Web
│ │
│ added │ shared
▼ ▼
Project Web
Model: fast, the organization's ◄── your code
Model: mini → gpt-5-mini, through the organization's OpenAI- A provider holds a key or cloud trust for a model provider, such as Anthropic, OpenAI, Google Gemini or Google Vertex AI, or the address and key of a server that speaks one of their APIs (on Pro, Team and Enterprise). kosem keeps keys encrypted, shows only their last four characters, and checks them with the provider before saving. Add a provider shows how.
- A model is an alias, such as
fast, for one of a provider's models, such asclaude-haiku-4-5, through one provider. Callers send a project's alias asmodel. Aliases are unique in the organization, for its models, and in each project, for the project's.
Your organization gives a project models in two ways:
- It shares a model. The organization adds a model through one of its providers and shares it with projects, which add it under an alias of their own. A project gets that model and no other, and the provider needs no sharing. A change to the organization's model, such as its pricing, applies in every project that added it. Add models to your organization shows how.
- It shares a provider. A project adds any of the provider's models itself, under its own aliases. A project can also add a provider of its own. Add models to a project shows how.
If the organization stops sharing a model or a provider with a project, the project's models through it show Needs attention and requests to them fail, until it's shared again. A policy, Turn off projects' own providers, keeps projects from adding keys of their own, and stops the models through those they added: such a project uses only what the organization shares with it.
Calling a model
Each project has its own base URLs, one for each provider's API:
| API | Base URL |
|---|---|
| Anthropic Messages | https://api.kosem.ai/v1/projects/{project}/llm/anthropic |
| OpenAI Chat Completions and Responses | https://api.kosem.ai/v1/projects/{project}/llm/openai/v1 |
Point a provider's SDK at one of them, with a kosem API key in place of the provider's, and send a model's alias as model. Call models from your code shows how, and LLM Gateway endpoints lists every endpoint and header.
- The gateway changes as little as it can. When the model's provider speaks the API you called, kosem replaces only
modelwith the provider's name for it. It streams the answer back with the provider's status, request ID and rate limit headers, which describe your organization's account at the provider. - Every model takes every API. When its provider speaks another, kosem translates the request and the answer, and adds the header
Kosem-Translated: true. So code written for Anthropic's SDK can call a model that OpenAI serves. Translation carries text, files, function tools and their results, and the common sampling and output settings. It refuses what it can't carry across, such as a provider's own hosted tools or a stored Responses conversation, rather than drop it. - Every answer names its request. The header
Kosem-Request-Idholds the ID that the project's request log shows it under.
What kosem records
For each request that the gateway sends to a provider, kosem records who sent it, the model, how it ended, how long it took and the tokens the provider reported. It never records a prompt or an answer.
- Usage sums requests and tokens by day, project, model and caller. The organization's and each project's LLM Gateway pages show it.
- The request log keeps each request for your plan's retention window (Limits and quotas), with its error if it failed. It also keeps the requests that kosem refused once it had found the model.
Your provider bills your organization for the tokens, as it would without kosem. Agent model calls are free on kosem; direct LLM Gateway API calls that the provider answers are billed past your plan's monthly allowance: Limits and quotas has the numbers.
Why it works this way
- Your keys, your accounts. Requests go to your accounts at the providers, so your contracts, data terms and rate limits with them apply, and the providers' own consoles show every request.
- Aliases keep code apart from providers. Code names
fastorsmart, and the project decides what that means today. - Shared models or shared keys. Sharing a model gives a project that model alone, on the organization's terms. Sharing a provider gives it the key, and every model the key reaches. The organization chooses for each project, and the policy keeps projects from bringing keys of their own.
Who can do what
| To | You need |
|---|---|
| Add the organization's providers and models, and change them | Owner or LLM Gateway Admin on the organization |
| Choose the projects the organization's providers and models are shared with | Owner or LLM Gateway Admin on the organization |
| Turn off projects' own providers | Owner or Policy Admin on the organization |
| Add a project's own providers and its models, or remove them | Owner or LLM Gateway Admin on the project |
| Call a project's models | Owner, Editor, LLM Gateway Admin or LLM Gateway User on the project |
| See usage and the request log | Any basic role, LLM Gateway Admin or LLM Gateway User |