# LLM Gateway

How projects get models, shared by your organization or through providers, and how your code calls them with the providers' SDKs and a kosem API key.

The LLM Gateway serves AI models to your projects. Your code, your tools and your agents call them with the providers' own SDKs and a kosem API key, and kosem records what each request used. kosem never provides model access itself: every request goes through a provider, your organization's or a project's own, with your own key at the provider.

## How it works [#how-it-works]

```text
Organization
  Provider: Anthropic, your key, shared with no project
    Model: fast → claude-haiku-4-5, shared with Web
  Provider: OpenAI, your key, shared with Web
        │                                   │
        │ added                             │ shared
        ▼                                   ▼
Project Web
  Model: fast, the organization's ◄── your code
  Model: mini → gpt-5-mini, through the organization's OpenAI
```

- **A provider** holds a key or cloud trust for a model provider, such as Anthropic, OpenAI, Google Gemini or Google Vertex AI, or the address and key of a server that speaks one of their APIs (on Pro, Team and Enterprise). kosem keeps keys encrypted, shows only their last four characters, and checks them with the provider before saving. [Add a provider](/guides/llm-gateway/add-a-provider) shows how.
- **A model** is an alias, such as `fast`, for one of a provider's models, such as `claude-haiku-4-5`, through one provider. Callers send a project's alias as `model`. Aliases are unique in the organization, for its models, and in each project, for the project's.

Your organization gives a project models in two ways:

- **It shares a model.** The organization adds a model through one of its providers and shares it with projects, which add it under an alias of their own. A project gets that model and no other, and the provider needs no sharing. A change to the organization's model, such as its pricing, applies in every project that added it. [Add models to your organization](/guides/llm-gateway/add-models) shows how.
- **It shares a provider.** A project adds any of the provider's models itself, under its own aliases. A project can also add a provider of its own. [Add models to a project](/guides/llm-gateway/add-models-to-a-project) shows how.

If the organization stops sharing a model or a provider with a project, the project's models through it show **Needs attention** and requests to them fail, until it's shared again. A policy, **Turn off projects' own providers**, keeps projects from adding keys of their own, and stops the models through those they added: such a project uses only what the organization shares with it.

## Calling a model [#calling-a-model]

Each project has its own base URLs, one for each provider's API:

| API                                   | Base URL                                                   |
| ------------------------------------- | ---------------------------------------------------------- |
| Anthropic Messages                    | `https://api.kosem.ai/v1/projects/{project}/llm/anthropic` |
| OpenAI Chat Completions and Responses | `https://api.kosem.ai/v1/projects/{project}/llm/openai/v1` |

Point a provider's SDK at one of them, with a kosem API key in place of the provider's, and send a model's alias as `model`. [Call models from your code](/guides/llm-gateway/call-models-from-code) shows how, and [LLM Gateway endpoints](/reference/llm-gateway-endpoints) lists every endpoint and header.

- **The gateway changes as little as it can.** When the model's provider speaks the API you called, kosem replaces only `model` with the provider's name for it. It streams the answer back with the provider's status, request ID and rate limit headers, which describe your organization's account at the provider.
- **Every model takes every API.** When its provider speaks another, kosem translates the request and the answer, and adds the header `Kosem-Translated: true`. So code written for Anthropic's SDK can call a model that OpenAI serves. Translation carries text, files, function tools and their results, and the common sampling and output settings. It refuses what it can't carry across, such as a provider's own hosted tools or a stored Responses conversation, rather than drop it.
- **Every answer names its request.** The header `Kosem-Request-Id` holds the ID that the project's request log shows it under.

## What kosem records [#what-kosem-records]

For each request that the gateway sends to a provider, kosem records who sent it, the model, how it ended, how long it took and the tokens the provider reported. It never records a prompt or an answer.

- **Usage** sums requests and tokens by day, project, model and caller. The organization's and each project's LLM Gateway pages show it.
- **The request log** keeps each request for your plan's retention window ([Limits and quotas](/reference/limits#llm-gateway)), with its error if it failed. It also keeps the requests that kosem refused once it had found the model.

Your provider bills your organization for the tokens, as it would without kosem. Agent model calls are free on kosem; direct LLM Gateway API calls that the provider answers are billed past your plan's monthly allowance: [Limits and quotas](/reference/limits#monthly-allowances-and-usage-prices) has the numbers.

## Why it works this way [#why-it-works-this-way]

- **Your keys, your accounts.** Requests go to your accounts at the providers, so your contracts, data terms and rate limits with them apply, and the providers' own consoles show every request.
- **Aliases keep code apart from providers.** Code names `fast` or `smart`, and the project decides what that means today.
- **Shared models or shared keys.** Sharing a model gives a project that model alone, on the organization's terms. Sharing a provider gives it the key, and every model the key reaches. The organization chooses for each project, and the policy keeps projects from bringing keys of their own.

## Who can do what [#who-can-do-what]

| To                                                                          | You need                                                            |
| --------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| Add the organization's providers and models, and change them                | Owner or LLM Gateway Admin on the organization                      |
| Choose the projects the organization's providers and models are shared with | Owner or LLM Gateway Admin on the organization                      |
| Turn off projects' own providers                                            | Owner or Policy Admin on the organization                           |
| Add a project's own providers and its models, or remove them                | Owner or LLM Gateway Admin on the project                           |
| Call a project's models                                                     | Owner, Editor, LLM Gateway Admin or LLM Gateway User on the project |
| See usage and the request log                                               | Any basic role, LLM Gateway Admin or LLM Gateway User               |

## Related [#related]

- [Add models to your organization](/guides/llm-gateway/add-models)
- [Add models to a project](/guides/llm-gateway/add-models-to-a-project)
- [Call models from your code](/guides/llm-gateway/call-models-from-code)
- [See what your models were used for](/guides/llm-gateway/see-usage-and-requests)
- [Limits and quotas](/reference/limits)
