Skip to Content
kosemdocs
GuidesLLM Gateway

Call models from your code

Add a model to your project, create an API key, and call the model with OpenAI's or Anthropic's SDK, or with curl.

Call your project's models from your own code with the providers' SDKs, through its LLM Gateway. To add a model to the project, you need Owner or LLM Gateway Admin on it. To create its API key, you need Owner on the project.

Add a model to your project

The project needs a model first, through its own provider or one your organization shares with it: Add models to a project shows how. The model shows among the project's, with its alias to copy.

Create an API key

  1. In your project's LLM Gateway, click API keys.
  2. Click Create API key, enter a name, such as Checkout service, and click Create API key.
  3. Copy the key, which starts with kosem_sk_. kosem shows it only once.
  4. Click I copied the key.

The key belongs to a new service account in the project with the LLM Gateway User role: it can call the project's models and see their usage, and nothing else.

Under the keys, Call a model shows the project's base URLs, and examples for OpenAI's and Anthropic's SDKs, Claude Code, Codex and curl, filled in with the project's own model aliases.

In your terminal, keep the key and your project's ID, which Settings › General shows, in variables:

export KOSEM_API_KEY='kosem_sk_…'
export KOSEM_PROJECT='prj_01m5r9dh9mf0ps98ktkap3562c'

Call a model

Send the model's alias as model. These examples call a model with the alias fast.

With OpenAI's SDK

import os

from openai import OpenAI

client = OpenAI(
    base_url=f"https://api.kosem.ai/v1/projects/{os.environ['KOSEM_PROJECT']}/llm/openai/v1",
    api_key=os.environ["KOSEM_API_KEY"],
)
response = client.chat.completions.create(
    model="fast",
    messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)

The same client calls the Responses API, with client.responses.create, and streams with stream: true.

With Anthropic's SDK

import os

from anthropic import Anthropic

client = Anthropic(
    base_url=f"https://api.kosem.ai/v1/projects/{os.environ['KOSEM_PROJECT']}/llm/anthropic",
    api_key=os.environ["KOSEM_API_KEY"],
)
message = client.messages.create(
    model="fast",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello"}],
)
print(message.content[0].text)

With curl

curl -s -X POST "https://api.kosem.ai/v1/projects/$KOSEM_PROJECT/llm/openai/v1/chat/completions" \
  -H "Authorization: Bearer $KOSEM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "fast", "messages": [{"role": "user", "content": "Hello"}]}'

The provider answers, through kosem:

{
  "id": "chatcmpl-AbC123",
  "object": "chat.completion",
  "model": "claude-haiku-4-5",
  "choices": [
    { "index": 0, "message": { "role": "assistant", "content": "Hello! How can I help?" }, "finish_reason": "stop" }
  ],
  "usage": { "prompt_tokens": 8, "completion_tokens": 9, "total_tokens": 17 }
}

Every model takes both SDKs. When the model's provider speaks another API than the one you called, kosem translates, and the answer has the header Kosem-Translated: true.

If a call fails

The gateway answers errors in the format of the API you called, so your SDK raises them as it would the provider's.

StatusWhat it meansWhat to do
401The API key is wrong, expired or deletedCreate a new key
404, model_not_foundThe project has no model by that alias, or it needs attentionCheck the alias under the project's LLM Gateway › Models, or add the model
403The key's service account can't call models, or a policy turns the LLM Gateway offAsk a project Owner
429The provider's rate limit for your organization's account, or a limit of kosem'sRetry later, as your SDK does, unless X-Should-Retry: false says not to
502The provider refused your organization's keyAsk an Owner or LLM Gateway Admin to replace the provider's key

Errors explains each in full. Each answer's Kosem-Request-Id header names the request in the project's request log.

On this page