Skip to Content
kosemdocs
GuidesLLM Gateway

Add a provider

Add your organization's or project's key or Google Cloud trust for Anthropic, OpenAI, Gemini, Vertex AI or a compatible server.

Add a provider, an API key for a model provider, so that the LLM Gateway can serve its models. Your organization's provider serves the models you add to the organization, which you share with projects, and the projects you share the provider itself with, which make models of their own through it: you need Owner or LLM Gateway Admin on the organization. A project's own provider serves only that project: you need Owner or LLM Gateway Admin on the project.

You need an API key from the provider, such as one created in Anthropic's or OpenAI's console. For a server that speaks a provider's API, such as a self-hosted model, you need its base URL too, and your organization needs the Pro, Team or Enterprise plan (Limits and quotas). For Google Vertex AI, kosem uses Google Cloud workload identity federation instead of an API key.

Add a provider

  1. In your organization, click LLM Gateway, then Providers.
  2. Click Add a provider, and select the provider. For a server that speaks a provider's API, select OpenAI-compatible Chat Completions, OpenAI-compatible Responses or Anthropic-compatible API, and enter its Base URL.
  3. Paste the provider's key into API key. kosem keeps it encrypted and never shows it again.
  4. Optionally, enter a Name (optional), such as Anthropic production, to tell this provider apart from others.
  5. Under Projects that can make models through it, select All projects or Selected projects, or keep No projects yet and share it later from its Sharing tab.
  6. Click Test to check that the provider takes the key without saving it, and then click Add provider.

kosem saves the provider only once its service accepts the key. It opens the provider on its Sharing tab. The projects it's shared with then add its models: Add models to a project. To give projects only some of its models instead, keep it shared with none, and add those models to your organization from its Models tab: Add models to your organization.

Add Google Vertex AI

Google Vertex AI uses workload identity federation, so kosem stores no key of yours.

  1. In your organization or project, click LLM Gateway, then Providers.
  2. Click Add a provider, select Google Vertex AI, optionally enter a name and click Continue.
  3. On the provider's Overview tab, enter your Google Cloud project and optional Location (global by default).
  4. Run the generated gcloud commands or Terraform in Google Cloud. They enable the Vertex AI API, create or reuse the kosem workload identity pool and create a service account with the Vertex AI User role (roles/aiplatform.user) that trusts only this provider's subject.
  5. Paste the Project ID, Location, Workload identity provider and Service account into the form and click Finish setup.

kosem verifies that Google Cloud accepts this provider's identity, refuses other kosem identities and lets the service account reach Vertex AI in your project before activating it.

Add a project's own provider

  1. In the project, click LLM Gateway, then Providers.
  2. Click Add a provider, and follow steps 2 to 4 and 6 above. A project's own provider isn't shared with other projects.

kosem opens the provider on its Models tab: Add models to a project. The Providers page lists the providers your organization shares with the project too. Your organization's Providers page lists every project's own provider under Added in projects, for those who can see providers across the organization.

If the page says a policy keeps projects from adding providers of their own, your organization turned on Turn off projects' own providers. Models through providers the project added before show Needs attention and stop serving requests until the policy is lifted.

Replace a key

Replace a provider's key before you revoke the old one at the provider, so that requests don't fail meanwhile.

  1. In your organization, click LLM Gateway, then Providers, then the provider.
  2. Open the menu beside Verify and click Replace key.
  3. Paste the new key and click Replace key.

kosem uses the new key within seconds. Models that go through the provider keep working.

Fix a provider that needs attention

When a provider's service refuses its key, as when someone revokes it, kosem marks the provider Needs attention and says what the service said. Requests to its models fail with 502 until it's fixed.

  1. Replace the key, as above, or fix what the service said, such as an account without credit.
  2. Click Verify to check the provider again.

Add a provider with the API

Set your API key (Call the API), your organization's ID and the provider's API key:

export KOSEM_API_KEY='kosem_sk_…'
export KOSEM_ORG='org_01m5r8w2k3e9vbt6g0d4x7qz1h'
export ANTHROPIC_API_KEY='sk-ant-…'

curl -s -X POST "https://api.kosem.ai/v1/organizations/$KOSEM_ORG/providers" \
  -H "Authorization: Bearer $KOSEM_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"type\": \"providerTypes/anthropic\", \"apiKey\": \"$ANTHROPIC_API_KEY\"}"

kosem answers with the provider, which shows only the key's last four characters:

{
  "name": "organizations/org_01m5r8w2k3e9vbt6g0d4x7qz1h/providers/prv_01m5raq8h2e4vbt6g0d4x7qz1j",
  "type": "providerTypes/anthropic",
  "displayName": "Anthropic",
  "keyHint": "…x7Qa",
  "availability": { "allProjects": false, "projects": [] },
  "state": "ACTIVE"
}

For a compatible server, the type is providerTypes/openai-compatible, providerTypes/openai-responses-compatible or providerTypes/anthropic-compatible, with its address in config, such as {"base_url": "https://llm.example.com/v1"}. A project's own provider is added the same way at /v1/projects/$KOSEM_PROJECT/providers, without availability.

On this page