Run Onyx on a custom OpenAI-compatible LLM provider.

Quick answer: Add Custom LLM Provider in the admin panel.

Onyx's documentation is explicit that a custom provider works as long as it exposes OpenAI-compatible endpoints, and its example Base URL shape is exactly a gateway-style https://yourprovider.com/v1. The flow: open the Admin Panel from your profile icon, go to Configuration, then Language Models, and choose Add Custom LLM Provider. Four decisions matter in that form. Display Name is cosmetic. Provider Name must match a LiteLLM provider key, because Onyx routes model calls through LiteLLM underneath; for an OpenAI-compatible gateway that is openai. Base URL is the gateway endpoint including the /v1 suffix. And the Model Configurations section is where you register each model id you want available, spelled exactly as the catalog serves it. Save, pick a default, and chats route through the gateway immediately.

Admin Panel -> Configuration -> Language Models
  -> Add Custom LLM Provider

Display Name:   APIsRouter
Provider Name:  openai            (LiteLLM provider key)
Base URL:       https://api.apisrouter.com/v1
API Key:        sk-YOUR-APISROUTER-KEY
Model Configurations:
  claude-sonnet-4-6
  claude-haiku-4-5-20251001
  deepseek-v4-pro

Where the LLM sits in Onyx's architecture.

Onyx (onyx-dot-app on GitHub, roughly 31K stars, formerly Danswer) is an open-source AI platform for company knowledge: it indexes sources like Slack, Google Drive, Confluence, and dozens of other connectors, then answers questions over them through a chat UI, assistants, and agent workflows. It is one of the most deployed self-hosted enterprise search stacks, which is exactly why its LLM bill deserves a routing decision rather than a default. The pipeline splits cleanly in two. Indexing and retrieval, including document embedding and reranking, run on Onyx's own model server with local models by default; none of that touches your LLM provider. Answer generation is the other half: once retrieval assembles the relevant passages, an LLM reads them and writes the grounded response, and that call goes through LiteLLM to whichever provider the admin configured. The custom provider flow swaps the destination of exactly this half. Because LiteLLM forwards the model id as a plain string to an openai-type provider, the ids you register in Model Configurations can be anything the endpoint behind the Base URL serves: Claude for careful grounded answers, DeepSeek for volume, Gemini for very long source contexts. Different assistants can default to different models, so a support assistant and an engineering assistant can ride different price points through the same provider entry.

Full setup, and what stays untouched.

The provider form is the whole integration; there is no config file to edit or container to rebuild for it. After saving, set the default model for the workspace, and optionally override the model per assistant where you want different quality tiers. What deliberately stays untouched: connectors keep their own credentials, the index is unaffected, and the embedding model configured for search does not move. That separation is worth spelling out because it makes this a low-risk change. If the gateway misbehaved, search and sources would still work; only answer generation would error, and switching the default back to a previous provider is one dropdown. For teams that automate deployments, the same provider definition can be seeded through Onyx's API rather than clicked through the UI, but the admin panel path is the documented and stable surface, and one-time setup rarely justifies more.

# confirm the gateway lists the ids you plan to register
curl -s https://api.apisrouter.com/v1/models \
  -H "Authorization: Bearer $APISROUTER_API_KEY" | head -50

# confirm a chat completion works end to end
curl -s https://api.apisrouter.com/v1/chat/completions \
  -H "Authorization: Bearer $APISROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-4-6",
       "messages":[{"role":"user","content":"ping"}]}'

Choosing models for grounded enterprise answers.

Model evaluation inside Onyx is unusually concrete: ask the same question against the same connectors with two different assistant defaults and compare which answer cites the right passages. The per-key usage log prices both candidates on your real question mix.

• Grounded answering is input-heavy: the model reads retrieved passages that dwarf the answer it writes. Per-input-token price therefore sets your cost per question more than output price does.
• claude-sonnet-4-6 is a strong workspace default: disciplined about staying inside the retrieved sources and resistant to inventing policy that is not in the documents.
• High-traffic assistants (IT helpdesk, HR FAQ) run well on claude-haiku-4-5-20251001 or deepseek-v4-pro, where volume pricing keeps per-seat cost predictable.
• Long source documents favor long-context ids; gemini-3.1-pro-preview is worth testing for assistants that pull large design docs or contracts into context.
• Register several ids in one provider entry and assign them per assistant. Quality tiers per team beat one global compromise model.

Pay-as-you-go · transparent per-model pricing

Selected models are priced below official list prices. Exact input, output, cache, and per-request prices are shown for each model.

Failure modes specific to Onyx.

Provider Name is not a free-text label. It must match a LiteLLM provider key, and for a gateway that key is openai. A made-up name fails at request time with a LiteLLM provider error even though the form saved fine. The Base URL wants the /v1 suffix. Onyx's own documentation shows endpoint shapes ending in /v1; without it, the chat-completions path resolves wrong and requests 404 at the gateway. Model ids live in Model Configurations. A model you never registered there cannot be selected as a default, and a typo in a registered id surfaces as a model-not-found error on first use, not at save time. The gateway's /v1/models listing is the authoritative spelling. If your admin UI is missing the Base URL field on the custom-models form, you have hit a reported UI regression in some 2026 releases rather than a missing feature; upgrading restores the field. And remember which half you moved: if search results look wrong or stale, that is indexing and connectors, which never touch the custom provider. Only generated answers route through the gateway.

Who routes Onyx through a gateway.

• Self-hosted teams replacing per-vendor accounts with one endpoint, one key, and per-key usage that maps cleanly to a workspace or department.
• Enterprises that standardized on Onyx for internal search and want Claude-quality grounded answers without a separate Anthropic billing relationship.
• Platform teams running several assistants at different quality tiers, priced per assistant through registered model ids on one provider.
• Evaluators comparing answer quality across model families on identical corpora, where each candidate is a registered id rather than a new provider integration.
• Developers without access to a given vendor's billing. Top-up based access with no card requirement removes the per-provider sign-up dependency.

Verify the endpoint and debug the first chat.

The two curl checks above cover the gateway half before you touch the form: the ids you plan to register must appear in /v1/models, and a direct chat completion should answer. Inside Onyx, failures localize quickly. A provider error naming LiteLLM means the Provider Name is not a valid key; set it to openai. An authentication error on first chat means the API Key does not belong to the endpoint in Base URL. A model-not-found error is an id mismatch between Model Configurations and the catalog. Answers that generate but ignore your documents are a retrieval or connector issue, upstream of the LLM provider entirely. Once chats flow, the APIsRouter console shows per-request model, token counts, and spend. For a workspace tool where every question carries retrieved context, that per-question token number is the honest basis for capacity planning, and one key per workspace turns the usage log into a department-level cost report.

FAQ