Run Quivr's RAG brain on a custom OpenAI-compatible endpoint.

Quick answer: llm_base_url in LLMEndpointConfig.

Current Quivr is quivr-core, a Python RAG library, and its LLM wiring is explicit. LLMEndpointConfig carries supplier (openai by default), model, llm_base_url, and llm_api_key; LLMEndpoint.from_config() builds the actual client from those fields, and for the openai supplier that client is LangChain's ChatOpenAI constructed with your base URL. Set llm_base_url to https://api.apisrouter.com/v1, set model to any catalog id, and hand the endpoint to your Brain. The key can come from the config field or the environment: when llm_api_key is not set, quivr-core resolves it from an environment variable named after the supplier, which for the openai supplier is OPENAI_API_KEY. Both paths are upstream behavior, readable in quivr_core/rag/entities/config.py and quivr_core/llm/llm_endpoint.py.

from quivr_core.llm import LLMEndpoint
from quivr_core.rag.entities.config import (
    DefaultModelSuppliers, LLMEndpointConfig)

llm = LLMEndpoint.from_config(LLMEndpointConfig(
    supplier=DefaultModelSuppliers.OPENAI,
    model="claude-sonnet-4-6",          # any catalog id
    llm_base_url="https://api.apisrouter.com/v1",
    llm_api_key=os.environ["APISROUTER_API_KEY"],
))

What Quivr is now, and where the LLM slot sits.

Quivr (QuivrHQ on GitHub, roughly 39K stars) began as a full second-brain application and pivoted into quivr-core: an opinionated RAG library you embed in your own product. You feed it files, it parses and chunks them, embeds the chunks into a vector store (FAISS by default, PGVector supported), and answers questions over them through a configurable retrieval workflow. The Brain object is the unit: Brain.from_files() ingests, brain.ask() retrieves and generates. Generation is the only step that needs a chat model. The retrieval workflow assembles context from your documents, and the LLMEndpoint you passed writes the grounded answer. That endpoint is built once from LLMEndpointConfig, so the base URL decision is made at construction time and applies to every ask() on that brain. Because ChatOpenAI forwards the model field as a plain string over /v1/chat/completions, the id can be Claude, DeepSeek, GPT, or Gemini when the endpoint behind llm_base_url serves them. One honest note on project status: the repository has been quiet since mid-2025, so treat quivr-core as a stable library rather than a fast-moving one. The config surface described here matches the latest main branch, and the quiet history means it is unlikely to shift under you; it also means old tutorials describing the retired full-stack app (backend .env files, a hosted frontend) no longer match the code.

Full setup: a brain with a gateway-routed LLM.

The complete pattern passes the configured LLMEndpoint into Brain.from_files. Everything else about the brain (parsing, chunking, the FAISS store, the retrieval workflow) is independent of the LLM endpoint and keeps its defaults. Mind the embedder. If you do not pass one, quivr-core builds LangChain's OpenAIEmbeddings with its own defaults, which authenticates with OPENAI_API_KEY and targets the stock OpenAI endpoint. That is a separate client from the chat LLM: routing generation through the gateway does not move it. Pass your own embedder (a local sentence-transformers wrapper, or any LangChain Embeddings instance you configure) if you do not want the embedding half depending on an OpenAI account.

import os
from quivr_core import Brain
from quivr_core.llm import LLMEndpoint
from quivr_core.rag.entities.config import (
    DefaultModelSuppliers, LLMEndpointConfig)

llm = LLMEndpoint.from_config(LLMEndpointConfig(
    supplier=DefaultModelSuppliers.OPENAI,
    model="claude-sonnet-4-6",
    llm_base_url="https://api.apisrouter.com/v1",
    llm_api_key=os.environ["APISROUTER_API_KEY"],
    max_output_tokens=2048,
    temperature=0.3,
))

brain = Brain.from_files(
    name="team-docs",
    file_paths=["handbook.pdf", "runbook.md"],
    llm=llm,
    # embedder=...  # separate component; see note above
)

print(brain.ask("What is the on-call escalation policy?").answer)

Choosing a generation model for RAG answers.

Comparing candidates is a construction-time change: build two LLMEndpoints against the same base URL, two brains over the same files, and diff the answers on a fixed question set. The per-key usage log prices each candidate run, so quality-per-token is measured rather than argued.

• RAG generation is input-heavy: retrieved chunks dominate the prompt. Per-input-token price sets the cost of an answer, which is why a fast id often halves the bill without touching retrieval quality.
• claude-sonnet-4-6 is the dependable default for grounded answers that respect the retrieved context and decline cleanly when the documents do not contain the answer.
• High-volume embedded products (Quivr's stated use case) run well on claude-haiku-4-5-20251001, deepseek-v4-flash, or gemini-3.5-flash for the everyday question mix.
• max_context_tokens in the same config governs how much retrieved context the pipeline packs; raising it pairs naturally with long-context ids and raises input spend proportionally.
• Unknown model prefixes fall back to a generic tokenizer for budgeting, which is cosmetic; the request itself carries your id unchanged to the endpoint.

Pay-as-you-go · transparent per-model pricing

Selected models are priced below official list prices. Exact input, output, cache, and per-request prices are shown for each model.

Corrections to common Quivr lore.

Guides in circulation describe surfaces Quivr no longer has, so it is worth stating what the current code actually does. quivr-core is LangChain-backed, not LiteLLM-backed. The supplier enum selects a LangChain chat class, and openai maps to ChatOpenAI with your llm_base_url. If a tutorial tells you to configure a LiteLLM proxy or api_base setting inside Quivr, it describes an older architecture; the current field is llm_base_url on LLMEndpointConfig. The full-stack app is retired. Instructions about a backend .env, Supabase setup, or an in-app model picker refer to the pre-pivot application, which is no longer what the repository ships. Configuration now happens in your Python code (or your own app around the library). The key env var is supplier-derived. For supplier openai it is OPENAI_API_KEY, even when the endpoint is not OpenAI. If you prefer not to overload that name, pass llm_api_key explicitly in the config, which takes precedence and keeps the environment clean. The embedder is separate. Generation routing does not move embeddings; the default embedder is OpenAIEmbeddings with its own credentials. Decide the two halves independently, and re-embedding an existing store is only needed if you change the embedding model itself.

Who routes quivr-core through a gateway.

• Product teams embedding RAG in their apps who want the generation model to be a config value, not a vendor commitment baked into the stack.
• Developers running many brains at different quality tiers: one key, one endpoint, model id per brain.
• Teams that want Claude-quality grounded answers behind an OpenAI-shaped config without adding a second SDK or provider account.
• Builders benchmarking generation models over a fixed corpus, where each candidate is one LLMEndpointConfig change.
• Developers without access to a given vendor's billing. Top-up based access with no card requirement removes the per-provider sign-up dependency.

Verify the endpoint and debug the first ask().

Confirm the gateway lists your model before ingesting anything; the model field must match a served id exactly. First-run failures are predictable. A warning that the API key for supplier openai is not set means neither llm_api_key nor OPENAI_API_KEY was visible when the config was constructed; the warning happens at construction, the failure at the first ask(). A 401 means the resolved key does not belong to the endpoint in llm_base_url. A model-not-found error is an id typo against /v1/models. And an embedding-related authentication error during Brain.from_files is the separate default embedder asking for its own OpenAI credentials, which no llm_base_url setting will fix; pass an embedder you control. Once answers flow, the APIsRouter console shows per-request model, token counts, and spend. For a library that packs retrieved chunks into every prompt, the tokens-per-answer number on your real corpus is the figure that should drive your model choice.

curl -s https://api.apisrouter.com/v1/models \
  -H "Authorization: Bearer $APISROUTER_API_KEY" | head -50

FAQ