Run AnythingLLM on any model via Generic OpenAI.

Quick answer: five fields in LLM Preference.

Open AnythingLLM's settings, go to the LLM section under AI Providers (the screen historically titled LLM Preference), and search the provider list for Generic OpenAI. Selecting it exposes five fields: Base URL, API Key, Chat Model Name, Token Context Window, and Max Tokens. Fill them as follows: Base URL https://api.apisrouter.com/v1 (the /v1 belongs in this field), your key, and the exact catalog id in Chat Model Name, for example claude-sonnet-4-6. Then the two numbers: Token Context Window is the model's total window and Max Tokens caps a single response, so take both from the model's documentation rather than guessing. Save, and every workspace that follows the system default now chats through the gateway. The docs flag this provider as developer-focused precisely because it trusts your inputs; that is not a warning against using it, just a statement that the five fields are a contract.

Base URL:              https://api.apisrouter.com/v1
API Key:               sk-YOUR-APISROUTER-KEY
Chat Model Name:       claude-sonnet-4-6
Token Context Window:  200000
Max Tokens:            8192

What the Generic OpenAI provider actually drives.

AnythingLLM (Mintplex Labs on GitHub, roughly 63K stars) is the all-in-one private document assistant: workspaces that embed your files, chat grounded in retrieved context, agents with tool use, available as a desktop app and a self-hosted server. The LLM preference configured above is the default brain for all of it. Requests leave as standard chat completions against your Base URL with the Chat Model Name as the model string, so vendor does not matter: a Claude id is as valid as a GPT one, and switching means editing one field. Workspaces can also override the system default with their own provider and model settings, which is how one deployment runs claude-sonnet-4-6 for the legal workspace and gpt-5.5 for engineering without either knowing about the other. The two token fields deserve respect because AnythingLLM uses them to budget context. The context window value decides how much retrieved document text and chat history gets packed into each request; understate it and your carefully embedded documents get trimmed out of their own answers, overstate it and requests bounce off the model's real limit. This is the field pair behind most "RAG feels dumb" reports on generic endpoints.

Embeddings are a separate decision.

AnythingLLM splits the LLM preference from the embedding preference, and the split is load-bearing. The chat model answers questions; the embedder turns your documents into vectors at upload time, and those vectors persist. Changing chat models is free, any day, per workspace. Changing embedders invalidates the geometry of everything already embedded and means re-embedding your documents. The practical setup: AnythingLLM ships a built-in local embedder that works offline and costs nothing, and many deployments simply keep it. If you point the embedding preference at a remote endpoint instead, confirm the specific embedding id is served there before uploading your document base, and treat that choice as married to the vector store. The freedom this buys on the chat side is the point of the architecture: with embeddings settled, the Generic OpenAI chat model is a low-stakes dial. Run deepseek-v4-pro for a month of heavy summarization, switch the field to claude-sonnet-4-6 for a quarter of client work, and nothing about your documents needs to move.

curl -s https://api.apisrouter.com/v1/models \
  -H "Authorization: Bearer $APISROUTER_API_KEY" | head -50
# Chat Model Name must match one of these ids exactly

Choosing models for document work.

Every id bills through the same key, so the comparison loop is a settings edit: same workspace, same documents, two weeks on each candidate, and the per-model spend in the APIsRouter console next to your own judgment of the answers.

• Grounded QA over retrieved chunks is input-heavy: claude-haiku-4-5-20251001 and gemini-3.5-flash answer citation-following questions well at volume prices.
• claude-sonnet-4-6 earns the default where answers ship to humans unedited: contract review, report drafting, anything with consequences.
• deepseek-v4-pro is the long-context workhorse for workspaces built on big documents, where the window and input pricing dominate the experience.
• Agent workspaces need dependable tool calling; start agents on claude-sonnet-4-6, then test whether a lighter id holds up on your actual flows.
• Use per-workspace overrides as policy: the default stays fast, and the expensive id lives only in the workspaces whose work justifies it.

Pay-as-you-go · transparent per-model pricing

Selected models are priced below official list prices. Exact input, output, cache, and per-request prices are shown for each model.

Failure modes specific to AnythingLLM.

The token fields cause the subtle failures. A context window set too low silently truncates retrieved context, which presents as a model that ignores your documents; set the documented window for the id in Chat Model Name. Max Tokens set too high for the model produces hard errors on long answers; set it to what the model actually allows for output. Hard errors are more honest. Authentication failures are the API Key field. Model-not-found is Chat Model Name drifting from the catalog spelling; the field is free text and the /v1/models listing is the source of truth. Connection errors on the desktop app usually mean a proxy or firewall between the app and the endpoint; on Docker deployments, remember it is the container that must reach the Base URL, not your browser. If chat works but a workspace behaves differently than expected, check its override settings; workspace-level provider settings win over the system default, and a forgotten override is the classic "same question, different model" mystery. The docs' developer-focused framing summarizes all of the above: the provider does exactly what its five fields say, no more and no less.

Who routes AnythingLLM through a gateway.

• Teams running private document assistants who want frontier answer quality without a vendor account per model family.
• Desktop users pointing their personal document base at Claude or GPT ids on a prepaid balance, no card required to start.
• Self-hosters who keep the built-in local embedder and treat remote chat as the one metered lane, read per model on one usage log.
• Deployments segmented by workspace, where per-workspace overrides plus per-key metering give each team its own model and its own bill.
• Builders comparing answer models over a fixed document base, where each candidate is one settings edit rather than a migration.

Verify the endpoint and debug the first workspace chat.

Curl the models listing and one chat completion first, using the exact id you intend to put in Chat Model Name. With both passing, the gateway half is proven and every remaining symptom lives in the five fields. Then save the provider settings and ask one question in a workspace with a document you know well. A grounded, citing answer means the whole pipeline works. An answer that ignores the document points at the context window value or at the workspace's own retrieval settings. Hard errors map cleanly: key, id spelling, Base URL shape. Once chats flow, the APIsRouter console shows per-request model, token counts, and spend. Document QA multiplies input tokens through retrieval, and the usage log is where you learn what your document base actually costs to query, per model, per day, and per workspace key if you split them.

curl -s https://api.apisrouter.com/v1/chat/completions \
  -H "Authorization: Bearer $APISROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-4-6",
       "messages":[{"role":"user","content":"ping"}]}'

FAQ