Run Letta agents on an OpenAI-compatible endpoint.

Quick answer: two environment variables on the server.

Letta's documented path for OpenAI-compatible endpoints is environment configuration on the self-hosted server: set OPENAI_API_BASE to the endpoint URL and OPENAI_API_KEY to its key when starting the server, and Letta registers the models that endpoint serves. For APIsRouter the base is https://api.apisrouter.com/v1. There is no per-agent base-URL field in the UI; the endpoint is a server-level decision, which is why the environment is the surface that matters. One requirement is non-negotiable and worth reading before anything else: Letta's docs state that OpenAI-compatible endpoints must support function calling, because the agent loop is built on tool calls. An endpoint that only does plain chat completions cannot run a Letta agent at all. Catalog models on APIsRouter speak standard tool calling over /v1/chat/completions, which is the shape Letta expects.

docker run \
  -v ~/.letta/.persist/pgdata:/var/lib/postgresql/data \
  -p 8283:8283 \
  -e OPENAI_API_KEY="$APISROUTER_API_KEY" \
  -e OPENAI_API_BASE="https://api.apisrouter.com/v1" \
  letta/letta:latest

Why Letta leans on its model harder than a chat app.

Letta (letta-ai on GitHub, roughly 24K stars) grew out of the MemGPT research project and builds stateful agents: agents with persistent, self-editing memory that survives across sessions. Where a chat client sends your message and prints the reply, a Letta agent runs an inner loop on every interaction, reasoning about what it knows, calling memory tools to read and rewrite its own core memory and archival storage, and only then producing a response. That architecture has two consequences for endpoint routing. First, every step of the loop is a tool-calling request, which is why function calling is a hard requirement rather than a nice-to-have; a model that fumbles tool schemas does not degrade gracefully here, it breaks the agent's ability to remember. Second, request volume per interaction is higher than the conversation transcript suggests, because memory management fires alongside the visible reply. The model id serving all of this is a plain string to the endpoint, so with a multi-vendor gateway behind OPENAI_API_BASE, a Claude id can run the agent loop while a fast id serves lighter agents on the same server, each addressed by its handle.

The honest support state, straight from upstream.

Letta's own documentation says OpenAI proxy endpoints are not officially supported and that you are likely to encounter errors, recommending direct provider connections instead. That warning deserves to be quoted rather than buried, because most pages on this topic pretend it does not exist. What it means in practice is narrower than it sounds: Letta tests against first-party APIs, and an endpoint that deviates from OpenAI semantics, especially around tool calling, produces failures that upstream will not prioritize. An endpoint that genuinely implements the spec, tool calls included, runs fine, and that is precisely the compatibility bar a gateway lives or dies by. The support history also had one real bug worth knowing about. Through early 2026, models registered via OPENAI_API_BASE were auto-prefixed as an openai-proxy provider while agent creation validated against a shorter list of accepted prefixes, so proxy models registered but could not be used to create agents. The issue was closed with a fix in January 2026; if you run a pinned older server and agent creation rejects models the server clearly lists, that mismatch is what you are hitting, and upgrading is the fix. One more moving target: Letta's product surface has been shifting, and its docs currently steer new users toward newer deployment modes while noting the classic Docker image is no longer the actively maintained surface. The environment variables above are the documented mechanism for the self-hosted server; check the current docs for which server artifact upstream recommends the week you deploy.

# after the server is up, list models Letta knows about
curl -s http://localhost:8283/v1/models/ | head -50
# use the handle exactly as listed when creating agents

Choosing models for stateful agents.

The evaluation that matters is loop fidelity: create a test agent, have a conversation that forces memory updates, then read the agent's core memory and verify it actually changed. A model can write charming replies and still fail the memory contract, and only the loop test catches that.

• Memory editing is structured tool work. claude-sonnet-4-6 and gpt-5.5 handle the rewrite-your-own-memory loop reliably, which is the core competence a Letta agent needs.
• Long-lived agents accumulate context. Models that stay coherent deep into a context window matter more here than in stateless chat, which is where claude-opus-4-7 earns its slot for high-stakes assistants.
• Fleets of lightweight agents, one per user or per task, are volume workloads. claude-haiku-4-5-20251001 keeps per-agent cost flat while still doing competent tool calls.
• deepseek-v4-pro is worth testing for agents that mix reasoning with bilingual traffic; the tool-calling requirement is the gate, so test the loop, not just the prose.
• Whatever you pick, pick per agent. The server registers the whole catalog, and each agent binds to a handle, so a memory-heavy concierge and a disposable task agent can run different ids side by side.

Pay-as-you-go · transparent per-model pricing

Selected models are priced below official list prices. Exact input, output, cache, and per-request prices are shown for each model.

Failure modes specific to Letta.

Agent creation rejecting a model the server lists is the historical prefix bug. Models registered through a proxy carried a provider prefix that agent creation refused to accept on affected versions. The fix landed in January 2026; on current releases the handle shown in the model listing is the handle that works. If you are pinned to an older image, this is the strongest single reason to upgrade before debugging anything else. An agent that replies but never remembers is a tool-calling failure. Either the endpoint does not implement function calling, or the model behind the id handles tool schemas poorly. The symptom is conversations that work while core memory never updates. Test the same agent on claude-sonnet-4-6 to separate endpoint problems from model problems. Environment variables set in the wrong place are the Docker classic: OPENAI_API_BASE exported in your shell does nothing for a container started without the -e flags. The variables must reach the server process itself. And because the endpoint is server-level, remember the blast radius: changing OPENAI_API_BASE moves every agent on that server. There is no per-agent endpoint override, so one server per gateway is the clean topology, with model choice per agent doing the differentiation.

Who routes Letta through a gateway.

• Builders of persistent assistants who want Claude-quality memory editing without a separate vendor account, key, and billing surface for each model they try.
• Teams running agent fleets where each user gets an agent, and per-key usage tracking turns the memory layer's real cost into a readable report.
• Researchers comparing how models handle self-editing memory, where each candidate is a handle change on a test agent rather than a provider migration.
• Self-hosters in environments where direct vendor API access is blocked and a single gateway endpoint is what the network policy allows.
• Developers without access to a given vendor's billing. Top-up based access with no card requirement removes the per-provider sign-up dependency.

Verify the endpoint and debug the first agent.

Verify the gateway before the server: list models with the key, and run one chat completion with a tool definition attached, because tool calling is the capability Letta actually depends on. If the tool-call round trip works in curl, the endpoint half is proven. Then start the server with the two variables and read its model listing. Models appearing there proves registration; an agent successfully created from a listed handle proves the prefix path; a conversation that updates core memory proves the loop end to end. Debug in that order, because each stage has a distinct failure set: the env vars, the server version, and the model's tool competence respectively. Once agents run, the APIsRouter console shows per-request model, token counts, and spend. Stateful agents bill more per interaction than their transcripts suggest, since memory management runs behind every reply, and the usage log is where that hidden multiplier becomes a number you can budget.

curl -s https://api.apisrouter.com/v1/chat/completions \
  -H "Authorization: Bearer $APISROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-4-6",
       "messages":[{"role":"user","content":"What is 2+3?"}],
       "tools":[{"type":"function","function":{
         "name":"calc","description":"add numbers",
         "parameters":{"type":"object","properties":{
           "a":{"type":"number"},"b":{"type":"number"}}}}}]}'

FAQ