AI API cost optimization: the complete guide to cutting your bill

Quick answer: what actually drives an AI API bill

A prototype answering a few hundred requests a day can run on pocket change, and the same workflow at production volume can turn into a five-figure monthly invoice without anyone changing the code. The gap is rarely the sticker price of the model itself. It is a handful of quiet defaults: the flagship model left as the default for every call, a system prompt that grew line by line over a year, no cache in front of repeat requests, and a rate that was never revisited after the first integration. Two pricing shapes cover almost everything a developer will meet. Chat and completion models bill per token, split into an input rate for what you send and a usually higher output rate for what comes back. A per-call image model such as GPT Image 2 charges a flat rate per generated image instead, independent of prompt length. For a typical text or agent workload, though, token pricing sets nearly all of the bill, so that is where the rest of this guide focuses.

Where the tokens actually go

None of this is exotic. It is the same arithmetic whether the workload is a support bot, a coding agent, or a batch labeling job: tokens in, tokens out, multiplied by a rate that depends entirely on which model answered. The next section prices one realistic month four different ways so the effect of each lever is visible instead of theoretical.

• System prompts and tool definitions ride along on every single call, so a bloated prompt is a recurring charge, not a one-time cost.
• Input tokens usually dominate a chat or agent workload: the running conversation, retrieved context, and tool output all count as input again on every follow-up call.
• Output tokens are billed at a meaningfully higher rate than input on nearly every provider, so an unbounded response costs more per token than the prompt that triggered it.
• A failed or timed-out request still consumes the tokens it already sent. The retry sends them again, so one flaky call can bill twice.
• Flagship reasoning models commonly carry three to five times the per-token rate of a mid-tier model doing the same classification or extraction task.

Worked example: the same 10 million tokens, priced four ways

Take a workload that sends 10 million tokens a month, split roughly three-quarters input to one-quarter output, a realistic ratio for a chat or agent product with moderate conversation length. The table below prices the same volume first with no optimization at all, then with three levers applied one at a time. Each stage keeps the workload's actual behavior unchanged; only the routing, prompt size, cache usage, and rate change. The order matters less than doing all four. Model tiering accounts for the largest single jump because it removes the flagship rate from most of the traffic. Trimming and caching then shrink the volume that gets billed at all. The rate cut in the last row is the only step that changes nothing about the application itself.

Why the bill surprises people every month

Each of these is fixable without a rewrite, and none of them require negotiating a better headline rate first. Fixing them in the order below is what produced the worked numbers above.

• Traffic outgrows the routing decisions made during the prototype, and "just call the best model" quietly becomes the production default.
• Nobody sets a max_tokens ceiling, so a model that should answer in fifty tokens occasionally writes six hundred, at the higher output rate.
• Retries double-bill by design: the timed-out attempt already spent its input tokens, and the retry spends them again.
• System prompts grow one edge case at a time, and every added line becomes a permanent charge on every future call, not just the one that needed it.
• Nobody looks at spend broken down by model or by feature, so the one workflow burning most of the budget stays invisible until finance asks about a number nobody can explain.

Comparison: what the same task costs on different model tiers

Model choice is the single biggest lever precisely because the spread between tiers is enormous for tasks that do not need a flagship model's reasoning depth. Classification, extraction, short-form drafting, and routing decisions rarely need the most expensive model on the list; they need a model that gets the answer right, cheaply, every time.

Six fixes that actually move the number

The first five fixes are provider-agnostic; they work identically whether the request lands on a vendor's own endpoint or a gateway in front of it. The sixth is different, since it is the only lever here that touches billing rather than behavior. APIsRouter is one such gateway, an OpenAI-compatible endpoint with pay-as-you-go billing and no subscription; global models there are priced about 20% below official list, and Chinese-origin models are priced below their own official rates. Layered on top of the first five stages, a rate cut like that is exactly what took the worked example from $36.00 down to $15.66 in the earlier table.

• Tier by task difficulty. Send classification, extraction, and short drafts to a budget model; reserve the flagship tier for reasoning that genuinely needs it.
• Cap max_tokens on every call. An unbounded response can cost more in output than the entire input side of the same request.
• Cache anything repeatable. Identical prompts, a static system prompt paired with short varying inputs, and classification with a small label set are the easiest wins.
• Trim the system prompt on a schedule, not just once. Move rarely-needed instructions into a lookup the model requests only when relevant, instead of resending them on every call.
• Batch what does not need a live answer. Nightly summarization, backfill labeling, and report generation do not need synchronous latency, and several providers price batch jobs lower than real-time calls.
• Lower the rate itself once the call pattern is efficient. Routing routine traffic through a discounted OpenAI-compatible endpoint changes nothing about the prompts or the model IDs above; it only changes what you pay per token.

Point an OpenAI-compatible client at the lower rate

None of the fixes above require touching the request format. An OpenAI-compatible client only needs a new base_url and a matching API key; the model IDs, the message shape, and the streaming behavior stay exactly the same as a direct integration. Point the client at https://api.apisrouter.com/v1 with any model ID from the catalog above, and run the smoke test below before switching a production workload over.

from openai import OpenAI

client = OpenAI(
    api_key="sk-your-key",
    base_url="https://api.apisrouter.com/v1",
)

response = client.chat.completions.create(
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Summarize this ticket in two sentences."}],
    max_tokens=200,
)
print(response.choices[0].message.content)

FAQ