How to reduce AI API costs without cutting quality

Quick answer: match the model to the task, then fix the tokens.

Two numbers explain most AI API bills: how many tokens a request uses, and what the model charges per token. Developers spend a lot of effort on the first number, trimming prompts, capping context, shortening instructions, and comparatively little on the second, even though model choice alone can move a bill by an order of magnitude. A request that costs a few cents on a flagship reasoning model can cost a fraction of a cent on a model built for straightforward classification or extraction, with no visible difference in the answer the user actually reads. The practical order of operations is: pick the cheapest model that still clears the task's quality bar, put a hard ceiling on output tokens so no single call runs away, cache the requests that repeat, and only then fine-tune prompt wording. Going straight to prompt-shortening while every request still hits the most expensive model in the catalog leaves the biggest lever completely untouched.

Where the money actually goes.

Input tokens tend to dominate for chat, retrieval-augmented, and agentic workloads, because the same system prompt and a growing history get resent on every call. Output tokens dominate for generation-heavy tasks like long-form drafting or summarizing into a large report. Either way, the bill is a multiplication of tokens by rate by volume, and the rate term is the one most teams never revisit after picking a default model on day one.

• Input tokens: the system prompt, any retrieved documents or knowledge-base snippets, and prior turns in a conversation, all billed again on every call that includes them.
• Output tokens: the model's own reply, which on most models costs several times more per token than input on that same model.
• Model rate: the price per token itself, which varies far more between models than most single-request tweaks can ever recover.
• Request volume: the plain number of calls a feature makes per day, which multiplies whatever the first three numbers add up to.

A worked example: one support chatbot, three models.

Take a support chatbot handling 50,000 conversations a month. Each conversation averages about 3,000 input tokens once the system prompt, a couple of retrieved FAQ snippets, and the recent turns are included, plus roughly 400 output tokens for the reply. That works out to 150 million input tokens and 20 million output tokens a month, before anything about the prompt itself changes. The table below prices that exact workload on three models from the same catalog. The flagship reasoning model is a common default for teams that started building on the most capable option and never revisited the choice; the mid-tier model is a reasonable everyday driver; the budget model is what most of those 50,000 conversations, being routine FAQ lookups, actually need.

Why the bill surprises people.

None of these require a redesign to fix. A max_tokens ceiling and a per-task model map catch most of the damage; the rest is a matter of checking what a request actually costs against what a pricing page says it should cost.

• No hard output cap: without max_tokens set, a model can generate far more than a task needed, and every extra token is billed.
• Retries double-count tokens: a client that retries after a timeout resends the same input and pays for it twice, sometimes before the first response even lands.
• One model for everything: routing a one-line classification and a multi-file refactor through the same flagship model, because that's what got wired up first and nobody revisited it.
• Context that only grows: chat history, tool output, and retrieved documents accumulate in the request and rarely get trimmed mid-conversation.
• Sticker price versus delivered price: the official per-token list price is the number developers memorize, but it is not always the number actually billed once a request goes through a routing layer.

What the same request costs across the catalog.

The table below lines up catalog price against official direct-provider list price for the same seven models, cheapest to priciest. Global models here list 20% below their official rate; Chinese-developed models undercut their own official rate too, by a smaller but still real margin. The spread between the top row and the bottom row is why picking a cheaper model outperforms most token-level tricks on its own.

Fixes that actually move the bill.

The routing ladder below makes the first fix concrete: five tiers spanning the catalog, each mapped to the kind of task it should actually handle.

• Route by task, not habit. Keep a flagship model for calls where a wrong answer is expensive, and move routine classification, extraction, and templated replies down to a budget model.
• Set a hard max_tokens on every call. This is the single most common runaway-cost bug, and it takes one line to fix.
• Cache deterministic and templated queries. A chatbot answering the same handful of FAQs hundreds of times a day should hit the model once and serve the rest from cache.
• Trim the system prompt and retrieved context down to what the task needs. Every token sitting in a request is billed again on every call that includes it.
• Batch non-interactive work. Nightly summarization or bulk classification can run through a single cheap model at whatever pace a rate limit allows, instead of firing one request per item at interactive latency.
• Lower the per-token rate itself. Pointing requests at a discounted OpenAI-compatible endpoint instead of billing at direct list prices is the one lever that costs no engineering time; a gateway such as APIsRouter lists global models 20% below official rates and Chinese-developed models below their own rates, with pay-as-you-go billing and no subscription.

Config example: route by tier and verify the price.

None of this requires switching frameworks. Any OpenAI-compatible client works once the base URL points at the catalog above; the model parameter is the only thing that changes per tier, and a hard max_tokens travels with every call regardless of which tier handled it. Verify a model's real per-call cost with a single curl request before wiring a router into production traffic:

from openai import OpenAI

client = OpenAI(
    api_key="sk-APIsRouter-...",
    base_url="https://api.apisrouter.com/v1",
)

# Match request complexity to the cheapest model that clears the bar
MODEL_BY_TIER = {
    "bulk": "deepseek-v4-flash",     # classification, extraction, high-volume chat
    "routine": "glm-5",              # everyday agent steps, templated replies
    "default": "claude-sonnet-4-6",  # coding, multi-step agents, careful writing
    "hard": "claude-opus-4-7",       # architecture, security review, high-stakes calls
}

def call(tier: str, messages: list, max_output_tokens: int = 400):
    model = MODEL_BY_TIER.get(tier, "glm-5")
    response = client.chat.completions.create(
        model=model,
        messages=messages,
        max_tokens=max_output_tokens,  # hard cap, no runaway generation
    )
    return response.choices[0].message.content, response.usage

reply, usage = call("bulk", [{"role": "user", "content": "Classify this ticket: refund request"}])
print(usage.prompt_tokens, usage.completion_tokens)

FAQ