Model routing for margin protection: stop paying flagship rates for utility work

Quick answer: route by task value, not by model loyalty.

If you charge per seat and pay per token, your gross margin is a race between two curves that have nothing to do with each other. Revenue per user is a flat line. Inference cost per user rises with every feature you ship and every habit your users build. The fix is not a better prompt or a quarterly renegotiation. It is routing: classify every AI touchpoint in your product by the value it creates, then pin each class to the cheapest model that passes your quality bar. The pattern that falls out is almost always the same. A small share of requests, the hero feature that justifies the subscription, deserves a flagship model. The long tail of utility calls does not: titles, tags, summaries, extraction, autocomplete, and most chat turns run fine on models priced well over 20x cheaper on input. Users judge the hero output. Nobody churns over which model wrote a list title. There are three levers on an inference bill: the mix (which model handles which request), the volume (tokens per request and requests per user), and the rate (what you pay per million tokens). Routing is the mix lever, and it is the one that moves COGS the most without touching the product surface or asking a vendor for anything.

Where per-user COGS actually goes.

Write the unit economics down before touching the router. Monthly inference COGS per active user is the sum, across features, of request count times average tokens times the per-token rate of whatever model answers. Most teams track the total bill and none of the terms, which is why the number only gets attention after it has eaten the margin. The request mix is the term with the leverage. A typical AI SaaS serves a few dozen high-value generations per user per month and hundreds of small utility calls around them. When everything defaults to the flagship, those utility calls are billed at rates built for the hard problem. A flagship at $3 per million input tokens against a budget model at $0.14 is a gap of over 20x on the exact traffic where the quality difference is hardest to detect. Input tokens make it worse. Chat-style features re-send the system prompt and conversation history on every turn, so input volume dwarfs output in almost every production trace. That means the rate that matters most is the input rate, and input rates are precisely where budget models undercut flagships hardest. Then there is the tail. Usage per seat is not normally distributed: the top decile of users routinely drives around half of total token spend while paying the same subscription as everyone else. A flagship-only stack hands your heaviest users the power to erase their own contribution margin, and a free tier on flagship defaults is pure COGS with no ceiling at all.

Worked example: one request mix, two bills.

Here is a concrete profile for a writing-assistant style product: one hero generation feature, a refinement chat, background summaries, and a pile of classification calls for titles and tags. Volumes are per active user per month, priced at official list rates so the comparison is vendor-neutral. Flagship-only means every row runs on claude-sonnet-4-6 at $3 input and $15 output per million tokens. Routed keeps the hero feature on the flagship and moves everything else down the ladder to the cheapest model that passes evals for that task. Two things make the routed column conservative. It ignores prompt caching, which discounts repeated prefixes on supported models, and it assumes every chat turn needs deepseek-v4-pro when a share of them are one-line edits a flash-class model would clear. The bigger point is structural: the tasks whose volume scales with user enthusiasm, chat turns and classification calls, are exactly the rows that moved to budget rates, so usage growth stops dragging COGS the way it used to.

Why AI SaaS margins erode faster than the forecast.

None of these leaks is fixed by a discount alone. A lower rate scales the whole bill down but leaves the structure intact: the free tier is still uncapped, the utility traffic still rides the flagship, and the heavy tail still owns your margin. Routing fixes the structure; the rate lever then multiplies whatever the routing saved. At a $29 seat, the flagship-only bill above is 17% of revenue on inference alone, before infra, support, and payment fees. The routed bill is 4.5%. For a p95 user running five times the median volume, flagship-only inference reaches about 85% of that seat's revenue, while the routed stack holds it near 23%. That difference is the gap between a power user you celebrate and one you quietly hope churns.

• Seat pricing is flat, usage is not. Power users pay the same subscription as everyone else while driving a disproportionate share of token spend, and they are also the users you least want to throttle.
• Input tokens dominate chat features. Every turn re-sends the system prompt and the full history, so a long conversation keeps billing its own past, and the bill grows quadratically with turn count.
• Free tiers default to flagship rates. The free plan usually calls whatever model client the founders wired in first, which makes every free user an uncapped cost center billed at premium prices.
• Feature creep compounds silently. Autocomplete, titles, summaries, and tag suggestions each look cheap in isolation, and each one defaults to the expensive client that already exists in the codebase.
• Regeneration doubles cost invisibly. A retry button re-runs the full prompt, and users press it most on the longest, most expensive outputs.
• Prompt bloat is a recurring tax. Every instruction added to the system prompt is billed on every call, in every feature, for every user, from now on.

Routing strategies compared: effort versus margin impact.

Routing is not one technique. The table below orders the common strategies by how much engineering they demand and what they return. The practical path for most teams: pin the free tier first, then static task routing, then an escalation ladder once eval tooling exists. Dynamic per-request routing is powerful but should never be the first move, because misroutes are silent and you need measurement infrastructure before you can trust it.

Seven moves that protect margin this quarter.

The last two levers compound with everything else. A summary call that moved from a flagship to a budget model already costs a fraction of what it did; running that budget model through a discounted endpoint cuts the remainder again without another line of product code. One gateway option is APIsRouter, an OpenAI-compatible API with pay-as-you-go billing and no subscription: global models are priced 20% below official list, Chinese models sit below their official rates, the first top-up adds +100% balance, and the no-signup checkout at /topup takes payment first and emails the key. A full routing ladder from the catalog looks like this:

• Inventory every AI touchpoint and tag it with a task class. You cannot route what you have not mapped, and most teams find forgotten features still pointed at the flagship.
• Pin the free tier to a budget model with a hard daily token cap. Free users are marketing spend; give the spend a budget like any other channel.
• Build a small eval set per task class before downgrading anything. Route on measured pass rates, not on vibes, and downgrades stop being scary.
• Trim prompts and history windows. Input tokens are most of a chat bill, so cap history depth and cut every system-prompt sentence that does not change outputs.
• Meter the heavy tail. Soft per-seat limits with usage-based overage turn your most expensive users back into your most profitable ones.
• Add an escalation ladder. Let the cheap model answer first and retry on the flagship only when validation fails; most requests never escalate.
• Lower the rate itself last. Once the mix is fixed, billing the same models through a cheaper OpenAI-compatible endpoint multiplies every saving above.

Config example: a router you can ship this week.

A margin-protecting router does not need a framework. It needs one function that owns model selection, a route table in config instead of scattered client calls, and cost logging tagged by task and plan so unit economics become a query instead of a quarterly archaeology project. Because the endpoint is OpenAI-compatible, model IDs are just strings: the ladder lives in one dict, and changing a route is a config edit, not a refactor. Log prompt and completion tokens on every response; per-user COGS is then a GROUP BY away. Roll it out like a pricing change: shadow-route first. Send a sample of live traffic to the candidate model, log both outputs, and score them against the task's eval set before flipping the route. Once routes are live, put the cost log on a weekly report split by plan and feature. The first report usually surfaces one forgotten endpoint still pinned to the flagship, and that discovery alone tends to pay for the whole exercise.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.apisrouter.com/v1",
    api_key=os.environ["LLM_API_KEY"],
)

ROUTES = {
    "generate": "claude-sonnet-4-6",   # hero feature, paid tiers
    "chat": "deepseek-v4-pro",         # refinement turns
    "summarize": "MiniMax-M2.7",       # background jobs
    "classify": "deepseek-v4-flash",   # titles, tags
}

FREE_TIER_MODEL = "deepseek-v4-flash"
FALLBACK_MODEL = "claude-sonnet-4-6"

def complete(task: str, plan: str, messages: list, **kwargs):
    if plan == "free":
        model = FREE_TIER_MODEL
    else:
        model = ROUTES.get(task, FALLBACK_MODEL)

    response = client.chat.completions.create(
        model=model,
        messages=messages,
        **kwargs,
    )

    usage = response.usage
    log_cost(  # your metrics sink: warehouse, StatsD, a table
        task=task,
        plan=plan,
        model=model,
        input_tokens=usage.prompt_tokens,
        output_tokens=usage.completion_tokens,
    )
    return response

FAQ