AI SaaS token COGS: what each user actually costs you

Quick answer: price every feature in tokens before you ship it.

Every request your product makes to a model API bills tokens in both directions, and that spend is cost of goods sold in the most literal sense: it scales linearly with usage, per user, per action. The core formula is short. Monthly token COGS per paying user equals actions per month times the cost of one action, where one action costs (input tokens times the input rate plus output tokens times the output rate) divided by one million. Then add the number most operators forget: free-tier drag. If free-to-paid conversion runs at 5%, every subscriber carries nineteen free users on their back, and each free user runs the same formula at their own usage level. True per-payer COGS is the payer's own cost plus nineteen times the free-user cost, and on a flagship model the second term is usually the bigger one. A workable health target: keep total token COGS, free tier included, under about a fifth of ARPU. That leaves room for hosting, vector databases, payment fees, and support while still landing near the gross margins software businesses are expected to have. Three levers get you there: fewer tokens per action, cheaper models per task, and a lower per-token rate. The rest of this page prices each lever with real numbers.

Where the tokens, and the dollars, actually go.

Input and output tokens bill at different rates, and the asymmetry shapes everything. Output typically costs five or six times more per token, yet input usually dominates total spend, because every request re-sends everything the model needs to see: the system prompt, formatting instructions, few-shot examples, retrieved RAG chunks, and any conversation history. A 900-token system prompt on a feature that fires 40 times per user per month is 36,000 billed input tokens per user before anyone types a word. One user action is also rarely one model call. A typical generate button can trigger a moderation pre-check, the main completion, a retry when the model returns malformed JSON, and a small follow-up call that writes a title or a summary for the sidebar. Each of those calls re-bills its own input. Teams that meter per feature routinely discover that a decorative sub-call, a tag generator or an auto-titler, quietly accounts for a large slice of the bill because someone pointed it at the same flagship model as the hero feature. Finally, tokens bill when they are generated, not when they are read. A user who closes the tab mid-stream still costs you the rest of the generation unless your backend cancels the stream on disconnect. None of this is exotic; it is just invisible until you attribute spend to users and features instead of eyeballing the monthly invoice.

Worked example: a $19/month product, user by user.

Take a concrete product: a $19/month writing assistant. Each generation sends about 3,000 input tokens (an 800-token system prompt, roughly 2,000 tokens of document context, and the user's instruction) and returns about 500 output tokens. An engaged subscriber runs 150 generations a month, which is 450K input and 75K output tokens. The table prices exactly that usage at official list rates, one model at a time, with the free tier deliberately ignored for now. The spread is the story. The same subscriber behavior costs $4.50 or eight cents depending on which model serves it, and every one of these models is a plausible engine for a drafting feature. Model choice is a pricing decision wearing an engineering costume.

Why the bill still surprises operators.

None of these appear in a per-request spot check; they only appear in per-user, per-feature metering. That is why the wrapper code at the bottom of this page matters more than any single optimization: you cannot fix a margin you cannot attribute. Once every request writes a row with a user id, a feature name, and a computed dollar cost, each item on this list becomes a query, an alert, and then a fix.

• Free-tier drag: at 5% conversion, each subscriber carries 19 free users. On a flagship model at list rates, a light free user (20 actions a month) costs about $0.60, which adds $11.40 of COGS per payer and flips a 76% token margin into 16%.
• Input re-billing: the system prompt, RAG context, and chat history are resent on every call, so prompt bloat is a recurring charge on all future traffic, not a one-time cost.
• Unbounded output: without max_tokens caps per feature, verbose models pad answers, and output is the expensive direction per token.
• Repair loops: schema validation failures and automatic retries can double the model calls behind one user click without anyone noticing.
• Power users under flat pricing: a small cohort at the top of the usage curve can burn several times the median subscriber cost while paying the same $19.
• Non-product traffic: staging environments, cron jobs, and eval suites running on the production key show up as COGS but produce no revenue.

Three serving architectures, one margin table.

Put the pieces together and the same product has wildly different unit economics depending on how requests are served. Scenario A sends every request, free and paid, to gpt-5.5 at list rates: the naive launch configuration. Scenario B routes by job at the same list rates: 120 of a subscriber's 150 monthly generations run on gpt-5.4-mini, the 30 that need flagship quality stay on gpt-5.5, and the free tier runs entirely on deepseek-v4-flash. Scenario C keeps exactly that routing but bills the same models through a below-list OpenAI-compatible endpoint instead of directly at list. Routing is the big win, and the rate cut compounds it. A to B is an architecture change that took this hypothetical team a week of eval work; B to C is a procurement change that took an afternoon and one config line.

How to protect the margin: fixes in order of payoff.

The last lever needs no refactor. One option is APIsRouter, an OpenAI-compatible endpoint with pay-as-you-go billing and no subscription: global models are priced 20% below official list, Chinese models sit below their official rates, the first top-up adds +100% balance, and the /topup checkout takes payment first and emails the key with no signup form. A routing ladder for the product above looks like this at those rates:

• Meter first: log user id, feature, model, input tokens, output tokens, and computed cost on every request. Per-user COGS should be a query, not a guess.
• Budget the free tier in tokens, not vibes: a hard monthly token allowance per free account, served by the cheapest capable model, with a visible usage meter.
• Route by job: flagship models only for the hero feature, mid-tier for drafts and rewrites, budget models for titles, tags, classification, and summaries.
• Shrink the input: cut the system prompt to the lines that change behavior, cap history depth, and retrieve fewer, better RAG chunks.
• Cap the output: set max_tokens per feature and ask for terse formats; a JSON list does not need prose around it.
• Cache the shared prefix: providers discount repeated prompt prefixes heavily, so keep stable content at the front of the prompt and variable content at the end.
• Lower the unit rate itself: the same models are often billed below list through an OpenAI-compatible gateway, which cuts COGS across every tier for a one-line base URL change.

Meter it in code: cost per request, per user.

Every OpenAI-compatible response carries a usage object with exact prompt and completion token counts, so COGS attribution is a wrapper, not a project. Wrap your model calls once, write one row per request, and per-user unit economics become a GROUP BY. The same wrapper is the natural home for routing: the model becomes a parameter chosen by feature instead of a constant, and a price map turns token counts into dollars at write time so dashboards never need to re-derive rates. Verify the endpoint and inspect the usage object with a one-off request before wiring it into the product:

from openai import OpenAI

client = OpenAI(
    base_url="https://api.apisrouter.com/v1",
    api_key="sk-APIsRouter-...",
)

# USD per 1M tokens: (input, output)
PRICES = {
    "gpt-5.5": (4.00, 24.00),
    "gpt-5.4-mini": (0.60, 3.60),
    "deepseek-v4-flash": (0.126, 0.252),
}

def metered_completion(user_id, feature, model, messages, max_tokens=600):
    resp = client.chat.completions.create(
        model=model, messages=messages, max_tokens=max_tokens
    )
    p_in, p_out = PRICES[model]
    usage = resp.usage
    cost_usd = (
        usage.prompt_tokens * p_in + usage.completion_tokens * p_out
    ) / 1_000_000
    log_usage_row(user_id, feature, model,
                  usage.prompt_tokens, usage.completion_tokens, cost_usd)
    return resp.choices[0].message.content

FAQ