How to build a character AI app: architecture, code, and cost math

Quick answer: the four-component architecture

You do not need to train or fine-tune a model. Every serious companion app on the market today, including the ones serving millions of users, is a thin orchestration layer over hosted LLM APIs. The character lives in the prompt and the database, not in the weights. The rest of this guide builds each component with runnable code against a standard OpenAI-compatible endpoint, then walks the token math so you know what a monthly active user actually costs before you launch.

• Persona prompt: a structured system prompt (500 to 1,000 tokens) that defines who the character is and how it speaks.
• Memory: a rolling window of recent turns plus a periodically refreshed summary, so conversations survive past the context limit.
• Moderation: an age gate, an input classifier, and content policies matched to the model you route to.
• Routing: a small function that picks a cheap model for casual chat and a stronger one for long scenes.

Component 1: the persona prompt

The persona prompt is the product. It is a system message that ships with every request and typically contains five parts: identity (name, age, backstory), personality traits, speech style with two or three example lines, hard behavioral rules (never break character, keep replies under a length cap), and the current scenario. Keep it between 500 and 1,000 tokens. Shorter personas drift; much longer ones burn input budget on every single message and rarely improve adherence. Store personas as rows in your database so users can create and edit characters, which is the core loop of Character.AI-style products. Here is a complete request. The same shape works with any OpenAI-compatible backend; this example uses a multi-model gateway endpoint so you can swap models by changing one string:

curl https://api.apisrouter.com/v1/chat/completions \
  -H "Authorization: Bearer $KEAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [
      {"role": "system", "content": "You are Mira, a sarcastic starship engineer in her 30s. Personality: dry wit, fiercely loyal, hates small talk. Speech style: short technical sentences, occasional teasing. Example: \"The coupling is fried. Again. Did you touch it?\" Rules: never break character, never mention being an AI, keep replies under 120 words."},
      {"role": "user", "content": "Mira, the reactor is making that noise again."}
    ],
    "temperature": 0.9,
    "max_tokens": 300,
    "stream": true
  }'

Component 2: memory that survives long conversations

Companion app sessions run long. A three-layer memory stack keeps context coherent without sending the entire history every time: Layer 1 is the rolling window: the last 10 to 14 raw turns, sent verbatim. Layer 2 is the running summary: every 20 turns or so, a cheap model compresses older messages into a paragraph that tracks names, relationship state, and open plot threads. Layer 3 is pinned facts: durable user details (name, preferences, past events the character should remember) extracted into your database and injected as a short system message. This is the whole implementation in Python with the official OpenAI SDK:

from openai import OpenAI

client = OpenAI(
    api_key="sk-...",
    base_url="https://api.apisrouter.com/v1",
)

def build_context(persona, summary, pinned_facts, recent_turns, user_msg):
    facts = "\n".join(f"- {f}" for f in pinned_facts)
    return [
        {"role": "system", "content": persona},
        *([{"role": "system", "content": f"Facts to remember:\n{facts}"}]
          if pinned_facts else []),
        *([{"role": "system", "content": f"Story so far: {summary}"}]
          if summary else []),
        *recent_turns[-12:],
        {"role": "user", "content": user_msg},
    ]

def refresh_summary(old_turns, prev_summary):
    prompt = (
        "Update this running summary of a roleplay chat. Keep names, "
        "relationship state, and open plot threads. Max 150 words.\n\n"
        f"Current summary: {prev_summary}\n\nNew messages: {old_turns}"
    )
    resp = client.chat.completions.create(
        model="deepseek-v4-flash",
        messages=[{"role": "user", "content": prompt}],
        max_tokens=400,
    )
    return resp.choices[0].message.content

Component 3: moderation and content policies

Moderation in a companion app has three distinct jobs, and conflating them is the most common launch mistake. First, gate your audience. If your app allows mature fictional themes, put them behind an adult age gate and keep a separate all-ages default. App Store and Play Store review both check this, and payment processors care too. Second, screen inputs. Run a cheap classifier pass over user messages before they reach the character model, blocking categories that no provider permits: content involving minors, credible real-world harm, and illegal instructions. A single deepseek-v4-flash call with a yes/no rubric costs a fraction of a cent and adds under a second of latency. Third, match content policy to model. Providers differ on fiction: xAI's published policy explicitly allows mature fictional themes for adult users, DeepSeek and other open-weight-derived models are community favorites for roleplay because their hosted policies are flexible for fiction, while Anthropic's policy prohibits explicit content, making Claude the right choice only for all-ages creative writing. Understanding these policies before you route traffic saves you from refusal storms in production. Whatever you build, it must respect the terms of service of every provider you call.

Component 4: model routing and the selection matrix

Most messages in a companion app are short casual turns that a budget model handles perfectly. A minority are long emotional scenes or creative set pieces where prose quality is the product. Routing by scene depth cuts costs dramatically without users noticing:

ROUTES = {
    "casual": "deepseek-v4-flash",
    "scene": "deepseek-v4-pro",
    "prose": "claude-sonnet-4-6",  # all-ages creative writing only
}

def pick_model(user_msg: str, turns_in_scene: int) -> str:
    if turns_in_scene > 6 or len(user_msg) > 400:
        return ROUTES["scene"]
    return ROUTES["casual"]

What a monthly active user actually costs

Token math before launch beats a billing surprise after. Using the context budget from the memory table: roughly 3,300 input tokens and 250 output tokens per message. Companion app users are heavy; community-reported figures put engaged users at dozens of messages a day, so assume approximately 40 messages a day across 20 active days, or about 800 messages per MAU per month. That gives 2.64M input tokens and 0.2M output tokens per MAU per month. Multiplying by catalog rates:

One endpoint for every model on the matrix

Signing up for five provider accounts to run one router is real overhead: five keys, five billing dashboards, five sets of rate limits. A multi-model gateway collapses that. APIsRouter exposes every model above through the single OpenAI-compatible base URL https://api.apisrouter.com/v1, so the router snippet works as-is by changing only the model string. Billing is pay-as-you-go with no subscription, global models run 20% below official pricing with Chinese models below official rates, and the no-signup checkout at /topup sends a key to your email after payment, with the first top-up adding +100% balance. GET /v1/models with your key returns the live model list, which is also handy for populating a model picker in your own admin UI.

Production tips that save real money and pain

• Stream everything. Time-to-first-token is the metric users feel; a streamed reply at 2 seconds feels faster than a complete reply at 4.
• Keep the stable prefix stable. Persona and pinned facts should be byte-identical across requests so backends that support prefix caching can exploit it.
• Cap max_tokens per route. Casual turns rarely need more than 300 output tokens; an uncapped model that rambles doubles your output bill.
• Log cost per user, not just per request. A small fraction of heavy users will dominate spend; you need to see them to design fair usage tiers.
• Build fallback into the router. If the primary model times out or refuses, retry once on a sibling model before surfacing an error.
• Trim the rolling window by tokens, not turns, so one long paste does not blow the budget.
• Handle refusals in character. Map a refusal to a graceful in-story deflection instead of showing raw API text.
• Do not store raw chats longer than you must, and say so in your privacy policy. Companion chat logs are sensitive by nature.

FAQ