The Single Detail That Decides Your LLM Bill

$0.000003.

That is the per-token input price for GPT-4o mini as of mid-2025 - roughly three millionths of a dollar. It looks like a rounding error. It is not. At the scale a real SaaS feature runs, even a cheap model will surprise you if you have not done the arithmetic before you scope the work.

I want to be precise about what I am covering here. This is not a comparison of which model is smartest. It is about the three numbers you need to pull from a pricing page before you commit to an architecture, because those numbers change what you can promise a client and when.

Which Numbers Actually Matter?

Every major provider publishes the same basic shape: input cost per million tokens, output cost per million tokens, and a context window ceiling. OpenAI publishes these at platform.openai.com/docs/pricing. Anthropic publishes theirs at anthropic.com/pricing. Google publishes Gemini pricing at ai.google.dev/pricing.

The input/output split matters more than most engineers expect. Output tokens cost two to four times more than input tokens across most providers. If your feature generates long completions - summaries, structured JSON payloads, multi-step plans - you are paying at the expensive rate for the bulk of your traffic.

The context window ceiling is the second trap. A 128k context sounds enormous until you are stuffing a user's document history, a system prompt, and a retrieval result into every call. Token count compounds fast.

My first instinct was to solve this with a simple token counter before each call - refuse the request if it would exceed a threshold. That works in a script. It breaks in production because you cannot know the output length before the model runs. You need a different approach.

The real solution is to budget at the feature level, not the call level.

# Rough monthly cost estimate before you commit
INPUT_PRICE_PER_M  = 0.15   # USD, GPT-4o as of mid-2025 (verify at pricing page)
OUTPUT_PRICE_PER_M = 0.60   # USD, GPT-4o as of mid-2025

avg_input_tokens  = 1_200   # per user action
avg_output_tokens = 400     # per user action
daily_actions     = 500     # your MAU estimate

monthly_cost = (
    (avg_input_tokens  / 1_000_000 * INPUT_PRICE_PER_M) +
    (avg_output_tokens / 1_000_000 * OUTPUT_PRICE_PER_M)
) * daily_actions * 30

print(f"Estimated monthly LLM cost: ${monthly_cost:.2f}")

Run that before you quote the project. Plug in the numbers from the provider's live pricing page, not from memory. Prices move.

Once you have the estimate, the next decision is model tier. A smaller, cheaper model handles classification, intent detection, and short-form extraction well. Reserve the expensive model for tasks where output quality directly affects whether the user trusts the product. That split is not a performance optimisation - it is a margin decision.

// Route by task complexity at the service layer
const model =
  task.type === "classification" || task.type === "extraction"
    ? "gpt-4o-mini"   // ~$0.15 / 1M input tokens
    : "gpt-4o";       // ~$2.50 / 1M input tokens

const completion = await openai.chat.completions.create({
  model,
  messages,
  max_tokens: task.maxOutputTokens, // always set this; never leave it open
});

Always set max_tokens. An open-ended completion on a slow user request can generate ten times what you expected. That is not a model bug - it is a missing constraint.

Rate limits are the third constraint and the one most likely to block a launch. Tier 1 accounts on OpenAI cap at 500 requests per minute for most models. If your feature fires multiple calls per user action - retrieval, rerank, generate - you can hit that wall with fewer than 200 concurrent users. Anthropic and Google have their own tier structures. Check the limits tab, not just the pricing tab, before you design the concurrency model.

# Check your current tier limits via the API (OpenAI example)
curl https://api.openai.com/v1/organization/limits \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json"

The rate limit is not a soft suggestion. It is a hard ceiling that returns a 429, and your users see an error if you have no retry budget built in.

For SaaS features where a launch date cannot move, I treat rate limits as infrastructure constraints the same way I treat RDS connection pools. You design around them before you write the first handler, not after the first load test fails.

For more on reading model releases from a builder's angle, the Gemini 2.5 Pro Is Out. Here Is What Actually Changes for People Who Ship. post covers a similar evaluation frame. And if you want the honest take on AI tooling that did not hold up under daily use, Cursor Earned Its Spot. Then It Slipped. Here Is Where It Stands Right Now. is worth reading alongside this one.

If you are building a feature where the LLM is core to the product - not a side utility - the architecture decisions get heavier. SaaS & Web App Development covers how I approach that scope.

One place this advice does not apply: internal tools with a fixed, small user base and a generous employer-paid API budget. If 15 people are using the tool and the monthly bill caps at $40 regardless of usage patterns, the optimisation work costs more in engineering hours than it saves. Spend that time on something that moves the product forward instead.