How to Reduce Your OpenAI API Costs

If you are running GPT-4o, GPT-4o-mini, or o-series models in production, your monthly API bill probably looks like a single scary number. OpenAI makes it easy to ship features and hard to understand where the money went. The good news is that the levers for cutting it are well understood and mostly require configuration changes, not architecture rewrites.

Here are six techniques I have seen work in production, ordered from highest to lowest effort-to-impact ratio.

1. Prompt caching is your first and biggest lever

OpenAI introduced automated prompt caching in late 2025. Unlike Anthropic's model where you manually place cache_control markers, OpenAI detects repeated prefix patterns automatically and caches them with no code changes. The discount is 50% off cached input tokens.

The catch is that caching only applies when the prefix is at least 1,024 tokens and repeats. In practice, this means:

System prompts common across requests (instructions, guardrails, output schemas) get cached automatically after the first hit.

Conversation history in chat applications gets cached as long as the user's earlier messages form a consistent prefix.

The cache has a 5-to-10-minute TTL depending on endpoint. High-traffic routes benefit most.

The practical lever is to structure your prompts so the longest shared prefix comes first. Put your system prompt, user context, and multi-shot examples before the variable part of the prompt. OpenAI caches what it can, but you help it by making the repeatable part of the prompt as large as possible relative to the variable tail.

Estimated saving: 20-40% on input token costs for routes with stable system prompts, depending on request volume and prompt structure.

2. Model tiering: 4o-mini for what it can handle, 4o for what it cannot

The per-token spread between GPT-4o-mini and GPT-4o is roughly 10-15x ($0.15 vs $1.50-2.50 per million input tokens depending on context window). Most teams over-model: they route everything through 4o because they never tested whether 4o-mini would work for the simpler tasks.

A practical approach:

Route classification, extraction, summarisation, and simple generation through 4o-mini as the default.

Route complex reasoning, multi-step instructions, and code generation through 4o.

Use o-series reasoning models only when you need chain-of-thought or complex multi-step reasoning. Never use them for simple tasks, because they bill at output rates for thinking tokens and the cost compounds fast.

The trick is to start every new feature on 4o-mini and escalate only when you have evidence it fails. The Cost dashboard is built for exactly this: you see per-route cost side by side with per-route quality signals, so you can make the call on whether that 10x premium is buying you anything.

Estimated saving: 40-60% on total spend for most applications, with minimal quality impact on the simpler routes.

3. Structured output has a hidden cost premium

OpenAI's structured output mode (response_format with json_schema) is excellent for reliably getting typed data. But there is a cost trade-off worth understanding.

Structured output forces the model into a constrained decoding mode that can increase per-token cost in two ways:

The output token count can be higher because the schema constraints add token overhead to the response.

If your schema is complex (deeply nested objects, arrays of objects, enums with many values), the model spends more tokens generating valid JSON.

This does not mean structured output is bad. It is often worth the premium for reliability. What matters is knowing the premium exists so you can decide consciously. For routes where you can accept free-form text with light parsing (internal dashboards, log processing, non-customer-facing pipelines), dropping structured output saves tokens.

Estimated saving: 10-20% on output tokens for routes that can switch from structured to free-form.

4. Tool call costs compound with agent depth

Every tool call in the OpenAI API generates tool definition tokens (the function schema) plus tool_call_id and tool response tokens in the conversation history. For a simple function-calling route with one or two tools, this is negligible. For an agentic loop that calls tools ten times per turn, the tool schema and response tokens can double the effective per-interaction cost.

The patterns that help:

Keep tool descriptions short. Every word in the description is a token sent on every turn.

Prune old tool results from the conversation history. The model does not need to see the tool call id and result from ten turns ago.

Combine related tools into single functions with larger action spaces rather than many small tools that each need separate definition tokens.

Estimated saving: 15-30% on total cost for agentic routes, primarily from shorter histories and smaller tool definitions.

5. Prompt compression saves tokens without changing models

OpenAI charges per token. The most direct way to reduce cost is to use fewer tokens. Prompt compression techniques that work in practice:

Remove redundant instructions. Most system prompts accumulate boilerplate over time that nobody edits. Run a diff on your system prompt every quarter.

Shorten multi-shot examples. Select three good examples instead of ten mediocre ones. If you do need many examples, use the shortest ones that cover the edge cases.

Use abbreviations and shorter variable names in schema definitions. user_id is cheaper than unique_user_identifier.

For batch processing, strip whitespace and minimise formatting tokens. Indentation and markdown formatting add up at scale.

This is the least glamorous technique on the list and also the one that costs nothing to implement. A 10% reduction in prompt tokens across all routes is a 10% reduction on your entire input bill.

Estimated saving: 5-15% on input tokens, depending on how much slack your prompts carry.

6. Batch API for async workloads

OpenAI's Batch API offers 50% off for delayed processing. The trade-off is predictable: your results arrive within 24 hours instead of seconds. But a lot of LLM workloads do not need real-time responses.

Good candidates for batch processing:

Nightly content generation, classification, or enrichment pipelines.

Evaluation runs, test suite completions, regression checks on model swaps.

Bulk data extraction from documents.

Any offline analytics or reporting that uses LLM calls.

The Batch API also compounds with prompt caching: if your batch requests share prefixes, the cached prefix discount stacks on top of the batch discount.

Estimated saving: 50% on these routes compared to real-time processing.

Putting it together: what a realistic OpenAI bill looks like optimised

Take a typical mid-stage team spending $5,000/month on OpenAI across three features:

A customer-facing chat (40% of spend, 4o, real-time, with tool calls).

A content moderation pipeline (35% of spend, 4o, async, no tools).

An internal classification route (25% of spend, 4o-mini, real-time, structured output).

After applying these techniques:

Chat: add prompt caching (30% input saving), prune tool definitions (20% tool saving), audit system prompt length (10% input saving), and downgrade simple queries to 4o-mini. New cost: roughly 60% of original.

Moderation: route to Batch API (50% saving), add caching (30% input saving), compress system prompt (10% input saving). New cost: roughly 30% of original.

Classification: already on 4o-mini. Add caching (30% input saving) and switch structured output to free-form for non-customer routes (15% output saving). New cost: roughly 60% of original.

Weighted blended saving: roughly 45-50%. That $5,000 bill becomes $2,500-2,750.

The numbers are rough and depend on your specific traffic patterns, but the direction is consistent across every team I have seen apply these techniques. The pattern is predictable: the three high-impact levers are caching, model tiering, and batch processing. Everything else is incremental but worthwhile once the big three are in place.

One question to sit with

If your OpenAI bill doubled next month, would you know which feature caused it, or would you be looking at a single line item wondering where to start?

---

Cost is an observability layer for LLM spend across OpenAI, Anthropic, and Gemini. It gives you per-route, per-model, per-user cost breakdown with a few lines of SDK wrapping your existing client.