The Token Diet

Anthropic moved to pay-per-overage. Windsurf replaced credits with quotas. GitHub Copilot moves to usage-based billing June 1. The subscription era lasted about 18 months and every vendor has walked it back. Developers who grew up on unlimited AI are learning what metered electricity felt like after rural electrification. The power still works. It just has a meter now.

1. Triage with cheap models

Mendral, a CI/CD analytics company, processes roughly 4,000 build failures per week. They use Haiku 4.5 as a triage layer that catches 80% of duplicate issues before a frontier model touches them. A triage match costs about 25x less than a full investigation. Only genuinely novel failures escalate to Opus.¹

Their formulation: "The expensive model thinks; the cheap model reads." Haiku processes about 65% of all tokens but represents only 36% of total costs. The orchestration layer (Opus) plans the investigation and writes precise sub-agent instructions that Haiku executes. The key insight: better frontier models enable tighter task scoping for cheaper models, so upgrading your planner can actually reduce your overall bill.¹

For individual developers, the same principle applies. Use GPT-5.4 or Opus for architecture decisions and debugging plans, then hand execution to Sonnet, Haiku, or DeepSeek. One developer in the Copilot billing thread reported using "5.4 for the main session/debugging/planning and then execute with 5.3" to cut costs while maintaining quality.²

2. Route deterministic work off the agent

Every grep, find, git log, and jq command your AI agent runs costs tokens. The agent reads the output, reasons about it, and responds. For deterministic operations with predictable outputs, the reasoning step is waste.

Write shell scripts for repeatable operations. If you run the same sequence of commands every morning (check status, pull logs, summarize), wrap it in a script and call it directly. The agent can invoke the script, but the script does the work. One file read through an agent consumes 400+ tokens of overhead (prompt, system messages, safety checks). A script reads the file for free.

This is especially true for file operations. If you know which file you need, cat it yourself. If you know which string you're looking for, grep it yourself. Reserve the agent for tasks where you genuinely need reasoning, not retrieval.

3. Kill hidden overhead

Your agent consumes tokens you never asked for. System prompts, tool definitions, safety injections, and context management all add up invisibly.

Claude Code injects a malware-scanning reminder into every file read: roughly 400 tokens per read, 80+ times per session. That's 20,000-40,000 tokens of overhead you pay for but didn't request. A bug in HERMES.md routing can push requests to extra-usage billing without warning.³⁴

Lighter harnesses give you more control. One developer switched to OpenCode specifically because it lets you edit the system prompt and choose cheaper models with a single config change. "Even with all my personal instructions, I start with about 15k context used" versus 30-40k with heavier alternatives.⁵

Check what your tools inject. If you can't see your system prompt, you can't optimize it.

4. Use prompt caching deliberately

Anthropic's prompt cache has two tiers: a 5-minute default and a 1-hour extended option at 2x the write cost. Cached reads cost 10% of the base input price. For Opus 4.7, that's $0.50 per million tokens instead of $5. The savings are real, but only if your workflow stays within the TTL window.⁷

The practical implication: batch your work. Five questions about a codebase asked in sequence hit the cache on turns 2 through 5. The same five questions spread across 30 minutes pay full price every time. For agentic workflows with gaps between tool calls, use the 1-hour TTL explicitly.

Three common mistakes drain the cache savings. Placing cache breakpoints on content that changes every request (like timestamps) guarantees misses. Forgetting that tool definition changes invalidate the entire cache. And letting conversations exceed 20 blocks without adding a new breakpoint, which causes the lookback window to miss earlier cached content.⁷

Simon Willison's Claude Token Counter tool revealed that Opus 4.7's tokenizer uses 1.46x the tokens of Opus 4.6 for the same text. Same pricing per million, 46% more tokens consumed. Know which model version you're running and what it costs at the token level.⁶

5. Run local models for routine tasks

DeepSeek V4 Flash runs at 50 tokens per second on consumer hardware. Qwen 3.6-27B scores 77.2% on SWE-bench Verified and fits a 32GB MacBook Pro. These models handle code completion, documentation, test generation, and routine refactoring without API calls.

The split: use cloud frontier models for tasks that require deep reasoning, long-context understanding, or access to the latest training data. Use local models for everything else. One commenter in the Copilot thread put it plainly: "Cancelling. Going with Codex $100, Kimi annual plan, DeepSeek API, and a local LLM once I get a Mac Studio."²

The tools exist. Ollama, LM Studio, and llama.cpp have made local inference a one-command setup. Local model quality narrows the gap with cloud every quarter, but the cost difference is permanent: local inference is free at the margin.

6. Write a good AGENTS.md (or don't write one at all)

A well-structured AGENTS.md file can reduce agent exploration overhead by 10-15%. A bad one makes things worse. Augment Code's research found that files exceeding 150 lines trigger diminishing returns as agents load irrelevant context and explore tangentially. One poorly structured documentation set caused agents to load 80,000 tokens of irrelevant context before starting work.⁸

The key principles: keep the main file to 100-150 lines covering common cases. Use numbered step-by-step workflows (one six-step deployment workflow reduced missing files from 40% to 10%). Pair every prohibition with a solution ("don't instantiate HTTP clients directly" plus "use the shared apiClient from lib/http"). And use decision tables to resolve architectural choices upfront instead of letting the agent explore both paths.⁸

If your project doesn't have an AGENTS.md, that's fine. The agent will explore on its own. But if you do have one, make sure it's worth the tokens it consumes on every session start.

7. Ask the model to be brief

The simplest optimization is two words. A developer benchmarked Claude Code's "caveman" compression plugin against the instruction "Be brief" across 24 prompts in six categories. The results: "Be brief" reduced output tokens by 34% (from 636 to 419 tokens per response) with zero quality loss. The caveman plugin achieved similar reduction (401 tokens) but with more complexity.⁹

Quality scores stayed consistent across all approaches (0.970-0.985 range). Every response hit required key points. The plugin's real value turned out to be consistent output structure and mid-session intensity switching, not token savings. For most developers, adding "Be brief" or "Be concise" to your system prompt or CLAUDE.md gets you most of the benefit at zero cost.

8. Monitor before you optimize

Most developers have no idea how many tokens they consume per session, which operations are expensive, or where their context window fills up.

Tools that help: ccusage tracks Claude consumption per session. Simon Willison's token counter compares tokenization across models. The /usage endpoint in Claude Code shows your 5-hour and 7-day consumption percentages. OpenRouter provides per-request cost breakdowns.⁶

Build a dashboard. Watch it for a week. The first time you see a 40,000-token session that produced three lines of code, you'll find the motivation to change your habits.

9. Treat the subscription as a budget, not a buffet

The mental model shift is the hardest part. Under flat-rate pricing, developers optimized for speed and convenience. Ask the AI everything. Let it read every file. Run the agent in a loop until it works. Token cost was someone else's problem.

Under usage-based pricing, every request has a visible cost. The developers who adapt fastest are the ones who ask: "Does this task need AI, or do I already know the answer?" A surprising number of agent interactions are retrieval tasks disguised as reasoning tasks. You don't need a frontier model to tell you what's in a file you wrote yesterday.

The developers who built habits during the unlimited era have the hardest adjustment. The ones who started after the meter arrived will never understand what they missed.

Share on Bluesky · Share via Email