codingplans.dev
Technical Research Brief • Dataset verified 2026-10-17

Thinking Tokens vs Output Tokens: the Test-Time Compute Trade

Reasoning models spend tokens before they answer. Those tokens are billed as output, they dominate latency, and on easy tasks they buy nothing. The right question is not "which model is smartest" but "how much test-time compute does this task justify". This brief compares current models on measured intelligence, cost per task, speed and token pricing.

Intelligence leader
Claude Opus 5.5 (58)
Artificial Analysis Intelligence Index v4.3.2
Cheapest strong multitask model
GLM-5.3 Flash at $0.28/task
Index 42 on the same suite
Fastest in the dataset
Nemotron 3.5 Super VL
388 output tokens per second

Measured quality, cost and speed

Artificial Analysis, 2026-10-17
ModelAA IntelligenceOutput speedCost / AA taskCheapest token price
Claude Opus 5.5
Anthropic		 58 97 t/s $5.98 $4.00 / $20.00 per M
Gemini 4 Argon
Google		 53 — $1.99 $2.00 / $10.00 per M
GPT-6.1 Sol
OpenAI		 52 66 t/s $0.72 $2.00 / $10.00 per M
Claude Sonnet 5.5
Anthropic		 56 141 t/s $5.46 $2.00 / $10.00 per M
GLM-5.3
Z.ai		 45 81 t/s $2.01 $0.079 / $4.80 per M
Claude Haiku 5.5
Anthropic		 43 235 t/s $0.21 $0.10 / $0.50 per M
DeepSeek V4.1 Flash
DeepSeek		 39 176 t/s $0.27 $0.15 / $0.60 per M
DeepSeek V4 Pro
DeepSeek		 36 79 t/s $0.67 $0.66 / $1.98 per M

A three-tier routing policy

Tier 1 — Mechanical edits

Renames, formatting, imports, test scaffolding. Use a fast non-reasoning model or a low reasoning effort. Paying frontier thinking-token prices for a rename is pure waste.

Tier 2 — Multi-file features

Medium effort on a strong mid-tier model. The model needs to plan and run tests, but the task does not require novel reasoning. This is the volume tier for most teams.

Tier 3 — Ambiguous, high-blast-radius work

Max effort on a frontier model, with a human review gate. The cost per task is high, but so is the cost of a wrong migration or a subtle concurrency bug.

What the numbers say

  • The intelligence leader costs $5.98 per Artificial Analysis task; the cheapest model in this table costs $0.21. That is a 28.5x spread for a 15-point intelligence difference.
  • Time to first answer token is the metric that tracks "feels slow". Frontier reasoning models can take minutes on hard prompts; small models answer in well under a second. Terminal agents that block on a single model inherit the slowest case.
  • Cost per task already includes reasoning tokens. A model with a cheap input price and a chatty thinking process can end up more expensive per task than a model with a higher sticker price.
  • Cache pricing changes the answer again. Repeated repository context dominates input tokens, and cache reads range from 2% to 10% of base input across current vendors. Measure your own hit rate before trusting any list price.
← Back to Models Comparison (39 Models) Read Prompt Caching Economics →