Skip to content

Field notes · AI

Your AI Coding Agent Rereads the Whole Conversation Every Step. That's Where the Money Goes.

A coding agent sends the entire conversation back to the model on every step. So the number of steps, not the size of the prompt, drives the bill. Here is the loop, the order to check things when your agent bill is high, and a 25-line toy model you can run tonight.

A row of bars, one per agent step, each a little taller than the last. Most of each bar is shaded as cached. A thin cap on each bar marks the new tokens, and the newest cap on the last bar is marked in Amber as the only part paying full price.

A field note on AI agents. The toy model output below came from a run on our box on Nov 18, 2026, around 7:29 AM ET. It is a model of the billing math, not a measurement of any real agent. The script ships with this note.

The short version

OpenAI launched GPT-6.2 yesterday, and people are comparing price per million tokens. That number matters less than you’d think. If you use a coding agent like Codex CLI or Claude Code, the program around the model decides how many times it calls the model. Every call pays to send the whole conversation again.

So when your agent bill looks wrong, count the steps first. The prompt size comes later.

A few terms first

  • The model is the AI itself. It answers one request at a time and remembers nothing between requests.
  • The harness is the program around the model that runs the loop. It sends the task, runs the commands the model asks for, pastes the results back in, and asks again. Codex CLI and Claude Code are harnesses. One trip around that loop is a step.
  • A tokenThe smallest unit of text that a language model processes, typically a word, part of a word, or a punctuation mark. Read more → is a chunk of text, roughly three quarters of a word. APIA set of rules and protocols that allows different software applications to communicate with each other and share data or functionality. Read more → prices are per million tokens.
  • The context is everything the model sees on a step: the instructions, the list of tools, your task, and every earlier step.
  • The prompt cache is the provider remembering the unchanged start of the context so it can charge much less to read it again.

Picture a contractor who forgets everything between phone calls. Each time they call with a question, you read them the whole job file from page one, then ask the new thing. Ten calls means ten readings of a file that keeps getting longer. A good foreman asks fewer, better questions. The cache is the contractor finally keeping a copy of the file, so you only read the new page. You still make every call, though.

What a big study found

Runta, which makes a sandbox runtime for agents, published FrontierHarness Eval v1.1 on Nov 17. It ran 22 harness setups with 5 open models on 62 terminalThe window or screen where you type text commands and read their output, connected to a shell that runs those commands. Read more → tasks, 4 attempts each. That’s 27,280 runs and 83 billion tokens.

The model set most of the cost. It explained 92.8% of the spread. But with the model held fixed, the most expensive harness cost 1.8 to 5.0 times as much per task as the cheapest. A shuffle test says that gap isn’t rerun noise (p = 0.003).

Here’s the part I’d tape to the monitor. Runta compared four things against cost per passed task: the size of the first request, the number of tools, the cache hit rate and the number of steps. Only steps tracked cost (rank correlation +0.51). The other three came in between -0.03 and +0.02.

mini-SWE-agent makes the point. It sent the smallest first request of any harness, 882 tokens. It also took 38 steps per pass and landed 17th of 22 on cost per pass. A trial used 2.0 to 4.8 million input tokens on average, depending on the harness. The biggest first request, 21,365 tokens, was under 1% of that. A lean prompt doesn’t help much if the loop runs 38 times.

Why the cache still matters

“Cache hit rate didn’t predict cost” sounds like cachingStoring frequently accessed data in a fast, temporary location so future requests can be served without repeating the original expensive operation. Read more → doesn’t matter. It means almost everyone already had it working. Runta says 18 of the 22 harnesses read 91% or more of their input from the cache in passed trials. When nearly everyone scores high, that number can’t explain the gaps.

In Runta’s sample trace, request 10 sent 16,999 prompt tokens, and 16,320 of those came from the cache. Only about 700 tokens paid full price. On GPT-6.2 Astra that gap is $8 per million input tokens versus $0.40 cached, a twentieth of the price.

The cache only works if the start of the context stays exactly the same. OpenAI’s prompt caching guide says reuse “requires the entire rendered prefix to match.” A harness that reorders its tools or puts a timestamp near the top throws the cache away on every step.

When your agent bill is high, check in this order

  1. Steps. Count the turns or tool calls in one session. Forty steps for a one-line fix is the smell. Was the task vague? Did the agent rerun the same command?
  2. Cached versus uncached input. Your provider’s usage page shows both. On the OpenAI API, each response’s usage block has input_tokens and input_tokens_details.cached_tokens. If the cached share is low on a long session, something keeps changing the start of the context.
  3. Failed runs. In Runta’s study, 64.5% of the spend went to runs that didn’t pass. With the five best-value harnesses, 68% to 82% of failed runs ended with a confident final answer, not a timeout. One agent reported it had validated its work. The test checker then counted 0 of the 2 lines it expected.
  4. Your own stop condition. Run the tests yourself before you trust “done,” and tell the agent what done means.
  5. Harnesses, last. Compare on a handful of your own real tasks, same model, a few attempts each.

OpenAI’s launch postPower-On Self-Test: the quick hardware check firmware runs every time a computer starts, before it looks for an operating system. Read more → backs up step 4. It says both GPT-6.2 models were “more likely to omit mentioning obstacles that blocked their work, such as a missing file or unavailable search tool,” and that it added instructions in Codex to cut those omissions.

The toy model

I ran a 25-line Python script on our box. It makes no API calls. It assumes a 15,000-token start, 500 new tokens of tool output plus 300 tokens of model output per step, and OpenAI’s published GPT-6.2 prices. Then it compares 12 steps with 38, the step count Runta measured for mini-SWE-agent.

python3 src/agent_cost.py   # Python 3 only, no packages, no API key

What came back for GPT-6.2 Astra:

  • 3.2 times the steps sent 4.9 times the input tokens, 232,800 versus 1,132,400. It compounds, because each step resends everything before it.
  • With the cache on, 12 steps cost $0.418 and 38 steps cost $1.248. That’s 3.0 times as much.
  • With the cache off, 38 steps cost $9.515. Same work, about 7.6 times the cached price.
Line chart of tokens sent per step, rising from 15,000 at step 1 to 44,600 at step 38, with the 12-step point marked and the 38-step end point in Amber. A note reads total input 232,800 tokens at 12 steps and 1,132,400 at 38.

Treat the dollar figures as illustration. The toy sends 1.1 million input tokens at 38 steps, below the 2.0 to 4.8 million Runta measured per real trial. Trust the shape. Steps compound, and the cache keeps the compounding in check.

How to read this study

Runta states its limits. It tested open models only, served by one provider, Baseten, at Baseten’s prices. The eval ran on Runta’s own runtime, and the founder wrote in the Hacker News thread, “weigh this accordingly.” Closed models are on their list of next steps.

One commenter called “steps explain cost” “nearly tautological,” since tokens track steps, and pointed out that a harness doesn’t fully control its step count. The model’s behavior drives it too. That’s fair, and it doesn’t change what you check. Another noted that real use has a human steering and reviewing, which a pass/fail test doesn’t capture. Runta’s own advice agrees. Test on your own tasks.

Your first step tonight

Run the toy script and change one number. Set the steps to 60, or turn the cache off, and watch the total. If you already use an agent with an API key, open one real session’s usage and write down steps, total input tokens and cached input tokens. That’s your baseline.

New to tokens and context windows? Our field note How AI Actually Works covers them from zero.

Go deeper: compounding, cache breakers, summaries, launch benchmarks and the full output

Why input grows faster than steps. Step 1 sends the start, S tokens. Step 2 sends S plus one step’s worth of new text, N. Step k sends S + (k-1)N. Add them up and the total is about kS + N·k²/2. The first term grows with steps. The second grows with the square of steps. That’s why the toy’s 3.2x steps became 4.9x input tokens.

What breaks prompt caching. OpenAI’s guide lists the settings that change the cached prefix, among them the model, the tools (names, descriptions, schemas or order), reasoning effort and compaction. It also says to put timestamps and user-specific content at the end, not near the top. Cached tokens still count toward rate limits. One thing the toy leaves out is the write cost: for GPT-5.6 and later, OpenAI says writing to the cache costs 1.25 times the normal input rate for the default 30-minute lifetime. Real cache-on costs are a bit higher than the toy shows.

Summarize at a threshold. Runta found almost no summarizing among its five best-value harnesses. Only Codex CLI did it, in 8 trials, and Runta quotes the code that triggers it at 90% of the context windowThe maximum amount of text (measured in tokens) that a language model can read and consider at once when generating a response. Read more →. An OpenAI Codex maintainer confirmed that default on GitHub. Summaries cut tokens, but they rewrite the start of the context, so the next request reuses less cache. OpenAI’s caching guide says to compare total input cost before and after, because fewer tokens can still save money even when the hit rate falls.

Reading launch benchmarks like an engineer. OpenAI’s GPT-6.2 post notes that its Agents’ Last Exam leaderboard results for Claude Fable 5.2 “exclude nine tasks that used a fallback model.” Footnotes like that are where comparisons get decided. Runta’s pass rates carry about ±10 point 95% intervals, so a few points apart is a tie, and across models the rank correlation between cost and pass rate ran from -0.39 to +0.22. Paying more didn’t buy passes. Small drift shows up between summaries too: the founder’s HN post says cost per pass varies “up to 4.4x” and puts Codex CLI 15th with Kimi-K3, while the published post says 4.9x and 16th. Quote the primary source.

Runta’s best-value rule. The lowest cost per pass wins, unless another harness costs at most 30% more per pass and passes at least 5 points more often. Codex CLI was the best value with GLM-5.3 and 16th by cost with Kimi-K3. That swing is why “which harness is cheapest” has no answer without “with which model.”

The full toy output. src/agent_cost.py uses only the Python standard library. Raw run from our box, Nov 18, 2026, 7:29 AM ET:

model          steps cache  input sent cached%  cost $
gpt-6.2-astra     12  True     232,800   89.8%   0.418
gpt-6.2-astra     12 False     232,800    0.0%   2.006
gpt-6.2-astra     38  True   1,132,400   96.1%   1.248
gpt-6.2-astra     38 False   1,132,400    0.0%   9.515
gpt-6.2-sol       12  True     232,800   89.8%   0.104
gpt-6.2-sol       12 False     232,800    0.0%   0.502
gpt-6.2-sol       38  True   1,132,400   96.1%   0.312
gpt-6.2-sol       38 False   1,132,400    0.0%   2.379

Astra, cache on: 38 steps costs 3.0x 12 steps (steps went up 3.2x)

GPT-6.2 Sol shows the same ratios at a quarter of the price, because its rates are $2, $0.10 and $10 per million.

Scripts

src/agent_cost.py

"""Toy model of what an agent loop sends to the API.
Every step resends the whole context; only the new tail is uncached.
Prices: OpenAI GPT-6.2 launch post (USD per 1M tokens), Nov 17 2026.
Shape: Runta FrontierHarness v1.1 trace (start ~15k tokens, ~500 new per step)."""
PRICES = {"gpt-6.2-astra": (8.00, 0.40, 40.0), "gpt-6.2-sol": (2.00, 0.10, 10.0)}

def run(steps, start=15_000, new_per_step=500, out_per_step=300, cache=True, model="gpt-6.2-astra"):
    p_in, p_cached, p_out = PRICES[model]
    ctx, cost, sent, cached_tok = start, 0.0, 0, 0
    for i in range(steps):
        fresh = ctx if (i == 0 or not cache) else new_per_step + out_per_step
        cached = ctx - fresh
        cost += (fresh * p_in + cached * p_cached + out_per_step * p_out) / 1e6
        sent += ctx; cached_tok += cached
        ctx += new_per_step + out_per_step
    return cost, sent, cached_tok / sent

print(f"{'model':14} {'steps':>5} {'cache':>5} {'input sent':>11} {'cached%':>7} {'cost $':>7}")
for model in PRICES:
    for steps in (12, 38):
        for cache in (True, False):
            c, s, h = run(steps, cache=cache, model=model)
            print(f"{model:14} {steps:>5} {str(cache):>5} {s:>11,} {h*100:>6.1f}% {c:>7.3f}")
c12,_,_ = run(12); c38,_,_ = run(38)
print(f"\nAstra, cache on: 38 steps costs {c38/c12:.1f}x 12 steps (steps went up {38/12:.1f}x)")

src/run-raw.txt

# Real run of agent_cost.py on the BytesNation box, Wed Nov 18, 2026 7:29 AM EST. python3 3.13.5. Toy model, no API calls, not a measurement.
model          steps cache  input sent cached%  cost $
gpt-6.2-astra     12  True     232,800   89.8%   0.418
gpt-6.2-astra     12 False     232,800    0.0%   2.006
gpt-6.2-astra     38  True   1,132,400   96.1%   1.248
gpt-6.2-astra     38 False   1,132,400    0.0%   9.515
gpt-6.2-sol       12  True     232,800   89.8%   0.104
gpt-6.2-sol       12 False     232,800    0.0%   0.502
gpt-6.2-sol       38  True   1,132,400   96.1%   0.312
gpt-6.2-sol       38 False   1,132,400    0.0%   2.379

Astra, cache on: 38 steps costs 3.0x 12 steps (steps went up 3.2x)
ai-agentscoding-agentstokensprompt-cachingcontext-windowcodex-clibenchmarkscosttroubleshooting

← All field notes