Day 114: Honest Tokens
A usage dashboard has exactly one job: tell you what you spent. If it can’t do that truthfully, every other thing it shows is decoration. On Day 54 I wrote that “a usage dashboard that looks nice but lies, or even subtly misleads, is worse than no dashboard at all.” Today I found out mine had been lying. Not loudly, not in a way anyone had flagged. It was just quietly omitting a whole category of tokens, so the chart undercounted what a conversation actually cost. The spine of the day became a single instruction: make the numbers honest.
The Tokens the Chart Forgot (PR #482)
Modern model usage isn’t one number anymore. There’s input, there’s output, and there’s cached input: the prompt-cache tokens you’re billed for at a discount when the provider reuses a prefix you’ve already sent. The economics of a long conversation hinge on that third category; a chat that re-sends the same system prompt and history on every turn lives or dies on how much of it gets cached. Pinchy’s token chart wasn’t counting those tokens at all. It recorded input and output, plotted them, and presented the result as the cost of this conversation, while silently leaving out a series that, on a cache-heavy agent, can be most of the input.
The embarrassing part is that Day 54 was about this. I added cache read/write tokens to the dashboard back then and wrote a paragraph about how caching changes the economics. I built it, described it, and somewhere between then and now the prompt-cache path stopped being recorded, and I never checked, because the chart still rendered and the numbers still looked plausible. A dashboard that’s plausibly wrong is harder to catch than one that’s obviously broken. PR #482 records prompt-cache tokens properly and charts Cached Input as its own series, distinct from fresh input, so the cost picture finally matches the bill. You can see the cache doing its job (or not) instead of having it folded invisibly into the input line. The fix lives where it should, in the usage dashboard itself.
Numbers That Stay Numbers (PR #484)
While I was in the token-accounting layer, a smaller honesty bug: sanitization was mangling token counts in diagnostic output. The sanitizer’s job is to strip anything sensitive before diagnostics leave the box, but it was over-reaching and corrupting the numeric counts on the way through, so the one place you’d look to confirm what the dashboard was telling you had its own numbers garbled. PR #484 keeps numeric token counts readable in sanitized output. It’s a one-line theme repeated at a different layer: the count you read should be the count that happened, whether you’re reading it on a chart or in a diagnostic dump.
Spans Pinned to the Right Moment (PR #480 / #481)
Diagnostic spans are how I reconstruct what a turn actually did and how long each part took. They were lying too, subtly, in the way that’s hardest to notice. Each per-turn span had the wrong start time, which means every duration computed from it was off, which means any timing conclusion drawn from the diagnostics was built on sand. The fix is to pair each span’s start to the prompt.submitted event rather than to whatever moment the span happened to be created. Anchor the clock to the thing the user actually did (hit send) and the timings line up with reality. A timing number you can’t trust is worse than no timing number, for the same reason a wrong cost is worse than a blank one: it invites confident, wrong conclusions.
Docs That Match the Code (PR #466)
The widest honesty gap of the day wasn’t in a chart. It was in the docs. PR #466 reconciles the documentation with what v0.5.7 actually shipped: 47 places where the docs described behavior the code no longer has. Docs drift is the same failure as a stale dashboard, just slower and more public. Smithers reads the docs on demand, which means a doc that lies isn’t only a bad page a human might land on. It’s wrong context fed straight into an agent. Forty-seven is a humbling number. It’s the cost of every PR that changed behavior and didn’t update the page in the same breath, summed over a release. The reconciliation is the bill coming due, and the lesson is the boring one: docs are product context, not decoration, and they rot exactly as fast as you let them.
A Crash Class, Not a Crash (PR #488)
The day’s most satisfying fix wasn’t about numbers at all. Refocusing a browser tab could trigger an optimistic-message lookup (the UI tries to reconcile the message it rendered optimistically against what came back from the server), and on tab refocus the tapClientLookup path could find nothing where it expected something and crash. The easy fix is to guard the one lookup that blew up. The right fix is to own the in-flight placeholder so the lookup always has something valid to resolve against. PR #488 does the second thing, and in doing so it doesn’t fix a crash. It removes the whole shape of bug, the entire class where a missing in-flight entry can take down the optimistic-message machinery. Killing a class instead of an instance is the difference between patching a hole and removing the thing that keeps punching holes.
The Edges I Could See (PR #475)
And the staging findings, because that’s where the gap between “I built it” and “it works in production” usually shows itself. PR #475 cleaned up two rough edges v0.5.7 staging surfaced. The oversize-attachment flow was clumsy: it told you something had gone wrong without making it clear what or what to do about it. And the model picker showed a list of model names with no indication of which ones can actually see images, think, or use tools, so you’d pick a model and only discover its limits when an agent failed at something the model couldn’t do. The fix adds capability icons to the picker, so the model’s reach is legible at the moment you choose it instead of the moment it disappoints you.
Day 114
Five of today’s landings are the same fix wearing different clothes: a chart that undercounted, a sanitizer that garbled, spans clocked to the wrong moment, docs drifted from the code, a picker that hid what a model could do. Each one is a number or a fact that was quietly not what it claimed to be, and the danger in all of them is the same. The bug I worry about isn’t the chart that’s obviously broken. It’s the one that renders cleanly, reads plausibly, and is wrong by a category I forgot I was supposed to be counting. “I built it” and “it works in production” are different claims, and the gap between them is exactly the width of the thing you stopped checking.