Google measures a 77% token premium when its coding agents chase a bug comment nobody asked about
Sam Saccone and Madhuri Garikapati, in a Friday post, say 13% of file-reading sessions exposed an internal agent to a TODO or FIXME with a bug id. In 6% of those encounters the agent acted on the id, and 83% of those actions were unprompted. After controls, those detours came with 70% more steps, 77% more tokens and 105% more wait time. A cleaner has deleted thousands of stale references in six months, with reviewer acceptance above 95%.
Feature2 minIn-depth
Maya Chen2 min read
- Coding agents
- Tokens
- Code

Photo: Google Developers BlogSource page
Google on Friday published a measurement of how often its own coding agents leave the task to chase a bug comment in the code. The post is dated 6 November. The authors are Sam Saccone, a senior staff software engineer, and Madhuri Garikapati, a senior staff data scientist. The data, they write, are distributed traces from thousands of engineers and hundreds of millions of agent steps. The post does not name the agent or the model. Model tier appears only as one of the variables they control for.
They used inline comments of the form TODO(bug/...) and FIXME(bug/...) as a proxy, because the comment often outlives the ticket. Of file-reading sessions, 13% exposed the agent to one of those references. In 94% of encounters the agent ignored the comment. In 6% it took a further action aimed at that bug id. Of those actions, 83% were unprompted: the developer had not asked about the bug.
Closed tickets and open ones
Among the unprompted detours, the post’s split is 32% already fixed or verified, 9% marked obsolete or won’t-fix, 2% duplicate or restricted, and 57% still open, with a status of new, assigned or accepted. The summary line in the post calls the closed share 41%, which is the 32 and the 9 added together. On the still-open tickets, 61% of the agent’s extra work was a lookup and 39% was an edit. The post describes two traces. In one, the agent opened SurfaceControlTest.java, saw a TODO whose bug had been fixed in 2023, and kept investigating it. In the other, it rewrote ModelDownloaderV1 for a TODO inside the bug it was already on. A person later called that a scope increase, and the model agreed.
Saccone and Garikapati then report a regression. They control for the developer, codebase complexity, task type, model tier, the number of files read, the length of the initial prompt, and the step at which the bug reference appeared. The method is a multi-way fixed-effects regression with standard errors clustered by developer. They mark every result p<0.0001. Per session, an unprompted detour is associated with 70% more steps, 77% more tokens and 105% more wait time. The percentages are relative to sessions without that detour, under those controls. The post does not give the baseline step count or the baseline token count.
A cleaner, with a person still accepting the change
The response they describe is another agent. It searched the code for bug references whose tickets were fixed or obsolete, then read the history of the bug, the file and the surrounding changes, sometimes across years, before proposing a deletion. A change went out for review only when the agent concluded that both the comment and the intent behind it were obsolete. Over the last six months, they write, that cleaner has removed thousands of references, with acceptance from human reviewers above 95%. The post also names presubmit analyzers and corpus scanners as the other tools. It does not say what share of the company’s TODO comments are gone, or whether the 77% token figure has moved since the cleaner started.


