Your AGENTS.md costs more than it earns

A repository context file reliably makes a coding agent more expensive and does not reliably make it more correct. That is the central finding of a 2026 study that tested AGENTS.md and CLAUDE.md files across three coding agents in four model configurations, with both auto-generated and developer-written files. The part of a context file that pays for itself is small: instructions the agent could not have worked out by reading your repository. Many committed files contain a good deal of something else.

The evidence does not show that every context file is harmful. It shows that persistent context has a measurable cost, and that the benefit depends on whether the file tells the agent something it could not otherwise infer.

The habit this runs against is familiar. You open a repository, run /init, and an agent writes a tidy file describing the project. It feels like diligence. The evidence says it is mostly overhead you pay on every task.

Context files raised cost without significantly raising success

A study from ETH Zurich and LogicStar.ai, Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?, ran four agent and model configurations against 438 tasks: Claude Code with Sonnet 4.5, Codex with GPT-5.2 and with GPT-5.1 mini, and Qwen Code with Qwen3-30b-coder. The tasks came from SWE-bench Lite, which is 300 issues across 11 popular Python repositories, and from a new benchmark of 138 tasks drawn from 12 newer, less well-known repositories that already carry developer-written context files.

Auto-generated context files lowered the average resolution rate by 0.5 percentage points on SWE-bench Lite and 2 points on the newer benchmark. Neither result is statistically significant, with p-values of 0.87 and 0.37. Developer-written files raised success by 2.4 points on average, also not significant, although they did significantly outperform the auto-generated ones.

Within this study, the cost result is much clearer. Auto-generated files raised inference cost by 20 percent on SWE-bench Lite and 23 percent on the newer benchmark, with p below 0.001 in both cases, and added 2.45 and 3.92 agent steps per task. Developer-written files raised cost by up to 19 percent.

So the honest summary is narrower than the one circulating, and more useful. A context file does not measurably break your agent. It measurably bills you for the privilege of reading it, and in exchange delivers a change in success that the study cannot distinguish from zero.

The paper was revised, and most coverage still quotes the first version

If you have seen this study reported as "context files cut success by 3 percent and developer files raise it by 4," that is the February version. The authors revised the paper in June. The current version reports the smaller figures above, renames the new benchmark from AGENTbench to CTXbench, and softens the abstract from saying context files tend to reduce success to saying they do not generally improve it.

The direction did not change between versions. The magnitudes shrank, and the statistical framing became more careful. That is exactly the kind of drift that makes secondary coverage unreliable in this area, and it is why we read the revision rather than the summaries. A claim about agent behavior without a version attached is not much of a claim.

Repository overviews are the section that does not work

The most commonly recommended section of a context file is the one the evidence supports least. The ETH team measured how quickly agents found the files they needed to change, and context files did not speed that up. Eight of the twelve developer-written files in the new benchmark included a codebase overview, and those overviews did not help agents locate relevant code faster.

The reason is redundancy. An overview restates what is already in the README, the directory tree, and the code itself, all of which an agent reads on its own. The study's ablation makes the point cleanly: when the researchers removed the repositories' other documentation, the same auto-generated context files improved success by 2.7 points on average and outperformed the developer-written ones. In a documented repository, the context file is a second copy of a map the agent already holds, and you pay for it to read both.

That is also why the conclusion is not "delete the file." The information in a context file is not the problem. Duplicated information is.

Agents follow the file closely, which is why the bill goes up

The cost increase is not a sign that agents ignore context files. It is a sign that they obey them. In the ETH traces, when a file mentioned the uv package manager, agents used it 1.6 times per task, against fewer than 0.01 times when it was not mentioned. Repository-specific tools named in a file were used 2.5 times per task, against fewer than 0.05 otherwise.

Obedience has a price. With a context file present, agents ran more tests, searched more, read more files, and wrote more. Reasoning tokens rose as well: auto-generated files added 22 percent for GPT-5.2 and 10 percent for GPT-5.1 mini on SWE-bench Lite. One model repeatedly re-read a context file that was already sitting in its context window, spending steps to do it. Every line in the file is an instruction the agent takes seriously, and every instruction it takes seriously has a cost whether or not it improves the result.

This is a pattern we keep running into. The system does precisely what it was told, nothing visibly fails, and the cost lands somewhere nobody was watching. We have written before about why the failure mode is not bad code.

A second study found context files make agents faster, not more correct

A January 2026 study from Singapore Management University and collaborators, On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents, appears at first to point the other way. Running Codex on 124 pull requests across 10 repositories, with each task run once with a developer-written AGENTS.md and once without, it found the file cut median runtime by 28.64 percent and median output tokens by 16.58 percent. Both results were statistically significant. Input tokens and total tokens were not, and median total tokens were essentially unchanged.

The two studies do not actually conflict, because they measured different things with different files. The ETH study counted total inference cost, including the cost of reading the file, using mostly auto-generated files in well-documented repositories. The Singapore study counted runtime and output tokens with developer-written files that maintainers had kept current. One measured the price of the map. The other measured time saved by a good map.

The Singapore study also leaves out the question that matters most. It did not evaluate correctness. The authors state that a full correctness evaluation was out of scope, and checked by hand only that the agent produced non-trivial changes on 50 sampled tasks. So the fair reading is narrow: a well-maintained context file can make an agent faster, and there is no evidence yet that it makes the agent more right.

Put the two studies side by side and the useful question changes. It is not whether AGENTS.md works. It is which kind of file works, for what kind of task, and at what cost.

Many committed context files carry avoidable problems

Setting effectiveness aside, the files people commit are full of issues that are easy to name. A June 2026 study from the Federal University of Minas Gerais, Configuration Smells in AGENTS.md Files, examined context files in the 100 most-starred open-source repositories that had one and catalogued six recurring problems. The most common was Lint Leakage, restating rules a linter or formatter already enforces, found in 62 percent of files. Context Bloat appeared in 42 percent and Skill Leakage in 35 percent. The problems travel together: files with both Skill Leakage and Conflicting Instructions were 83 percent more likely to also show Context Bloat.

Formatting does not rescue a file that is mostly noise. A separate study of instruction adherence, covering 1,650 Claude Code sessions, found that file length from 25 to 500 lines, instruction position, how instructions were split across files, and conflicting instructions produced no detectable effect on how well the agent followed them. Restructuring a bloated file does not help. Cutting it might.

The vendors already agree with half of this

Official guidance is closer to the evidence than the /init habit suggests. Anthropic's Claude Code documentation recommends keeping each CLAUDE.md under 200 lines because longer files consume more context and reduce adherence, and favors specific instructions over general ones. OpenAI's Codex best practices say a short, accurate AGENTS.md is more useful than a long file of vague rules, and describe /init output as a starting point that should be edited to match how the team actually builds and ships.

The vendors agree that persistent instructions should be short and specific. The research goes a step further and questions whether some of what the vendors recommend putting there deserves persistent context at all. Anthropic, OpenAI, and the agents.md site all still suggest a project overview or repository layout section, which is the section the ETH study found least useful. The workable position is to take the vendors' advice on brevity and specificity and set aside their advice on overviews.

The format is not going anywhere. OpenAI released AGENTS.md in August 2025, drawing on work across Codex, Amp, Jules, Cursor, and Factory, and in December 2025 it was donated to the Linux Foundation's Agentic AI Foundation alongside the Model Context Protocol. The agents.md site reports it in use across more than 60,000 open-source projects, a figure the format's own site reports and its sponsor repeats rather than an independent count. The question the research answers is not whether to have a file. It is what earns a line in it.

What earns a line in the file

One test does most of the work: could an agent have figured this out by reading the repository? If it could, the line is probably costing you tokens and buying nothing. If it could not, the line is probably worth keeping.

That leaves a short list worth keeping. Exact build, test, and migration commands, particularly any that differ from the defaults for your stack. Tooling an agent would not guess, such as a required package manager or a test suite that needs a service running first. Constraints that reflect real decisions, including the things you never want an agent to touch. Conventions that exist in your team but are not visible in the code.

And a longer list worth cutting. The repository overview that restates your README. Style and formatting rules your linter already enforces. General engineering advice any capable model already follows. Anything /init generated that nobody has checked against how the team actually works. The ETH authors put their own recommendation plainly: skip auto-generated context files for now, and limit human-written ones to instructions that are not already in the README, evaluated before adoption.

The best line in our own file is one we did not write

Our own context file for this website mostly passes the test above, and the line that passes it best is one our tooling wrote for us. The file runs to 102 lines, well inside Anthropic's guidance, in two parts: a block that the Next.js tooling writes and manages, and a section below it that is ours.

The managed block tells the agent that this version of the framework has breaking changes, that its APIs and conventions may differ from the agent's training data, and that it should read the documentation shipped in the installed package before writing code. By the ETH study's own mechanism, that instruction costs money, because it sends the agent off to read. It is also the one thing an agent cannot infer from the repository, since no repository can tell a model that the model's own knowledge is out of date. The research recommends skipping auto-generated context files, but its evidence is really against redundant ones. This generated line is not redundant. It is the least inferable sentence in the file.

Our section ended up organized around the same test, more by accident than design. Its central block is headed "Facts that are not obvious from the code," and nearly every line in it exists because something went wrong or nearly did. The build container has no database, so any page rendered at build time ships without data, which once put placeholder content into production. The site uses three different rendering modes on purpose, and the file says so because the obvious improvement, making them consistent, is the wrong one. A single shared filter decides which posts are publicly visible, and the file warns that a hand-written copy of it is how a draft leaks. The test suite fetches the live site and fails offline. Our CSS tooling silently drops a style rule once its last use leaves the source, which once hid an experiment with no error at all. None of that is visible from reading the code, and each of it cost something to learn.

The style section works the same way. It does not describe the site's design, which an agent can read for itself. It records what design elements mean, which the code cannot show. A rounded pill containing text means something clickable, so a pill used as a label promises a click that never happens. One background treatment belongs to the home page only. Nothing on the site is numbered. The file notes that, repeatedly, a treatment that felt native to the site turned out not to exist, or to exist with a different meaning.

Two parts are the ones the research would question. The first is a short list of documents to read before starting, including an architecture document an agent could find unaided. We keep it for one reason the studies do not cover: one of those documents is a decision log that records why things were not done, and an absence is the one thing reading code can never reveal. The second is a set of working expectations, such as verify rather than assert, and say what you did not check. That is general advice, the category the evidence suggests cutting, and we keep it on belief rather than measurement.

Which is the gap our own file tells us to report. We have not measured its effect on cost or on success. We have not run our tasks with the file and without it. Everything above is the test applied by reading, not by experiment, and the ETH authors are right that a context file should be evaluated before it is trusted. Ours has not been yet.

The file is a cost center until proven otherwise

The useful way to think about a context file is not as documentation. It is as a standing instruction you pay to have re-read on every task, by an agent that will follow it faithfully whether or not it helps. Treated that way, the right size is whatever survives the question of whether each line changes the outcome. For most repositories, that is much shorter than the file they have now.

Written by Synthetixis, an AI-native product studio. More on what most AI software gets wrong.