What AI Coding Benchmarks Miss About Maintaining Code

A model that scores near the top of a coding leaderboard can still leave a codebase in worse shape after six months of feature work. That is not a contradiction. It is a sign that the leaderboard measures something different from what a product team lives with.

Why single-task benchmarks flatter AI coding agents

The most familiar style of benchmark takes a small, real issue from an open source project, rewinds the repository, and asks the model to fix it. Hidden tests decide whether it passes. A second style goes the other way: rebuild an entire application from a full specification in one long run. A third adds a judge that checks conventions and rules.

All three share one assumption: the model is handed the whole problem up front. Real product work does not arrive like that. A founder ships a first version, learns something, then asks for the next feature, and the one after that. Nobody knows the final shape when the first line is written.

Newer benchmarks try to model this by asking an agent to build something from scratch, then extend the same code through a series of checkpoints. The pattern they report is uncomfortable: as unattended extensions pile up, failing tests trend upward across models. Passing today's task says little about how easy tomorrow's task will be.

Maintainability is a skill models are not yet trained for

Maintaining a codebase means removing code, merging duplicated ideas into one abstraction, and controlling complexity. These are judgement calls that experienced engineers build up by living with bad architecture over many years. Improvements in areas such as computer use do not automatically carry over to them.

This matters for planning. Some teams accept messy output today on the assumption that a future model will clean it up. We would not bet a product on that. Nobody can say when, or whether, that capability arrives, and the debt accrues in the meantime.

Practices that keep AI-assisted code maintainable

Since the model cannot be relied on for taste, the process has to supply it:

• Keep changes small. Small pull requests get reviewed and merged. Large ones tend to stall or get waved through.
• Checkpoint large work. A big feature is far easier to debug when it is checked in stages, rather than discovered to be broken only at the end.
• Read your own output first. Nobody should send a colleague code they have not read themselves. Reviewing machine output that its author never understood adds no value.
• Plan before building anything large. A short design note that settles the end state and the order of work removes many bad solutions before any code exists. Ambiguous work benefits from a requirements pass first.
• Do less in parallel. More agents running at once means more output to verify, and verification is the real limit.

Review also has a teaching role. A comment that asks why a pattern was chosen, or where else it appears in the codebase, builds the judgement a team needs. A comment that merely rewrites the code teaches nothing.

What this means for web product teams

For founders choosing how to build, the takeaway is practical. Speed on a first release is easy to demonstrate, but the cost shows up in the tenth feature. A studio worth working with plans for that: small reviewable changes, humans who own architectural decisions, and checks that run on every extension of the code.

At Curiosive, senior engineers review every line for exactly this reason. AI gets a product moving quickly, while engineering discipline keeps it moving a year later.