- Published on
Coding Agents Do Complicated. Old Codebases Are Complex.
I started with a guess: AI is good at complex problems and not so good at complicated ones. After reading around, I think that's about half right. And the wrong half is the one that matters when you point an agent at an old codebase.
This post leans on two people in particular: Addy Osmani, whose post Brownfield Agentic Engineering gives it most of its practical advice, and Ryan MacDonald, whose talk Prompts to Pipelines is where its example of a green check over a real bug comes from.
Complicated isn't a smaller version of complex
The two words sound like points on one difficulty scale. They aren't. Sholom Glouberman and Brenda Zimmerman drew the distinction in a 2002 paper on health care reform, with three examples. Baking a cake is simple: follow the recipe and it mostly works. Sending a rocket to the moon is complicated. It takes hard calculations and precise engineering, but the principles are known, and expertise makes the next launch better. Raising a child is complex. No recipe guarantees the result, and what worked with one child may not work with the next, even with twins.
Dave Snowden's Cynefin framework draws the same line by asking when you can know cause and effect. In a complicated problem you can know it up front, through analysis, if you have the expertise. So you gather the facts, analyze them, and respond. In a complex problem you only know it in hindsight. Analysis won't get you there, so you run small experiments that are safe to fail, watch what happens, and do more of what works.
Here's how I'd map that onto code:
| The 2002 example | In a codebase | What an agent needs | |
|---|---|---|---|
| Simple | baking a cake | rename a variable in a well-tested module | a clear instruction |
| Complicated | sending a rocket to the moon | implement a known algorithm, or port a module that has a full test suite | the spec and the tests |
| Complex | raising a child | change how billing works in a ten-year-old system | what the people around it know and never wrote down |
In plenty of ways the rocket is harder than the child. What separates them is whether you can tell an answer is right before you ship it.
Where agents land
In The Shifted Complexity, Susanne Pfeiffer argues that AI now handles much of the simple-to-complicated work junior developers used to learn on. What's left for the developer is more of the complex part, like requirements and the people around the system. She also warns that "prompt in, code out" makes a task look simple when it isn't. In Cynefin, that's the cliff between clear (the framework's current name for simple) and chaotic, where work that looks routine falls into chaos because nobody noticed its uncertainty.
Chris Richardson goes a step further in Why GenAI-based coding agents are a complex domain in Cynefin. Most developer tools are complicated. A compiler or a database follows stable rules you can learn. An agent doesn't. The same prompt can give different results, and a small change in wording, context or model version can change a lot. So using one is a complex activity in itself. His warning is that teams who treat agents as fancy automation will trust the output too much and spend too little on feedback loops and guardrails.
That's the half of my guess that holds up. Agents can work on complex problems, and they make the experimenting part of complex work cheap. Osmani makes this point about rewrites: teams can now have agents try several versions, in different languages or frameworks, test and profile each one, and pick, often for much less than it used to cost. So agents changed the price of trying. In his words, "They haven't changed the evidence required to choose one."
That sentence is where I'd draw the line. An agent is a fast source of candidates. Whether that helps depends on whether you can check a candidate. In complicated work you can, by analysis, as long as the rules are written down somewhere a machine can read them, like a spec or a test suite. In complex work you only find out by trying.
METR makes a related point from the other side. In the study I come back to below, it warns that benchmarks may overstate what models can do, because they only measure well-scoped tasks that can be scored automatically. A task you can score is a task you can check.
Why old codebases are complex
Osmani's Brownfield Agentic Engineering starts from a definition I like. In a brownfield system, the repository no longer fully describes how the thing behaves. The rest lives in people's heads and in what other teams expect of it. Let an agent loose on that code unsupervised, he says, and you can end up with something that works but has the wrong design and brittle tests.
To me that's the complex domain. An agent can read every file and still miss the rule that matters, because the rule isn't in a file. You learn it when a change breaks something.
Not every unwritten rule is out of reach, though. Some live in an old issue, a pull request thread or an incident write-up, and an agent with access can find them. Osmani suggests a read-only research pass before risky changes, one that writes a short memo where every claim points at a file, issue, ownership record or dashboard. The rules that live only in someone's head are the ones no research pass will find.
There's an old rule for this. In The Thing (1929), G. K. Chesterton describes a reformer who finds a fence across a road, sees no use for it and wants it gone. His advice: go and find out why it was put there, and then we can talk about removing it. An agent cleaning up odd-looking code is that reformer, and usually there's nobody around to ask.
There's some evidence for this in a randomized trial METR ran in early 2025. Sixteen experienced open-source developers worked real issues in their own repositories, which averaged ten years old and over a million lines of code. With AI allowed, they took 19% longer, and afterwards they believed it had made them 20% faster. Among the factors the paper thinks contributed are large, complex repositories and implicit repository context.
Two caveats matter. METR says its result doesn't show that AI slows down most developers. Its February 2026 update also says developers are probably sped up more now, though its data is only weak evidence for how much. What I take from it is that whatever the code doesn't say gets expensive, and that feeling faster isn't a measurement.
Turning complex into complicated
If a brownfield change is complex because the rules aren't written down, the fix is to write them down in a form something can check. A post on executable specifications puts this in Cynefin terms. A vague prompt leaves an agent with loose constraints, and the result only shows itself once it runs. A precise, executable spec gives it constraints you can verify before shipping, which is what the complicated domain looks like. The author is careful to say Snowden hasn't endorsed this reading. I'd hold it loosely too, but it's a useful way to read Osmani's advice, most of which comes down to getting rules out of people's heads:
- Write down what the code can't say: business rules, trade-offs, the history behind odd-looking code, conventions no tool enforces. Leave out anything the agent can work out from the code itself.
- Pin today's behavior with characterization tests before changing anything, ugly parts included, because some of that ugliness is what the business runs on. Don't let the session that makes the tests pass be the only one that wrote them, or you get a green suite that encodes whatever it just invented.
- When the same review comment keeps coming back, turn it into a lint rule, a hook, a type or a test. Keep prose for the constraints nothing can enforce.
- Split the codebase into zones. Well-tested, isolated code is green, and an agent can work there in a tight loop. Mixed code is yellow and gets characterization tests first. Auth, billing and permissions are red, where a person pairs on every step. A person draws the map, not the agent.
The ugly behavior in that second point has a name. Hyrum's Law, from Hyrum Wright, says that once a system has enough users, someone will depend on every behavior they can observe, whatever the documentation promises. xkcd has the best picture of it:
xkcd 1172, "Workflow", by Randall Munroe, CC BY-NC 2.5.
Here's that idea as a toy. Turn the practices on and see which rules move out of people's heads:
An agent is asked to clean up an old invoicing API. The rules below are made-up examples of what a codebase like that knows but its code doesn’t say.
Practices in place
Invoice totals round half up. It looks like a bug, but billing depends on it.
only in someone’s head
A partner’s nightly export reads the total column by name.
only in someone’s head
The generated API clients are never edited by hand.
only in someone’s head
An old mobile app version still sends the previous field name.
only in someone’s head
All four live in people’s heads. A change can pass every test and still break them, and you find out after it ships.
The note is the one that never becomes a check. Some rules live outside anything you can run, like a partner's export job, and the best you can do is write them down where a reviewer will see them. That's the case Osmani keeps prose for.
Even "checked on every change" is weaker than it sounds. A check answers the question someone wrote it to ask, and only while it's switched on. Bash has a classic example: in local result=$(validate_input "$path"), the $? that follows reports the exit status of local, which is always 0, so an error handler that tests it never runs, and every test that doesn't feed in bad input still passes. ShellCheck does flag this by default, as SC2155, so whether you're covered depends on which rules your project actually runs. The example comes from Ryan MacDonald's talk Prompts to Pipelines (August 2026), where it shows checks staying green over a real failure. His answer has two parts: adversarial review, and a rule that an agent can't call work done without evidence for each claim. Osmani's advice on review fits here: start fresh and work backward from the acceptance criteria, because a clean reviewer is more likely to notice a test that proves the implementation but misses the requirement. A reviewer who didn't write the change is the check for questions nobody encoded.
So, complex or complicated?
My answer to my own question: agents do well on complicated work once someone has written the rules down, and they're useful in complex work because they make experiments cheap. What they don't bring is the evidence. In a brownfield system that evidence starts in people's heads, and getting it out into tests and notes is most of the job.
Osmani also points out that agents make the cost of ambiguity visible. The conventions a team never wrote down always cost something, in onboarding and in review, but quietly. With an agent, each one turns into the same correction made by hand, over and over. He sees that as a stronger case for maintenance work teams already knew was worth doing, and I agree.
Before I hand a task to an agent
Now I ask myself two questions:
- If the agent gave me three versions, could I tell which one is right without shipping any of them?
- Is there anyone I'd want to ask before changing this code?
If I can pick the right version and there's nobody I'd need to ask, the task is complicated, and I let the agent run with it. Otherwise I do the complex part myself first: I go ask that person, write down what they tell me, and then hand over what's left in small pieces.
Enjoyed this post?
Subscribe to get notified about new posts and updates. No spam, unsubscribe anytime.
By subscribing, you agree to our Privacy Policy. You can unsubscribe at any time.
Discussion
Loading comments...
This website is still under development. If you encounter any issues, please contact me.
Found this useful? Support the blog
- Authors

- Name
- Mamun Rashid
