When an AI makes a bad programming decision, the obvious conclusion is that the AI made a mistake.
Sometimes that's exactly what happened.
It misunderstood the requirement.
It didn't inspect enough code.
It invented something that wasn't there.
It found a quick local solution instead of the existing shared one.
Those are all real AI problems.
But lately I've started asking another question.
What if the mistake is also telling us something about the software?
What if the AI didn't create the architectural problem?
What if it exposed it?
The wrong answer can still be reasonable
Suppose an AI is asked to fix a problem in a large application.
It searches for the failing behavior.
It finds the code associated with it.
It reads the surrounding implementation.
It sees a place where a small change would correct the problem.
So it makes the change.
The code compiles.
The test passes.
The visible problem disappears.
Then somebody who knows the larger system looks at the result and says:
Why did you put that there?
We already solve that in the shared layer.
That's an AI failure.
But it may also be something else.
From the part of the codebase the AI inspected, the local solution looked legitimate.
That should make us curious.
Why did it look legitimate?
Architecture is partly about making ownership obvious
We usually think about architecture in terms of separation.
This layer owns the data.
That package owns rendering.
This component handles Markdown.
That service owns persistence.
Those boundaries may be perfectly clear in an architecture document.
They may be perfectly clear to the programmer who designed the system.
They may even be perfectly clear to the team that has maintained it for years.
But are they clear from the code?
That's a different question.
If an AI can enter the codebase through a bug report, follow a reasonable trail through the source, and arrive at the wrong owner for a responsibility, then perhaps ownership isn't as obvious as we thought.
That doesn't excuse the AI for failing to inspect further.
It does give us useful information.
A new human programmer might make the same mistake.
AI doesn't carry the history the way we do
This becomes particularly important in mature software.
A human developer who has worked on a product for ten years knows things that may never have been written down.
They know that a particular file looks important but is really legacy code.
They know that another component became the source of truth after a refactor three years ago.
They remember that somebody once tried implementing a feature in one place and later moved it somewhere else.
They know which code is authoritative because they lived through the decisions that made it authoritative.
AI doesn't necessarily have that history.
Even if the history exists somewhere in the repository, the AI still has to find it and recognize that it matters to the current problem.
So AI enters the code much more like a new programmer.
It looks at what's there now.
It follows the structure that's visible now.
It makes inferences from the evidence that's available now.
That can be a weakness.
It can also be a very useful architectural test.
Would a new programmer make the same mistake?
This has become one of my favorite questions.
When an AI puts a change in the wrong place, I ask:
Would a competent programmer who was new to this project have any reason to think that was the right place?
If the answer is no, then we probably have an AI problem.
The naming may be clear.
The ownership may be obvious.
The shared implementation may be directly referenced.
The AI simply failed to do the work necessary to understand it.
But what if the answer is yes?
What if there really are two places that look like they own the same responsibility?
What if the shared implementation is hidden behind three layers of indirection while a product-local implementation sits right next to the bug?
What if the only thing identifying one file as authoritative is a sentence in a project note?
What if the directory structure doesn't match the architecture anymore?
Now the AI mistake is telling us something.
The codebase can contain more than one truth
This is one of the easiest problems for a mature application to accumulate.
A behavior starts in one place.
Later, another product needs something similar.
Some code gets copied.
Then one copy is improved.
Another one is patched.
A third version gets wrapped.
A fourth becomes the one everybody is supposed to use.
Eventually there may be one official source of truth and three unofficial ones still sitting around looking perfectly functional.
Humans who know the history understand the distinction.
AI sees four implementations.
Then we become angry when it chooses the wrong one.
Again, the AI should have inspected further.
But perhaps the better question is why the wrong implementation still looks authoritative.
Local fixes are architectural probes
This is where AI failures can become valuable.
An AI tends to start near the problem.
That's not always a bad strategy.
If a button doesn't work, look at the button.
If an image doesn't render, look at image rendering.
If a copied page loses content, look at the copy operation.
The problem is that the visible failure and the architectural owner of the behavior aren't always in the same place.
When the AI makes a local fix and we discover that the real responsibility belongs somewhere else, we've learned something about the path from symptom to authority.
Maybe that path is too obscure.
Maybe ownership is split.
Maybe the API allows behavior that should only be possible through another layer.
Maybe a product still contains code that should have been removed when the shared implementation was introduced.
The wrong fix can act like a probe.
It shows us where the architecture invites the wrong conclusion.
This isn't an argument for accepting bad work
There's an important distinction here.
I'm not suggesting that we excuse an AI because its mistake teaches us something.
Wrong code is still wrong code.
If a responsibility belongs in the shared layer, a product-local patch doesn't become acceptable just because it was understandable.
The point is what happens next.
We can simply reject the patch and tell the AI:
Don't do that again.
Or we can reject the patch and ask:
Why was that available as a reasonable choice?
Those lead to very different outcomes.
The first corrects one worker.
The second may improve the system.
Last week we talked about guards
In the previous Guide, Mostly Guidelines, I argued that important AI programming rules eventually need to move out of prose and into guards.
Don't just tell the AI not to cross a dependency boundary.
Make the dependency rule enforceable.
Don't merely tell it which implementation is canonical.
Where possible, structure the system so there is only one implementation to use.
That still matters.
But there's a step before writing the guard.
We need to understand why the wrong choice was available.
Sometimes the answer is simple.
The AI ignored a clear rule.
Fine. Guard the rule.
Other times the violation reveals that the architecture itself contains ambiguity.
Now the guard is only part of the cure.
The architecture may need attention too.
A guard around ambiguity isn't the same as removing ambiguity
Imagine we have two implementations of the same responsibility.
One is old.
One is canonical.
We can write a guard that prevents the AI from modifying the old one.
That may solve the immediate problem.
But why is the old implementation still there?
Maybe it can't be removed yet.
Maybe there's a legitimate compatibility reason.
Maybe the two aren't really duplicates after all.
Those are architectural questions.
A guard can keep the AI from making the wrong move while we answer them.
But the cleaner solution may eventually be to remove the false choice altogether.
That's where AI can become a surprisingly useful design critic.
Not because it understands the architecture better than we do.
Because it doesn't.
Forgetting the history can be useful
Humans often compensate for weak architecture with institutional memory.
We know where the bodies are buried.
We know which folders not to touch.
We know which class name is misleading.
We know that this API exists but shouldn't really be used anymore.
We know that one implementation is technically public but effectively obsolete.
That knowledge lets us function inside a system that may not communicate its intent very well.
AI doesn't reliably carry all of that baggage.
So it trips over things we learned to walk around.
That's annoying.
It's also informative.
Every time an AI makes a mistake that a knowledgeable maintainer would never make, we have the opportunity to ask whether the knowledgeable maintainer is compensating for something the architecture should communicate on its own.
Repeated AI mistakes deserve special attention
One mistake can be random.
Two mistakes can be coincidence.
But if different AI workers keep arriving at the same wrong solution, I pay attention.
Especially if different models do it.
Especially if the workers were given different prompts.
Especially if we strengthen the instructions and the same pattern appears again later.
At that point, we may not be looking at an instruction problem.
We may be looking at a path the architecture naturally encourages.
That's valuable information.
The machine is repeatedly telling us:
Given what I can see, this looks like the obvious place to make this change.
We shouldn't automatically believe it.
But we should probably investigate why it keeps saying that.
There is a difference between hiding a mistake and fixing its cause
Suppose an AI repeatedly adds product-specific behavior to a component that should remain generic.
We could put a warning in the prompt.
Then another warning in the architecture document.
Then a guard script.
Then a qualification check.
All of those may be useful.
But if the generic component exposes exactly the extension point that keeps inviting product-specific behavior, perhaps the interface itself deserves another look.
The guard stops the symptom.
The architecture determines whether the temptation remains.
That's the larger lesson.
AI is becoming part of the feedback loop
We usually think of AI as a tool that operates on our architecture.
I think it can also become a tool for evaluating that architecture.
Not by asking it:
Is this good architecture?
That's far too vague.
Instead, watch what happens when independent agents work inside the system.
Where do they repeatedly get confused?
Where do they duplicate behavior?
Where do they invent missing abstractions?
Where do they choose the wrong authority?
Where do they keep trying to cross a boundary?
Where do they repeatedly need a paragraph of special instructions before they can work safely?
Those are signals.
Not every signal represents a design flaw.
But repeated signals deserve investigation.
The goal isn't to make architecture AI-friendly
I don't want software designed around the limitations of this year's AI model.
That would be another kind of mistake.
Models will improve.
Their search behavior will improve.
Their ability to maintain context will improve.
The tools around them will improve.
But architecture that clearly communicates ownership and responsibility is useful regardless of who is working on the code.
Humans benefit from one source of truth.
Humans benefit from clear boundaries.
Humans benefit when obsolete implementations disappear.
Humans benefit when dependency direction is obvious.
Humans benefit when the system doesn't require tribal knowledge to understand where a change belongs.
If AI pressures us to improve those things, that's not designing for AI.
That's AI exposing work we probably should have done anyway.
Treat the mistake as evidence
I don't think every AI failure deserves an architectural review.
Sometimes the model simply got it wrong.
But I no longer want to automatically treat an incorrect implementation as the end of the story.
I want to know why it looked correct.
What did the AI see?
What didn't it see?
Which implementation appeared authoritative?
Which boundary wasn't visible?
Which local clue outweighed the broader architecture?
Could another programmer reasonably reach the same conclusion?
Those questions turn a failed change into evidence.
And sometimes the answer will be uncomfortable.
The AI didn't create the architectural problem.
It just wandered into the part of the system where we'd been hiding it.
Related field note
For the experience that made this lesson particularly obvious to me, read The Wrong Fix Was the Right Warning.