Sometimes You Have to Stop the Train

PageSnip was getting close.

Not small-project close, where you’ve got three things left on a checklist and you’re pretty sure two of them are cosmetic.

This is a large application that has been growing for a long time. It does a lot. We’d already spent substantial time going back through older parts of it, improving performance, cleaning things up, testing the ugly cases, and making sure the product could handle the kind of scale we’d been promising.

Some of that work had turned out spectacularly well.

The indexing and storage side, for example, had reached the point where the size of a normal user’s collection simply wasn’t something I was worried about anymore.

Then I started seeing little problems in the sharing system.

None of them, by themselves, looked like a reason to stop.

That was the problem.

These bugs were too small

The individual glitches were fixable.

That almost made them more dangerous.

Something would behave incorrectly in one path. We’d find it. The AI would inspect the code. We’d make the correction. The tests would pass.

Then something nearby would act strangely.

Fix it.

Test it.

Move on.

Except I finally found myself asking a different question.

Why am I seeing this many little problems here?

These weren’t brand-new experimental features. Some of this code had been around for months.

At this point, I wasn’t as interested in the bug I was looking at as I was in the fact that another one had appeared.

That changed the assignment.

I stopped the release work.

Sharing wasn't really sharing anymore

There was another realization happening at about the same time.

PageSnip’s sharing system had started life with a pretty modest idea behind it.

One of the examples I used early on was a recipe.

You’ve cleaned up a recipe in Markdown on your computer. You’re cooking dinner. You want to pull the recipe up on your phone in the kitchen.

PageSnip could share it across the local network.

Simple.

Except the product didn’t stay there.

Over time we added more ways to get information out of PageSnip.

There was the LAN-based Share system with its own server.

There was hosted sharing.

There was static publishing so somebody could publish material to their own website.

There were different kinds of endpoints. A person might publish an individual Page, a Library, or another point inside a much larger structure.

At some point I realized that calling all of this “sharing” was understating what the product had become.

Publishing wasn’t another feature anymore.

Publishing had become one of the things PageSnip fundamentally did.

That mattered because once the product changes, the architecture has to be willing to change with it.

You can’t keep treating a first-class responsibility as an attachment to an older feature just because that’s where it started.

I didn't want to fix this after release

There was also a much more practical reason to stop.

If I’m going to discover a structural problem in a system that manages people’s information, I want to discover it while the only data I can damage is mine.

This is the good time.

This is the cheap time.

Once you ship, somebody somewhere is going to build a repository you never imagined.

Then somebody else is going to build one ten times larger.

People are going to depend on it.

They’re going to keep things in it that matter to them.

If I discover after that point that we need to change something fundamental underneath their data, the job becomes much harder.

Now I’m not just repairing software.

I’m protecting everybody’s history while I do it.

So when the little glitches started looking like a pattern, I wasn’t interested in getting PageSnip out the door and congratulating myself for making the date.

I wanted to know what was underneath them.

One giant AI audit wasn't the answer

The obvious thing to do was an architectural audit.

But that exposed another version of the same problem.

PageSnip had become large enough that asking one AI agent to:

Go through this repository and tell me everything that's architecturally wrong with it.

would have sounded impressive and probably produced an impressive report.

I wasn't convinced it would produce the depth I wanted.

There were too many different questions hiding inside that one question.

So we didn't do one audit.

We did several.

Different AI agents got fresh copies of the repository.

Each one got a specific job.

One could look at how important mutations moved through the application.

Another could look for cases where multiple pieces of code independently implemented what was really the same logical operation.

Another could look at module responsibilities and places where unrelated concerns had accumulated together.

The point wasn't to get several AIs to vote on whether PageSnip was good code.

The point was to keep each investigation narrow enough that the agent could actually dig into it.

That turned out to be important.

I'd reached the point where even the audit needed architecture.

The AI was looking through a keyhole

One of the patterns that came back helped explain some of what we'd been seeing.

Several PageSnip source files had grown very large.

We already knew that.

Earlier, we'd made a conscious decision not to break all of them apart immediately because doing a big structural refactor while trying to stabilize a release had its own risks.

That decision wasn't crazy.

But the size had created another problem.

An AI working on a particular function might find the function, inspect the surrounding code, understand what was happening, and make a perfectly reasonable change.

The problem was that the relevant truth might be much farther away.

The AI wasn't necessarily misunderstanding what it saw.

It was looking through a keyhole.

The piece visible through the keyhole made sense.

What it couldn't always see was that another part of the room contained another implementation of essentially the same responsibility.

That explained a lot.

We had more than one person doing the same job

This led to one of the architectural rules we're putting in place:

One logical mutation. One semantic implementation.

I tend to think of this in terms of people.

Suppose one person in a company has a particular job.

Everybody who needs that job done can call that person.

They can call a hundred times a day.

That's fine.

What you don't want is five different people quietly deciding that the same job belongs to them and each developing their own slightly different rules for doing it.

Software isn't much different.

If PageSnip has one logical operation, there needs to be one semantic authority for that operation.

Every workflow that needs it calls that authority.

That way, when we fix the behavior, we fix the behavior.

Not one copy of the behavior.

It sounds obvious when stated that way.

It isn't necessarily what naturally happens while a large AI-assisted codebase grows one assignment at a time.

Then I walked away

The audits gave us good information.

The AI made good recommendations.

Then I did something highly technical.

I walked away from the computer.

I needed to think about what PageSnip had become rather than just what the code currently looked like.

I thought about publishing.

I thought about the user sitting in front of the application.

I thought about which capabilities should be exposed and how complicated some of them had become.

I thought about what should happen if somebody had a million documents and changed one item in the middle.

I didn't want that change to mean the entire universe needed to be regenerated.

I thought about the fact that somebody might publish an endpoint from almost anywhere in that structure.

When I came back, I had some changes I wanted to make to the proposed architecture.

Some of those ideas were good as they were.

Some got better after I put them back in front of the AI and let it work through the consequences.

That's the part of AI-assisted programming I think people sometimes miss.

The answer wasn't for me to out-code the AI.

It wasn't for me to sit down and manually rewrite the publishing system to prove I was still necessary.

My contribution was changing the question.

Then we audited the answer

Once we had a better idea of where we wanted the architecture to go, we didn't immediately start ripping code apart.

We went back and audited again.

This time the question wasn't:

What's wrong with this code?

It was closer to:

Given these architectural goals, what do we already have that's good, what conflicts with the goals, and what actually needs to change?

That produced another useful result.

We didn't need to throw the baby out with the bathwater.

There was plenty of good code in PageSnip.

There were components that were proven and doing their jobs well.

There was no reason to rewrite those just because other parts of the architecture needed to be cleaned up.

The job became more surgical.

Keep the things that already have the right responsibility.

Remove the competing paths.

Establish the authority boundaries.

Break responsibilities apart where the current concentration makes them unsafe to maintain.

Then make those rules explicit enough that the next AI session doesn't have to rediscover them.

That's when the programmer has to be in charge

It's easy to look at AI-assisted development and think the important question is who wrote the code.

I don't think that's a particularly interesting question anymore.

The AI writes a lot of my code.

Good.

It does it faster than I could.

In many cases, it does it better than I would.

Also good.

But none of that would have stopped PageSnip from shipping.

The individual fixes were available.

The tests could keep turning green.

There was always another patch we could make.

Somebody had to look at the pattern and say:

No. We're stopping here.

Somebody had to decide that sharing had become publishing.

Somebody had to decide that publishing had become important enough to deserve first-class architectural treatment.

Somebody had to decide that one broad audit wasn't enough.

Somebody had to divide the questions among several independent agents.

Somebody had to read the answers together.

Somebody had to decide what code deserved to survive.

And somebody had to decide that having the product out a little sooner wasn't worth discovering these same problems later with customer data sitting on top of them.

That's the part of programming I don't see disappearing.

The AI can drive an incredible amount of the work.

But somebody still has to conduct it.

And sometimes the most important thing the conductor can do is stop the train.

That's what we did with PageSnip.

And I'm glad we did it before anybody else's data was along for the ride.

Related Guide

For the broader lesson behind this experience, see Why Being the Programmer in Charge Still Matters.