I recently listened to an episode of The OpenAI Podcast with Tejal Patwardhan, who leads OpenAI's frontier evaluations team.
Her job is to help test the newest models and determine how capable they're becoming. One of the most interesting things she talked about was how hard it's getting to design tests that remain difficult enough to tell them very much.
The old tests are getting too easy.
The models aren't just improving. They're improving quickly enough that the people evaluating them have to keep finding new ways to push them to their limits.
As the conversation was wrapping up, she made a point that stuck with me.
If someone has tried ChatGPT and asked it to do something but wasn't happy with the result, they should try it again a week or two later. There's a very real chance that the result will be different.
I think that's some of the best practical advice I've heard about AI.
The model you tried may not exist anymore
When someone says, “I tried ChatGPT and it wasn't very good,” the first question that comes to my mind is:
When?
I'm not saying their experience wasn't real. It probably was.
But a conclusion reached six months ago may be based on a model, a system, and a set of capabilities that have changed several times since then.
We still talk about these products as if they're conventional software. We use the same names, so we assume we're using the same thing.
But the name on the screen may be one of the few things that hasn't changed.
The underlying model may be different. The reasoning may be deeper. The model may have access to tools it didn't have before. It may be better at deciding when to inspect something, when to run a test, and when its first answer isn't good enough.
The AI someone tried a few months ago may no longer exist in any meaningful sense.
What I'm seeing between Claude and ChatGPT
I use both Claude and ChatGPT regularly.
With Claude, I normally use Sonnet. It's more token-effective for the subscription I have, and I'm not interested in paying for a much more expensive plan or burning through a monthly allowance in a few hours.
Sonnet usually does a good job for me. It's written some good code, and it's helped with marketing, planning, and other kinds of work. Depending on the task, I'll use medium thinking or high thinking.
With ChatGPT, I tend to use the highest reasoning model available.
That's a good news, bad news situation.
The bad news is that the higher reasoning models are slower. They spend more time thinking through the problem. They don't always start producing code as quickly as a faster model.
But that's also the good news.
They're spending more time thinking through the problem.
A few years ago, people often talked about Claude as the model that appeared to sit with a difficult programming task, work through it, build things, test things, and keep examining the problem before it answered.
I'm now seeing that same kind of behavior regularly from the higher reasoning models in ChatGPT.
The answer may take longer, but the result is usually better.
The search that was fast until it wasn't
We recently made an enhancement to the search feature in PageSnip.
The search itself was extremely fast, but we discovered that it was only returning the first 200 matching rows.
For many searches, that's plenty. But a broad search can produce more than 200 results, so we needed a way to return the first group and then let the user load more.
The first change solved that problem. It returned the first 200 and provided a way to continue through the remaining results.
But it created another one.
Before displaying those first 200 rows, it waited until the program had determined how many total matches existed. On a small result set, that wasn't noticeable. On a large one, it meant that the user could be left waiting even though the first 200 matches were already available.
We needed the first results to appear immediately.
The complete count could come later.
Then the AI built a 10,000-row test
On the next pass, I watched ChatGPT work through the problem.
It didn't just read the code and suggest a change.
It created a test containing 10,000 matching rows. It ran the search against that data and discovered that calculating everything up front could exceed the two-minute timeout.
That gave it a real failure to examine instead of a theoretical one.
From there, it changed the logic.
The program would find the first 200 results and return them immediately. Then it could continue working on the full count without blocking the user from seeing the results already found.
That struck me.
I wasn't using a separate coding agent. I was working through the regular browser-based ChatGPT conversation, just as I do every day.
But the model was doing the kind of work people associate with agentic programming. It was creating realistic test data, running the software, measuring the behavior, finding the bottleneck, and revising the design based on what the test revealed.
It didn't merely write code faster than I could type it.
It created a way to prove whether the code worked.
That's not the ChatGPT many people remember
I spend a lot of time around programmers, particularly in the Clarion community, and I still hear comments such as:
I tried ChatGPT, but I wasn't happy with it. I moved to Claude, and Claude works better for me.
That's fine.
Claude may work better for them. It may be a better match for the kind of code they write, the way they explain a problem, or the amount they want to spend.
My concern isn't that someone prefers Claude.
My concern is that they may be comparing the Claude they're using today against a version of ChatGPT they tried months ago.
That isn't a current comparison.
It's a memory.
The reverse is true too. Someone who tried Claude a year ago and preferred ChatGPT shouldn't assume the comparison still holds.
Use whichever model works best for the job. But every now and then, go back and test the others again.
The field isn't waiting for our opinions to catch up
As I was thinking about this article, Moonshot AI in China released Kimi K3, another model creating considerable attention for its coding, reasoning, context size, and open-weight release.
Before that, DeepSeek had been the Chinese model everyone was talking about.
Now there's already another name competing for that attention.
Kimi K3 may prove to be as important as its early reception suggests, or another model may pass it before most people have even had time to try it. That's almost beside the point.
The important part is the pace.
New models aren't arriving after we've had years to understand the previous generation. They're arriving while we're still arguing about conclusions we reached from models that are already old.
The field isn't waiting for our opinions to catch up.
We're changing too
The model isn't the only part of this relationship that's improving.
I've been doing session-based programming with AI almost every day for the past couple of years. During that time, I've learned a great deal about how to work with it.
I've learned to pay attention to thread length. I've learned that a conversation can accumulate enough history and abandoned ideas that it becomes harder for the AI to see what currently matters.
I've learned when to prepare a handoff and move the next stage of the work into a clean conversation.
I've also made a habit of asking the AI how we're doing. Do we have enough clean context left to begin something large? Are earlier instructions starting to interfere with the current work? Is this a good stopping point?
Those are questions most casual users probably never think to ask.
Someone trying AI for the first time may give it a vague paragraph, receive a vague answer, and decide that the technology isn't useful.
When they return a few months later, the model may be better.
But they may also be better at explaining what they need.
That's why retrying the same task can produce such a dramatic difference. The technology has moved forward, and so has the person using it.
Don't confuse disappointment with a verdict
There are still things AI doesn't do well.
There are times when it produces the wrong answer, chooses a poor design, misses an instruction, or confidently heads in the wrong direction.
I'm not suggesting that anyone keep repeating the same request until the AI happens to produce something they like.
I'm suggesting that we stop treating disappointment as a permanent verdict.
Save the task.
Save the result.
Remember what went wrong.
Then come back later and try it again.
Try the original request and see whether the model has improved. Then try it with what you've learned about providing context, setting constraints, managing the conversation, and testing the result.
You may discover that the tool still isn't ready for the job.
You may also discover that the job it couldn't do a few months ago has become routine.
Try, try again
We're used to technology improving from one year to the next.
We're not used to it changing enough in a few weeks that an old failure may already be obsolete.
That's the adjustment people need to make with AI.
Regardless of whether you're happy with the model you're using or disappointed by one you've abandoned, go back every now and then and reinvestigate it.
Run the old test.
Ask the old question.
See what happens now.
At the rate these models are changing, one thing seems certain:
We ain't seen nothing yet.
Related guide
The companion guide, A Failed AI Experiment Isn't a Final Answer, provides a practical way to save, repeat, and evaluate an AI task instead of letting one disappointing result become a permanent conclusion.