Six months ago, every thread in every AI channel or feed was about the perfect CLAUDE.md. Then it was AGENTS.md. Then it was the prompt library, the skill catalogue, the custom MCP server, the bespoke agent loop. We treated the model like a junior developer that needed detailed instructions taped to every wall.
I’m starting to think we got it backwards.
That harness is trying to fix yesterday’s model
Every skill, every prompt, every workaround, every behavioural rule encodes an assumption about a limitation in a specific generation of AI. We find a weakness, build a process around it, and congratulate ourselves for making the agent more reliable. Then six months later the model no longer has that weakness, and our harness is still forcing yesterday’s behaviour.
A concrete example from my own setup: I had a custom skill that told the model to “always run npm test before claiming a fix worked.” It was good advice for an older model that would confidently hallucinate green checkmarks. Newer models actually run the tests themselves, read the output, and fix the real failure. The skill now just slows things down, because every prompt pays for a rule the model has internalised.
The effort does not seem worth it
The uncomfortable fact is that a lot of carefully crafted harnesses are turning into a new form of technical debt. Not because they’re wrong, but because the model improves faster than the configuration does.
Every layer of instruction the model has to read is tokens paid before it does any real work. A prompt that says “be careful, double-check, never do X” is teaching the model the wrong thing if X stopped being a problem six months ago — you’ve capped its ceiling. And you’re now maintaining a system whose capabilities double every few months, which no software framework has ever evolved at this pace.
CTOs and engineering leads have never had to manage assets that age out faster than they can be documented.
Treat the harness like code, not furniture
The shape of the fix is the same shape as the fix for any kind of accumulation:
- Delete by default. If a skill, prompt, or rule can’t justify itself in the current model, it goes. No “but it was useful in March.”
- Retest on every model upgrade. When you swap models, you re-evaluate the whole configuration, not just the bits that broke.
- Treat every customisation as having an expiration date. When you add something, write down what it’s compensating for. When the compensation is no longer needed, the customisation follows.
The emerging question is whether the optimal harness is the most sophisticated one, or the thinnest one that still does the job.
The risk
The biggest risk in this space right now might be believing that your six-month-old AI setup is still helping. It might be the thing holding you back — a set of guardrails for a model that’s already grown past them, paid for in tokens and capped behaviour on every run.
The fix isn’t more cleverness in the harness. It’s treating customisations like code with an expiration date: delete what the model no longer needs, retest what you keep, and assume the rest will follow.
