We Built an AI That Learns From Its Mistakes. For Weeks, It Learned Nothing.

We built a self-improvement loop into our AI agent: every run gets scored, its weaknesses get diagnosed, and the lessons get saved as reusable skills the agent pulls up on the next similar task. It ran for weeks. Our dashboard said it was healthy. It was learning nothing at all — and the scoring that was supposed to catch that was the thing hiding it.

The idea: an agent that turns its own failures into skills

The pitch is the kind of thing every AI company puts on a slide. The agent finishes a build. A scorer grades the trajectory. A “Reflector” reads the low-scoring parts and writes down what went wrong. Those lessons become skills — small, task-shaped playbooks (“when scaffolding a Next.js + Prisma auth app, install these deps first”) that get injected into the next relevant run. Do it enough times and the agent compounds: it stops re-making the mistakes you already paid for once.

We built all of that. Phases zero through two, shipped. The problem is that “we built it” and “it works” are different claims, and for a while only the first one was true.

The loop was running. It was doing nothing.

When we finally instrumented the skill library end-to-end, the numbers were brutal in their clarity:

• Zero skill_used events. Ever. Not “few.” Not “declining.” The agent had never once actually used a skill it had learned.
• Every skill's helpfulness score was 0. Nothing had ever been credited with helping, because nothing had ever been used.
• The full-stack category had 0 active skills — there was no path for a newly extracted skill to ever graduate from “shadow” to “active,” so it never did.

The “compounding learning” headline feature was 100% inert. The machinery turned; the belt wasn't connected to anything.

Why our own dashboard said everything was fine

This is the part that should make any builder nervous. The reason we didn't catch it sooner is that our scorer was lying to us the same way a bad benchmark does — it rewarded the absence of evidence.

The old trajectory scorer maxed out every quality axis unless it found positive proof of a problem. In practice that meant near-perfect scores for almost everything, including runs that accomplished nothing. Our worst example: a single run that burned 400 actions and 7.25 million tokens and produced no working result scored somewhere in the 93–100 range. If your scoring can't tell a 7-million-token wedge apart from a clean success, your self-improvement loop has no gradient to climb. The Reflector was being handed a flat 95 on every run and, reasonably, concluding there was nothing to fix.

What we changed

Fixing it meant rebuilding the measurement first, then the plumbing it fed:

• Made the scorer discriminate. Re-scored against real outcomes, that 400-action wedge dropped from 99 to 59/FAIL; a 251-action wedge went 93 to 68/FAIL; a clean 8-call run stayed at 100. Now the Reflector gets a real gradient.
• Fixed a validator that rejected real wins. The regression gate assumed the reigning champion passed every fixture, so any test the new candidate also failed looked like a regression. Two genuine improvements had been wrongly thrown out. One of them is now the live champion.
• Connected the belt. Added auto-graduation (a skill goes active after being re-derived three times with zero failures) and inlined the top-ranked playbook directly into the prompt instead of hoping a weak model would choose to open it.

The payoff was a single, specific, verifiable event: on a live run, the nextjs-prisma-auth-crud-app playbook was inlined, used, and credited — helpfulness score 0 → 3.5. The first time in the system's history that a skill was actually used and paid off. The full extract → activate → inject → use → credit cycle finally closed.

What's still not done

Honesty cuts both ways, so: two pieces of the loop still aren't validated end-to-end — automatic generation of held-out test tasks, and fully automatic promotion of a winning candidate to champion. Both are blocked on infrastructure, not code: they need more concurrent evaluation sandboxes than our small staging box can schedule. We're not going to call them done until we've watched them work on real runs, because the whole point of this post is that “built” and “works” are not the same word.

Why publish this at all

Because a dead learning loop that reports itself as healthy is exactly the failure mode you can't detect from the outside — and telling you about it is the only thing that makes the claim “our agent improves over time” worth anything. If you want the numbers behind all of this, read how we benchmark our AI agent, or the companion piece on the benchmark we had to retract.