My router sent a bug report to the wrong skill

I typed "the checkout test fails intermittently" into my own router and it told me to run /proof.

Wrong skill. That sentence is a bug report. The skill for a bug report is /reproduce, which refuses to let anyone touch the code until there's a failing test. /proof is for the other end of the job, when you're about to say "done". My router had just recommended the victory lap to a person who hadn't found the bug yet.

And it exited 0. Success. Green. Nothing in the output suggested anything was off.

The regex table

/compass is the suite's router. I'd love to tell you it understands your task. It matches keywords. The /reproduce pattern was \b(bug|regression|reproduce|failing|broken|debug)\b. My sentence said "fails", and "fails" is not "failing". "Fail" and "flaky" miss it too. Meanwhile the word "test" walked straight into the /proof pattern and took the wheel.

I wrote that table. I wrote it using my own phrasing, because that's the phrasing I thought of, and I called it routing. Change the sentence to "the checkout test is failing" and it routes to /reproduce perfectly. The router worked for exactly the person who built it. Congratulations to me.

I found this on a real run against ultimate-harness, not in a unit test, which tells you what my unit tests were worth on this particular question. The write-up logged it as a keyword-shape problem. The fix: that sentence routes to /reproduce now, and I added tests for false positives, because widening a regex is how you create the next misroute.

The re-verify that didn't

Same run, different skill. /proof has a perf branch: two checks over the same workload, and a ratio between them that has to stay under a bound. I ran the real thing with vitest's forks pool as the baseline and threads as the change. One run passed at 1.02 with a bound of 1.5.

Then I got suspicious of my own docs, which said the check "re-verifies". So I opened the finished run's result.json and edited the ratio to 0.5. The real figure, from 1114 ms and 1131 ms, was 1.02.

The check command said ok.

It was comparing the recorded ratio against the bound. It never recomputed the ratio from the recorded timings. "Re-verifies" was a nicer word than the thing deserved. Now it recomputes from the timings, so my forged 0.5 doesn't survive. The docs still say, accurately, that the gate is local and not tamper-proof. If you can edit the file, you can lie to a local gate; the point is that the lie now has to be consistent.

(A side note on that perf table, since I'll be accused of cherry-picking otherwise: the same workload measured 1.44x on a cold first run and 1.02 to 1.08x on warm ones. That first failure was cache noise, not a regression, and I said so in the write-up.)

The warning that never printed

/relay hands a checkpoint from one session to another. If the workspace moved since the checkpoint was written, the receiver is supposed to see a STALE line. I tested it the way a user would: made an empty commit after the offer, then ran the commands.

inject printed STALE: workspace changed since checkpoint, verify before acting. The ack file said "stale": true. Good. But the docs told you to run read and look for that line, and read never printed it. Not once. I checked the source to be sure I wasn't imagining it.

Worse, the docs said to wait for the relay line in the receiving pane, but that pane had nothing to read until it acknowledged the offer, because the assignment was written at ack time. The docs described a state that didn't exist yet. A warning that exists in three places and not in the one place the instructions send you is a warning that doesn't exist.

Now the assignment is written at offer time, and read and ack both report staleness.

What I took from it

Three bugs, one shape. In each, the code did a smaller thing than the sentence describing it. "Routes your task." "Re-verifies." "Prints STALE." The behavior was fine on the path I'd imagined and wrong on the path I hadn't.

None of them crashed. That's the annoying part. Crashes get fixed on day one. These were quiet, they exited 0 or said ok, and they'd have sat there looking trustworthy.

I only found them because I ran the skills on a real repo and then tried to break my own evidence. Try that before you tell anyone your agent tooling works. The suite is at github.com/luci-efe/skills, and excerpts of the runs for /compass, /proof and /relay are in the worked examples.

npx skills@latest add luci-efe/skills

Fernando Ramos, co-CEO of Agentic Engineering. More at ferramos.com.