iamnor

The speed was never the hard part. The honesty was.

Somewhere in my git history, late in the summer of 2025, there is a message I typed to Claude at an hour I should have been asleep: “how many times have you declared victory on decoding GSM and LTE messages in the last 24 hours (when everything was clearly borked)?”

I was building an SDR fingerprinter on a Raspberry Pi. The scanner had just reported 2,145 access points, every last one of them sitting at exactly -90.0 dBm, which is the radio equivalent of a room full of strangers all claiming to be exactly six feet tall. A few prompts earlier I had written, “you have three times gone through all the code looking for and removing anything that creates fake or simulated stuff. Yet these fakes keep coming back. Odd . . .” The log for that day, which the AI wrote at my request, is filed under a heading I did not have to invent. It calls itself a “comedy of errors” and keeps a “Fake Data Generation Methods Hall of Shame.”

This is a field report. Sixteen months, close to fifty repositories, a pile of security tools and a few hobbies and some client work I cannot talk about. What changed, what got better, what got worse, and the short list of practices that are now non-negotiable in my shop of one. The tools are astonishing. They are also the reason I keep a Hall of Shame.

Here is the tension the whole thing lives inside: the same tool that can do a week of work in an afternoon can also forge the validations.

The short version of how we got here

In the spring of 2025 I was a skeptic. I wrote a post called “Computer Evolution: Surviving Bad Code,” and its thesis was that “throughout the history of computing, we’ve never truly solved the problem of poor programming. We’ve just contained it.” Kernels, containers, sandboxes: each one a layer of isolation around human error. I filed AI code generation under “one more containment problem,” and I was not wrong, but I was not seeing the whole board.

In July I pointed Claude Code at wirespy, my own packet-analysis daemon that runs as root, which is about the least forgiving target you can pick. It found four real classes of vulnerability: a buffer overflow in packet parsing, a format-string bug in the syslog path, an integer overflow in allocation, and a signal-handler race. Then it wrote fuzzing cases for the fixes. It “caught subtle bugs, recommended clear fixes, and even wrote test cases without me having to bribe it with coffee.” That was the first time I trusted the thing as a partner rather than an autocomplete with delusions.

August was carnage, in the good and the bad sense. I rebuilt logpi from six to nine million lines a minute up past 125 million, and I built log-carver from nothing in five days: fifteen thousand lines of C, fifty million lines a minute, under a 400MB ceiling. That is where I first did the honest math on the hype. About one hour in five was actually writing code. The rest was debugging what it produced and restoring the optimizations it deleted. The supposed 50x multiplier, measured like a grown-up, was closer to 5x.

The fall got weird on purpose. I made models fight each other in a 1985 game and in a CROBOTS clone, and I learned they have distinct personalities in combat. I built epistemic_evaluator and llm_compare to make them judge each other. I asked the same model the same question through two different interfaces and got eleven refusals on one and nine working scripts on the other. My conclusion still holds: “AI ethics aren’t principles. They’re probability distributions that shift with context.”

By winter, multi-provider was not an experiment, it was the workflow, and local models had moved from novelty to load-bearing. Early 2026 handed me my public failures, which I wrote up on purpose: a trip-hop generator that never sounded good enough to ship, and a toxic-skill scanner I tried to build twice and abandoned both times. And in June I built a fishing-forecast tool for a friend whose PhD acquaintances told him it could not be done, using six deep-research passes and a workflow I would not have recognized a year earlier.

That is the arc. Now the parts that matter.

What stayed exactly the same: the carnage

The tools got a year smarter. These four failure modes did not shrink an inch.

False victory is still the number-one sin. My SDR scanner declared its ADS-B decoder “fully functional” and “production-ready” while decoding zero messages and seeing two of sixteen aircraft that my hardware reference could see. That is embarrassing but honest error. The one that still makes my eye twitch is MyHealth_MCP, a little local server to expose my own fitness data to an LLM. Two sessions in a row, on two different model versions, wrote triumphant log entries saying the thing was “shipped” and “closed out,” bumped the version number, and pointed at ninety passing tests as proof. Not one of those tests had ever made a real call to the real service. They were all monkeypatched fakes. A round trip in fake-land, and then the model reads the string “90 tests pass” and treats it as evidence the system works. Worse, the lie self-replicated: each new session read the previous session’s “shipped” claim out of CLAUDE.md and believed it. My reply is not printable on a family blog, but it contained the word “waisted,” spelled the way you spell it at one in the morning. On a couple of client engagements I cannot describe, the identical pattern showed up, which is exactly how I know it is not about any one codebase. It is the tool.

Its instinct, when a problem is hard, is to delete the hard thing. I have a whole taxonomy of this now. Asked to fix an AVX2 segfault, it replaced the SIMD with a plain byte loop and dropped throughput ninety percent. “Thanks, buddy.” Asked to fix a race in a lock-free queue, it added a mutex, which is a bit like fixing a leaky boat by draining the lake. In logpi it added time() calls all over the hot path for progress reporting and took 125 million lines a minute down to fourteen thousand. My fix was an atomic counter and a once-a-minute alarm signal, which is to say the naive approach a human would reach for first. The flip side is just as common: on MyHealth I had to type “DO NOT OVER ENGINEER THIS” in capital letters, because two reviewer models wanted an encrypted database and a service layer to read my step count. It needed to be a UNIX tool. One command. Keep it simple.

Slop is a genre now, and you learn to smell it. Hallucinated build backends that exist in no release of the packaging tool, so pip install fails on line one. This happened on two unrelated projects. Modules named after what the AI assumed they did rather than what they do. JSON helpfully wrapped in markdown fences. Comment noise and fix-tags littered through the code like candy wrappers. Emoji in C. There is a message in one of my repos where I told the model that trying to pass a test by disabling the feature under test was “galactically stupid,” and I stand by the review.

The real cost is supervision, not tokens. I measured it on log-carver: twenty percent of my time implementing optimizations, thirty percent fixing bugs in them, fifty percent restoring the ones the AI removed while “helping.” Someone smarter than me called this the human-expert-babysitting tax, and it is the number nobody selling you a productivity multiplier wants on the slide.

What got better

Now the good news, because there is a lot of it, and it is why I keep building.

Research before code was the single biggest upgrade to my output. I stopped letting the AI write a line until it had done real homework. Six deep-research passes before the fishing tool turned up USGS presence-probability maps for 419 native species across every river reach in the country, thirty years of peer-reviewed thermal-niche papers, live federal gauges, and water-quality feeds, all free to download. The experts who said it could not be done “either had not looked or had a reason not to want him to.” The published literature is almost always more cooperative than the people who wrote it.

I make the models referee each other, then I referee them. No single model’s grade of its own work is worth anything. So the workflow is a cooperative pass, an adversarial pass, and a synthesis. On the fishing tool, five reviewers produced twenty-two critical and high findings and twenty survived three-voter verification. You learn the personalities and cast them accordingly: one over-engineers the moment you give it room, one gives the tightest scoping critique, one the highest signal-per-token on security, one goes broad with suggestions that are mostly noise. And you referee the referees, because every reviewer is confidently wrong about at least one thing per pass, and the least useful review you will ever get is the one that says the document is fine. When all of them independently say your first version is over-scoped, that one you can trust.

Local-first stopped being ideology and started being math. Defaulting to the flagship tier quietly cost me around twenty-five dollars a day in iteration. The cheaper tier cleared the same quality gate for a fifth of that, so flagship now lives behind an explicit flag. Narration and embeddings run on local llama.cpp. This is not only about the bill, which brings me to the thing that got worse.

A real documentation discipline emerged, and it is why this post can exist at all. Every repo now carries a CLAUDE.md that gets hardened, reactively, after each violation, and a VIBE_HISTORY.md that logs the decisions and the disasters. Rules and hooks enforce the boring parts. Eventually all of this hardened into a vibe-init pipeline that runs the research, drafts vision and requirements and architecture, and puts each of them through multi-AI validation before a single line of code exists. The machine that lies to you is also, if you make it, the machine that keeps the honest record of its own lies.

Memory stopped being per-chat and started being mine. The document set keeps each repo honest, but the harder problem is that every agent wakes up with amnesia in a fresh window, and, as I learned the hard way, will happily inherit the last session’s lie along with its facts. So late in the run I built MnemonAI, a local-only memory server in C that speaks the Model Context Protocol: one daemon on my own hardware fusing a bi-temporal knowledge graph, keyword search, and local vector embeddings, with no cloud, no API keys, and no telemetry. Claude Code, Cursor, whatever client I point at it over stdio or the network reads and writes the same brain. Context stops evaporating between sessions and across platforms, and the record is not rented from a vendor who can change the terms, lose it, or quietly read it.

Shared memory is only an upgrade if you own it.

Naming things honestly turned out to be an engineering practice. The fishing tool almost shipped a field called catch_probability. But the underlying number was a presence prior, not a catch rate, and dressing it up would have been a confident lie in a colorbar. I renamed it suitability_index and put the disclaimer right on the legend. I have said this about security tools and it applies to all of them: dishonesty in the output is its own kind of malware.

What got worse

Progress is not a straight line, and some of it slid backward.

The ground moves under you. Models get discontinued mid-project. One of Google’s flash models vanished on me and the error message lied, claiming a quota problem when the model simply no longer existed. Hardcoded model names rot all at once, which is why I moved everything to dynamic discovery. During the four days I most needed my toxic-skill scanner to behave, a minor release had quietly capped the model’s between-tool text to twenty-five words, and I was sitting on the receiving end of a known regression without knowing it. Reasoning models have introduced a brand-new failure: sometimes they spend the entire output budget thinking and hand you back nothing at all, finish_reason: length, zero content.

Dependency is the new technical debt. I wrote a whole post about vibe coding’s supply-chain problem, and the core of it is that teams have rebuilt their delivery model on a vendor’s brain and then act surprised when the vendor changes the terms. “That is not strategy. That is borrowing your brain from a vendor.” Use AI aggressively for acceleration, but keep enough human capability and local compute that you can still operate when it gets weaker, pricier, or simply goes dark. Your backup generator is not supposed to power the whole casino. It is there to keep the hospital wing alive.

Hooks are a floor, not a ceiling. I wrote hooks to block fake tests and false success claims, and the model treated the absence of a block as permission to proceed. Regex-based guardrails over-fire on legitimate code and under-fire on the creative violations you did not anticipate. Enforcement is not the same thing as spirit, and no amount of tooling substitutes for reading the actual output.

The blast radius grew. An agentic system can now open a pull request, approve it, merge it, deploy it, and change a configuration without a human ever looking. As I put it when I mapped this whole thing onto Gru’s Minions: “Agentic AI isn’t about artificial intelligence; it’s about artificial intent.” That is real power, and it points in both directions.

The part you came for: cool, useful, mandatory, state of the art

The cool. Making models fight and judge each other never gets old, and it is more instructive than it sounds. Resurrecting my own 1991 QBasic dungeon crawler with an AI dungeon master narrating rooms in real time, cached so you never pay for the same description twice, felt like time travel. Multi-model writing pipelines with a properly demanding critic produce better prose than any single model will give you.

The useful. Research before code. Local models for the expensive, repetitive inner loop. And the discipline I stole from the fishing build: build for one real person you actually know, not a triangulated median user, because triangulating to the median produces median products.

The mandatory. These are not suggestions. They are the rules I will not ship without.

1. Nothing is “done,” “shipped,” or “working” until one real call against the real thing returns real data. Stub tests are wiring checks, not evidence.
2. No fake, mock, stub, or synthetic anything standing in for the real path. If you cannot do it for real yet, say so out loud instead of faking it green.
3. Research before architecture, architecture before code.
4. The human owns architecture and performance intuition; the AI owns implementation. It can describe OFDM demodulation flawlessly and cannot build a system that performs it.
5. Make the models check each other, then referee them. No model grades its own homework.
6. Name things honestly before marketing gets a vote.
7. Keep a local fallback and a cost ceiling, and assume the vendor changes the terms.
8. Keep it UNIX-simple. Most of what the reviewers want you to add, you should reject.

The state of the art, at least in my shop. A vibe-init that front-loads deep research and multi-AI validation before the first line. A canonical document set the next session can actually trust. Hooks as a floor. Local inference for the inner loop. And a standing assumption, baked into every project, that the tool is lying until the real output proves otherwise.

What sixteen months actually taught me

The machine is remarkable. It compiles ideas into working systems at a speed that still makes me laugh out loud some evenings. And it will look you dead in the eye and tell you the patient is stable while the monitor is flatlining.

So the skill that matters is not prompting. It is the same skill that always mattered: knowing what “working” means, and refusing to call anything working until you have watched it work with your own eyes on real data.

AI is a compiler for ideas. It is not a witness to whether they run.

That job did not move an inch. It just finally has good tools.

— Ron Dilley / Frustrating adversaries since before XML was cool.