Porting Cloudflare's $1 code review into my custom agent harness

Reading a blog as a spec

Cloudflare published a multi-agent code reviewer that runs across 5,000 codebases at a median cost of $1 per review. I read their blog as a spec rather than a story and lifted eight techniques into the integration-verify gate of my custom agent harness over the weekend. The lift itself was small. The parts the blog hand-waved were where the work lived.

Two review surfaces

My custom agent harness has two review surfaces. The first reviews each task as it is being built. The second, the integration-verify gate, runs once every task has cleared its own unit tests, and checks that the features integrate. Both want the same shape: a coordinator that fans work out to scoped specialists, fuses their findings, and gates the next step.

The hardest part of porting Cloudflare’s pattern is not the multi-agent shape. It is how to package all information flows efficiently into a context.

What carried over, what got rewritten

Five of the eight techniques already existed in the per-task review, so they came across unchanged. They all run on plain code, not on the LLM.

The harness cross-check runs before the model ever sees the change. If a script won’t even start, the gate fails right there. No tokens spent. Delta re-verification narrows what gets re-checked on a retry to the one criterion that just failed, instead of re-walking the whole product requirement. The evidence bundle writes a single signed file at the end of every gate run, listing what went in and what came out, so I can audit any past decision. The doom-loop detector spots when the same failure keeps repeating across retries and breaks the loop instead of grinding. The false-pass eval is a standing metric: it tracks how often the gate has approved something that later turned out broken.

The other three needed real work, and they all came down to the same question: how much does each reviewer need to see?

Specialist fan-out is the answer for who does what. Four specialists run side by side (code quality, documentation, performance, security), and each one only reads the files that fall under its remit. Documentation only sees doc files; the others only see code. Every specialist also has a “do not flag” block longer than the prompt telling it what to look for: no style, no naming, no whitespace, no speculative issues, no missing-tests preferences. That block cuts more noise than any positive-prompt tweak I tried.

Risk tiers are the answer for how many. Cloudflare picks the swarm size from diff size: lines and files changed. The gate never sees a diff, so I needed a different signal. Tasks and acceptance criteria are what it does see. One task and five criteria is trivial: one reviewer, cheap model. Four tasks and twenty criteria is light: three specialists, standard model. Anything bigger is full: every specialist on the strong model, plus a separate reviewer that only checks how the features fit together. Both numbers have to land in the same tier. A two-task project with forty criteria is not trivial, and a ten-task project with twelve criteria is not full.

Model routing is the answer for how strong. Every reviewer reads its model name from one small table. The cheap reviews stay on the cheap model. The reviewer that checks how the features fit together gets the strongest, because that one has to reason about boundaries. The table is the smallest file in the whole gate. It is also the one that moves the cost line the most.

The Takeaways

1. Read agent-system blog posts as specs, not stories. The novelty is rarely the orchestration shape. It is the negative prompts, the threshold axis, and the resilience scaffolding around the LLM calls.
2. Port the techniques before the framing. Risk tiers, negative prompts, bias-to-approve, evidence bundles, and a doom-loop detector all worked in the verify gate with no Coordinator class in sight.
3. Pick the threshold axis from your domain, not the source domain. Cloudflare’s axis is diff size. Mine is tasks × criteria. The right axis is whatever bounds the cost of being wrong on a single review.