Porting RHO when you're not actually in the dark

Reading a paper as a spec

RHO, short for Retrospective Harness Optimisation, is the method from the “Evolving Agents in the Dark”. It turns an agent’s own past sessions into a recipe for improving the agent. There is a reference implementation on GitHub that ports the paper to both Codex CLI and Claude Code. I lifted one full RHO cycle into my harness over the past few days. The interesting part was not the port. It was the constraint the paper works under, and my setting doesn’t have.

The setting the paper assumes away

A harness is the persistent shell around the agent: the per-phase instruction templates, the knobs that bound how many turns each phase gets, the helper scripts the agent calls. RHO improves the harness with no ground-truth labels and no validation set. Every signal it uses is something the agent generates by reading its own trajectories: does this trajectory look internally consistent, do parallel trajectories agree, would the agent prefer one rewritten harness over another. Label-free is the headline. It is also the hard constraint that shapes every other choice the paper makes.

My harness is not in the dark. Every task already runs through coded gates: a verify step that knows whether the unit tests pass, a quality gate that knows how many fix rounds the last phase burned, an integration step that knows whether the features composed. Those gates produce real outcomes, real retry counts, and real cost numbers. The cleanest port was not the one that preserved label-free purity. It was the one that used the labels everywhere a label exists, and saved self-preference for the place where there was genuinely no other signal.

The paper’s headline constraint is label-free. The first question of the port was which of the paper’s constraints my setting still has.

What carried over, what got rewritten

The paper’s cycle has a clear shape. Pick a small representative subset of past tasks (the paper calls this a coreset, chosen by a method that balances “hard tasks” with “varied tasks”). Diagnose what went wrong on each one. Sample a few candidate edits to the harness. Pick the candidate the agent prefers over the current version, and apply it. I kept the shape. Inside the shape I kept some pieces verbatim and replaced others.

What carried over was the maths. The coreset selector uses the paper’s greedy method on the same task-similarity score (a Jaccard overlap between task fingerprints) and the same difficulty-vs-diversity weight (θ = 0.7). The code is short and there was no reason to vary it. The diagnose stage carried over too: one read-only agent per coreset task, self-validation plus cross-task self-consistency, written down as a failure mode and an improvement direction.

What got rewritten was anything the paper had to estimate that my harness already measures. Difficulty is the cleanest example. The paper asks an LLM to rate how hard each past task was. My harness already counts the failures, the repeat runs of the same phase, and the cost in dollars. I replaced the LLM judge with a six-line function that adds those three numbers and clamps to ten. Real labels are cheaper than self-judgement and they do not drift.

Two other changes were operational. The single self-preference score became two judge lenses: does this edit address the diagnosed failure, and does it preserve the placeholders and JSON output contracts the runtime depends on. Every candidate writes its proposed harness into a per-run directory it cannot escape. Only the final accept step copies the winner into the live harness, after a full backup.

The paper’s group rollouts went in too. Each coreset task gets re-run live under every candidate harness in an ephemeral detached worktree, with a scope-aware start at the earliest phase the candidate’s edits touched. Probe evidence is required for acceptance, same bar as the paper. The two judge lenses still run, but they got demoted to tiebreak and veto. When probe evidence isn’t available (unreplayable tasks, judge-only templates), the baseline holds and the judges report as diagnostics.

The Takeaways

1. Read agent-system papers as specs, not stories. The cycle shape rarely matters. What matters is which constraints the paper had to invent its way around, and whether your setting still has them.
2. Swap the real labels in. If the source method estimates something your domain already measures, replace the estimator. Self-judged difficulty against six lines of arithmetic is not a fair fight.
3. The place with no signal doesn’t stay that way. I picked self-preference for the accept decision because there was nothing else. Once the probes went in, there was. Self-preference is the tiebreak now.