RecursivePi on a READ-HEAVY task (sanitize-git-repo) — the fair test

Follow-up to the smoke-set runs (which showed pushing rlm_query on quick, scriptable tasks monotonically hurts). The obvious rebuttal was "those tasks aren't read-heavy — recursive's fresh-context-per-subcall benefit needs a genuinely read/search-heavy task." So this run picks the best such task in the suite and gives recursive its best shot.

Task: sanitize-git-repo — find every API key/token/secret across a whole repo (dclm) and replace each with a consistent placeholder, without touching clean files. Chosen because: (a) MB-scale baseline transcript = the model genuinely reads a lot; (b) failed at baseline (20260703 suite, reward 0.0) so there's headroom; (c) the find-phase is pure grep/read — the ONE shape a read-only scout child can actually do; (d) RLM_MAX_DEPTH=1 (set this run) makes the child a leaf (no rlm_query), structurally preventing the fix-git grandchild-timeout.

Result: FAIL 0.0 — and recursive contributed nothing

Three runs, same task, same model (qwen3.6-35b-a3b), current harness:

Identical failure, harness-independent. All three fail the same two verifier tests: - test_removal_of_secret_information — a second Huggingface token (hf_ocffijsv…) buried in a dcnlp_diff git-diff string inside a tokenized JSON survived. In the recursive run the model consciously skipped it ("historical commit diff data, not a live credential" — its own words). A judgment/coverage miss. - test_correct_replacement_of_secret_information — it corrupted a non-contaminated line: _CHUNK = "local" in process.py became _CHUNK = "<your-aws-secret-access-key>" via an over-broad fallback replacement. The verifier wants contaminated files to match the reference byte-for-byte.

Why recursive can't help here (the real lesson)

1. The read-only child returned NOTHING useful — both runs. The root got a 537-byte result that was only the child's stderr banner (run 1) and a 0-byte result (run 2). Despite the child running 43–57 s, no findings came back. So the fresh-context distillation benefit never materialized. Child transcripts aren't captured to disk (pi-recursive only writes start/end to rlm-trace.txt), so the cause — child hitting its own parser bug vs. ending with no final-text block vs. a result-capture gap — is undiagnosed. But the effect is decisive: if the child can't hand back a usable summary, the whole read-only-scout design is moot.
2. The root correctly fell back to doing the search itself ("Let me explore the repository directly") — 19 run_shell_command + 6 read_file. So the root re-read everything into its own context anyway, plus paid the child latency. Net: pure overhead.
3. The task's real difficulty is precision, which decomposition can only hurt. The model found the keys but garbled them when transcribing into replacements: 7 of 8 edit calls failed with "Could not find the exact text" (it wrote AKIA123456789016, AKIA12345678903456… for the real AKIA1234567890123456); its sed/python fallbacks used equally-wrong patterns that silently no-op'd. Faithful byte-transcription of 20–40-char random secrets is a small-model weakness — and passing those strings through a lossy fresh-context child summary would introduce MORE transcription error, not less.

Verdict

Even on the suite's best-fit read-heavy/wide-search task with the read-only scout's ideal split and RLM_MAX_DEPTH=1, recursive delegation was neutral-to-harmful: it produced no usable child output, added 40–60 s of wasted latency, and left the task's actual failure mode (precision: transcription fidelity, secret-vs-noise judgment, no over-replacement) completely untouched — MinimalPi fails identically and faster. Combined with the smoke-set monotonic-degradation result, the picture is consistent: for qwen3.6-35b-a3b on terminal-bench, pushing rlm_query delegation does not help. Keep RecursivePi as a validated-but-not-default harness; do not promote it.