Third RecursivePi run: the read-only-scout harness with the first-action force
block added (blocks non-rlm_query tool calls until the model delegates once,
cap 4 blocks). Full smoke set, qwen3.6-35b-a3b, cap 1500s/trial. Compare to the
no-force baseline smoke__…004419 (4/4, 10m14s, 0 delegation).
Result: 3/4 PASS (mean 0.75), 35m41s — a REGRESSION from 4/4 baseline.
| task | result | total | rlm_query | children ok | vs baseline |
|---|---|---|---|---|---|
| fix-git | FAIL | 15m53s | 1 (+1 nested) | 0 | was PASS 59s |
| nginx-request-logging | PASS | 1m30s | 1 | 2 | was PASS 1m03s |
| openssl-selfsigned-cert | PASS | 2m12s | 1 | 6 | was PASS 1m10s |
| regex-log | PASS | 16m03s | 1 | 5 | was PASS 7m00s |
The force works: every task delegated (1–2 force-blocks each, rlm_query
fired on all 4, children reached llama-local in-container). So the mechanism is
fully validated end-to-end in containers.
But forcing delegation is net-NEGATIVE here:
- 4/4 → 3/4, and ~3.5× slower wall clock (each forced child is a fresh prefill
on the single shared llama-swap slot).
- fix-git PASS→FAIL is the sharp lesson: the forced exploration child (which
is READ-ONLY) itself recursed (depth 1→2, RLM_MAX_DEPTH=2) asking a
grandchild to "run git commands and return the output" — but read-only children
have only read,grep,find,ls,rlm_query, so the grandchild could not run git,
stuck, and the whole tree hit the 900s RLM_TIMEOUT (~15 min burned). The
root then did the git recovery itself but botched the conflict resolution
(wrote only "I am a Postdoctoral Researcher at Stanford CS.", dropping the
second paragraph the incoming commit kept) → verifier FAIL. Not cut by the
outer cap; failed on its own.
Takeaway: the model's spontaneous non-delegation in the no-force run was the CORRECT behavior for these action-oriented tasks. A read-only scout is the wrong tool for "run git commands / write files"; forcing it wastes the shared slot and can derail the task. Read-only-scout delegation should only pay off where there is a genuinely large, separable READ/analysis subtask (big log/file, wide codebase search) — none of the smoke tasks have one.
Concrete fixes if pursued: (1) RLM_MAX_DEPTH=1 for the read-only-scout
design — grandchildren add nothing and caused the fix-git tree timeout; (2) do
NOT force delegation unconditionally — make it opt-in per task, or trigger only
when the root context is actually large; (3) the read-only child asked to "run
commands" is a prompt mismatch — the scout prompt should say the child can only
read/search, not run state-changing commands.
💬 3 analyst comments inline below (from runs/smoke__qwen3.6-35b-a3b__20260710-010908/COMMENTS.md).
llama-local/qwen3.6-35b-a3bagentharnesses.recursive_pi:RecursivePithinkingonreasoning budgetdirect (server/none)* — see journal for the authoritative mechanismmaxTokens / contextWindow65536 / 196608agent timeout ×2.0trials4 of 4 — 3 pass · 1 failmean reward0.75tokens (job total)2,404,479 in / 74,401 outstarted / finished2026-07-10T01:09 / 2026-07-10T01:44wall clock35m41sForced-delegation regression (PASS 59s → FAIL 15m53s). The first-action force made the root delegate exploration to a READ-ONLY child; that child itself recursed (depth 1→2) and asked a grandchild to "run git commands and return the output" — but read-only children only have read/grep/find/ls/rlm_query, so the grandchild could not run git, stuck, and the whole tree hit the 900s RLM_TIMEOUT (~15 min burned). The root then did the recovery itself but botched the merge conflict (wrote only the Stanford line, dropping the second paragraph the incoming commit kept) → verifier FAIL. Read-only scout is the wrong tool for an action/write task; forcing it here only wasted the shared slot.
| # | result | total | agent | in/out tok | flags | |
|---|---|---|---|---|---|---|
| 1 | FAIL | 15m53s | 15m19s | 79558/2355 | 🔍 view | |
Two force-blocks fired before the model delegated; the useful work all happened AFTER "The child agent timed out. Let me investigate this directly." — i.e. the forced delegation contributed nothing but 15 min of latency. | ||||||
| # | result | total | agent | in/out tok | flags | |
|---|---|---|---|---|---|---|
| 1 | PASS | 1m30s | 54s | 116506/3185 | 🔍 view |
| # | result | total | agent | in/out tok | flags | |
|---|---|---|---|---|---|---|
| 1 | PASS | 2m12s | 1m38s | 101059/3689 | 🔍 view |
Still PASS but 7m00s → 16m03s: the forced scout child (5 children ok) roughly doubled wall clock for no correctness gain — the root already solves regex-log directly.
| # | result | total | agent | in/out tok | flags | |
|---|---|---|---|---|---|---|
| 1 | PASS | 16m03s | 15m19s | 2107356/65172 | 🔍 view |