| Field | Value |
|---|---|
| Date | 2026-09-12 |
| Harness version | → 2026-09-12-checklists-9 |
| Models | configured local model (disposable test) |
| Evidence grade | B (one before/after live pair) + C (10 regression tests) |
| Pre-registered | no (backfilled 2026-09-13) |
| Source record | PLANNING.md |
Observation
Compound requests collapsed into a single action. Live, disposable database: "Save two separate notes: one containing alpha and one containing beta" wrote one combined note.
Hypotheses (reconstructed)
- H1 — One JSON planning call before execution, with completion state owned by the host, makes the model execute each requested action separately.
Intervention
src/agent/RunChecklist.ts: up to four actions; states owned by the host; the
model proposes actions, never completion. Planning only for explicit compound
phrasing ("two/three/four separate", "and then"). One continuation chance after
a premature final reply. Cached duplicate calls cannot complete another item.
Method
Same request, same disposable setup, before and after. Ten regression tests
(multi-action, early finals, duplicate replay, bounds, invalid plans, simple-call
cost, approvals, failed state). scripts/check-checklist.mjs for rendering.
Results
| Condition | Notes written | n |
|---|---|---|
| Without pre-planning | 1 (combined) | 1 |
| With pre-planning | 2 (distinct, each confirmed) | 1 |
Threats to validity
n = 1; one phrasing; compound detection covers a narrow set of sentences.
Conclusion
H1 supported at grade B for the reproduced case only.
Follow-ups
Repeat across phrasings and models; measure planning-call cost.
Reproduce
npm test
npx electron scripts/check-checklist.mjs