| Field | Value |
|---|---|
| Date | 2026-09-09 |
| Harness version | before the turn journal existed |
| Models | qwen2.5:1.5b-instruct, 4,096-token context |
| Evidence grade | B |
| Pre-registered | no (backfilled 2026-09-13) |
| Source record | CONTEXT.md, context-evaluation.json |
Observation
Answer turns spend most of a 4k window on retrieved rows. It was unknown whether a compact encoding would save tokens without losing answer accuracy.
Hypotheses (reconstructed)
- H1 — A schema-plus-array row encoding uses fewer prompt tokens than readable named fields at equal accuracy.
- H2 — Aggressive case-specific selection preserves accuracy while cutting tokens further.
Intervention
src/agent/ContextCompiler.ts: a fresh, bounded packet with explicit coverage
(scope=retrieved available included omitted), JSON-escaped cells, whole-row
fitting, no silent trimming.
Method
scripts/check-context.mjs: four synthetic cases (task status, exact timestamp,
negation, missing information), one run per format, server-reported token usage.
Results
| Format | Exact checks | Prompt tokens (server) |
|---|---|---|
| Readable named fields | 4/4 | 2406 |
| Repeated-key JSON | 3/4 | 2258 |
| Schema + array rows | 3/4 | 1922 |
| Case-specific selected rows | 2/4 | 474 |
| Shipped: rows for tasks, readable for notes | 4/4 | 2066 |
Compact note formats sometimes answered "Friday" when no time was present. Selection returned a bracket-wrapped ID, failing the output contract. Resident VRAM was unchanged (1,166,236,712 bytes): fewer prompt tokens did not reduce allocated model memory.
Threats to validity
n = 1 per format; four hand-built cases; warm-cache latency; one model.
Conclusion
- H1: refuted for notes, supported for typed task rows (accuracy held at the shipped mix, −14% tokens).
- H2: refuted — accuracy fell to 2/4.
Follow-ups
Repeat with n ≥ 3 and with the 7B coder and thinking models now in daily use.
Reproduce
node --import tsx scripts/check-context.mjs