| Field | Value |
|---|---|
| Date | 2026-09-10 |
| Harness version | → 2026-09-10-desktop-search-4 |
| Models | qwen2.5-coder:7b (live) |
| Evidence grade | B (journal-confirmed defect) + C (actual-host fixture) |
| Pre-registered | no (backfilled 2026-09-13) |
| Source record | DESKTOP-SEARCH-FIX.md, chicken-followup-evaluation.json |
Observation
Saved logs showed search for chickens returned six results, yet the
desktop answered that there were none. does your search work? was sent to the
provider as the query current date and time.
Hypotheses (reconstructed)
- H1 — The desktop host resolves web source IDs through a different reader than CLI/eval, so the defect is desktop-only and invisible to existing evals.
Intervention
One source reader for notes, tasks and web evidence in the desktop host; an
explicit source-loading error when returned IDs cannot be resolved; search
status questions answered from the recorded outcome with no inference; page
briefs keep their research query (chickens) separate from requirements.
Method
tests/desktopSearch.test.ts launches the actual desktop host as a child
process with synthetic network responses; npm run sim -- chicken-followup 3.
Results
Root cause confirmed: web IDs fell through to workspace lookup in the desktop only. After: actual-host fixture passes; simulator 3/3 repeats, 12 turns, no drift. Unit tests 392.
Threats to validity
The simulator covers only the unavailable-provider case; successful research is covered by the fixture, not a live provider.
Conclusion
H1 supported. Methodological finding: CLI/eval results do not validate the desktop path; tests must exercise the real host.
Follow-ups
Live checks through Electron for every new workflow (adopted in RL-011, RL-012).
Reproduce
npm test
npm run sim -- chicken-followup 3