Research log / RL-019

RL-019 Explicit implementation of a proposed draft improvement

Evidence grade DC

Field Value
Date 2026-09-13
Harness version 2026-09-13-workspace-map-7 → 2026-09-13-short-form-draft-implementation-9
Models observed: deepseek-r1:14b; tests: MockLlm
Evidence grade D (journal) + C (routing and harness tests)
Pre-registered yes: observation, baseline and hypotheses written before implementation
Source record journal turn 125 / run 396

Observation

After the model suggested “Implement a more robust navigation system,” the user replied: “yes implement a more robust navigation system. make sure you keep a checklist.” The harness classified the message as ordinary chat. The model returned another four-item prose plan and asked “Would you like me to proceed with implementing this?” No code changed and no host checklist existed.

Baseline

Hypotheses

Intervention

Method

  1. Replay the reported sentence through classifyIntent with a complete HTML draft open.
  2. Use MockLlm to return one valid exact replacement and run the real code conversation path.
  3. Assert that the draft changes, the bounded edit prompt is used, and both host checklist items finish.
  4. Classify the same sentence in an empty conversation and assert that it stays ordinary chat with no mutation route.
  5. Run the full suite, typecheck and build.

Results

Threats to validity

Conclusion

H1–H3 are supported by deterministic replay and harness tests. The reported requests, including the final “yes, implement this” form, no longer ask for permission twice and now reach implementation when a draft is active.

Follow-ups

  1. Repeat the navigation request in the desktop app and inspect the real model's exact edits and browser checks.
  2. Keep the model-proposed prose checklist as conversation context only; the host checklist remains the authority for work actually performed.

Reproduce

node --import tsx --test tests/codeConversation.test.ts
npm test
npm run typecheck
npm run build