RL-015 Served context, tail continuation for reasoning models, failure attribution

Observation

After the context default was raised to a per-model profile (8,192 for qwen3) by a concurrent, unlogged change, the user sent a third continue on the unfinished 5,532-char dogs draft (RL-014).

/api/ps after the run: qwen3:8b context_length=4096, although the harness resolved 8,192.

Five defects, found by reading the code paths involved:

1. Whole rewrite selected. The fit test compared reply room (≈ 4,024 allowed: 3,000 code reply + 1,024 reasoning reserve; derived from the budget formula, not logged) with the draft (≈ 2,588 needed). It "fitted", so the tail mode of RL-014 was not used — but the reasoning alone was ≈ 2,500–3,000 tokens.
2. Keep-warm reloads at 4,096. ping() used /v1/chat/completions, which cannot carry num_ctx.
3. Category lost twice. The code path's catch called finish without a failure; separately the desktop host's second finishRun (when website checks were planned) wrote failure = null over whatever was recorded.
4. Mislabelled category. The client raised reasoning exhaustion as context_overflow, advising LOCAL_LLM_MAX_REPLY_TOKENS, which the code path does not use.
5. Attribution. The turn recorded llm.model at the end of the run; the model was switched mid-run.

Hypotheses

• H1 — A compatibility-endpoint ping makes Ollama serve the model at its 4,096 default, and a native load with num_ctx restores 8,192; alternating between them forces a full reload each time. Refuted if /api/ps context does not change with the request type.
• H2 — For a reasoning model, continuing from the tail (remainder only) instead of rewriting the draft removes the draft-sized output cost that exhausted the budget. Refuted (at grade C) if the prompt still carries the whole draft; at grade B if live continuations still exhaust at a similar rate.
• H3 — A computed outline (ids written, in-page links with no section, classes) supplies the structure the model spent reasoning to recover. Refuted if the outline misses the missing sections on a representative draft.
• H4 — Classifying in the code path and passing the failure through the host's second finishRun gives every such run a non-null category.

Baseline

Intervention

• LocalLlmClient.ping() — native Ollama: POST /api/generate with an empty prompt and options.num_ctx; other servers unchanged.
• AgentRunner — tailMode when continuing an unfinished draft and (the model reasons — thinking: always, or optional with thinking switched on — or a rewrite cannot fit). LlmClient.thinkingRequested added (optional).
• CodeConversation.draftOutline + buildContinuationMessages(…, outline) — bounded to three lines of ≤ 300 chars; "keep the existing structure, ids and class names".
• failures.ts — reasoning_budget_exhausted category, explain and fix text, classify pattern; client raises it; code-path catch passes classify(err).
• agentHost — turn model captured at run start; setModel refused while a run is active; second finishRun carries answer.failure.
• Unchanged: whole-file continuation for non-reasoning models with room; patch edits for complete drafts (the other session's change).

Method

1. Live, controlled (H1): local Ollama, qwen3:8b, alternating requests, /api/ps read after each: compatibility ping (max_tokens: 1) → native /api/generate (num_ctx: 8192) → compatibility → native. n = 2 per request type.
2. Harness (H2–H4): tests/reasoningContinuation.test.ts with scripted clients carrying real capability profiles and 6,000 tokens of room (so only reasoning can select tail mode); tests/ollamaNative.test.ts ping body.
3. Full suite, typecheck, build.

Results

H1 — served context (live)

Every switch between request types reloaded the model (8–11 s). With keep-warm pinging every 4 minutes during use, a real request after a ping paid a reload and a ping after a real request undid the profiled window.

H2/H3 — harness

Outline for the test draft: ids already written: labrador, beagle / links with no section yet: border / classes in use: breed.

H4 — a thrown reasoning-exhaustion error in the code path is recorded with category reasoning_budget_exhausted; explain names the model.

Suite — 683 tests, 0 failures (5 new); typecheck clean; build passes.

Threats to validity

• H2's live effect is unmeasured. A reasoning model may still reason at length about a tail; tail mode removes the draft-sized output, not the reasoning.
• Which context run 389 actually received is unknown. The request sent 8,192, but a keep-warm ping could have left the model at 4,096 before or after; /api/ps was read only afterwards.
• Run 389 changed model mid-run; its behaviour cannot be attributed to one model.
• The agentHost changes (attribution, switch guard, failure pass-through) are covered by typecheck only, not by a host test.
• The concurrent profiled-context-patch-edit-1 change (context default, patch edits) is unlogged; turns under it mix both sessions' changes.
• n = 2 for the live measurement; one machine, one Ollama version.

Conclusion

• H1 supported (B).
• H2 supported for prompt shape (C); effect on exhaustion not tested live.
• H3 supported on the test draft (C).
• H4 supported in the runner (C); host pass-through by construction only.

Follow-ups

1. Live: restore turn 117's draft; continue on qwen3:4b-thinking, qwen3:8b (thinking on) and qwen2.5-coder:7b, n = 5 each; record outcome, reasoning tokens (server eval_count), elapsed, seam validity, verifier pass count.
2. After a week of use, snapshot failure categories: expect no new unrecorded or unknown for reasoning exhaustion.
3. Consider salvaging a fenced block from reasoning when the answer is empty, gated by the same no-progress and verifier checks — only with live evidence it helps.
4. Log the concurrent context-profile and patch-edit change as its own entry.

Reproduce

npm test
npm run typecheck
Invoke-RestMethod http://127.0.0.1:11434/api/ps