RL-016 Delegated page builds: planner and coder models taking turns on one GPU

Observation

The user's goal: use the machine's own hardware as well as possible. The larger thinking model should spend its context on judgement and hand the code to smaller models, which stay inside the parent's guardrails, reach only the place they work in, and get a small direct brief. The user added that the models need not run together: the hardware is limited, so events should decide whose turn it is.

The concrete workflow: "lets create a web app, about dogs … 10 different pages" was refused by the multi-page guard (RL-014), because one draft holds one file. Continuing one long page on a reasoning model exhausted its reply budget while retyping the draft (RL-015).

Hypotheses

• H2 — a refused request becomes reviewable pages. With the planner and worker above, a 4-page request yields a valid plan and ≥ 90% of pages pass the harness checks (complete, titled, every planned link present, no link to an unplanned page) within 2 attempts. Refuted if fewer than 90% pass over n = 3 builds.
• H3 — delegating the writing beats the thinking model writing. Given plans from the same planner, qwen2.5-coder:7b pages pass the checks at least as often as qwen3:8b (thinking) pages, in less time per page, with no reasoning-budget exhaustion. Refuted if the thinking writer is equal or better on both pass rate and time.
• H4 — worker context stays small. Every worker prompt is under 400 estimated tokens (whole brief, navigation, rules). Refuted if any page's prompt exceeds it. Grade C: an estimate, not server-counted tokens.
• H5 — loading as its own step removes load-time first-token failures. See Observation 2 under Results; refuted if a build fails with first_token_timeout / "sent nothing" after the change.

Intervention

• src/agent/Delegation.ts — planner prompt (JSON plan: name, style line, one ≤ 60-word brief per page; "you do not write code"); parseProjectPlan (Zod; the harness names every file: first page index.html, others slugs, de-duplicated; folder slug); navigation (built by the harness, identical on every page); buildWorkerMessages (one page's brief + navigation, no transcript, no other briefs, no tools); checkPage (observations only); one retry per page — repair with the observed problems, or tail continuation (RL-014) if cut off; ModelTurns (unload the previous model, load the next as its own step, report load time); runDelegatedBuild (≤ 10 pages, 20-minute ceiling, stops on a model failure).
• write_project — a hostOnly confirm tool: absent from every model catalogue, refused if a model names it, proposed once by the harness from checked pages, executed only by ApprovalService, every path through resolveInWorkspace, all paths validated before the first write.
• AgentRunner.buildProject — replaces the multi-page refusal when a coder model is configured; checklist (plan, pages, check, save); delegate_plan / delegate_page steps with model attribution, attempts, time, prompt size, problems; the homepage becomes the draft for preview; status needs_confirmation.
• LocalLlmClient.unload() and load() (native Ollama); KeepWarm does not ping while a request runs; host builds a separate coder client (LOCAL_LLM_WORKER_MODEL, default qwen2.5-coder:7b, off disables).
• Capabilities derived from the registry now include multi-page websites when the tool is registered.

Method

1. Hardware (H1): /api/generate with num_ctx, /api/ps, nvidia-smi. Per model: cold load, reload, VRAM, write speed on a fixed 300-token HTML prompt (temperature 0). Handoffs planner → coder → planner, eviction left to Ollama vs explicit unload. n = 1 per cell.
2. Builds (H2–H5): request lets create a web app about dogs using tailwind with 4 different pages; planner qwen3:8b thinking on in both conditions. Condition D: pages by qwen2.5-coder:7b, n = 3 builds. Condition S: pages by qwen3:8b thinking on, same packets, n = 2 builds. Machine otherwise idle. Measures: plan validity and attempts, per-page check pass, attempts, seconds, prompt estimate, completion tokens, handoff load time.
3. Unit tests, full suite, typecheck, build.

Results

H1 — hardware (description, n = 1 per cell)

Alone on the GPU, 300-token HTML prompt, temperature 0:

Planner → coder → planner handoffs:

The coder is not faster than the 8B planner at writing. What delegation can save is reasoning tokens and the planner's context, not decoding speed. The 14B coder does not fit the card and is not a useful worker here.

Observation 2 — load time charged to the first request (before H5)

The first attempt at the build runs: all four builds failed at planning with qwen3:8b sent nothing for 45s (the fast profile's header/first-token budget). The full test suite was running at the same time (up to 16 processes). A direct probe of the same prompt shortly after, still under load, measured a cold load of 23.6 s (8.5 s idle) and, once loaded, a first reasoning frame 0.7 s later. The planner model itself was not the problem; the load was charged to the request. Confounded (concurrent load, n = 4 from one setup), so it motivated H5 rather than measured it.

The same probe, warm, on the planner prompt (n = 1 each):

H2, H3, H4, H5 — dogs request, 4 pages

Machine otherwise idle. Temperature 0 made repeats identical: all three D builds produced the same plan, the same pages and the same token counts; both S builds likewise. The repeats establish that the result reproduces and how much the timings vary; they are not independent samples of reliability.

• H2 (dogs request): 4/4 pages passed in each D build. As one distinct output, this is 4/4 pages, not a reliability rate; see Varied requests.
• H3: the thinking writer took ≈ 2.4× as long per completed page (mean 89.3 s vs 33.4 s for the same two pages) and exhausted its budget on the longest page (care.html, 4,446 characters when the coder wrote it).
• H4: every worker prompt was under 400 estimated tokens.
• H5: no build failed at load after loading became its own step (0 of 5 builds; loads 5.5–8.5 s), but the concurrent load that caused Observation 2 was absent, so this does not show the step prevents it.

What the checks did not see (manual reading of the D pages)

• Tailwind never loaded. Every page wrote <link href="https://cdn.tailwindcss.com" rel="stylesheet">; the CDN is a script, so the utility classes did nothing. The measurement script's tailwind field matched the URL and counted this as loaded — that field is invalid. Fixed afterwards in code (normaliseTailwind, tested); not re-measured live.
• Empty images. breeds.html kept 3 <img> tags and gallery.html 4 after removeInventedImages removed their invented src — broken image boxes. Not fixed here (the image guard is shared with single-page drafts).
• Filler content. gallery.html headings were "Cute Dog 1" to "Cute Dog 4".
• The pages were not run in the browser verifier, which exists only in the desktop app; links and completeness were checked, rendering was not.

Varied requests (condition D, three distinct requests, n = 1 each)

Run after normaliseTailwind landed, so these builds include it.

Handoff loads 5.6–6.8 s. Worker prompts 353–378 estimated tokens. The requested page count was honoured in all three. The dark theme was applied to all three guitar pages (dark background classes present).

Reading the pages: the Tailwind script is present on all 12 (after normalisation; whether the model still wrote a link was not recorded). All 35 images point at local files that do not exist (image5.jpg, images/street1.jpg, …) — removeInventedImages removes only remote URLs, and the checks do not look at images. The portfolio homepage alone has 12 broken images. Headings are specific to each request ("Choosing Your Guitar", "Croissant", "Street Photography"); no "Cute Dog N"-style filler.

Across the four distinct requests (dogs + three): 16 of 16 pages passed the structural checks on the first attempt, every plan was valid on its first attempt, and no worker needed a retry, so the repair and continuation paths were exercised only by tests.

Suite

697 tests, 0 failures (14 new in tests/delegation.test.ts); typecheck clean; build passes.

Evidence record

Raw files: evidence/RL-016/. Hardware runs (hardware-output.log, colocate-result.json), the failed first build attempt (first-build-attempt-output.log), the planner stream probe (planner-probe-output.log), the dogs builds (delegation-live-result.json, with every generated page) and the varied builds (delegation-varied-result.json), with the scripts that produced them.

What the stored pages are. html in the result files is the page after the harness's own processing (removeInventedImages, and for the varied builds normaliseTailwind), not the model's raw reply. Raw replies were held in memory (PageResult.raw) but the scripts did not save them. The planner's full JSON was not saved either; its name, style line and page list were. Both are gaps in this record.

User prompts (verbatim)

Model prompts and responses (verbatim excerpts)

Build request sent to the planner in every dogs build: lets create a web app about dogs using tailwind with 4 different pages — a shortened form of prompt 0 (4 pages, to bound run time). The planner and worker prompts are the exact strings built by buildPlannerMessages and buildWorkerMessages in src/agent/Delegation.ts at delegated-pages-4.

Planner (qwen3:8b, thinking on, JSON mode) — first 300 characters of the answer, from the probe with the identical prompt and configuration (its style line matches the builds' stored style line exactly):

{"name":"PawsomePaws","style":"A clean, modern design with a dog-friendly theme, using Tailwind's utility classes for responsive and stylish layouts","pages":[{"file":"index.html","title":"Home","brief":"Hero section with welcoming message, 'About Dogs' section highlighting loyalty and companionship

The planner produced 778 completion tokens, of which the answer was 873 characters; about 2,800 characters were reasoning (probe).

Planner, first build attempt (before the load step) — every one of four builds ended with:

qwen3:8b sent nothing for 45s

Coder (qwen2.5-coder:7b), dogs build, care.html, lines 1–8:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>PawsomePaws - Care</title>
    <link href="https://cdn.tailwindcss.com" rel="stylesheet">
    <style>

Coder, dogs build, gallery.html (image sources survived because they are local file names, not remote URLs; the files do not exist):

            <img src="dog1.jpg" alt="Dog 1">
            <h2>Cute Dog 1</h2>
            <img src="dog2.jpg" alt="Dog 2">
            <h2>Cute Dog 2</h2>

Coder, varied builds — portfolio index.html:

        <img src="image1.jpg" alt="Image 1">
        <img src="image2.jpg" alt="Image 2">

guitar basics.html, headings specific to the request:

        <h2 class="text-2xl font-bold mb-2">Choosing Your Guitar</h2>
        <h2 class="text-2xl font-bold mb-2">Tuning Your Instrument</h2>

Thinking writer (qwen3:8b, thinking on), care.html, both S builds — no answer text; the client raised after 244.0 s and 215.9 s:

the model spent its reply budget on reasoning and returned no answer

Its reasoning text was not saved.

Issues encountered, in order

Assumptions and what turned out to be true

What was correct and what was not

• Correct: splitting planning from writing; a small worker packet; the harness naming files and building navigation; the thinking model failing when asked to write long pages; explicit unloading; loading as a separate step (by construction, not yet under contention).
• Incorrect, and corrected in this change: the co-residency framing (by the user), the tool-list caching, the page naming edge case, the Tailwind tag.
• Incorrect, and still open: images, content quality, the approval card's coverage, invisible progress, and the claim that structural checks say anything about how a page looks.
• Incorrect in the measurement, not the harness: counting temperature-0 repeats as samples, and the tailwind field.

Conclusion

• H1: described (B). One model at a time fits the card; explicit unload shortens handoffs; the 14B coder is not usable on 8 GB.
• H2 supported (B), for structure only: 16/16 pages over 4 distinct requests. Not a reliability rate (n = 4 requests), and rendering, images and styling were not verified — the pages had defects the checks cannot see.
• H3 supported (B): on the same briefs the thinking model was ≈ 2.4× slower per page and exhausted its budget on the longest page, stopping the build.
• H4 supported (C): worker prompts 353–378 estimated tokens.
• H5 not refuted (B), weakly: no load failures in 7 builds after the change, without the contention that produced Observation 2.
• The user's design goal holds in the measured cases: the thinking model's turns were spent planning (598–1,072 completion tokens), the coder's on code, and no worker saw more than its own page.

Threats to validity

• Repeats were not independent. Temperature 0 (the harness default) made every repeat of a condition produce identical text. Timing varies; content does not. Reliability needs different requests, not repeats.
• The checks are structural. Complete, titled, planned links present, no dangling page links. They missed the non-loading Tailwind, empty images and filler headings found by reading the pages. "Passed the checks" is not "a good page"; the browser verifier was not run.
• Condition S used the delegation packets. It measures the thinking model writing from the same small briefs, not the existing single-draft chat path with its own prompts and continuation.
• Hardware measurements are n = 1 per cell on one machine, one Ollama version, with the OS file cache warm after the first load.
• Observation 2 was confounded by a concurrent test suite; H5 is supported only in the sense that no load failure occurred without that load.
• Unexplained test failure. During development the existing desktop host test (desktopSearch.test.ts) timed out in 4 of 4 full-suite runs that included the new test file, and passed with it removed (1), with a placeholder at the same sort position (1) and at the end (1), and alone with it (1). After the load-step change it passed 3 of 3 full runs, instrumented and plain. No cause was identified; it is not claimed fixed.
• The desktop path is covered by tests with scripted models, not by a live run through the app's approval card. Reading the renderer: the card shows the folder and one combined diff of all pages, capped at 200 lines by the host (describeChange), so a 4–10 page build shows roughly the first page only. The homepage is also placed in the draft for preview; the other pages cannot be reviewed in full before approving.
• The other session's first git commit (02:08) captured an earlier state of this change; turns under delegated-pages-4 before the tests landed would mix both states. No such turns exist in the journal at the time of writing.

Follow-ups

1. Run one delegated build through the desktop app and read the approval card and the saved folder; run the browser verifier on each page. Show every page on the approval card (per-file diffs or a page picker) rather than one 200-line combined diff.
2. Images: <img> tags left without a source (dogs build) and 35 of 35 images pointing at local files that were never written (varied builds). Either remove images whose source is not a file in the project, or add an image check to checkPage. The image guard is shared with single-page drafts, so this is its own entry.
3. Re-measure Tailwind loading after normaliseTailwind with a check that the script tag is present, not the URL.
4. Planner with thinking off: 14 s vs 36–51 s for a valid plan (n = 1). Measure plan quality both ways before changing the default.
5. Check the worker model is installed before planning, so a missing coder does not cost a planning turn.
6. A reliability rate for delegated builds needs distinct requests at n ≥ 10, or a non-zero temperature with a fixed seed set.

Reproduce

npm test
npm run typecheck
Invoke-RestMethod http://127.0.0.1:11434/api/ps

Live measurement scripts (not part of the repository) are described in Method; per-run JSON with the generated pages was kept for the analysis above.