Spec 07 · On-demand episodes
← back to index
# On-demand bespoke episodes + iterative edit (Fable 5 spec)
*stop=end_turn · out_tokens=8665*
---
# SPEC 07 — ON-DEMAND EPISODES ("Harness Instructions")
**Verdict up front on substance vs theater:** the feature is real and the plumbing exists. The theater risk is the *content*: "tell me how I expand my Chinese influence" with zero zh collectors built will produce advice-shaped filler unless we force the RAI posture through. So this spec makes the generator **say what it doesn't have** — a China episode today will open with "coverage is thin: 2 independent items, both REPORTED" rather than confabulating a strategy. That's not a limitation to hide; it's the demo of the $1M/yr job.
---
## 1. Components & Files
| File | Role |
|---|---|
| `lib/ondemand.js` | NEW. Episode-session engine: create/edit/research/regenerate/commit/discard. Owns the drafts Map + disk persistence. |
| `lib/research.js` | NEW. `assembleResearch(instruction, opts)` — targeted per-episode research assembly with provenance/confidence stamping and the ≥30% independence check. |
| `lib/budget.js` | EXTRACT. Pull the realtime-voice reserve guard out of server.js into a shared module: `estimate()`, `reserve(usd)`, `commit(reservationId, actualUsd)`, `release(id)`. Daily ledger: `signal-data/spend-ledger.jsonl` (append-only, same writer as curation audit). |
| `lib/curation.js` | EXTEND, don't fork: export `writeAtomic(path, obj)` and `appendAudit(file, entry)` so ondemand.js reuses them. `addObjective()` already exists — spawn path uses it verbatim. |
| `server.js` | New route block `// --- ON-DEMAND EPISODES ---`; new voice tools registered beside `set_advisory_mode` etc. |
| `signal-data/drafts/<id>.json` | Persisted session state (survives launchd restart). |
| `signal-data/episode-audit.jsonl` | Append-only edit log, one line per op. Separate file from curation-audit — different lifecycle, keeps curation audit clean. |
| `signal-data/episodes/YYYY-MM-DD-od-<slug>.mp3` + `.json` sidecar | Committed output. Date-first keeps existing sort. |
| `/signal/console` | Adds a **Drafts** pane (read-WRITE). The curation view stays read-only per spec 06 — do not soften that; the write surface is drafts only. |
`buildEpisodes` regex change: `^\d{4}-\d{2}-\d{2}-(am|pm|od-[a-z0-9-]+)\.mp3$`, and read the `.json` sidecar for display title + `kind:"ondemand"` badge. **This touches the working player — it's the highest-regression-risk line in the spec; it gets its own acceptance test.**
---
## 2. Endpoints
All behind the existing cookie/Bearer gate. On-demand additionally requires the **owner** key (single check: bearer === `SIGNAL_OWNER_KEY`); the funnel is public-adjacent and these endpoints spend money.
```
POST /signal/episode/estimate {instruction} -> {usd, seconds, researchQueries, withinCaps:bool}
POST /signal/episode/draft {instruction, objectiveId? | spawnObjective?:{text,keywords[]}, confirmUsd}
GET /signal/episode/draft/:id
POST /signal/episode/draft/:id/edit {op:"remove_segment"|"restore_segment"|"pin_segment"|"add_segment"|"reorder",
segmentId?, directive?, order?[]}
POST /signal/episode/draft/:id/research {query}
POST /signal/episode/draft/:id/regenerate {steering?}
POST /signal/episode/draft/:id/commit {confirmUsd}
DELETE /signal/episode/draft/:id
GET /signal/episode/drafts -> active drafts (max 2 concurrent)
```
`confirmUsd` is the honest-UX enforcement: client must echo back the estimate it was shown; server rejects if actual estimate now exceeds echoed value by >20%. No silent spend drift.
---
## 3. Session State (draft shape)
```json
{
"id": "od_20250612_x7k2",
"status": "researching|drafted|committing|committed|discarded",
"createdAt": "2025-06-12T14:03:11Z",
"expiresAt": "2025-06-13T14:03:11Z",
"instruction": "China-market briefing; how do I exercise/expand my Chinese-language influence",
"objectiveId": "obj_zh_influence",
"research": {
"items": [
{"id":"r1","query":"china LLM market state","title":"...","url":"...","summary":"...",
"provenance":"INDEPENDENT","confidence":"REPORTED","addedBy":"initial","ts":"..."}
],
"independentRatio": 0.4,
"gaps": ["no FILED-confidence zh-market items; collectors not built"]
},
"segments": [
{"id":"s1","title":"State of play","text":"...","sources":["r1","r3"],
"origin":"generated","pinned":false,"removed":false}
],
"spend": {"reservationId":"rsv_...","estimatedUsd":0.42,"actualUsd":0.31,
"researchCalls":3,"researchCallsMax":6,"regens":1,"regensMax":3},
"steeringHistory": ["punchier","cut the throat-clearing"]
}
```
Segments are **soft-deleted** (`removed:true`) so voice "restore segment two" works and regenerate can be told what was rejected. `pinned:true` = regenerate must not touch it. Every mutation: `appendAudit('episode-audit.jsonl', {draftId, op, via:"voice"|"console", before, after, ts})`.
Runtime: `ondemandSessions` Map in `lib/ondemand.js`, hydrated from `signal-data/drafts/` on boot; every mutation → `writeAtomic`. Reuse the pattern from your existing sessions Map; do NOT keep draft state only in the /signal/ask conversation context.
---
## 4. Generate Path (the pipeline)
1. **Parse instruction** — one cheap Fable call → `{topic, angle, objectiveHint, suggestedQueries[≤4]}`.
2. **Research assembly** (`lib/research.js`):
- **Layer A (free, TODAY):** filter today's existing digest items by keyword overlap with topic/objective keywords. Tagged with their existing provenance/confidence.
- **Layer B (paid, TODAY):** ≤4 model research queries (Fable + web tooling). Every result stamped `INDEPENDENT` + `REPORTED` or `UNVERIFIED` — nothing from this path may claim `FILED|API` because no collector filed it. This is the honesty line.
- **Layer C (STAGED):** real collectors (jobs/market/discourse, zh-language sources) register per-topic; assembly fans out to them and items earn `API/FILED`.
- Enforce quota: if `independentRatio < 0.30` or `items < 4`, do NOT block — record in `research.gaps` and the synthesis prompt is REQUIRED to state the gap in segment 1. Dissent slot mandatory as in the daily path.
3. **Synthesis — reuse, don't fork:** build a digest-shaped text from research items (provenance/confidence tags inline, exactly the format the daily digest uses) and call **`signalScriptsP1(assembledText, mutedTopics, contextLines, steeringFromInstruction)`** with curation from `signal-settings.json` injected as today. The instruction becomes the brief; the objective's `advisoryMode` applies. Staged: a dedicated on-demand prompt variant inside `fableSignalScript`.
4. **Draft returned** — no TTS yet. TTS is the expensive irreversible step; it only runs at commit.
5. **Commit:** Signal-Live TTS path (the pregen renderer — explicitly NOT the Gia container path, which is curation-blind) → `signal-data/episodes/2025-06-12-od-china-market.mp3` + sidecar `{title, instruction, objectiveId, segments, research, spend, kind:"ondemand"}`. `buildEpisodes` picks it up; it appears in the PWA Episodes list with a badge.
## 5. Iterative Edit Loop
Console Drafts pane: segment cards (remove/restore/pin), free-text "add a segment on…" box, "go deeper: ___" research box (shows remaining research budget "3 of 6 queries left"), regenerate-with-steering, and a commit button that displays `~$0.18 TTS · ~40s`. Removed segments render struck-through, restorable.
Voice (`/signal/ask` tools, registered beside the spec-06 curation tools):
| Tool | Phrase |
|---|---|
| `create_episode(instruction, objective?)` | "Build me an episode: China market, how I expand my Chinese-language influence." Fable answers with the estimate and asks to confirm — spend never triggers on one utterance. |
| `read_draft()` | "Where are we?" → Fable speaks segment titles + gaps. |
| `remove_segment(ref)` / `add_segment(directive)` | "Kill segment three." / "Add a segment on zh-language RAI discourse." |
| `add_research(query)` | "Go deeper on Baidu's model licensing." Appends cited items, tells him what came back and its confidence. |
| `regenerate(steering?)` | "Regenerate, punchier, keep segment one." (pin resolution via `pinned`) |
| `commit_episode()` / `discard_episode()` | "Ship it." / "Toss it." |
The ask-session context pins `activeDraftId` so "segment three" resolves without IDs. If two drafts exist, Fable asks which. Both voice and console hit the **same** `/signal/episode/draft/:id/*` endpoints — one engine, one audit trail, `via` field distinguishes.
## 6. Objective Tie-In
- `POST /draft` with `objectiveId` attaches; with `spawnObjective` it calls `curation.addObjective({text:"Expand Chinese-language influence", keywords:["china","zh","中文",...], status:"active", advisoryMode:"full"})` and logs to curation-audit as today.
- On **commit**, top-cited keywords from the episode's `FILED|API`-confidence items only get proposed as `keywords.boost` additions on the linked objective (console shows "adopt these boosts?" — one click, not automatic; REPORTED/UNVERIFIED research must not silently steer the daily digest — that's the RAI line between "Byron explored a topic" and "unvetted web results now shape his canon").
- STAGED: objective "deepening" — daily pregen notices an objective with ≥2 committed on-demand episodes and allocates it a recurring digest slot.
## 7. Cost / Abuse Bounding
- Env: `SIGNAL_ONDEMAND_DAILY_USD=3.00`, `SIGNAL_ONDEMAND_EP_USD=0.75`, `SIGNAL_ONDEMAND_MAX_DRAFTS=2`.
- Per-draft counters: 4 initial queries, 6 total research calls, 3 regens, 1 commit. Hard-stop with a spoken/console reason, never silent failure.
- `lib/budget.js` reserve pattern (same as realtime voice): reserve at draft-create for research+synthesis, second reserve at commit for TTS; `commit(actual)` or `release` on discard/expiry. Realtime-voice and on-demand draw from **separate** daily ledger lines — one feature exhausting its cap must not brick the other.
- Owner-gated + `confirmUsd` echo + voice confirm-before-spend. Drafts auto-expire in 24h and release reservations (launchd-friendly sweep on boot + hourly).
---
## TENSION — ADJUDICATED
**Committed on-demand episodes are FIRST-CLASS. Drafts are ephemeral. Commit is the membrane.**
A pure-ephemeral design is dead UI incarnate: you'd pay real money for research, listen once, and the artifact evaporates — plus you'd need a *second* parallel player for drafts, which is exactly the duplicated-surface rot you hate. First-class costs you two real things, stated honestly: (1) Episodes-list pollution — mitigated by the `od-` badge and a delete affordance in the console; (2) contamination risk of future briefings by low-confidence web research — mitigated by the FILED/API-only, opt-in boost rule above. The draft layer keeps the one-off exploration cheap to abandon (auto-expiry, reservation release); the moment TTS money is spent, it's canon. One player, one Episodes list, one delete key.
## BUILD ORDER for 4.8
**FUNCTIONING-TODAY SLICE = items 1–7** (runs on live quip-service, leans on today's digest + spec-06 curation, before real collectors):
1. Extract `lib/budget.js` from the realtime guard; add `spend-ledger.jsonl` + on-demand caps. *(Do first — nothing spends before this exists.)*
2. `lib/curation.js`: export `writeAtomic` + `appendAudit`.
3. `lib/research.js` — Layer A (digest filter) + Layer B (≤4 model queries, `REPORTED/UNVERIFIED` stamping, independence check, `gaps[]`).
4. `lib/ondemand.js` — session engine, drafts dir persistence, soft-delete/pin/regen, synthesis via `signalScriptsP1` with digest-shaped assembly, commit via Signal-Live TTS, `episode-audit.jsonl`.
5. `server.js` route block + owner gate + estimate/confirm handshake.
6. `buildEpisodes` regex + sidecar support + PWA badge. **Regression test the existing am/pm list before deploying.**
7. Console Drafts pane (create box, estimate, segment cards, research box, commit).
8. *(Staged)* Voice tools for the edit loop in `/signal/ask` — worth doing next, but console-only is fully usable today.
9. *(Staged)* Real collectors registering into research Layer C with `API/FILED` confidence; zh-language sources for the China objective.
10. *(Staged)* Objective deepening — recurring digest slots for episode-backed objectives; dedicated on-demand prompt variant in `fableSignalScript`.