diff --git a/docs/superpowers/specs/2026-10-07-source-citations-design.md b/docs/superpowers/specs/2026-10-07-source-citations-design.md new file mode 100644 index 0000000..67aac29 --- /dev/null +++ b/docs/superpowers/specs/2026-10-07-source-citations-design.md @@ -0,0 +1,247 @@ +# Source citations in the transcript + +Status: draft for review + +Date: 2026-10-07 + +## Problem + +A turn that searches the web leaves no verifiable provenance behind. + +- `web_search` numbers its own results from 1 on every call. Three searches in + one turn produce three separate runs of `1.`, `2.`, `3.`, so a number in the + answer cannot identify a source across the turn. +- `web_fetch` prints no number at all, so a page the model actually read cannot + be cited by number. +- Nothing tells the model to cite, so it either pastes bare URLs into prose + (which the window draws as dead text if they land in a code span) or says + "according to the search results" with nothing to click. +- Vicinae cannot render markdown footnotes. `markdown-model.cpp` calls + `cmark_parser_new(CMARK_OPT_DEFAULT)` and attaches only the `table` and + `strikethrough` extensions (`src/server/src/ui/quick/markdown-model.cpp:330-340`), + so `[^1]` reaches the pane as literal text. Any footnote behaviour has to be + produced on our side of the boundary. + +## Goals + +1. Every source a turn consults gets one number, stable for that turn and the + same number for the model and for the reader. +2. The model is told to cite with that number, and the number it writes is the + number the reader sees. +3. The transcript turns a citation marker into a link to that source, and lists + the sources underneath the answer it belongs to. +4. Nothing is invented. A marker that does not match a known source is left + exactly as written. +5. A reopened session shows the same sources, because they are stored with the + turn rather than derived at render time from a tool result string. + +## Non-goals + +- Native markdown footnotes. The renderer has no such mode, and this spec does + not wait for one. +- Renumbering the model's citations to be contiguous. +- Favicons, snippets, timestamps or reading time in the list. +- Sources for any tool other than the web tools. Nothing else returns sources. +- Detecting and deleting a sources list the model wrote by hand. The prompt + forbids it; a heuristic that strips prose is a worse failure than a duplicate. + +## Assumptions + +The clarifying questions were delegated for this cycle, so the decisions that +would have been asked are recorded here instead. + +- **A1. The list shows what the answer cites, falling back to everything the + turn gathered.** An uncited entry is noise in a narrow pane, and a long + research turn can gather twenty. Listing only cited entries also keeps a + twenty five result turn from ending in a twenty five line list. When the + answer cites nothing while sources exist, the list shows all of them, because + then it is the only evidence of what was consulted. Rejected alternatives are + in "Approaches considered". +- **A2. Numbers are never renumbered.** Contiguous numbering from 1 in citation + order is the academic convention, but it would make the transcript disagree + with the model's own reasoning text and with the tool output a curious reader + can inspect. Keeping the model's numbers means one number means one thing + everywhere in the turn. +- **A3. The list is answer content, not a tool log.** It is shown whether or not + the Show Tool Calls preference is on. Hiding tool calls is about the activity + line; a citation list is part of the answer's credibility. +- **A4. The copied conversation includes the list**, in the same form, so + provenance survives a paste into a document. +- **A5. The marker is `[1]`, not a superscript.** A superscript inside a link is + a smaller target and, at this pane width, harder to read than the bracket form + that is already the convention in most model output. + +## Design + +### Module layout + +One new module, `src/lib/citations.ts`, with no Vicinae imports so it unit tests +without a runtime. It exports: + +- `interface Source { index: number; title: string; url: string }`, the shape + stored on a record and rendered in the list. +- `createSourceList()`, returning an object with `add(title, url)`, which gives + back the number to print plus the `Source` when that URL was new and nothing + when it was already known. +- `collectSources(calls)`, flattening a turn's records into one list ordered by + number, which is what the transcript renders from. +- `linkCitations(markdown, sources)`, returning the rewritten text and the set of + numbers the answer actually cited, since the list needs that set (A1). +- `renderSources(sources)`, the list block itself. + +### Numbering + +One source list per turn, created in `runAgent` and threaded through +`ToolContext`. It assigns the next free integer the first time it sees a URL, +and reuses the existing number afterwards, so a page found by a search and read +again with `web_fetch` keeps one number. A turn is the whole scope: numbers +restart at 1 on the next question, which is what makes them safe to renumber +never (A2). + +Deduplication is an exact match on the trimmed URL. Normalising further (case, +trailing slash, tracking parameters) risks merging two genuinely different +pages, and the cost of a miss is only a duplicate entry. + +### Tool output + +Both web tools print the number so the model can cite it. + +``` +[1] Result title + https://example.com/page + Snippet text … +``` + +A fetched page keeps its labelled header, because the body after it can run for +thousands of characters, and gains the same numbered first line: + +``` +[2] Page title +URL: https://example.com/page + + + +Links on page: +- https://example.com/other +``` + +`executeTool` returns the new entries with the result (`ToolResult.sources`), and +`runAgent` puts them on the `ToolCallRecord` it already creates. Nothing else +about the tool protocol changes, and a caller that supplies no list (a unit +test, say) gets a throwaway one so the output format is identical either way. + +### Storage + +`ToolCallRecord` gains an optional `sources?: Source[]` holding only the entries +that call added, so a turn's full list is the flattening of its records. The +zod schema gains the same optional field: zod strips unknown keys by default, so +without it the field would be silently lost on the first reload and a reopened +conversation would lose every link. + +A turn stored before this change has no field, so its markers stay literal and +it shows no list. That is the honest outcome: those turns never had numbered +sources. + +### Rendering + +`renderModelOutput` takes an optional `{ sources }` and runs a citation pass +first, before the maths and HTML passes, so a marker that the maths pass or the +HTML allowlist then walks over is already a plain markdown link. + +The pass follows the rules the rest of the pipeline already follows. + +- **Prose only.** Fenced segments and code spans are skipped, using the same + machinery the maths pass uses. A `[1]` inside a snippet is text. +- **`[n]`, `[^n]` and comma lists** (`[1, 3]`) are recognised. +- **All or nothing.** A marker is linked only when every index it names is a + known source. `[1, 9]` where 9 was never seen stays exactly as written, rather + than becoming half a link. +- **`http` and `https` only.** A destination is linked only when its scheme is + `http` or `https` and it contains no whitespace, no angle bracket, no + backslash and no control character, since any of those would break the link + form or smuggle a second one. Anything else is left as literal text. This is + the same reasoning as `SAFE_URL` in `html.ts`: a search result is untrusted + input, and a clickable `javascript:` link would be a real vulnerability. +- **Markdown structure wins.** A marker followed by `(` or `[` is part of + something the model wrote, such as a link, so it is left alone. +- **Duplicate definitions are dropped.** A `[^n]:` definition line for a known + index is removed, because the authoritative entry is in our list and the + model's copy would be a second line saying the same thing. An unknown index + keeps its line, since there is nothing to replace it with. + +The answer then gains a list, after the body and before the `_Tools:_` line: + +``` +_Sources_ + +- [[1]](https://example.com/page) Result title +- [[2]](https://example.com/page-two) Page title +``` + +The marker is the link, matching the inline marker character for character, so +the same token means the same thing in both places. An entry whose URL cannot be +linked keeps its title and shows no marker link. + +### Prompt + +The system prompt gains a paragraph next to the existing link rule, because both +are about the same thing: what a pointer in this window does. It asks for a +number the tool actually printed, one number at a time, no footnote definitions, +and no sources list of its own. + +## Error handling + +| Situation | Behaviour | +| ---------------------------------------- | ------------------------------------------------------ | +| No sources gathered | The pass is the identity function and no list is added | +| Tool failed, or returned no results | Registers nothing | +| Same URL found twice | One number, reused | +| Model cited a number that does not exist | Left as literal text | +| Model wrote `[1](https://…)` | Left alone, the `(` marks it as structure | +| Answer cites nothing while sources exist | Every source is listed (A1) | +| Session stored before this change | No links, no list | +| Unsafe or malformed destination | Marker and entry stay text, nothing is clickable | + +## Testing + +- `citations.test.ts`, new: numbering, deduplication, order, and reuse across + calls; linkification of `[1]`, `[^1]` and `[1, 3]`; a marker left alone for an + unknown index, an unsafe scheme, a fenced block, a code span, and a following + `(`; definition line removal; the cited set; list rendering; record + flattening. +- `tools.test.ts`: numbering continues across two calls in one turn; `web_fetch` + registers its page; the same URL keeps its number; the format with no list + supplied. +- `render.test.ts`: a citation marker survives the maths and HTML passes intact. +- `markdown.test.ts`: an answer with sources renders markers and a list; an + answer without sources is byte for byte what it was before; the list is shown + with Show Tool Calls off; the plain text export carries the list. +- Existing tool format assertions are updated for the new first line. + +## Approaches considered + +- **Parse the stored tool result strings at render time.** Rejected. It needs no + schema change and would work on old sessions, but it means re-deriving + numbering from formatted prose, and it breaks the moment the tool output + format changes for any other reason. +- **Use native markdown footnotes.** Rejected, and now known to be impossible: + see the Problem section. +- **Always list every source gathered.** Rejected as the default because a long + research turn ends in a wall of uncited links, though it is the fallback when + the answer cites nothing (A1). +- **Renumber citations to be contiguous.** Rejected in A2. +- **Link only the first citation of each source.** Rejected: every marker should + be clickable, since the reader may arrive at any one of them. +- **Superscript markers.** Rejected in A5. + +## Risks + +- Linkification cannot know which sentence a number supports. It makes the + number clickable and lists it; the accuracy of the mapping is the model's job. + This is why the prompt is specific about citing the number the tool printed + rather than a number chosen by the model. +- A model that ignores the instruction and writes its own sources list produces + two lists. The alternative, guessing at a section to delete, risks deleting an + answer. +- A turn with many results produces a long list. It is capped only by the tool + round budget, and nothing is silently truncated.