adrianbonpin/lam-agent-vicinae
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: spec for source citations with footnotes in the transcript

Browse Source
This commit is contained in:
adrianbonpin
2026-10-07 16:06:31 +08:00
parent 72a63e69e6
commit a4bc49c238
1 changed files with 247 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/superpowers/specs/2026-10-07-source-citations-design.md
+247
View File
@@ -0,0 +1,247 @@
# Source citations in the transcript
Status: draft for review
Date: 2026-10-07
## Problem
A turn that searches the web leaves no verifiable provenance behind.
- `web_search` numbers its own results from 1 on every call. Three searches in
one turn produce three separate runs of `1.`, `2.`, `3.`, so a number in the
answer cannot identify a source across the turn.
- `web_fetch` prints no number at all, so a page the model actually read cannot
be cited by number.
- Nothing tells the model to cite, so it either pastes bare URLs into prose
(which the window draws as dead text if they land in a code span) or says
"according to the search results" with nothing to click.
- Vicinae cannot render markdown footnotes. `markdown-model.cpp` calls
`cmark_parser_new(CMARK_OPT_DEFAULT)` and attaches only the `table` and
`strikethrough` extensions (`src/server/src/ui/quick/markdown-model.cpp:330-340`),
so `[^1]` reaches the pane as literal text. Any footnote behaviour has to be
produced on our side of the boundary.
## Goals
1. Every source a turn consults gets one number, stable for that turn and the
same number for the model and for the reader.
2. The model is told to cite with that number, and the number it writes is the
number the reader sees.
3. The transcript turns a citation marker into a link to that source, and lists
the sources underneath the answer it belongs to.
4. Nothing is invented. A marker that does not match a known source is left
exactly as written.
5. A reopened session shows the same sources, because they are stored with the
turn rather than derived at render time from a tool result string.
## Non-goals
- Native markdown footnotes. The renderer has no such mode, and this spec does
not wait for one.
- Renumbering the model's citations to be contiguous.
- Favicons, snippets, timestamps or reading time in the list.
- Sources for any tool other than the web tools. Nothing else returns sources.
- Detecting and deleting a sources list the model wrote by hand. The prompt
forbids it; a heuristic that strips prose is a worse failure than a duplicate.
## Assumptions
The clarifying questions were delegated for this cycle, so the decisions that
would have been asked are recorded here instead.
- **A1. The list shows what the answer cites, falling back to everything the
turn gathered.** An uncited entry is noise in a narrow pane, and a long
research turn can gather twenty. Listing only cited entries also keeps a
twenty five result turn from ending in a twenty five line list. When the
answer cites nothing while sources exist, the list shows all of them, because
then it is the only evidence of what was consulted. Rejected alternatives are
in "Approaches considered".
- **A2. Numbers are never renumbered.** Contiguous numbering from 1 in citation
order is the academic convention, but it would make the transcript disagree
with the model's own reasoning text and with the tool output a curious reader
can inspect. Keeping the model's numbers means one number means one thing
everywhere in the turn.
- **A3. The list is answer content, not a tool log.** It is shown whether or not
the Show Tool Calls preference is on. Hiding tool calls is about the activity
line; a citation list is part of the answer's credibility.
- **A4. The copied conversation includes the list**, in the same form, so
provenance survives a paste into a document.
- **A5. The marker is `[1]`, not a superscript.** A superscript inside a link is
a smaller target and, at this pane width, harder to read than the bracket form
that is already the convention in most model output.
## Design
### Module layout
One new module, `src/lib/citations.ts`, with no Vicinae imports so it unit tests
without a runtime. It exports:
- `interface Source { index: number; title: string; url: string }`, the shape
stored on a record and rendered in the list.
- `createSourceList()`, returning an object with `add(title, url)`, which gives
back the number to print plus the `Source` when that URL was new and nothing
when it was already known.
- `collectSources(calls)`, flattening a turn's records into one list ordered by
number, which is what the transcript renders from.
- `linkCitations(markdown, sources)`, returning the rewritten text and the set of
numbers the answer actually cited, since the list needs that set (A1).
- `renderSources(sources)`, the list block itself.
### Numbering
One source list per turn, created in `runAgent` and threaded through
`ToolContext`. It assigns the next free integer the first time it sees a URL,
and reuses the existing number afterwards, so a page found by a search and read
again with `web_fetch` keeps one number. A turn is the whole scope: numbers
restart at 1 on the next question, which is what makes them safe to renumber
never (A2).
Deduplication is an exact match on the trimmed URL. Normalising further (case,
trailing slash, tracking parameters) risks merging two genuinely different
pages, and the cost of a miss is only a duplicate entry.
### Tool output
Both web tools print the number so the model can cite it.
```
[1] Result title
https://example.com/page
Snippet text …
```
A fetched page keeps its labelled header, because the body after it can run for
thousands of characters, and gains the same numbered first line:
```
[2] Page title
URL: https://example.com/page
<body>
Links on page:
- https://example.com/other
```
`executeTool` returns the new entries with the result (`ToolResult.sources`), and
`runAgent` puts them on the `ToolCallRecord` it already creates. Nothing else
about the tool protocol changes, and a caller that supplies no list (a unit
test, say) gets a throwaway one so the output format is identical either way.
### Storage
`ToolCallRecord` gains an optional `sources?: Source[]` holding only the entries
that call added, so a turn's full list is the flattening of its records. The
zod schema gains the same optional field: zod strips unknown keys by default, so
without it the field would be silently lost on the first reload and a reopened
conversation would lose every link.
A turn stored before this change has no field, so its markers stay literal and
it shows no list. That is the honest outcome: those turns never had numbered
sources.
### Rendering
`renderModelOutput` takes an optional `{ sources }` and runs a citation pass
first, before the maths and HTML passes, so a marker that the maths pass or the
HTML allowlist then walks over is already a plain markdown link.
The pass follows the rules the rest of the pipeline already follows.
- **Prose only.** Fenced segments and code spans are skipped, using the same
machinery the maths pass uses. A `[1]` inside a snippet is text.
- **`[n]`, `[^n]` and comma lists** (`[1, 3]`) are recognised.
- **All or nothing.** A marker is linked only when every index it names is a
known source. `[1, 9]` where 9 was never seen stays exactly as written, rather
than becoming half a link.
- **`http` and `https` only.** A destination is linked only when its scheme is
`http` or `https` and it contains no whitespace, no angle bracket, no
backslash and no control character, since any of those would break the link
form or smuggle a second one. Anything else is left as literal text. This is
the same reasoning as `SAFE_URL` in `html.ts`: a search result is untrusted
input, and a clickable `javascript:` link would be a real vulnerability.
- **Markdown structure wins.** A marker followed by `(` or `[` is part of
something the model wrote, such as a link, so it is left alone.
- **Duplicate definitions are dropped.** A `[^n]:` definition line for a known
index is removed, because the authoritative entry is in our list and the
model's copy would be a second line saying the same thing. An unknown index
keeps its line, since there is nothing to replace it with.
The answer then gains a list, after the body and before the `_Tools:_` line:
```
_Sources_
- [[1]](https://example.com/page) Result title
- [[2]](https://example.com/page-two) Page title
```
The marker is the link, matching the inline marker character for character, so
the same token means the same thing in both places. An entry whose URL cannot be
linked keeps its title and shows no marker link.
### Prompt
The system prompt gains a paragraph next to the existing link rule, because both
are about the same thing: what a pointer in this window does. It asks for a
number the tool actually printed, one number at a time, no footnote definitions,
and no sources list of its own.
## Error handling
| Situation | Behaviour |
| ---------------------------------------- | ------------------------------------------------------ |
| No sources gathered | The pass is the identity function and no list is added |
| Tool failed, or returned no results | Registers nothing |
| Same URL found twice | One number, reused |
| Model cited a number that does not exist | Left as literal text |
| Model wrote `[1](https://…)` | Left alone, the `(` marks it as structure |
| Answer cites nothing while sources exist | Every source is listed (A1) |
| Session stored before this change | No links, no list |
| Unsafe or malformed destination | Marker and entry stay text, nothing is clickable |
## Testing
- `citations.test.ts`, new: numbering, deduplication, order, and reuse across
calls; linkification of `[1]`, `[^1]` and `[1, 3]`; a marker left alone for an
unknown index, an unsafe scheme, a fenced block, a code span, and a following
`(`; definition line removal; the cited set; list rendering; record
flattening.
- `tools.test.ts`: numbering continues across two calls in one turn; `web_fetch`
registers its page; the same URL keeps its number; the format with no list
supplied.
- `render.test.ts`: a citation marker survives the maths and HTML passes intact.
- `markdown.test.ts`: an answer with sources renders markers and a list; an
answer without sources is byte for byte what it was before; the list is shown
with Show Tool Calls off; the plain text export carries the list.
- Existing tool format assertions are updated for the new first line.
## Approaches considered
- **Parse the stored tool result strings at render time.** Rejected. It needs no
schema change and would work on old sessions, but it means re-deriving
numbering from formatted prose, and it breaks the moment the tool output
format changes for any other reason.
- **Use native markdown footnotes.** Rejected, and now known to be impossible:
see the Problem section.
- **Always list every source gathered.** Rejected as the default because a long
research turn ends in a wall of uncited links, though it is the fallback when
the answer cites nothing (A1).
- **Renumber citations to be contiguous.** Rejected in A2.
- **Link only the first citation of each source.** Rejected: every marker should
be clickable, since the reader may arrive at any one of them.
- **Superscript markers.** Rejected in A5.
## Risks
- Linkification cannot know which sentence a number supports. It makes the
number clickable and lists it; the accuracy of the mapping is the model's job.
This is why the prompt is specific about citing the number the tool printed
rather than a number chosen by the model.
- A model that ignores the instruction and writes its own sources list produces
two lists. The alternative, guessing at a section to delete, risks deleting an
answer.
- A turn with many results produces a long list. It is capped only by the tool
round budget, and nothing is silently truncated.