From a804afd56a6e889b5da5af1500390ad9241b9971 Mon Sep 17 00:00:00 2001 From: Adrian Bonpin Date: Wed, 7 Oct 2026 07:26:40 +0000 Subject: [PATCH] feat: render LaTeX, custom HTML and diagrams in the transcript (#19) --- README.md | 38 +- src/__tests__/code.test.ts | 60 +++ src/__tests__/html.test.ts | 81 +++ src/__tests__/latex.test.ts | 106 ++++ src/__tests__/markdown.test.ts | 10 + src/__tests__/render.test.ts | 135 +++++ src/lib/code.ts | 275 +++++++++- src/lib/constants.ts | 4 + src/lib/html.ts | 450 +++++++++++++++++ src/lib/latex.ts | 895 +++++++++++++++++++++++++++++++++ src/lib/markdown.ts | 17 +- src/lib/render.ts | 502 ++++++++++++++++++ 12 files changed, 2543 insertions(+), 30 deletions(-) create mode 100644 src/__tests__/html.test.ts create mode 100644 src/__tests__/latex.test.ts create mode 100644 src/__tests__/render.test.ts create mode 100644 src/lib/html.ts create mode 100644 src/lib/latex.ts create mode 100644 src/lib/render.ts diff --git a/README.md b/README.md index 3228a63..da70145 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,7 @@ Hit a global shortcut, type a question, get a streamed answer without leaving wh - **Persistent conversations.** Every chat is saved locally. The sidebar lists them by recency, and reopening the command picks up where you left off, unless it has been idle longer than you allow. - **Readable code.** Fenced blocks render in a dark panel with the language named in a header, syntax colouring, and a Copy button. - **Real markdown.** Tables, bulleted and numbered lists, headings, blockquotes, bold, italics and links all render natively, and the default prompt asks for the shape that fits the answer rather than flattening everything into prose. +- **Maths, real HTML, and diagrams.** LaTeX renders, so an equation arrives as an equation instead of a wall of backslashes. A small amount of HTML draws as well, for the status chip or tiny grid a sentence is worse at, and a diagram drawn with box characters keeps its alignment instead of being reflowed. See [Rendering](#rendering). - **Tools.** Lam decides when to call them: current time, location context, web search, and page fetching. A turn that uses up its tool budget still ends in an answer, written from what it gathered, rather than a notice that it stopped. - **Tool calls you can watch and stop.** What Lam is doing right now stays on screen, including when it asks the same question twice, the round limit is a preference, Force Stop is always one keystroke away, and a call that repeats itself with identical arguments ends the turn instead of running forever. - **Models, from a list.** Browse every model your host can reach, pick one and start using it immediately, and on a server you run, pull new ones or delete old ones. Tag names are picked, never typed. @@ -262,6 +263,36 @@ The photo clean-up applies exactly the same rule as the sweep, including the one When a conversation cannot be read there is no way to know which captures are still in use, so that action says it could not check and deletes nothing, rather than risking a screenshot a surviving chat still shows. +## Rendering + +The answer is model output, and Vicinae draws a fixed set of things. Everything between the model and the pane happens in one pass, in `render.ts`, and every step of it is there because the renderer cannot do that part alone. + +### LaTeX + +Vicinae has no maths renderer. `MarkdownModel` maps cmark nodes to a fixed set of block components (`MdHeading`, `MdCodeBlock`, `MdTable`, `MdImage` and friends) and there is no maths block among them, so `$x^2$` used to reach the pane as literal TeX. `latex.ts` converts it instead. + +Two facts about the renderer decided the design. A paragraph is a `TextEdit` with `textFormat: RichText` (`MdParagraph.qml`), and `markdown-model.cpp` passes `CMARK_NODE_HTML_INLINE` through untouched, so `sup`, `sub`, `b` and `i` are available. And the pane is a launcher-width column, so a formula has to read linearly: a stacked fraction could not survive a narrow column anyway. + +So this is a linearisation rather than a typesetter. `\frac{a}{b}` becomes `a/b`, `\sqrt{x+1}` becomes `√(x+1)`, `\sum_{i=1}^{n}` becomes `∑ᵢ₌₁ⁿ`, and only what Unicode has no form for falls back to a tag, so `x^{a/b}` becomes `xa/b`. `$...$`, `\(...\)`, `$$...$$` or `\[...\]` on a line of its own, and a fence labelled `math`, are all read. + +The one guard is worth knowing about. A lone `$` pair counts as maths only when the inside looks like maths, because `x=$a$b` and `cd $A; echo $B` are not formulas, and turning them into one is the single way this feature can make an answer worse than the dollar signs it replaced. `$5 and $10` is left exactly as written. + +### Custom HTML + +Raw HTML is not merely tolerated by the renderer, it is the only route to anything past what GFM offers: a `` with a colour or a size draws, and so does a table. That is the extension point for a model that would rather show a status chip than describe one, and the default prompt says the capability exists. + +**A block of HTML has to be moved before it can draw anything.** `markdown-model.cpp` has no HTML renderer. An HTML *block* goes through `processHtmlNodes`, which pulls out every `` as a block image and keeps the trimmed text of everything else with the markup thrown away, so `
ab
` arrives as the word `ab`. A *paragraph*, by contrast, keeps its raw HTML, because `CMARK_NODE_HTML_INLINE` is passed straight through to the `TextEdit`. So a well-formed block is joined onto one line and wrapped in a ``, which is not on CommonMark's list of block tags, and it arrives as the paragraph the rich text engine can read. Two rules keep that rewrite from doing damage: markup that does not balance is left to the block path, which at least keeps the words, and a block holding an `` is left alone entirely, because the extractor is the one place an image written as HTML is loaded for real. + +It is also a hole, because the same route carries whatever else the model writes, so `html.ts` filters it. The tags and style declarations the renderer acts on are kept, `script`, `style` and the other element types whose body is not content are removed with it, and an event handler, a `url()` in a style value and a `javascript:` destination are dropped. An `` is rewritten as a markdown link, because that is the only form Vicinae makes clickable. An `` keeps its tag and has its source judged instead, since the block extractor reads the attribute rather than any markdown this pass could write, and `data:` counts only when it carries an image. + +What the renderer cannot draw is kept as text rather than dropped, which is the deliberate choice in there. A tag that is not on the list, `` say, is shown literally: ugly, but honest, and better than a mystery about where the markup went. + +### Everything else + +The rest of the pass is defensive, and stays away from guessing what the model meant. A diagram drawn with box characters is fenced so it keeps its alignment, because `MdParagraph.qml` wraps and a directory tree reflowed is a directory tree scrambled. A fence label KSyntaxHighlighting would print but never resolve is renamed (`console` and `terminal` become `bash`) and one that carries no information at all (`text`, `plaintext`) is dropped, which is what stops a header naming a language nobody highlighted. A fence whose body is all URLs still becomes links, a real GFM table is left to the table renderer, and markup inside a code span is shown rather than applied. + +The whole pass is plain TypeScript with no Vicinae imports, so it unit tests without a runtime, and copying a conversation to the clipboard uses the source text rather than the rendering. + ## Architecture ``` @@ -285,7 +316,10 @@ src/ location.ts # time/place, derived locally thread.ts # turns -> flat message list markdown.ts # messages -> detail-panel markdown - code.ts # code fences and links, repaired + render.ts # the rendering pass: fences, maths, markup + latex.ts # TeX -> readable inline markup + html.ts # raw HTML -> the subset the renderer draws + code.ts # code fences, diagrams and links, repaired preferences.ts # preferences, normalised hooks/useChat.ts # the chat state machine hooks/usePreferences.ts # preferences + the model pick, loaded once @@ -320,7 +354,7 @@ A few decisions worth knowing about before you change things: The rendered height stays capped at 200px whatever is asked for (`Math.min(_rawH, 200, maxImageHeight)`), so this is a thumbnail rather than something readable, which is why the action panel still offers Open Screenshot for the full image. The alt text is short for the same reason it is not empty: whatever is there is drawn centred under the image, so a sentence reads as a stray line of prose, and it is the only thing left if the capture file has since been deleted. A preview of an unsent attachment is shown the same way, so a snip is visible before the question about it exists. -**Code fences are labelled so Vicinae can colour them.** `MdCodeBlock.qml` already provides the dark panel, a header naming the language, and a Copy button, so none of that is built here. The language string is what turns on highlighting: `syntax::highlight` resolves it with `definitionForName(language)` and then `definitionForFileName("file." + language)`. So `bash` and `ts` colour correctly, while an unrecognised string such as `terminal` still renders a label but produces no highlighting at all, so a wrong label is worse than none. `code.ts` therefore labels bare fences conservatively, only when a signal is clear, and the default prompt asks for a language on every fence. +**Fences are labelled so Vicinae can colour them, and a diagram is fenced so it stays readable.** `MdCodeBlock.qml` already provides the dark panel, a header naming the language, and a Copy button, so none of that is built here. The language string is what turns on highlighting: `syntax::highlight` resolves it with `definitionForName(language)` and then `definitionForFileName("file." + language)`. So `bash` and `ts` colour correctly, while an unrecognised string such as `terminal` still renders a label but produces no highlighting at all, so a wrong label is worse than none. `code.ts` therefore labels bare fences conservatively, only when a signal is clear, the default prompt asks for a language on every fence, and an alias that resolves to nothing (`console`, `terminal`) is renamed to one that does. The same module fences a paragraph drawn with box characters, because a paragraph wraps and a boxed diagram that reflows is a boxed diagram that no longer says anything. **A URL is only clickable as a markdown link, so `code.ts` takes them back out of code.** Vicinae's parser attaches the `table` and `strikethrough` GFM extensions and nothing else, so the `autolink` extension is off: a bare URL is plain dead text. Clickability comes from `MarkdownModel` turning `CMARK_NODE_LINK` into an ``, which the selection controller opens on release. A URL the model fenced is therefore the worst possible presentation, a dark panel with a Copy button and no way to open the address. `restoreUrlLinks` recovers three narrow shapes (a fence whose entire body is URLs, a lone `` `https://...` `` span, and a line that is only a URL) into `[url](url)` links, and the default prompt now says never to fence a URL in the first place. Anything that is not one of those three shapes is copied through, so a `curl` invocation containing an address is never rewritten. diff --git a/src/__tests__/code.test.ts b/src/__tests__/code.test.ts index 1c14202..2368dfc 100644 --- a/src/__tests__/code.test.ts +++ b/src/__tests__/code.test.ts @@ -1,7 +1,10 @@ import { describe, expect, it } from "vitest"; import { + fenceAsciiArt, inferLanguage, labelCodeFences, + mapOutsideCodeSpans, + normalizeFenceLabels, promoteStandaloneCodeSpans, repairCodeBlocks, restoreUrlLinks, @@ -416,3 +419,60 @@ describe("labelCodeFences", () => { expect(labelCodeFences(" ```\n ls\n ```")).toContain(" ```bash"); }); }); + +describe("mapOutsideCodeSpans", () => { + it("applies the transform everywhere a span is not", () => { + expect(mapOutsideCodeSpans("a `b` c", (text) => text.toUpperCase())).toBe("A `b` C"); + }); + + it("matches a delimiter of the same length only", () => { + // A single-backtick span inside a double-backtick span is content, not a + // closer, so the inner one is left exactly as written. + expect(mapOutsideCodeSpans("`` `a` ``", (text) => text.toUpperCase())).toBe("`` `a` ``"); + }); + + it("treats an unclosed run as text and carries on after it", () => { + expect(mapOutsideCodeSpans("a ` b", (text) => text.toUpperCase())).toBe("A ` B"); + }); +}); + +describe("normalizeFenceLabels", () => { + it("names a language that resolves in place of one that does not", () => { + expect(normalizeFenceLabels("```console\nls\n```")).toBe("```bash\nls\n```"); + }); + + it("drops a label that carries no information", () => { + expect(normalizeFenceLabels("```text\nplain\n```")).toBe("```\nplain\n```"); + }); + + it("leaves a language the model chose alone", () => { + const input = "```rust\nfn main() {}\n```"; + expect(normalizeFenceLabels(input)).toBe(input); + }); +}); + +describe("fenceAsciiArt", () => { + it("fences a tree so its columns stay monospaced", () => { + expect(fenceAsciiArt("src/\n├── a.ts\n└── b.ts")).toBe("```\nsrc/\n├── a.ts\n└── b.ts\n```"); + }); + + it("fences an ASCII table drawn with borders", () => { + const art = "+---+---+\n| a | b |\n+---+---+"; + expect(fenceAsciiArt(art)).toBe("```\n" + art + "\n```"); + }); + + it("leaves a real GFM table to the table renderer", () => { + const table = "| a | b |\n| --- | --- |\n| 1 | 2 |"; + expect(fenceAsciiArt(table)).toBe(table); + }); + + it("leaves prose that contains one pipe alone", () => { + const prose = "Here is what it does:\n| a | b |"; + expect(fenceAsciiArt(prose)).toBe(prose); + }); + + it("leaves a diagram that is already fenced alone", () => { + const fenced = "```\n├── a.ts\n└── b.ts\n```"; + expect(fenceAsciiArt(fenced)).toBe(fenced); + }); +}); diff --git a/src/__tests__/html.test.ts b/src/__tests__/html.test.ts new file mode 100644 index 0000000..197ed4e --- /dev/null +++ b/src/__tests__/html.test.ts @@ -0,0 +1,81 @@ +import { describe, expect, it } from "vitest"; +import { sanitizeInlineHtml } from "../lib/html"; + +describe("sanitizeInlineHtml", () => { + it("keeps the tags a rich text paragraph draws", () => { + expect(sanitizeInlineHtml("bold and italic")).toBe("bold and italic"); + expect(sanitizeInlineHtml("x2 and ai")).toBe("x2 and ai"); + }); + + it("keeps a style declaration it can use and drops the rest", () => { + const input = 'x'; + expect(sanitizeInlineHtml(input)).toBe('x'); + }); + + it("drops a style value that reaches outside the document", () => { + expect(sanitizeInlineHtml('x')).toBe( + "x", + ); + expect(sanitizeInlineHtml('x')).toBe( + 'x', + ); + }); + + it("drops an event handler", () => { + expect(sanitizeInlineHtml('x')).toBe("x"); + }); + + it("removes a script with the code inside it", () => { + expect(sanitizeInlineHtml("ok")).toBe("ok"); + expect(sanitizeInlineHtml("ok")).toBe("ok"); + expect(sanitizeInlineHtml('ok')).toBe("ok"); + expect(sanitizeInlineHtml("" })]); + expect(html.trim()).toBe("> hi"); + }); + it("groups an answer with its question, dividing only between exchanges", () => { const markdown = buildThreadMarkdown([ message({ id: "t2:assistant", role: "assistant", text: "second answer" }), diff --git a/src/__tests__/render.test.ts b/src/__tests__/render.test.ts new file mode 100644 index 0000000..b94a69b --- /dev/null +++ b/src/__tests__/render.test.ts @@ -0,0 +1,135 @@ +import { describe, expect, it } from "vitest"; +import { renderInlineContent, renderModelOutput } from "../lib/render"; + +describe("renderModelOutput", () => { + it("renders inline maths in a sentence", () => { + expect(renderModelOutput("The answer is $x^2 + 1$, obviously.")).toBe("The answer is x² + 1, obviously."); + expect(renderModelOutput("So \\(x^2\\) holds.")).toBe("So x² holds."); + }); + + it("leaves money and shell variables alone", () => { + expect(renderModelOutput("Costs $5 and $10 today.")).toBe("Costs $5 and $10 today."); + expect(renderModelOutput("cd $HOME && echo $PATH")).toBe("cd $HOME && echo $PATH"); + }); + + it("puts a display formula on a paragraph of its own", () => { + expect(renderModelOutput("$$\\frac{a}{b} = c$$")).toBe("\na/b = c\n"); + expect(renderModelOutput("$$\nx = 1\n$$")).toBe("\nx = 1\n"); + expect(renderModelOutput("\\[\\sum_{i=1}^{n} i\\]")).toBe("\n∑ᵢ₌₁ⁿ i\n"); + }); + + it("does not let a stray marker swallow the rest of the answer", () => { + // An opening `$$` with no closer only counts when what follows reads as a + // formula, so a price is not a formula and the paragraphs after it survive. + expect(renderModelOutput("The price is\n$$5 per unit.\n\nThat is cheap.")).toBe( + "The price is\n$$5 per unit.\n\nThat is cheap.", + ); + }); + + it("renders a maths fence as a formula rather than as code", () => { + expect(renderModelOutput("```math\n\\sum_{i=1}^{n} i\n```")).toBe("\n∑ᵢ₌₁ⁿ i\n"); + expect(renderModelOutput("```latex\nx^2\n```")).toBe("\nx²\n"); + }); + + it("leaves maths inside a fence alone", () => { + const markdown = "```ts\nconst x = $a$;\n```"; + expect(renderModelOutput(markdown)).toBe(markdown); + }); + + it("leaves a dollar inside a code span alone", () => { + expect(renderModelOutput("`$x^2$` stays.")).toBe("`$x^2$` stays."); + }); + + it("renames a fence language the renderer would print but not colour", () => { + expect(renderModelOutput("```console\nls\n```")).toBe("```bash\nls\n```"); + expect(renderModelOutput("```terminal\nls\n```")).toBe("```bash\nls\n```"); + }); + + it("drops a fence label that carries no information", () => { + expect(renderModelOutput("```text\nplain\n```")).toBe("```\nplain\n```"); + }); + + it("leaves a label that does resolve exactly as written", () => { + const markdown = "```python\nprint(1)\n```"; + expect(renderModelOutput(markdown)).toBe(markdown); + }); + + it("fences a diagram so its alignment survives", () => { + expect(renderModelOutput("src/\n├── a.ts\n└── b.ts")).toBe("```\nsrc/\n├── a.ts\n└── b.ts\n```"); + }); + + it("leaves a real table to the table renderer", () => { + const table = "| a | b |\n| --- | --- |\n| 1 | 2 |"; + expect(renderModelOutput(table)).toBe(table); + }); + + it("leaves a diagram inside a fence alone", () => { + const markdown = "```text\n├── a.ts\n└── b.ts\n```"; + expect(renderModelOutput(markdown)).toBe("```\n├── a.ts\n└── b.ts\n```"); + }); + + it("filters HTML in the flow of an answer", () => { + expect(renderModelOutput("still here")).toBe("still here"); + expect(renderModelOutput('x')).toBe( + 'x', + ); + expect(renderModelOutput('Read the docs.')).toBe( + "Read [the docs](https://example.com).", + ); + }); + + it("leaves the markup inside a fence as the code it is", () => { + const markdown = "```html\nx\n```"; + expect(renderModelOutput(markdown)).toBe(markdown); + }); + + it("hands a block of HTML to the engine as a paragraph", () => { + // A line that starts with a block tag is an HTML *block*, and Vicinae's block + // path keeps only the text of one. Wrapped in a span it is a paragraph, which + // is the path that keeps raw HTML. + expect(renderModelOutput("
ab
")).toBe( + "
ab
", + ); + }); + + it("joins a block of HTML onto one line", () => { + // `class` is dropped, because there is no stylesheet for it to name, and the + // style attribute is the only thing the renderer reads. + expect(renderModelOutput('
\nbold\n
')).toBe( + "
bold
", + ); + }); + + it("leaves markup that does not balance where the text it carries still shows", () => { + expect(renderModelOutput("
hello")).toBe("
hello"); + }); + + it("leaves a block holding an image alone, because that path loads it", () => { + expect(renderModelOutput('
')).toBe( + '
', + ); + }); + + it("leaves a lone inline tag alone, because it is not a block", () => { + const markdown = "Try bold for that."; + expect(renderModelOutput(markdown)).toBe(markdown); + }); + + it("keeps an answer that has nothing to render exactly as written", () => { + const plain = "Just a sentence, with a [link](https://example.com) and `code`."; + expect(renderModelOutput(plain)).toBe(plain); + }); + + it("repairs fences in model output but not in what a person typed", () => { + // A lone command is a fence when a model writes it, and a question about it + // is left as the person typed it. + expect(renderModelOutput("`pwd`")).toBe("```bash\npwd\n```"); + expect(renderInlineContent("`pwd`")).toBe("`pwd`"); + }); + + it("still renders maths and markup in what a person typed", () => { + expect(renderInlineContent("what is $x^2$?")).toBe("what is x²?"); + expect(renderInlineContent("hi")).toBe("hi"); + expect(renderInlineContent("```ts\nconst x = $a$;\n```")).toBe("```ts\nconst x = $a$;\n```"); + }); +}); diff --git a/src/lib/code.ts b/src/lib/code.ts index 429bc36..718b7ef 100644 --- a/src/lib/code.ts +++ b/src/lib/code.ts @@ -82,10 +82,14 @@ const FENCE = "```"; * bare ``` are the same two lines apart from context, so a pattern that only * matched bare fences treated every closing fence as a new opening one and * swallowed the rest of the answer. + * + * Exported because the maths and HTML passes have to skip fenced code for the + * same reason this file does, and a second copy of the pattern is a second + * place to get it wrong. */ -const OPENING_FENCE = /^(\s*)(`{3,}|~{3,})\s*([^\s`~]*)\s*$/; +export const OPENING_FENCE = /^(\s*)(`{3,}|~{3,})\s*([^\s`~]*)\s*$/; /** A closing fence is the marker and nothing else. */ -const CLOSING_FENCE = /^\s*(`{3,}|~{3,})\s*$/; +export const CLOSING_FENCE = /^\s*(`{3,}|~{3,})\s*$/; const STANDALONE_CODE_SPAN = /^\s*`([^`\n]+)`\s*[.!]?\s*$/; /** @@ -107,7 +111,7 @@ const STANDALONE_URL_SPAN = /^`([^`\n]+)`[.!]?$/; const LIST_MARKER = /^([ \t]*(?:[-*+]|\d+[.)])[ \t]+)(.*)$/; /** A GFM table row: it opens and closes with a pipe. */ -const TABLE_ROW = /^\s*\|.*\|\s*$/; +export const TABLE_ROW = /^\s*\|.*\|\s*$/; /** * The delimiter row under a header. * @@ -115,7 +119,72 @@ const TABLE_ROW = /^\s*\|.*\|\s*$/; * column. It is that the line could be nothing else: pipes, dashes, colons and * spaces, with at least one dash to tell it apart from a row of empty cells. */ -const TABLE_DELIMITER_ROW = /^\s*\|?[\s:|-]*-[\s:|-]*\|?\s*$/; +export const TABLE_DELIMITER_ROW = /^\s*\|?[\s:|-]*-[\s:|-]*\|?\s*$/; + +/** + * A run of backticks, and where the span it opens ends. + * + * The run length is part of the delimiter in markdown, and `` `a` `` inside a + * double-backtick span is content rather than a closer, so the search is for a + * run of exactly the same length. A run with no closer is literal text, so only + * the backticks are consumed. + */ +export function readCodeSpan(text: string, start: number): { raw: string; end: number } { + let length = 0; + while (text[start + length] === "`") length += 1; + + const marker = "`".repeat(length); + let cursor = start + length; + + while (cursor < text.length) { + const found = text.indexOf(marker, cursor); + if (found === -1) break; + + if (text[found + length] === "`") { + cursor = found + length; + while (text[cursor] === "`") cursor += 1; + continue; + } + + return { raw: text.slice(start, found + length), end: found + length }; + } + + return { raw: marker, end: start + length }; +} + +/** + * Applies `transform` to everything outside a code span, leaving the spans + * exactly as they were written. + * + * Every pass that rewrites a message needs this for the same reason: a + * `` `x^2` `` or a `` `` `` is something being shown to the reader, not + * something to convert. Shared so the rule for what a span is exists once. + */ +export function mapOutsideCodeSpans(text: string, transform: (text: string) => string): string { + let out = ""; + let index = 0; + let start = 0; + let found = false; + + while (index < text.length) { + if (text[index] !== "`") { + index += 1; + continue; + } + + found = true; + const span = readCodeSpan(text, index); + out += transform(text.slice(start, index)); + out += span.raw; + index = span.end; + start = index; + } + + if (!found) return transform(text); + + out += transform(text.slice(start)); + return out; +} function significantLines(code: string): string[] { return code @@ -308,22 +377,30 @@ function asBareUrl(candidate: string): string | undefined { return trimmed.replace(TRAILING_SENTENCE_PUNCTUATION, ""); } +/** + * A markdown link, escaped the way the parser needs it. + * + * Shared with the HTML pass, which rewrites an `` into the one shape + * Vicinae makes clickable. Square brackets in the *label* would end the link + * text early, and a destination containing parentheses (plenty of Wikipedia + * articles) has to be wrapped in pointy brackets or the parser reads the first + * `)` as the end of the destination and leaves the rest as trailing prose. + */ +export function markdownLink(label: string, destination: string): string { + const escaped = label.replace(/[[\]]/g, "\\$&"); + const target = /[()\s]/.test(destination) ? `<${destination}>` : destination; + return `[${escaped}](${target})`; +} + /** * Wraps a URL as a markdown link, using the address itself as the label so the * destination stays readable rather than turning into a generic word. - * - * A destination containing parentheses (plenty of Wikipedia articles) has to - * be wrapped in pointy brackets or the parser reads the first `)` as the end of - * the destination and leaves the rest as trailing prose. */ function asLink(url: string): string { - // Square brackets in the *label* would end the link text early, which is why - // an IPv6 literal such as `http://[::1]/` needs them escaped. The destination - // tolerates brackets, but not parentheses, which is what the pointy form is - // for. - const label = url.replace(/[[\]]/g, "\\$&"); - const destination = /[()]/.test(url) ? `<${url}>` : url; - return `[${label}](${destination})`; + // An IPv6 literal such as `http://[::1]/` carries the brackets in the label, + // which `markdownLink` escapes. The destination tolerates brackets but not + // parentheses, which is what the pointy form is for. + return markdownLink(url, url); } /** @@ -455,16 +532,172 @@ export function separateTables(markdown: string): string { return out.join("\n"); } +/** + * Fence languages that draw a label but no colour, and what to say instead. + * + * `syntax::highlight` in Vicinae resolves a language with + * `definitionForName(language)` and then `definitionForFileName("file." + language)`, + * so a name KSyntaxHighlighting has never heard of still gets printed above the + * block while producing no highlighting at all. A model reaching for "terminal" + * or "console" is describing a shell, so it is told a name that resolves. The + * `undefined` entries are the ones that carry no information whatsoever, and a + * bare fence is the honest rendering of those. + */ +const LANGUAGE_ALIASES: Record = { + console: "bash", + terminal: "bash", + shell: "bash", + commandline: "bash", + "command-line": "bash", + none: undefined, + output: undefined, + plain: undefined, + plaintext: undefined, + raw: undefined, + text: undefined, +}; + +/** + * Rewrites a fence label the renderer would print but not colour. + * + * A label the model chose is otherwise left alone: this only touches names that + * are known not to resolve. + */ +export function normalizeFenceLabels(markdown: string): string { + const lines = markdown.split("\n"); + const out: string[] = []; + let index = 0; + + while (index < lines.length) { + const opener = OPENING_FENCE.exec(lines[index]); + if (!opener) { + out.push(lines[index]); + index += 1; + continue; + } + + const [, indent, marker, existing] = opener; + const key = existing.toLowerCase(); + + if (existing && key in LANGUAGE_ALIASES) { + const replacement = LANGUAGE_ALIASES[key]; + out.push(replacement ? `${indent}${marker}${replacement}` : `${indent}${marker}`); + } else { + out.push(lines[index]); + } + + index += 1; + while (index < lines.length && !CLOSING_FENCE.test(lines[index])) { + out.push(lines[index]); + index += 1; + } + + if (index < lines.length) { + out.push(lines[index]); + index += 1; + } + } + + return out.join("\n"); +} + +/** Box drawing and block elements, which is what a diagram is drawn with here. */ +const BOX_DRAWING = /[\u2500-\u257f\u2580-\u259f]/g; + +/** + * Whether a line is part of a picture rather than part of a sentence. + * + * Two box-drawing characters is the clear signal. The ASCII spelling of the same + * thing is a line that starts and ends with a border character, which is how a + * hand-written `+---+---+` table or an `| a | b |` row begins. + */ +function isArtLine(line: string): boolean { + const trimmed = line.trim(); + if (trimmed.length === 0) return false; + + if ((trimmed.match(BOX_DRAWING)?.length ?? 0) >= 2) return true; + + return /^[+|].*[+|]$/.test(trimmed); +} + +/** + * Whether a paragraph is a picture that has to keep its alignment. + * + * Deliberately cautious, because fencing prose is worse than leaving a diagram + * to reflow: at least two art lines are required and they have to make up half + * the paragraph, a real GFM table is handed to the table renderer instead, and + * an indented block is already a code block in markdown. + */ +function isAsciiArt(paragraph: string[]): boolean { + if (paragraph.length < 2) return false; + + const isTable = paragraph.some( + (line, index) => TABLE_ROW.test(line) && TABLE_DELIMITER_ROW.test(paragraph[index + 1] ?? ""), + ); + if (isTable) return false; + + if (paragraph.some((line) => /^\s{4,}\S/.test(line))) return false; + + const art = paragraph.filter(isArtLine).length; + return art >= 2 && art * 2 >= paragraph.length; +} + +/** + * Fences a diagram that was written as plain text. + * + * A directory tree or a boxed diagram is drawn with spaces that carry the + * meaning, and a paragraph reflows them: `MdParagraph.qml` wraps, so the + * columns of an ASCII table land wherever the pane width puts them and the + * picture arrives scrambled. A fence is monospace and scrolls instead of + * wrapping, which is the one rendering that keeps the drawing intact. + */ +export function fenceAsciiArt(markdown: string): string { + const lines = markdown.split("\n"); + const out: string[] = []; + let index = 0; + + while (index < lines.length) { + if (OPENING_FENCE.test(lines[index])) { + index = copyFence(lines, out, index); + continue; + } + + // A paragraph is the run of non-blank lines, which is the unit that has to + // stay together: half a tree in a fence is no better than none of it. + let end = index; + while (end < lines.length && lines[end].trim().length > 0 && !OPENING_FENCE.test(lines[end])) end += 1; + + const paragraph = lines.slice(index, end); + if (isAsciiArt(paragraph)) out.push(FENCE, ...paragraph, FENCE); + else out.push(...paragraph); + + if (end === index) { + // A blank line, which is already a block boundary. + out.push(lines[end]); + index = end + 1; + continue; + } + + index = end; + } + + return out.join("\n"); +} + /** * The full pre-render pass, in order. * * Tables are separated first because they change block boundaries: a blank line - * has to exist before anything else looks at the line. Links are restored next - * so a recovered URL leaves no span for the promoter to fence again. Promotion - * then runs before labelling, so a promoted block is seen as an ordinary fence - * by the labeller, which is also harmless when promotion already inferred a - * language. + * has to exist before anything else looks at the line. Diagrams are fenced next, + * so every pass after this one sees them as code. Links are restored third so a + * recovered URL leaves no span for the promoter to fence again. Promotion then + * runs before labelling, so a promoted block is seen as an ordinary fence by the + * labeller, which is also harmless when promotion already inferred a language. + * Labels are normalised last, which is the only pass that looks at a fence the + * model wrote itself. */ export function repairCodeBlocks(markdown: string): string { - return labelCodeFences(promoteStandaloneCodeSpans(restoreUrlLinks(separateTables(markdown)))); + return normalizeFenceLabels( + labelCodeFences(promoteStandaloneCodeSpans(restoreUrlLinks(fenceAsciiArt(separateTables(markdown))))), + ); } diff --git a/src/lib/constants.ts b/src/lib/constants.ts index 4962ec8..274cf61 100644 --- a/src/lib/constants.ts +++ b/src/lib/constants.ts @@ -80,6 +80,10 @@ Lead with the answer, then the detail. This renders in a narrow window, so stay Use the markdown this window actually draws rather than flattening everything into prose. It renders tables, bulleted and numbered lists, headings, blockquotes, bold, italics, inline code and fenced code, so pick the shape that fits. A table is the right shape for comparing three or more things across the same few attributes, so write the table itself: never hand back a table as a bulleted list of rows, and never describe a table you could simply write. Keep it to about four narrow columns so it fits, give it a header row, and leave a blank line before it. A list is for steps or for independent items. Reach for a heading only when an answer genuinely has sections, since at this size a heading usually costs a line and buys nothing. +Maths is drawn, so write it as LaTeX rather than as flattened text: $x^2$ inside a sentence, and a formula worth a line of its own written as $$...$$. Fractions, roots, sums, integrals, products, Greek letters, and subscripts and superscripts all render. Stacked notation does not, because the window is narrow, so \\frac{a}{b} is drawn as a/b and a matrix comes out linear. Prefer a formula that reads on one line, and keep whole formulas out of code fences. + +A small amount of HTML is drawn too, for the rare case where a visual carries it better than a sentence. Bold, italic, underline, strike, sub, sup and a span with a colour or a size all work, for example new, and a small HTML table is fine. Reach for that only when it earns its place, a status chip or a tiny grid, never to wrap an answer, and still write a comparison as a markdown table. A diagram drawn with characters is fine, but fence it so the columns stay monospaced. + Write plainly: - Never use an em dash. Use a comma, a colon, or a full stop. - Skip stock service-desk phrasing: "Certainly", "I'd be happy to help", "Great question", "I hope this helps", "Let me know if you need anything else". Warmth that means something beats politeness that means nothing. diff --git a/src/lib/html.ts b/src/lib/html.ts new file mode 100644 index 0000000..2afa1eb --- /dev/null +++ b/src/lib/html.ts @@ -0,0 +1,450 @@ +import { mapOutsideCodeSpans, markdownLink } from "./code"; + +/** + * Custom HTML rendering for the detail panel, and the fence around it. + * + * Raw HTML is not merely tolerated by Vicinae's markdown renderer, it is the + * only route to anything beyond what GFM offers. `markdown-model.cpp` passes + * `CMARK_NODE_HTML_INLINE` through unescaped, and `MdParagraph.qml` hands the + * result to a `TextEdit` with `textFormat: RichText`, so a `` with a + * colour, a ``, or a small `` really does draw. That is the + * extension point for a model that wants to show a status chip or a compact + * comparison instead of describing one. + * + * It is also a hole, because the same route carries whatever else the model + * writes: a `