feat: render the attached screenshot instead of just naming it

The transcript said "_Screenshot attached_" and left it at that, on a note in
the test file claiming Vicinae could not resolve a local image. It can. That
claim was true of a URL the model invented, not of a file this extension
captured itself.

Checked against the renderer rather than assumed:

- `imageProviderUrl` in `markdown-model.cpp` routes `file://`, and a
  scheme-less string, through `ImageURL::local`. Anything it does not
  recognise comes back empty, which is why a bare path is not good enough.
- `MdImage.qml` draws it block-level and centred, using the alt text as an
  italic caption, and caps the rendered height at 200px regardless of the
  `raycast-width`/`raycast-height` hint it also accepts.

So `attachments.ts` gains `imageSource`, which builds a `file://` URL with
`pathToFileURL` so a path with a space or non-ascii still resolves, and
`markdown.ts` gains `renderAttachments`, used for a turn's attachments and
for an unsent pending snip. That second use is the one that matters most:
the capture is now visible before the question about it exists.

The alt text carries the label, so a deleted file degrades to what the
transcript used to say. The 200px cap is a Vicinae limit, so the action panel
keeps Open Screenshot for the full image.

Tests that asserted the old behaviour are inverted, and imageSource is
covered for spaces and non-ascii.

Verified: 280 tests, tsc, eslint, prettier, and vici build.
This commit is contained in:
2026-10-04 15:37:14 +08:00
parent e00b49bbe5
commit c1212192c5
6 changed files with 109 additions and 23 deletions
+4 -2
View File
@@ -16,7 +16,7 @@ Hit a global shortcut, type a question, get a streamed answer without leaving wh
- **Persistent conversations.** Every chat is saved locally. The sidebar lists them by recency, and reopening the command picks up where you left off, unless it has been idle longer than you allow.
- **Readable code.** Fenced blocks render in a dark panel with the language named in a header, syntax colouring, and a Copy button.
- **Tools.** Lam decides when to call them: current time, location context, web search, and page fetching.
- **Screenshots, two ways.** Snip a region into the conversation you are already in from the action panel, or run **Snip to Lam** to start a new chat about it. The launcher window gets out of the way first, so it never ends up in the shot. Neither is offered unless the current model can actually see images.
- **Screenshots, two ways.** Snip a region into the conversation you are already in from the action panel, or run **Snip to Lam** to start a new chat about it. The launcher window gets out of the way first, so it never ends up in the shot, and the capture appears in the transcript as a thumbnail with Open Screenshot for the full image. Neither is offered unless the current model can actually see images.
- **Chat only.** Lam answers questions. It does not touch your files or run commands. The one exception is capture, and it only ever happens because you pressed the key for it.
## Requirements
@@ -187,7 +187,9 @@ A few decisions worth knowing about before you change things:
**The transcript reads newest message first.** The search bar sits at the top of the window, so the most recent message, an answer once one has been written, is the first line under it rather than below the fold. Within a single exchange that means the answer appears above the question it responds to. `buildThread` produces that order, and `buildThreadMarkdown` regroups messages back into exchanges so the horizontal rule separates one exchange from the next instead of dividing every message.
**The transcript is text-only, and nothing in it is bold except an error.** The answer is what gets read, so it is plain body text carrying no label; the question is a blockquote, which the renderer draws dimmed, italic, and with a left rule. A `**You**` / `**Lam**` header on each exchange gave both speakers identical weight and made the answer sit behind chrome. Tool calls collapse to one italic line (_Tools: `web_search`_) rather than a bold header plus a list, so they stay below the answer too. Raycast solves the same problem with alignment, but markdown offers no alignment control.
**The transcript is text plus the attached images, and nothing in it is bold except an error.** The answer is what gets read, so it is plain body text carrying no label; the question is a blockquote, which the renderer draws dimmed, italic, and with a left rule. A `**You**` / `**Lam**` header on each exchange gave both speakers identical weight and made the answer sit behind chrome. Tool calls collapse to one italic line (_Tools: `web_search`_) rather than a bold header plus a list, so they stay below the answer too. Raycast solves the same problem with alignment, but markdown offers no alignment control.
**An attached screenshot is rendered, not described.** This is a correction: the transcript used to say `_Screenshot attached_` and leave it at that, on the belief that Vicinae could not resolve a local image. It can. `imageProviderUrl` in `markdown-model.cpp` routes `file://` (and a scheme-less string) through `ImageURL::local`, and `MdImage.qml` draws it block-level and centred with the alt text as an italic caption underneath. The one real limit is that the rendered height is capped at 200px regardless of any size hint, so it is a thumbnail rather than something readable, which is why the action panel still offers Open Screenshot for the full image. The alt text is not decoration either: it is what remains if the capture file has since been deleted. A preview of an unsent attachment is shown the same way, so a snip is visible before the question about it exists.
**Code fences are labelled so Vicinae can colour them.** `MdCodeBlock.qml` already provides the dark panel, a header naming the language, and a Copy button, so none of that is built here. The language string is what turns on highlighting: `syntax::highlight` resolves it with `definitionForName(language)` and then `definitionForFileName("file." + language)`. So `bash` and `ts` colour correctly, while an unrecognised string such as `terminal` still renders a label but produces no highlighting at all, so a wrong label is worse than none. `code.ts` therefore labels bare fences conservatively, only when a signal is clear, and the default prompt asks for a language on every fence.
+21 -1
View File
@@ -2,7 +2,7 @@ import { mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, describe, expect, it } from "vitest";
import { mimeForPath, readAttachment } from "../lib/attachments";
import { imageSource, mimeForPath, readAttachment } from "../lib/attachments";
const tempDirs: string[] = [];
@@ -30,6 +30,26 @@ describe("mimeForPath", () => {
});
});
describe("imageSource", () => {
it("produces a file URL, which is the only scheme the renderer resolves for a local file", () => {
expect(imageSource("/tmp/snip.png")).toBe("file:///tmp/snip.png");
});
it("escapes a path with a space, which the user's own screenshot directory has", () => {
expect(imageSource("/Users/x/Pictures/Screenshot 2026-10-04.png")).toBe(
"file:///Users/x/Pictures/Screenshot%202026-10-04.png",
);
});
it("escapes non-ascii", () => {
expect(imageSource("/tmp/caf\u00e9.png")).toBe("file:///tmp/caf%C3%A9.png");
});
it("keeps a bare path out of the markdown, because that resolves to nothing", () => {
expect(imageSource("/tmp/snip.png")).not.toBe("/tmp/snip.png");
});
});
describe("readAttachment", () => {
it("reads the bytes as bare base64, with no data URL prefix", async () => {
const dir = await tempDir();
+36 -8
View File
@@ -75,9 +75,9 @@ describe("buildThreadMarkdown", () => {
expect(markdown.indexOf("second question")).toBeLessThan(markdown.indexOf("first answer"));
});
it("renders no images, because markdown cannot resolve them", () => {
// Images render block-level and centred at up to 200px (`MdImage.qml`), and
// every candidate icon source fails. Keep the transcript text-only.
it("renders no images for a message that has no attachment", () => {
// Images only ever come from an attachment this extension took itself. A URL
// the model invented is not resolvable and renders as an empty block.
const markdown = buildThreadMarkdown([
message({ role: "user", text: "q" }),
message({ role: "assistant", text: "a" }),
@@ -290,15 +290,43 @@ describe("sessionToPlainText", () => {
describe("attachments in the transcript", () => {
const screenshot = { path: "/tmp/snip.png", mime: "image/png" };
it("says a screenshot is attached, because the image cannot be rendered", () => {
// Images render block-level, centred and capped at 200px (`MdImage.qml`),
// and no form of `![](path)` resolves to a local capture. Saying it is there
// is the honest option; the file itself is opened from the action panel.
it("shows the attachment instead of just naming it", () => {
// Vicinae resolves a local file through `ImageURL::local`, so the capture is
// rendered rather than described. `MdImage.qml` caps the height at 200px, so
// it is a thumbnail, and the alt text is what is left if the file is gone.
const markdown = buildThreadMarkdown([
message({ role: "user", text: "what is this?", attachments: [screenshot] }),
]);
expect(markdown).toContain("_Screenshot attached_");
expect(markdown).toContain("![Screenshot attached](file:///tmp/snip.png)");
});
it("keeps the question above its image", () => {
const markdown = buildThreadMarkdown([
message({ role: "user", text: "what is this?", attachments: [screenshot] }),
]);
expect(markdown.indexOf("> what is this?")).toBeLessThan(markdown.indexOf("!["));
});
it("renders every attachment on a turn", () => {
const markdown = buildThreadMarkdown([
message({
role: "user",
text: "compare",
attachments: [screenshot, { path: "/tmp/second.png", mime: "image/png" }],
}),
]);
expect(markdown).toContain("file:///tmp/snip.png");
expect(markdown).toContain("file:///tmp/second.png");
});
it("never puts an image on an answer, which has no attachments", () => {
const markdown = buildThreadMarkdown([
message({ role: "assistant", text: "a cat", attachments: [screenshot] }),
]);
expect(markdown).not.toContain("![");
});
+15
View File
@@ -1,5 +1,6 @@
import { readFile } from "node:fs/promises";
import { extname } from "node:path";
import { pathToFileURL } from "node:url";
import type { PendingAttachment } from "./types";
/**
@@ -26,6 +27,20 @@ export function mimeForPath(path: string): string {
return MIME_TYPES[extname(path).toLowerCase()] ?? "application/octet-stream";
}
/**
* A markdown image source for a local file.
*
* A `file://` URL rather than a bare path, because Vicinae's markdown renderer
* only resolves a scheme it recognises: `imageProviderUrl` in
* `markdown-model.cpp` routes `file://`, and a scheme-less string, through
* `ImageURL::local`, and anything it does not recognise it returns as empty.
* `pathToFileURL` does the escaping, so a capture under a directory with a space
* in its name still resolves.
*/
export function imageSource(path: string): string {
return pathToFileURL(path).href;
}
/**
* Reads an image into the shape a request needs.
*
+9 -5
View File
@@ -3,7 +3,7 @@ import { useCallback, useEffect, useMemo, useState } from "react";
import { readAttachment } from "../attachments";
import { useChat } from "../hooks/useChat";
import { useVisionSupport } from "../hooks/useVision";
import { buildThreadMarkdown, type TranscriptOptions } from "../markdown";
import { buildThreadMarkdown, renderAttachments, type TranscriptOptions } from "../markdown";
import { getLamPreferences } from "../preferences";
import { deleteSession, listSessions, loadSession, summarizeSession } from "../sessions";
import { resolveStartup } from "../startup";
@@ -59,7 +59,7 @@ const EMPTY_TRANSCRIPT = "**Ask Lam anything**\n\nType a question in the search
const DEFAULT_IMAGE_PROMPT = "What is in this image?";
/** The reminder shown while a snip is attached but not yet sent. */
const PENDING_NOTE = "_Screenshot attached. Ask a question about it, or press Return to send it as it is._";
const PENDING_NOTE = "_Ask a question about it, or press Return to send it as it is._";
export interface ChatViewProps {
/** Opens a specific conversation instead of resuming the latest one. */
@@ -182,9 +182,13 @@ export function ChatView({
* already looking.
*/
const threadMarkdown = useMemo(() => {
const notes = [launchNote, snipNote, pending ? PENDING_NOTE : undefined].filter((note): note is string =>
Boolean(note),
);
const notes = [
launchNote,
snipNote,
// The preview comes first so the captured image is visible before the user
// decides what to ask about it, rather than only after sending.
pending ? `${renderAttachments([pending])}\n\n${PENDING_NOTE}` : undefined,
].filter((note): note is string => Boolean(note));
const transcript =
thread.length === 0 ? EMPTY_TRANSCRIPT : buildThreadMarkdown(thread, transcriptOptions);
+24 -7
View File
@@ -1,5 +1,6 @@
import { imageSource } from "./attachments";
import { repairCodeBlocks } from "./code";
import type { ChatTurn, ThreadMessage } from "./types";
import type { Attachment, ChatTurn, ThreadMessage } from "./types";
/**
* Markdown rendering for the detail panel.
@@ -22,6 +23,24 @@ import type { ChatTurn, ThreadMessage } from "./types";
export const STREAMING_CURSOR = "▍";
/**
* Renders attachments as markdown images.
*
* Vicinae does resolve a local file: `MdImage.qml` draws it block-level and
* centred, uses the alt text as an italic caption underneath, and caps the
* height at 200px no matter what is asked for. So this is a thumbnail rather
* than something readable, which is why the action panel also offers Open
* Screenshot for the full image.
*
* The alt text is not decoration: it is what remains if the file has since been
* deleted, so it says what the picture was rather than leaving a blank space.
*/
export function renderAttachments(attachments: Attachment[]): string {
return attachments
.map((attachment) => `![Screenshot attached](${imageSource(attachment.path)})`)
.join("\n\n");
}
/** Which optional sections the transcript includes. Driven by preferences. */
export interface TranscriptOptions {
showToolCalls: boolean;
@@ -50,12 +69,10 @@ function renderTools(message: ThreadMessage): string {
function renderMessage(message: ThreadMessage, options: TranscriptOptions): string {
// The question is identified by its styling alone.
if (message.role === "user") {
// An attached screenshot cannot be shown here: images render block-level,
// centred and capped at 200px (`MdImage.qml`), and there is no form of
// `![](path)` that resolves to a local capture. The transcript says the
// image is there instead, and the action panel opens it.
const attached = message.attachments.length > 0 ? "\n\n_Screenshot attached_" : "";
return `${quote(message.text)}${attached}`;
const images = renderAttachments(message.attachments);
const asked = quote(message.text);
return images ? `${asked}\n\n${images}` : asked;
}
// While a tool is running there is no answer text yet, so the activity line