feat: snip a region into chat, and make links clickable #14
@@ -16,7 +16,7 @@ Hit a global shortcut, type a question, get a streamed answer without leaving wh
|
|||||||
- **Persistent conversations.** Every chat is saved locally. The sidebar lists them by recency, and reopening the command picks up where you left off, unless it has been idle longer than you allow.
|
- **Persistent conversations.** Every chat is saved locally. The sidebar lists them by recency, and reopening the command picks up where you left off, unless it has been idle longer than you allow.
|
||||||
- **Readable code.** Fenced blocks render in a dark panel with the language named in a header, syntax colouring, and a Copy button.
|
- **Readable code.** Fenced blocks render in a dark panel with the language named in a header, syntax colouring, and a Copy button.
|
||||||
- **Tools.** Lam decides when to call them: current time, location context, web search, and page fetching.
|
- **Tools.** Lam decides when to call them: current time, location context, web search, and page fetching.
|
||||||
- **Screenshots, two ways.** Snip a region into the conversation you are already in from the action panel, or run **Snip to Lam** to start a new chat about it. The launcher window gets out of the way first, so it never ends up in the shot. Neither is offered unless the current model can actually see images.
|
- **Screenshots, two ways.** Snip a region into the conversation you are already in from the action panel, or run **Snip to Lam** to start a new chat about it. The launcher window gets out of the way first, so it never ends up in the shot, and the capture appears in the transcript as a thumbnail with Open Screenshot for the full image. Neither is offered unless the current model can actually see images.
|
||||||
- **Chat only.** Lam answers questions. It does not touch your files or run commands. The one exception is capture, and it only ever happens because you pressed the key for it.
|
- **Chat only.** Lam answers questions. It does not touch your files or run commands. The one exception is capture, and it only ever happens because you pressed the key for it.
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
@@ -187,7 +187,9 @@ A few decisions worth knowing about before you change things:
|
|||||||
|
|
||||||
**The transcript reads newest message first.** The search bar sits at the top of the window, so the most recent message, an answer once one has been written, is the first line under it rather than below the fold. Within a single exchange that means the answer appears above the question it responds to. `buildThread` produces that order, and `buildThreadMarkdown` regroups messages back into exchanges so the horizontal rule separates one exchange from the next instead of dividing every message.
|
**The transcript reads newest message first.** The search bar sits at the top of the window, so the most recent message, an answer once one has been written, is the first line under it rather than below the fold. Within a single exchange that means the answer appears above the question it responds to. `buildThread` produces that order, and `buildThreadMarkdown` regroups messages back into exchanges so the horizontal rule separates one exchange from the next instead of dividing every message.
|
||||||
|
|
||||||
**The transcript is text-only, and nothing in it is bold except an error.** The answer is what gets read, so it is plain body text carrying no label; the question is a blockquote, which the renderer draws dimmed, italic, and with a left rule. A `**You**` / `**Lam**` header on each exchange gave both speakers identical weight and made the answer sit behind chrome. Tool calls collapse to one italic line (_Tools: `web_search`_) rather than a bold header plus a list, so they stay below the answer too. Raycast solves the same problem with alignment, but markdown offers no alignment control.
|
**The transcript is text plus the attached images, and nothing in it is bold except an error.** The answer is what gets read, so it is plain body text carrying no label; the question is a blockquote, which the renderer draws dimmed, italic, and with a left rule. A `**You**` / `**Lam**` header on each exchange gave both speakers identical weight and made the answer sit behind chrome. Tool calls collapse to one italic line (_Tools: `web_search`_) rather than a bold header plus a list, so they stay below the answer too. Raycast solves the same problem with alignment, but markdown offers no alignment control.
|
||||||
|
|
||||||
|
**An attached screenshot is rendered, not described.** This is a correction: the transcript used to say `_Screenshot attached_` and leave it at that, on the belief that Vicinae could not resolve a local image. It can. `imageProviderUrl` in `markdown-model.cpp` routes `file://` (and a scheme-less string) through `ImageURL::local`, and `MdImage.qml` draws it block-level and centred with the alt text as an italic caption underneath. The one real limit is that the rendered height is capped at 200px regardless of any size hint, so it is a thumbnail rather than something readable, which is why the action panel still offers Open Screenshot for the full image. The alt text is not decoration either: it is what remains if the capture file has since been deleted. A preview of an unsent attachment is shown the same way, so a snip is visible before the question about it exists.
|
||||||
|
|
||||||
**Code fences are labelled so Vicinae can colour them.** `MdCodeBlock.qml` already provides the dark panel, a header naming the language, and a Copy button, so none of that is built here. The language string is what turns on highlighting: `syntax::highlight` resolves it with `definitionForName(language)` and then `definitionForFileName("file." + language)`. So `bash` and `ts` colour correctly, while an unrecognised string such as `terminal` still renders a label but produces no highlighting at all, so a wrong label is worse than none. `code.ts` therefore labels bare fences conservatively, only when a signal is clear, and the default prompt asks for a language on every fence.
|
**Code fences are labelled so Vicinae can colour them.** `MdCodeBlock.qml` already provides the dark panel, a header naming the language, and a Copy button, so none of that is built here. The language string is what turns on highlighting: `syntax::highlight` resolves it with `definitionForName(language)` and then `definitionForFileName("file." + language)`. So `bash` and `ts` colour correctly, while an unrecognised string such as `terminal` still renders a label but produces no highlighting at all, so a wrong label is worse than none. `code.ts` therefore labels bare fences conservatively, only when a signal is clear, and the default prompt asks for a language on every fence.
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
|||||||
import { tmpdir } from "node:os";
|
import { tmpdir } from "node:os";
|
||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
import { afterEach, describe, expect, it } from "vitest";
|
import { afterEach, describe, expect, it } from "vitest";
|
||||||
import { mimeForPath, readAttachment } from "../lib/attachments";
|
import { imageSource, mimeForPath, readAttachment } from "../lib/attachments";
|
||||||
|
|
||||||
const tempDirs: string[] = [];
|
const tempDirs: string[] = [];
|
||||||
|
|
||||||
@@ -30,6 +30,26 @@ describe("mimeForPath", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("imageSource", () => {
|
||||||
|
it("produces a file URL, which is the only scheme the renderer resolves for a local file", () => {
|
||||||
|
expect(imageSource("/tmp/snip.png")).toBe("file:///tmp/snip.png");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("escapes a path with a space, which the user's own screenshot directory has", () => {
|
||||||
|
expect(imageSource("/Users/x/Pictures/Screenshot 2026-10-04.png")).toBe(
|
||||||
|
"file:///Users/x/Pictures/Screenshot%202026-10-04.png",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("escapes non-ascii", () => {
|
||||||
|
expect(imageSource("/tmp/caf\u00e9.png")).toBe("file:///tmp/caf%C3%A9.png");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps a bare path out of the markdown, because that resolves to nothing", () => {
|
||||||
|
expect(imageSource("/tmp/snip.png")).not.toBe("/tmp/snip.png");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe("readAttachment", () => {
|
describe("readAttachment", () => {
|
||||||
it("reads the bytes as bare base64, with no data URL prefix", async () => {
|
it("reads the bytes as bare base64, with no data URL prefix", async () => {
|
||||||
const dir = await tempDir();
|
const dir = await tempDir();
|
||||||
|
|||||||
@@ -75,9 +75,9 @@ describe("buildThreadMarkdown", () => {
|
|||||||
expect(markdown.indexOf("second question")).toBeLessThan(markdown.indexOf("first answer"));
|
expect(markdown.indexOf("second question")).toBeLessThan(markdown.indexOf("first answer"));
|
||||||
});
|
});
|
||||||
|
|
||||||
it("renders no images, because markdown cannot resolve them", () => {
|
it("renders no images for a message that has no attachment", () => {
|
||||||
// Images render block-level and centred at up to 200px (`MdImage.qml`), and
|
// Images only ever come from an attachment this extension took itself. A URL
|
||||||
// every candidate icon source fails. Keep the transcript text-only.
|
// the model invented is not resolvable and renders as an empty block.
|
||||||
const markdown = buildThreadMarkdown([
|
const markdown = buildThreadMarkdown([
|
||||||
message({ role: "user", text: "q" }),
|
message({ role: "user", text: "q" }),
|
||||||
message({ role: "assistant", text: "a" }),
|
message({ role: "assistant", text: "a" }),
|
||||||
@@ -290,15 +290,43 @@ describe("sessionToPlainText", () => {
|
|||||||
describe("attachments in the transcript", () => {
|
describe("attachments in the transcript", () => {
|
||||||
const screenshot = { path: "/tmp/snip.png", mime: "image/png" };
|
const screenshot = { path: "/tmp/snip.png", mime: "image/png" };
|
||||||
|
|
||||||
it("says a screenshot is attached, because the image cannot be rendered", () => {
|
it("shows the attachment instead of just naming it", () => {
|
||||||
// Images render block-level, centred and capped at 200px (`MdImage.qml`),
|
// Vicinae resolves a local file through `ImageURL::local`, so the capture is
|
||||||
// and no form of `` resolves to a local capture. Saying it is there
|
// rendered rather than described. `MdImage.qml` caps the height at 200px, so
|
||||||
// is the honest option; the file itself is opened from the action panel.
|
// it is a thumbnail, and the alt text is what is left if the file is gone.
|
||||||
const markdown = buildThreadMarkdown([
|
const markdown = buildThreadMarkdown([
|
||||||
message({ role: "user", text: "what is this?", attachments: [screenshot] }),
|
message({ role: "user", text: "what is this?", attachments: [screenshot] }),
|
||||||
]);
|
]);
|
||||||
|
|
||||||
expect(markdown).toContain("_Screenshot attached_");
|
expect(markdown).toContain("");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps the question above its image", () => {
|
||||||
|
const markdown = buildThreadMarkdown([
|
||||||
|
message({ role: "user", text: "what is this?", attachments: [screenshot] }),
|
||||||
|
]);
|
||||||
|
|
||||||
|
expect(markdown.indexOf("> what is this?")).toBeLessThan(markdown.indexOf("!["));
|
||||||
|
});
|
||||||
|
|
||||||
|
it("renders every attachment on a turn", () => {
|
||||||
|
const markdown = buildThreadMarkdown([
|
||||||
|
message({
|
||||||
|
role: "user",
|
||||||
|
text: "compare",
|
||||||
|
attachments: [screenshot, { path: "/tmp/second.png", mime: "image/png" }],
|
||||||
|
}),
|
||||||
|
]);
|
||||||
|
|
||||||
|
expect(markdown).toContain("file:///tmp/snip.png");
|
||||||
|
expect(markdown).toContain("file:///tmp/second.png");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("never puts an image on an answer, which has no attachments", () => {
|
||||||
|
const markdown = buildThreadMarkdown([
|
||||||
|
message({ role: "assistant", text: "a cat", attachments: [screenshot] }),
|
||||||
|
]);
|
||||||
|
|
||||||
expect(markdown).not.toContain("![");
|
expect(markdown).not.toContain("![");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
import { readFile } from "node:fs/promises";
|
import { readFile } from "node:fs/promises";
|
||||||
import { extname } from "node:path";
|
import { extname } from "node:path";
|
||||||
|
import { pathToFileURL } from "node:url";
|
||||||
import type { PendingAttachment } from "./types";
|
import type { PendingAttachment } from "./types";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -26,6 +27,20 @@ export function mimeForPath(path: string): string {
|
|||||||
return MIME_TYPES[extname(path).toLowerCase()] ?? "application/octet-stream";
|
return MIME_TYPES[extname(path).toLowerCase()] ?? "application/octet-stream";
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A markdown image source for a local file.
|
||||||
|
*
|
||||||
|
* A `file://` URL rather than a bare path, because Vicinae's markdown renderer
|
||||||
|
* only resolves a scheme it recognises: `imageProviderUrl` in
|
||||||
|
* `markdown-model.cpp` routes `file://`, and a scheme-less string, through
|
||||||
|
* `ImageURL::local`, and anything it does not recognise it returns as empty.
|
||||||
|
* `pathToFileURL` does the escaping, so a capture under a directory with a space
|
||||||
|
* in its name still resolves.
|
||||||
|
*/
|
||||||
|
export function imageSource(path: string): string {
|
||||||
|
return pathToFileURL(path).href;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Reads an image into the shape a request needs.
|
* Reads an image into the shape a request needs.
|
||||||
*
|
*
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ import { useCallback, useEffect, useMemo, useState } from "react";
|
|||||||
import { readAttachment } from "../attachments";
|
import { readAttachment } from "../attachments";
|
||||||
import { useChat } from "../hooks/useChat";
|
import { useChat } from "../hooks/useChat";
|
||||||
import { useVisionSupport } from "../hooks/useVision";
|
import { useVisionSupport } from "../hooks/useVision";
|
||||||
import { buildThreadMarkdown, type TranscriptOptions } from "../markdown";
|
import { buildThreadMarkdown, renderAttachments, type TranscriptOptions } from "../markdown";
|
||||||
import { getLamPreferences } from "../preferences";
|
import { getLamPreferences } from "../preferences";
|
||||||
import { deleteSession, listSessions, loadSession, summarizeSession } from "../sessions";
|
import { deleteSession, listSessions, loadSession, summarizeSession } from "../sessions";
|
||||||
import { resolveStartup } from "../startup";
|
import { resolveStartup } from "../startup";
|
||||||
@@ -59,7 +59,7 @@ const EMPTY_TRANSCRIPT = "**Ask Lam anything**\n\nType a question in the search
|
|||||||
const DEFAULT_IMAGE_PROMPT = "What is in this image?";
|
const DEFAULT_IMAGE_PROMPT = "What is in this image?";
|
||||||
|
|
||||||
/** The reminder shown while a snip is attached but not yet sent. */
|
/** The reminder shown while a snip is attached but not yet sent. */
|
||||||
const PENDING_NOTE = "_Screenshot attached. Ask a question about it, or press Return to send it as it is._";
|
const PENDING_NOTE = "_Ask a question about it, or press Return to send it as it is._";
|
||||||
|
|
||||||
export interface ChatViewProps {
|
export interface ChatViewProps {
|
||||||
/** Opens a specific conversation instead of resuming the latest one. */
|
/** Opens a specific conversation instead of resuming the latest one. */
|
||||||
@@ -182,9 +182,13 @@ export function ChatView({
|
|||||||
* already looking.
|
* already looking.
|
||||||
*/
|
*/
|
||||||
const threadMarkdown = useMemo(() => {
|
const threadMarkdown = useMemo(() => {
|
||||||
const notes = [launchNote, snipNote, pending ? PENDING_NOTE : undefined].filter((note): note is string =>
|
const notes = [
|
||||||
Boolean(note),
|
launchNote,
|
||||||
);
|
snipNote,
|
||||||
|
// The preview comes first so the captured image is visible before the user
|
||||||
|
// decides what to ask about it, rather than only after sending.
|
||||||
|
pending ? `${renderAttachments([pending])}\n\n${PENDING_NOTE}` : undefined,
|
||||||
|
].filter((note): note is string => Boolean(note));
|
||||||
|
|
||||||
const transcript =
|
const transcript =
|
||||||
thread.length === 0 ? EMPTY_TRANSCRIPT : buildThreadMarkdown(thread, transcriptOptions);
|
thread.length === 0 ? EMPTY_TRANSCRIPT : buildThreadMarkdown(thread, transcriptOptions);
|
||||||
|
|||||||
+24
-7
@@ -1,5 +1,6 @@
|
|||||||
|
import { imageSource } from "./attachments";
|
||||||
import { repairCodeBlocks } from "./code";
|
import { repairCodeBlocks } from "./code";
|
||||||
import type { ChatTurn, ThreadMessage } from "./types";
|
import type { Attachment, ChatTurn, ThreadMessage } from "./types";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Markdown rendering for the detail panel.
|
* Markdown rendering for the detail panel.
|
||||||
@@ -22,6 +23,24 @@ import type { ChatTurn, ThreadMessage } from "./types";
|
|||||||
|
|
||||||
export const STREAMING_CURSOR = "▍";
|
export const STREAMING_CURSOR = "▍";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Renders attachments as markdown images.
|
||||||
|
*
|
||||||
|
* Vicinae does resolve a local file: `MdImage.qml` draws it block-level and
|
||||||
|
* centred, uses the alt text as an italic caption underneath, and caps the
|
||||||
|
* height at 200px no matter what is asked for. So this is a thumbnail rather
|
||||||
|
* than something readable, which is why the action panel also offers Open
|
||||||
|
* Screenshot for the full image.
|
||||||
|
*
|
||||||
|
* The alt text is not decoration: it is what remains if the file has since been
|
||||||
|
* deleted, so it says what the picture was rather than leaving a blank space.
|
||||||
|
*/
|
||||||
|
export function renderAttachments(attachments: Attachment[]): string {
|
||||||
|
return attachments
|
||||||
|
.map((attachment) => `})`)
|
||||||
|
.join("\n\n");
|
||||||
|
}
|
||||||
|
|
||||||
/** Which optional sections the transcript includes. Driven by preferences. */
|
/** Which optional sections the transcript includes. Driven by preferences. */
|
||||||
export interface TranscriptOptions {
|
export interface TranscriptOptions {
|
||||||
showToolCalls: boolean;
|
showToolCalls: boolean;
|
||||||
@@ -50,12 +69,10 @@ function renderTools(message: ThreadMessage): string {
|
|||||||
function renderMessage(message: ThreadMessage, options: TranscriptOptions): string {
|
function renderMessage(message: ThreadMessage, options: TranscriptOptions): string {
|
||||||
// The question is identified by its styling alone.
|
// The question is identified by its styling alone.
|
||||||
if (message.role === "user") {
|
if (message.role === "user") {
|
||||||
// An attached screenshot cannot be shown here: images render block-level,
|
const images = renderAttachments(message.attachments);
|
||||||
// centred and capped at 200px (`MdImage.qml`), and there is no form of
|
const asked = quote(message.text);
|
||||||
// `` that resolves to a local capture. The transcript says the
|
|
||||||
// image is there instead, and the action panel opens it.
|
return images ? `${asked}\n\n${images}` : asked;
|
||||||
const attached = message.attachments.length > 0 ? "\n\n_Screenshot attached_" : "";
|
|
||||||
return `${quote(message.text)}${attached}`;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// While a tool is running there is no answer text yet, so the activity line
|
// While a tool is running there is no answer text yet, so the activity line
|
||||||
|
|||||||
Reference in New Issue
Block a user