fix: lay the screenshot preview out to the image, not to the request
The preview rendered, but with dead space around it. `MdImage.qml` sizes its container to the request rather than to the image, so with no hint it is the full pane width and 200px tall and the image is aspect-fitted inside that. A wide capture sat in a letterbox: measured against the real captures in the support directory, a 4:1 crop wasted about 106px of the 200px container and a 6.7:1 crop about 140px, which is what made the transcript look gappy and put the caption far below the image. - `readAttachment` now reads the PNG `IHDR` chunk, so `Attachment` carries the real dimensions and they are persisted with the turn. - `previewBox` computes an exact box from those dimensions, scaled to fit 400x200 with the scale capped at 1 so a small capture is not upscaled, and `imageSource` emits it as the `raycast-width`/`raycast-height` hints `parseImageSize` looks for. The path is resolved from the URL before the query, so the hints never reach the file name. - The alt text is now "Screenshot". It is drawn centred under the image, so the sentence "Screenshot attached" read as a stray line of prose floating in the middle of the pane. - A pending note no longer stacks on top of the empty-transcript placeholder. Both said the same thing, one line under the other, once a snip was attached. A capture with no readable header simply falls back to the old sizing. Verified: 293 tests, tsc, eslint, prettier, and vici build.
This commit is contained in:
@@ -189,7 +189,11 @@ A few decisions worth knowing about before you change things:
|
|||||||
|
|
||||||
**The transcript is text plus the attached images, and nothing in it is bold except an error.** The answer is what gets read, so it is plain body text carrying no label; the question is a blockquote, which the renderer draws dimmed, italic, and with a left rule. A `**You**` / `**Lam**` header on each exchange gave both speakers identical weight and made the answer sit behind chrome. Tool calls collapse to one italic line (_Tools: `web_search`_) rather than a bold header plus a list, so they stay below the answer too. Raycast solves the same problem with alignment, but markdown offers no alignment control.
|
**The transcript is text plus the attached images, and nothing in it is bold except an error.** The answer is what gets read, so it is plain body text carrying no label; the question is a blockquote, which the renderer draws dimmed, italic, and with a left rule. A `**You**` / `**Lam**` header on each exchange gave both speakers identical weight and made the answer sit behind chrome. Tool calls collapse to one italic line (_Tools: `web_search`_) rather than a bold header plus a list, so they stay below the answer too. Raycast solves the same problem with alignment, but markdown offers no alignment control.
|
||||||
|
|
||||||
**An attached screenshot is rendered, not described.** This is a correction: the transcript used to say `_Screenshot attached_` and leave it at that, on the belief that Vicinae could not resolve a local image. It can. `imageProviderUrl` in `markdown-model.cpp` routes `file://` (and a scheme-less string) through `ImageURL::local`, and `MdImage.qml` draws it block-level and centred with the alt text as an italic caption underneath. The one real limit is that the rendered height is capped at 200px regardless of any size hint, so it is a thumbnail rather than something readable, which is why the action panel still offers Open Screenshot for the full image. The alt text is not decoration either: it is what remains if the capture file has since been deleted. A preview of an unsent attachment is shown the same way, so a snip is visible before the question about it exists.
|
**An attached screenshot is rendered, not described.** This is a correction: the transcript used to say `_Screenshot attached_` and leave it at that, on the belief that Vicinae could not resolve a local image. It can. `imageProviderUrl` in `markdown-model.cpp` routes `file://` (and a scheme-less string) through `ImageURL::local`, and `MdImage.qml` draws it block-level and centred with the alt text as an italic caption underneath.
|
||||||
|
|
||||||
|
**The preview is asked for as an exact box, which is why the PNG header is parsed.** `MdImage.qml` sizes its container to the *request* rather than to the image: with no hint it is the full pane width and 200px tall, and the image is aspect-fitted inside that, so a wide capture sits in a letterbox with dead space above and below. Measured on real captures here, a 4:1 crop wasted about 106px of a 200px container and a 6.7:1 crop wasted about 140px. `previewBox` therefore computes a box from the image's real dimensions, scales it to fit 400x200, and emits it as the `raycast-width` and `raycast-height` hints the renderer reads, so the container matches the image and nothing is wasted. The scale is capped at 1 so a small capture is not blown up.
|
||||||
|
|
||||||
|
The rendered height stays capped at 200px whatever is asked for (`Math.min(_rawH, 200, maxImageHeight)`), so this is a thumbnail rather than something readable, which is why the action panel still offers Open Screenshot for the full image. The alt text is short for the same reason it is not empty: whatever is there is drawn centred under the image, so a sentence reads as a stray line of prose, and it is the only thing left if the capture file has since been deleted. A preview of an unsent attachment is shown the same way, so a snip is visible before the question about it exists.
|
||||||
|
|
||||||
**Code fences are labelled so Vicinae can colour them.** `MdCodeBlock.qml` already provides the dark panel, a header naming the language, and a Copy button, so none of that is built here. The language string is what turns on highlighting: `syntax::highlight` resolves it with `definitionForName(language)` and then `definitionForFileName("file." + language)`. So `bash` and `ts` colour correctly, while an unrecognised string such as `terminal` still renders a label but produces no highlighting at all, so a wrong label is worse than none. `code.ts` therefore labels bare fences conservatively, only when a signal is clear, and the default prompt asks for a language on every fence.
|
**Code fences are labelled so Vicinae can colour them.** `MdCodeBlock.qml` already provides the dark panel, a header naming the language, and a Copy button, so none of that is built here. The language string is what turns on highlighting: `syntax::highlight` resolves it with `definitionForName(language)` and then `definitionForFileName("file." + language)`. So `bash` and `ts` colour correctly, while an unrecognised string such as `terminal` still renders a label but produces no highlighting at all, so a wrong label is worse than none. `code.ts` therefore labels bare fences conservatively, only when a signal is clear, and the default prompt asks for a language on every fence.
|
||||||
|
|
||||||
|
|||||||
@@ -2,10 +2,21 @@ import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
|||||||
import { tmpdir } from "node:os";
|
import { tmpdir } from "node:os";
|
||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
import { afterEach, describe, expect, it } from "vitest";
|
import { afterEach, describe, expect, it } from "vitest";
|
||||||
import { imageSource, mimeForPath, readAttachment } from "../lib/attachments";
|
import { imageSource, mimeForPath, pngSize, previewBox, readAttachment } from "../lib/attachments";
|
||||||
|
|
||||||
const tempDirs: string[] = [];
|
const tempDirs: string[] = [];
|
||||||
|
|
||||||
|
/** A 24-byte PNG header, which is all `pngSize` reads. */
|
||||||
|
function pngHeader(width: number, height: number): Buffer {
|
||||||
|
const bytes = Buffer.alloc(24);
|
||||||
|
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]).copy(bytes, 0);
|
||||||
|
bytes.writeUInt32BE(13, 8);
|
||||||
|
bytes.write("IHDR", 12, "ascii");
|
||||||
|
bytes.writeUInt32BE(width, 16);
|
||||||
|
bytes.writeUInt32BE(height, 20);
|
||||||
|
return bytes;
|
||||||
|
}
|
||||||
|
|
||||||
async function tempDir(): Promise<string> {
|
async function tempDir(): Promise<string> {
|
||||||
const dir = await mkdtemp(join(tmpdir(), "lam-attach-"));
|
const dir = await mkdtemp(join(tmpdir(), "lam-attach-"));
|
||||||
tempDirs.push(dir);
|
tempDirs.push(dir);
|
||||||
@@ -30,23 +41,83 @@ describe("mimeForPath", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("pngSize", () => {
|
||||||
|
it("reads the dimensions out of the IHDR chunk", () => {
|
||||||
|
expect(pngSize(pngHeader(1600, 400))).toEqual({ width: 1600, height: 400 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("refuses anything that is not a PNG", () => {
|
||||||
|
// Guessing at another format would put a wrong aspect ratio into the layout.
|
||||||
|
expect(pngSize(Buffer.from("not a png at all, just some text"))).toBeUndefined();
|
||||||
|
expect(pngSize(Buffer.alloc(0))).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("refuses a header that is present but truncated", () => {
|
||||||
|
expect(pngSize(pngHeader(10, 10).subarray(0, 20))).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("refuses degenerate dimensions", () => {
|
||||||
|
expect(pngSize(pngHeader(0, 100))).toBeUndefined();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("previewBox", () => {
|
||||||
|
it("fits a wide capture to the width limit and keeps the aspect ratio", () => {
|
||||||
|
// This is the case that produced the dead space: 4:1 at full pane width was
|
||||||
|
// letterboxed into a 200px-tall container it barely used.
|
||||||
|
expect(previewBox(1600, 400)).toEqual({ width: 400, height: 100 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("fits a 16:9 capture to the height limit instead", () => {
|
||||||
|
expect(previewBox(1920, 1080)).toEqual({ width: 356, height: 200 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("fits a portrait capture by its height", () => {
|
||||||
|
expect(previewBox(400, 1600)).toEqual({ width: 50, height: 200 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not blow a small capture up into a blurry one", () => {
|
||||||
|
expect(previewBox(100, 50)).toEqual({ width: 100, height: 50 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("has nothing to say when the size is unknown", () => {
|
||||||
|
expect(previewBox(undefined, undefined)).toBeUndefined();
|
||||||
|
expect(previewBox(1600, undefined)).toBeUndefined();
|
||||||
|
expect(previewBox(1600, 0)).toBeUndefined();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
describe("imageSource", () => {
|
describe("imageSource", () => {
|
||||||
it("produces a file URL, which is the only scheme the renderer resolves for a local file", () => {
|
it("produces a file URL, which is the only scheme the renderer resolves for a local file", () => {
|
||||||
expect(imageSource("/tmp/snip.png")).toBe("file:///tmp/snip.png");
|
expect(imageSource({ path: "/tmp/snip.png" })).toBe("file:///tmp/snip.png");
|
||||||
});
|
});
|
||||||
|
|
||||||
it("escapes a path with a space, which the user's own screenshot directory has", () => {
|
it("escapes a path with a space, which the user's own screenshot directory has", () => {
|
||||||
expect(imageSource("/Users/x/Pictures/Screenshot 2026-10-04.png")).toBe(
|
expect(imageSource({ path: "/Users/x/Pictures/Screenshot 2026-10-04.png" })).toBe(
|
||||||
"file:///Users/x/Pictures/Screenshot%202026-10-04.png",
|
"file:///Users/x/Pictures/Screenshot%202026-10-04.png",
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("escapes non-ascii", () => {
|
it("escapes non-ascii", () => {
|
||||||
expect(imageSource("/tmp/caf\u00e9.png")).toBe("file:///tmp/caf%C3%A9.png");
|
expect(imageSource({ path: "/tmp/caf\u00e9.png" })).toBe("file:///tmp/caf%C3%A9.png");
|
||||||
});
|
});
|
||||||
|
|
||||||
it("keeps a bare path out of the markdown, because that resolves to nothing", () => {
|
it("keeps a bare path out of the markdown, because that resolves to nothing", () => {
|
||||||
expect(imageSource("/tmp/snip.png")).not.toBe("/tmp/snip.png");
|
expect(imageSource({ path: "/tmp/snip.png" })).not.toBe("/tmp/snip.png");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("carries the size hints the renderer reads", () => {
|
||||||
|
// `parseImageSize` looks for these names, and resolves the path from the URL
|
||||||
|
// before the query, so the hints never reach the file name.
|
||||||
|
expect(imageSource({ path: "/tmp/snip.png", width: 1600, height: 400 })).toBe(
|
||||||
|
"file:///tmp/snip.png?raycast-width=400&raycast-height=100",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("omits the hints when the dimensions are unknown", () => {
|
||||||
|
expect(imageSource({ path: "/tmp/snip.png", width: undefined, height: undefined })).toBe(
|
||||||
|
"file:///tmp/snip.png",
|
||||||
|
);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -78,6 +149,28 @@ describe("readAttachment", () => {
|
|||||||
expect(Buffer.from(attachment.base64, "base64")).toEqual(bytes);
|
expect(Buffer.from(attachment.base64, "base64")).toEqual(bytes);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("keeps the dimensions, which the transcript needs long after this returns", async () => {
|
||||||
|
const dir = await tempDir();
|
||||||
|
const path = join(dir, "snip.png");
|
||||||
|
await writeFile(path, pngHeader(1600, 400));
|
||||||
|
|
||||||
|
const attachment = await readAttachment(path);
|
||||||
|
|
||||||
|
expect(attachment.width).toBe(1600);
|
||||||
|
expect(attachment.height).toBe(400);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("omits the dimensions for an image it cannot measure", async () => {
|
||||||
|
const dir = await tempDir();
|
||||||
|
const path = join(dir, "snip.png");
|
||||||
|
await writeFile(path, Buffer.from([0x89, 0x50, 0x4e, 0x47]));
|
||||||
|
|
||||||
|
const attachment = await readAttachment(path);
|
||||||
|
|
||||||
|
expect(attachment.width).toBeUndefined();
|
||||||
|
expect(attachment.height).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
it("fails loudly when the file is gone", async () => {
|
it("fails loudly when the file is gone", async () => {
|
||||||
await expect(readAttachment("/nonexistent/snip.png")).rejects.toThrow();
|
await expect(readAttachment("/nonexistent/snip.png")).rejects.toThrow();
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -298,7 +298,7 @@ describe("attachments in the transcript", () => {
|
|||||||
message({ role: "user", text: "what is this?", attachments: [screenshot] }),
|
message({ role: "user", text: "what is this?", attachments: [screenshot] }),
|
||||||
]);
|
]);
|
||||||
|
|
||||||
expect(markdown).toContain("");
|
expect(markdown).toContain("");
|
||||||
});
|
});
|
||||||
|
|
||||||
it("keeps the question above its image", () => {
|
it("keeps the question above its image", () => {
|
||||||
|
|||||||
+92
-8
@@ -1,15 +1,14 @@
|
|||||||
import { readFile } from "node:fs/promises";
|
import { readFile } from "node:fs/promises";
|
||||||
import { extname } from "node:path";
|
import { extname } from "node:path";
|
||||||
import { pathToFileURL } from "node:url";
|
import { pathToFileURL } from "node:url";
|
||||||
import type { PendingAttachment } from "./types";
|
import type { Attachment, PendingAttachment } from "./types";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Reading attachments off disk.
|
* Reading attachments off disk, and describing them to the renderer.
|
||||||
*
|
*
|
||||||
* Kept separate from `capture.ts` because the two answer different questions:
|
* Kept separate from `capture.ts` because the two answer different questions:
|
||||||
* capture is about getting an image *onto* the disk, this is about getting it
|
* capture is about getting an image *onto* the disk, this is about getting it
|
||||||
* back off again. It also means a future file picker or clipboard-file path can
|
* back off again and telling Vicinae how to draw it.
|
||||||
* reuse this without touching the capture backends.
|
|
||||||
*
|
*
|
||||||
* Imports no Vicinae module, so it unit tests with no runtime.
|
* Imports no Vicinae module, so it unit tests with no runtime.
|
||||||
*/
|
*/
|
||||||
@@ -23,10 +22,67 @@ const MIME_TYPES: Record<string, string> = {
|
|||||||
".gif": "image/gif",
|
".gif": "image/gif",
|
||||||
};
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The box a preview is fitted into.
|
||||||
|
*
|
||||||
|
* The height is not a preference: `MdImage.qml` clamps rendered height to 200px
|
||||||
|
* whatever is asked for (`Math.min(_rawH, 200, maxImageHeight)`), so a taller
|
||||||
|
* request is silently ignored. The width is here to keep a very wide capture
|
||||||
|
* from being scaled up past the detail pane it sits in.
|
||||||
|
*/
|
||||||
|
const PREVIEW_MAX_WIDTH = 400;
|
||||||
|
const PREVIEW_MAX_HEIGHT = 200;
|
||||||
|
|
||||||
export function mimeForPath(path: string): string {
|
export function mimeForPath(path: string): string {
|
||||||
return MIME_TYPES[extname(path).toLowerCase()] ?? "application/octet-stream";
|
return MIME_TYPES[extname(path).toLowerCase()] ?? "application/octet-stream";
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** The eight bytes every PNG starts with. */
|
||||||
|
const PNG_SIGNATURE = [0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Pixel dimensions from a PNG's `IHDR` chunk, or `undefined` for anything else.
|
||||||
|
*
|
||||||
|
* Only PNG is handled, because that is the only thing a snip produces and
|
||||||
|
* guessing at other formats would be a lie. The header is 24 bytes, so this
|
||||||
|
* costs nothing on top of the read that is already happening.
|
||||||
|
*/
|
||||||
|
export function pngSize(bytes: Buffer): { width: number; height: number } | undefined {
|
||||||
|
if (bytes.length < 24) return undefined;
|
||||||
|
if (!PNG_SIGNATURE.every((byte, index) => bytes[index] === byte)) return undefined;
|
||||||
|
if (bytes.toString("ascii", 12, 16) !== "IHDR") return undefined;
|
||||||
|
|
||||||
|
const width = bytes.readUInt32BE(16);
|
||||||
|
const height = bytes.readUInt32BE(20);
|
||||||
|
|
||||||
|
return width > 0 && height > 0 ? { width, height } : undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The exact box to ask the renderer for, or `undefined` when the size is unknown.
|
||||||
|
*
|
||||||
|
* This is the whole point of reading the header. With no hint, `MdImage.qml`
|
||||||
|
* sizes its container to the pane width and 200px tall and then aspect-fits the
|
||||||
|
* image inside it, so a wide screenshot sits in a letterbox with dead space
|
||||||
|
* above and below it. Asking for a box that already matches the image's aspect
|
||||||
|
* ratio leaves none.
|
||||||
|
*
|
||||||
|
* The scale is capped at 1 so a small capture is not blown up into a blurry one.
|
||||||
|
*/
|
||||||
|
export function previewBox(
|
||||||
|
width: number | undefined,
|
||||||
|
height: number | undefined,
|
||||||
|
): { width: number; height: number } | undefined {
|
||||||
|
if (!width || !height || width <= 0 || height <= 0) return undefined;
|
||||||
|
|
||||||
|
const scale = Math.min(1, PREVIEW_MAX_WIDTH / width, PREVIEW_MAX_HEIGHT / height);
|
||||||
|
|
||||||
|
return {
|
||||||
|
width: Math.max(1, Math.round(width * scale)),
|
||||||
|
height: Math.max(1, Math.round(height * scale)),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* A markdown image source for a local file.
|
* A markdown image source for a local file.
|
||||||
*
|
*
|
||||||
@@ -36,18 +92,46 @@ export function mimeForPath(path: string): string {
|
|||||||
* `ImageURL::local`, and anything it does not recognise it returns as empty.
|
* `ImageURL::local`, and anything it does not recognise it returns as empty.
|
||||||
* `pathToFileURL` does the escaping, so a capture under a directory with a space
|
* `pathToFileURL` does the escaping, so a capture under a directory with a space
|
||||||
* in its name still resolves.
|
* in its name still resolves.
|
||||||
|
*
|
||||||
|
* The query is read from the URL before it is resolved to a path, so the size
|
||||||
|
* hints travel here without ever reaching the file name. They are spelled the
|
||||||
|
* way Raycast spelled them, which is what the parser looks for.
|
||||||
*/
|
*/
|
||||||
export function imageSource(path: string): string {
|
export function imageSource(attachment: Pick<Attachment, "path" | "width" | "height">): string {
|
||||||
return pathToFileURL(path).href;
|
const url = pathToFileURL(attachment.path);
|
||||||
|
const box = previewBox(attachment.width, attachment.height);
|
||||||
|
|
||||||
|
if (box) {
|
||||||
|
url.searchParams.set("raycast-width", String(box.width));
|
||||||
|
url.searchParams.set("raycast-height", String(box.height));
|
||||||
|
}
|
||||||
|
|
||||||
|
return url.href;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Reads an image into the shape a request needs.
|
* Reads an image into the shape a request needs.
|
||||||
*
|
*
|
||||||
* Ollama wants base64 with no `data:` URL prefix, which is why the payload is
|
* Ollama wants base64 with no `data:` URL prefix, which is why the payload is
|
||||||
* built here rather than handed straight to the renderer.
|
* built here rather than handed straight to the renderer. The dimensions are
|
||||||
|
* kept for the same reason the path is: the transcript needs them long after
|
||||||
|
* this call is over, and the preview layout depends on them.
|
||||||
*/
|
*/
|
||||||
export async function readAttachment(path: string): Promise<PendingAttachment> {
|
export async function readAttachment(path: string): Promise<PendingAttachment> {
|
||||||
const bytes = await readFile(path);
|
const bytes = await readFile(path);
|
||||||
return { path, mime: mimeForPath(path), base64: bytes.toString("base64") };
|
const size = pngSize(bytes);
|
||||||
|
|
||||||
|
return {
|
||||||
|
path,
|
||||||
|
mime: mimeForPath(path),
|
||||||
|
base64: bytes.toString("base64"),
|
||||||
|
width: size?.width,
|
||||||
|
height: size?.height,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The persistable half of a pending attachment: everything except the base64 payload. */
|
||||||
|
export function toStoredAttachment(attachment: PendingAttachment): Attachment {
|
||||||
|
const { path, mime, width, height } = attachment;
|
||||||
|
return { path, mime, width, height };
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -190,10 +190,14 @@ export function ChatView({
|
|||||||
pending ? `${renderAttachments([pending])}\n\n${PENDING_NOTE}` : undefined,
|
pending ? `${renderAttachments([pending])}\n\n${PENDING_NOTE}` : undefined,
|
||||||
].filter((note): note is string => Boolean(note));
|
].filter((note): note is string => Boolean(note));
|
||||||
|
|
||||||
|
// The placeholder is what tells a user that the search bar is the composer.
|
||||||
|
// When a note above already says that, printing both put the same
|
||||||
|
// instruction twice, one line under the other.
|
||||||
|
const empty = notes.length === 0 ? EMPTY_TRANSCRIPT : "";
|
||||||
const transcript =
|
const transcript =
|
||||||
thread.length === 0 ? EMPTY_TRANSCRIPT : buildThreadMarkdown(thread, transcriptOptions);
|
thread.length === 0 ? empty : buildThreadMarkdown(thread, transcriptOptions);
|
||||||
|
|
||||||
return [...notes, transcript].join("\n\n");
|
return [...notes, transcript].filter((part) => part.length > 0).join("\n\n");
|
||||||
}, [launchNote, snipNote, pending, thread, transcriptOptions]);
|
}, [launchNote, snipNote, pending, thread, transcriptOptions]);
|
||||||
|
|
||||||
const rows: SessionRow[] = useMemo(() => {
|
const rows: SessionRow[] = useMemo(() => {
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
import { useCallback, useEffect, useRef, useState } from "react";
|
import { useCallback, useEffect, useRef, useState } from "react";
|
||||||
import { runAgent } from "../agent";
|
import { runAgent } from "../agent";
|
||||||
|
import { toStoredAttachment } from "../attachments";
|
||||||
import { buildMessages, composeSystemPrompt } from "../history";
|
import { buildMessages, composeSystemPrompt } from "../history";
|
||||||
import { OllamaError } from "../ollama";
|
import { OllamaError } from "../ollama";
|
||||||
import type { ResolvedPreferences } from "../preferences";
|
import type { ResolvedPreferences } from "../preferences";
|
||||||
@@ -126,9 +127,11 @@ export function useChat(options: UseChatOptions): UseChat {
|
|||||||
const preferences = preferencesRef.current;
|
const preferences = preferencesRef.current;
|
||||||
const base = sessionRef.current;
|
const base = sessionRef.current;
|
||||||
|
|
||||||
// Only the path and mime type are kept on the turn. The base64 payload is
|
// Only the path, mime type and dimensions are kept on the turn. The base64
|
||||||
// used for this one request and then dropped, so it never reaches storage.
|
// payload is used for this one request and then dropped, so it never
|
||||||
const attachments = attachment ? [{ path: attachment.path, mime: attachment.mime }] : [];
|
// reaches storage. The dimensions have to survive because the transcript
|
||||||
|
// lays the preview out long after the payload is gone.
|
||||||
|
const attachments = attachment ? [toStoredAttachment(attachment)] : [];
|
||||||
|
|
||||||
// The turn is inserted immediately with an empty response so the UI shows
|
// The turn is inserted immediately with an empty response so the UI shows
|
||||||
// the question while the answer streams in beneath it.
|
// the question while the answer streams in beneath it.
|
||||||
|
|||||||
+5
-5
@@ -32,13 +32,13 @@ export const STREAMING_CURSOR = "▍";
|
|||||||
* than something readable, which is why the action panel also offers Open
|
* than something readable, which is why the action panel also offers Open
|
||||||
* Screenshot for the full image.
|
* Screenshot for the full image.
|
||||||
*
|
*
|
||||||
* The alt text is not decoration: it is what remains if the file has since been
|
* The alt text is short on purpose. Whatever is here is drawn centred under the
|
||||||
* deleted, so it says what the picture was rather than leaving a blank space.
|
* image, so a sentence reads as a stray line of prose floating in the middle of
|
||||||
|
* the pane. It is also the only thing left if the file has since been deleted,
|
||||||
|
* which is why it is not empty.
|
||||||
*/
|
*/
|
||||||
export function renderAttachments(attachments: Attachment[]): string {
|
export function renderAttachments(attachments: Attachment[]): string {
|
||||||
return attachments
|
return attachments.map((attachment) => `})`).join("\n\n");
|
||||||
.map((attachment) => `})`)
|
|
||||||
.join("\n\n");
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Which optional sections the transcript includes. Driven by preferences. */
|
/** Which optional sections the transcript includes. Driven by preferences. */
|
||||||
|
|||||||
@@ -17,6 +17,10 @@ export const ToolCallRecordSchema = z.object({
|
|||||||
export const AttachmentSchema = z.object({
|
export const AttachmentSchema = z.object({
|
||||||
path: z.string(),
|
path: z.string(),
|
||||||
mime: z.string().default("image/png"),
|
mime: z.string().default("image/png"),
|
||||||
|
// Absent on sessions written before the header was read, which is fine: a
|
||||||
|
// preview without dimensions falls back to the renderer's own sizing.
|
||||||
|
width: z.number().optional(),
|
||||||
|
height: z.number().optional(),
|
||||||
});
|
});
|
||||||
|
|
||||||
export const ChatTurnSchema = z.object({
|
export const ChatTurnSchema = z.object({
|
||||||
|
|||||||
+10
-1
@@ -60,6 +60,16 @@ export interface Attachment {
|
|||||||
path: string;
|
path: string;
|
||||||
/** Sent to Ollama next to the base64 payload. */
|
/** Sent to Ollama next to the base64 payload. */
|
||||||
mime: string;
|
mime: string;
|
||||||
|
/**
|
||||||
|
* Pixel dimensions, read from the file's header.
|
||||||
|
*
|
||||||
|
* Kept because the transcript has to lay the preview out long after the file
|
||||||
|
* was read, and Vicinae sizes an image's container to the request rather than
|
||||||
|
* to the image, which leaves dead space unless the request has the right
|
||||||
|
* aspect ratio. Absent for a format whose header we do not parse.
|
||||||
|
*/
|
||||||
|
width?: number;
|
||||||
|
height?: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -74,7 +84,6 @@ export interface PendingAttachment extends Attachment {
|
|||||||
/** Base64 of the image, with no `data:` URL prefix. Ollama wants it bare. */
|
/** Base64 of the image, with no `data:` URL prefix. Ollama wants it bare. */
|
||||||
base64: string;
|
base64: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** What a tool actually returned during a turn, kept for display and debugging. */
|
/** What a tool actually returned during a turn, kept for display and debugging. */
|
||||||
export interface ToolCallRecord {
|
export interface ToolCallRecord {
|
||||||
name: string;
|
name: string;
|
||||||
|
|||||||
Reference in New Issue
Block a user