feat: snip a region into chat, and make links clickable
Two changes: screenshot input gated on the model really reporting vision, and a fix for URLs that arrived as code. Links - A fence whose body is all URLs, a lone `https://...` inline span, and a line that is only a URL are rewritten into [url](url) markdown links. Vicinae attaches only the table and strikethrough GFM extensions, so autolink is off and a bare URL is dead text; clickability comes from MarkdownModel turning CMARK_NODE_LINK into an <a href>. A fenced URL was a dark panel with a Copy button and no way to open it. - promoteStandaloneCodeSpans was itself promoting a lone URL span into a fence, so it made this worse. Link restoration now runs first. - The default prompt says never to fence a URL. - Anything that is not one of those three shapes is copied through, so a curl invocation containing an address is never rewritten. Screenshots - capture.ts shells out to an auto-detected tool: grimblast, grim + slurp, spectacle, maim, scrot, flameshot, or screencapture on macOS. There is no screenshot call in @vicinae/api and Clipboard moves no binary, so the extension owns the capture and reads the file it wrote rather than guessing at the clipboard. Cancel is detected by whether an image landed on disk, because the tools disagree on exit codes. - In chat: "Snip a Region" in the action panel (cmd+shift+S) attaches to the current conversation. Standalone: "Snip to Lam" starts a new one. - vision.ts caches the /api/show capability answer per host and model. A failed check is "unknown", never "no", so a dropped connection is not reported as a fact about the user's model. Actions still fail closed. - Only the path is stored, base64 is read at send time, and the image goes on its own turn only. Capture is never a model-callable tool. Verified: 244 tests, tsc, eslint, and vici build (both entrypoints).
This commit is contained in:
@@ -37,6 +37,42 @@ export interface ApiMessage {
|
||||
tool_calls?: ToolCall[];
|
||||
/** Only set on `role: "tool"` messages, naming the tool that produced this content. */
|
||||
tool_name?: string;
|
||||
/**
|
||||
* Base64 image data, for vision models.
|
||||
*
|
||||
* Ollama only reads these on a `user` message (`gpt-oss` style tool roles have
|
||||
* no images). They are the one genuinely large thing in a request, so they are
|
||||
* attached to the question they belong to and never replayed later, the same
|
||||
* way tool output is not replayed.
|
||||
*/
|
||||
images?: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* A file the user attached to a question.
|
||||
*
|
||||
* Only the path is stored. A screenshot runs to megabytes, and
|
||||
* `MAX_STORED_CHARS_PER_TURN` exists precisely so one turn cannot dominate
|
||||
* LocalStorage, so the bytes stay on disk and are read when the turn is sent.
|
||||
*/
|
||||
export interface Attachment {
|
||||
/** Absolute path to the image on disk. */
|
||||
path: string;
|
||||
/** Sent to Ollama next to the base64 payload. */
|
||||
mime: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* An attachment that has been read and is ready to send, held in the view while
|
||||
* the user types their question.
|
||||
*
|
||||
* The base64 payload lives here and nowhere else. It is needed for exactly one
|
||||
* request, so it is deliberately not part of `Attachment` and never reaches
|
||||
* storage.
|
||||
*/
|
||||
export interface PendingAttachment extends Attachment {
|
||||
/** Base64 of the image, with no `data:` URL prefix. Ollama wants it bare. */
|
||||
base64: string;
|
||||
}
|
||||
|
||||
/** What a tool actually returned during a turn, kept for display and debugging. */
|
||||
@@ -58,6 +94,8 @@ export interface ChatTurn {
|
||||
/** Set when the turn failed, so a stored session can show what went wrong. */
|
||||
error?: string;
|
||||
toolCalls: ToolCallRecord[];
|
||||
/** Images the user attached, kept for the transcript. The bytes are not stored. */
|
||||
attachments: Attachment[];
|
||||
}
|
||||
|
||||
export interface Session {
|
||||
@@ -104,4 +142,6 @@ export interface ThreadMessage {
|
||||
error?: string;
|
||||
/** Activity line shown while this message is streaming, e.g. "Searching the web". */
|
||||
status?: string;
|
||||
/** Images attached to this question. Only ever non-empty on a user message. */
|
||||
attachments: Attachment[];
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user