Two changes: screenshot input gated on the model really reporting vision, and a fix for URLs that arrived as code. Links - A fence whose body is all URLs, a lone `https://...` inline span, and a line that is only a URL are rewritten into [url](url) markdown links. Vicinae attaches only the table and strikethrough GFM extensions, so autolink is off and a bare URL is dead text; clickability comes from MarkdownModel turning CMARK_NODE_LINK into an <a href>. A fenced URL was a dark panel with a Copy button and no way to open it. - promoteStandaloneCodeSpans was itself promoting a lone URL span into a fence, so it made this worse. Link restoration now runs first. - The default prompt says never to fence a URL. - Anything that is not one of those three shapes is copied through, so a curl invocation containing an address is never rewritten. Screenshots - capture.ts shells out to an auto-detected tool: grimblast, grim + slurp, spectacle, maim, scrot, flameshot, or screencapture on macOS. There is no screenshot call in @vicinae/api and Clipboard moves no binary, so the extension owns the capture and reads the file it wrote rather than guessing at the clipboard. Cancel is detected by whether an image landed on disk, because the tools disagree on exit codes. - In chat: "Snip a Region" in the action panel (cmd+shift+S) attaches to the current conversation. Standalone: "Snip to Lam" starts a new one. - vision.ts caches the /api/show capability answer per host and model. A failed check is "unknown", never "no", so a dropped connection is not reported as a fact about the user's model. Actions still fail closed. - Only the path is stored, base64 is read at send time, and the image goes on its own turn only. Capture is never a model-callable tool. Verified: 244 tests, tsc, eslint, and vici build (both entrypoints).
148 lines
4.2 KiB
TypeScript
148 lines
4.2 KiB
TypeScript
/**
|
|
* Shared domain types for Lam Agent.
|
|
*
|
|
* These are deliberately split into two families:
|
|
*
|
|
* - `ApiMessage` and friends mirror exactly what Ollama's `/api/chat` accepts.
|
|
* - `Session`, `ChatTurn` and friends are Lam's own storage model.
|
|
*
|
|
* Keeping them separate matters because the storage model is derived from the
|
|
* API model, not identical to it: the tool-calling messages that Ollama needs
|
|
* mid-turn are not persisted, so a reloaded session replays cleanly as plain
|
|
* user/assistant pairs.
|
|
*/
|
|
|
|
export type ChatRole = "system" | "user" | "assistant" | "tool";
|
|
|
|
export interface ToolCall {
|
|
function: {
|
|
name: string;
|
|
arguments: Record<string, unknown>;
|
|
};
|
|
}
|
|
|
|
export interface ToolDefinition {
|
|
type: "function";
|
|
function: {
|
|
name: string;
|
|
description: string;
|
|
parameters: Record<string, unknown>;
|
|
};
|
|
}
|
|
|
|
/** A message in the shape Ollama's `/api/chat` endpoint expects. */
|
|
export interface ApiMessage {
|
|
role: ChatRole;
|
|
content: string;
|
|
tool_calls?: ToolCall[];
|
|
/** Only set on `role: "tool"` messages, naming the tool that produced this content. */
|
|
tool_name?: string;
|
|
/**
|
|
* Base64 image data, for vision models.
|
|
*
|
|
* Ollama only reads these on a `user` message (`gpt-oss` style tool roles have
|
|
* no images). They are the one genuinely large thing in a request, so they are
|
|
* attached to the question they belong to and never replayed later, the same
|
|
* way tool output is not replayed.
|
|
*/
|
|
images?: string[];
|
|
}
|
|
|
|
/**
|
|
* A file the user attached to a question.
|
|
*
|
|
* Only the path is stored. A screenshot runs to megabytes, and
|
|
* `MAX_STORED_CHARS_PER_TURN` exists precisely so one turn cannot dominate
|
|
* LocalStorage, so the bytes stay on disk and are read when the turn is sent.
|
|
*/
|
|
export interface Attachment {
|
|
/** Absolute path to the image on disk. */
|
|
path: string;
|
|
/** Sent to Ollama next to the base64 payload. */
|
|
mime: string;
|
|
}
|
|
|
|
/**
|
|
* An attachment that has been read and is ready to send, held in the view while
|
|
* the user types their question.
|
|
*
|
|
* The base64 payload lives here and nowhere else. It is needed for exactly one
|
|
* request, so it is deliberately not part of `Attachment` and never reaches
|
|
* storage.
|
|
*/
|
|
export interface PendingAttachment extends Attachment {
|
|
/** Base64 of the image, with no `data:` URL prefix. Ollama wants it bare. */
|
|
base64: string;
|
|
}
|
|
|
|
/** What a tool actually returned during a turn, kept for display and debugging. */
|
|
export interface ToolCallRecord {
|
|
name: string;
|
|
arguments: Record<string, unknown>;
|
|
result: string;
|
|
isError: boolean;
|
|
}
|
|
|
|
/** One question and its answer. The unit the transcript is rendered from. */
|
|
export interface ChatTurn {
|
|
id: string;
|
|
prompt: string;
|
|
response: string;
|
|
thinking: string;
|
|
model: string;
|
|
createdAt: number;
|
|
/** Set when the turn failed, so a stored session can show what went wrong. */
|
|
error?: string;
|
|
toolCalls: ToolCallRecord[];
|
|
/** Images the user attached, kept for the transcript. The bytes are not stored. */
|
|
attachments: Attachment[];
|
|
}
|
|
|
|
export interface Session {
|
|
id: string;
|
|
title: string;
|
|
model: string;
|
|
createdAt: number;
|
|
updatedAt: number;
|
|
turns: ChatTurn[];
|
|
}
|
|
|
|
/** The lightweight record kept in the session index, so listing sessions never loads full transcripts. */
|
|
export interface SessionSummary {
|
|
id: string;
|
|
title: string;
|
|
model: string;
|
|
createdAt: number;
|
|
updatedAt: number;
|
|
turnCount: number;
|
|
preview: string;
|
|
}
|
|
|
|
/** A model as reported by `GET /api/tags`. */
|
|
export interface ModelInfo {
|
|
name: string;
|
|
parameterSize?: string;
|
|
}
|
|
|
|
/**
|
|
* One rendered line of the conversation.
|
|
*
|
|
* A turn is stored as a question/answer pair, but the transcript is displayed as
|
|
* a flat sequence of messages so each side can carry its own icon and weight.
|
|
*/
|
|
export interface ThreadMessage {
|
|
id: string;
|
|
role: "user" | "assistant";
|
|
/** Full text, used for the detail panel and for copying. */
|
|
text: string;
|
|
createdAt: number;
|
|
isStreaming: boolean;
|
|
thinking: string;
|
|
toolCalls: ToolCallRecord[];
|
|
error?: string;
|
|
/** Activity line shown while this message is streaming, e.g. "Searching the web". */
|
|
status?: string;
|
|
/** Images attached to this question. Only ever non-empty on a user message. */
|
|
attachments: Attachment[];
|
|
}
|