Add llama.cpp backend, model gateway, URL relay and browser view

Four features, plus a latent bug fix. llama.cpp backend. Claude Code only ever speaks the Anthropic Messages API — confirmed empirically by pointing it at a logging server, which received POST /v1/messages?beta=true. llama-server implements that natively (verified in its README, alongside --port default 8080), so this is a plain base-URL backend with no translation shim, the same shape as Ollama. Its --api-key defaults to none, so the auth token is a placeholder Claude Code requires and llama-server ignores. Model alias fix. ANTHROPIC_DEFAULT_HAIKU_MODEL is documented as "also used for background functionality", and Triple-C set none of the alias vars. So on every custom-endpoint backend, Claude Code resolved `haiku` to an Anthropic model id and sent it to a local server that does not have it — background features failed silently. All four ANTHROPIC_DEFAULT_{OPUS,SONNET,HAIKU,FABLE}_MODEL vars are now pinned to the backend's configured model, with an optional Haiku override, and blanked for Anthropic and Bedrock so those keep Claude Code's defaults. The deprecated ANTHROPIC_SMALL_FAST_MODEL is never emitted. Existing Ollama and OpenAI-Compatible containers are recreated once so the new env reaches them; the snapshot is preserved. Model gateway. Optional LiteLLM sibling container, off by default, mirroring stt.rs — this is what makes real OpenAI usable, since api.openai.com has no /v1/messages. Pinned to v1.96.0 by tag and digest: the 1.82.7/1.82.8 malware was PyPI-only and never affected the official images, which is precisely why this builds FROM the image rather than pip-installing, but 1.84.0 is still the floor for proxy CVEs (API-key SQLi, Host-header auth bypass, MCP auth bypass). Binds 0.0.0.0 because project containers consume it, and therefore always sets a master_key — LiteLLM without one accepts any key. The provider key lives in the OS keychain and is uploaded into a volume, never an image layer or label. URL relay. A container-side xdg-open/BROWSER shim opens URLs in the host's browser. Uses an OSC sequence to /dev/tty rather than a printed sentinel, because the shim usually runs as a grandchild of a process capturing its children's output. Degrades to printing the URL when no terminal is attached, so scheduled tasks do not hang. Only http/https, with control characters rejected before new URL() — which strips newlines, so java\nscript: would otherwise parse as javascript:. Nothing auto-opens; the user confirms. The web terminal shows a tap-to-open banner instead, since that browser may be a phone across a tunnel. Browser view. A Project Home tab that watches and takes over the browser Claude drives with Playwright, using Playwright's own dashboard. Zero image cost — Playwright stays user-installed. It does not reuse the auth bridge's PortForward, which binds an unauthenticated port: correct for a throwaway OAuth listener, wrong for mouse and keyboard control of a browser in a passwordless-sudo container. Instead a token-gated loopback proxy checks Host, then token or a forbidden-header origin signal, before a byte reaches the container. Host ports are confined to 47820..=47827 so CSP frame-src can enumerate them rather than widening to a wildcard, with a test asserting the two agree. 188 frontend tests, 107 Rust tests, both builds clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 16:55:28 -07:00
parent 7d00390e1f
commit cc5f691677
46 changed files with 6194 additions and 61 deletions
@@ -1,6 +1,10 @@
 import { describe, it, expect, vi, beforeEach } from "vitest";
 import { render, screen, fireEvent } from "@testing-library/react";
-import ModelSection from "./ModelSection";
+import ModelSection, {
+  DEFAULT_LLAMACPP_CONFIG,
+  DEFAULT_OLLAMA_CONFIG,
+} from "./ModelSection";
+import { CUSTOM_ENDPOINT_BACKENDS } from "../../../../lib/types";
 import type { Backend, Project } from "../../../../lib/types";

 const baseProject: Project = {
@@ -12,6 +16,7 @@ const baseProject: Project = {
  backend: "anthropic",
  bedrock_config: null,
  ollama_config: null,
+  llamacpp_config: null,
  openai_compatible_config: null,
  allow_docker_access: false,
  sandbox_mode_enabled: true,
@@ -55,7 +60,7 @@ describe("ModelSection — shared auth token toggle", () => {
    expect(screen.getByRole("switch", { name: TOGGLE })).toBeInTheDocument();
  });

-  it.each<Backend>(["bedrock", "ollama", "open_ai_compatible"])(
+  it.each<Backend>(["bedrock", "ollama", "llama_cpp", "open_ai_compatible"])(
    "is hidden for the %s backend",
    (backend) => {
      renderSection({ backend });
@@ -93,3 +98,118 @@ describe("ModelSection — shared auth token toggle", () => {
    expect(screen.getByRole("switch", { name: TOGGLE })).toBeDisabled();
  });
 });
+
+describe("ModelSection — llama.cpp backend", () => {
+  beforeEach(() => vi.clearAllMocks());
+
+  it("is offered as a backend choice", () => {
+    renderSection();
+    expect(
+      screen.getByRole("option", { name: "llama.cpp" }),
+    ).toBeInTheDocument();
+  });
+
+  it("seeds llama-server's default port when the backend is first chosen", () => {
+    renderSection();
+    fireEvent.change(screen.getByLabelText("Backend"), {
+      target: { value: "llama_cpp" },
+    });
+    expect(save).toHaveBeenCalledWith({
+      backend: "llama_cpp",
+      llamacpp_config: DEFAULT_LLAMACPP_CONFIG,
+    });
+    expect(DEFAULT_LLAMACPP_CONFIG.base_url).toContain(":8080");
+  });
+
+  it("does not clobber an existing config when re-selected", () => {
+    renderSection({
+      backend: "ollama",
+      llamacpp_config: {
+        base_url: "http://gpu-box:9090",
+        model_id: "mine",
+        haiku_model_id: null,
+      },
+    });
+    fireEvent.change(screen.getByLabelText("Backend"), {
+      target: { value: "llama_cpp" },
+    });
+    expect(save).toHaveBeenCalledWith({ backend: "llama_cpp" });
+  });
+
+  it("saves the base URL and model on blur", () => {
+    renderSection({ backend: "llama_cpp" });
+
+    const url = screen.getByLabelText("Base URL");
+    fireEvent.change(url, { target: { value: "http://gpu-box:8080" } });
+    fireEvent.blur(url);
+    expect(save).toHaveBeenCalledWith({
+      llamacpp_config: { ...DEFAULT_LLAMACPP_CONFIG, base_url: "http://gpu-box:8080" },
+    });
+
+    const model = screen.getByLabelText("Model");
+    fireEvent.change(model, { target: { value: "qwen3.5-coder-30b" } });
+    fireEvent.blur(model);
+    expect(save).toHaveBeenCalledWith({
+      llamacpp_config: { ...DEFAULT_LLAMACPP_CONFIG, model_id: "qwen3.5-coder-30b" },
+    });
+  });
+});
+
+describe("ModelSection — background (haiku) model override", () => {
+  beforeEach(() => vi.clearAllMocks());
+
+  it("covers exactly the backends that point at a custom endpoint", () => {
+    expect([...CUSTOM_ENDPOINT_BACKENDS]).toEqual([
+      "ollama",
+      "llama_cpp",
+      "open_ai_compatible",
+    ]);
+  });
+
+  it.each([...CUSTOM_ENDPOINT_BACKENDS])("is offered for the %s backend", (backend) => {
+    renderSection({ backend });
+    const field = screen.getByLabelText("Background model");
+    expect(field).toBeInTheDocument();
+    // Blank is the documented default — it reuses the main model.
+    expect(field).toHaveValue("");
+    expect(field).toHaveAttribute("placeholder", "(same as the model above)");
+  });
+
+  it.each<Backend>(["anthropic", "bedrock"])(
+    "is not offered for the %s backend, which keeps Claude Code's defaults",
+    (backend) => {
+      renderSection({ backend });
+      expect(screen.queryByLabelText("Background model")).not.toBeInTheDocument();
+    },
+  );
+
+  it("saves a trimmed override, and clears it back to null when blanked", () => {
+    renderSection({ backend: "ollama" });
+    const field = screen.getByLabelText("Background model");
+
+    fireEvent.change(field, { target: { value: "  qwen3.5:3b  " } });
+    fireEvent.blur(field);
+    expect(save).toHaveBeenCalledWith({
+      ollama_config: { ...DEFAULT_OLLAMA_CONFIG, haiku_model_id: "qwen3.5:3b" },
+    });
+
+    fireEvent.change(field, { target: { value: "   " } });
+    fireEvent.blur(field);
+    expect(save).toHaveBeenCalledWith({
+      ollama_config: { ...DEFAULT_OLLAMA_CONFIG, haiku_model_id: null },
+    });
+  });
+
+  it("shows an existing override and explains what it is for", () => {
+    renderSection({
+      backend: "llama_cpp",
+      llamacpp_config: {
+        base_url: "http://host.docker.internal:8080",
+        model_id: "big",
+        haiku_model_id: "small",
+      },
+    });
+    expect(screen.getByLabelText("Background model")).toHaveValue("small");
+    expect(screen.getByText(/background work/i)).toBeInTheDocument();
+  });
+});