CyberCoveLLC/Triple-C
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 35 Wiki Activity

Add llama.cpp backend, model gateway, URL relay and browser view

Browse Source 
Four features, plus a latent bug fix.

llama.cpp backend. Claude Code only ever speaks the Anthropic Messages
API — confirmed empirically by pointing it at a logging server, which
received POST /v1/messages?beta=true. llama-server implements that
natively (verified in its README, alongside --port default 8080), so
this is a plain base-URL backend with no translation shim, the same
shape as Ollama. Its --api-key defaults to none, so the auth token is a
placeholder Claude Code requires and llama-server ignores.

Model alias fix. ANTHROPIC_DEFAULT_HAIKU_MODEL is documented as "also
used for background functionality", and Triple-C set none of the alias
vars. So on every custom-endpoint backend, Claude Code resolved `haiku`
to an Anthropic model id and sent it to a local server that does not
have it — background features failed silently. All four
ANTHROPIC_DEFAULT_{OPUS,SONNET,HAIKU,FABLE}_MODEL vars are now pinned to
the backend's configured model, with an optional Haiku override, and
blanked for Anthropic and Bedrock so those keep Claude Code's defaults.
The deprecated ANTHROPIC_SMALL_FAST_MODEL is never emitted. Existing
Ollama and OpenAI-Compatible containers are recreated once so the new
env reaches them; the snapshot is preserved.

Model gateway. Optional LiteLLM sibling container, off by default,
mirroring stt.rs — this is what makes real OpenAI usable, since
api.openai.com has no /v1/messages. Pinned to v1.96.0 by tag and digest:
the 1.82.7/1.82.8 malware was PyPI-only and never affected the official
images, which is precisely why this builds FROM the image rather than
pip-installing, but 1.84.0 is still the floor for proxy CVEs (API-key
SQLi, Host-header auth bypass, MCP auth bypass). Binds 0.0.0.0 because
project containers consume it, and therefore always sets a master_key —
LiteLLM without one accepts any key. The provider key lives in the OS
keychain and is uploaded into a volume, never an image layer or label.

URL relay. A container-side xdg-open/BROWSER shim opens URLs in the
host's browser. Uses an OSC sequence to /dev/tty rather than a printed
sentinel, because the shim usually runs as a grandchild of a process
capturing its children's output. Degrades to printing the URL when no
terminal is attached, so scheduled tasks do not hang. Only http/https,
with control characters rejected before new URL() — which strips
newlines, so java\nscript: would otherwise parse as javascript:. Nothing
auto-opens; the user confirms. The web terminal shows a tap-to-open
banner instead, since that browser may be a phone across a tunnel.

Browser view. A Project Home tab that watches and takes over the browser
Claude drives with Playwright, using Playwright's own dashboard. Zero
image cost — Playwright stays user-installed. It does not reuse the auth
bridge's PortForward, which binds an unauthenticated port: correct for a
throwaway OAuth listener, wrong for mouse and keyboard control of a
browser in a passwordless-sudo container. Instead a token-gated loopback
proxy checks Host, then token or a forbidden-header origin signal,
before a byte reaches the container. Host ports are confined to
47820..=47827 so CSP frame-src can enumerate them rather than widening
to a wildcard, with a test asserting the two agree.

188 frontend tests, 107 Rust tests, both builds clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
shadow-test
2026-08-09 16:55:28 -07:00
co-authored by Claude Opus 5
parent 7d00390e1f
commit cc5f691677
46 changed files with 6194 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/src/components/projects/home/config/ModelSection.test.tsx
+122 -2
View File
@@ -1,6 +1,10 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import { render, screen, fireEvent } from "@testing-library/react";
import ModelSection from "./ModelSection";
import ModelSection, {
DEFAULT_LLAMACPP_CONFIG,
DEFAULT_OLLAMA_CONFIG,
} from "./ModelSection";
import { CUSTOM_ENDPOINT_BACKENDS } from "../../../../lib/types";
import type { Backend, Project } from "../../../../lib/types";
const baseProject: Project = {
@@ -12,6 +16,7 @@ const baseProject: Project = {
backend: "anthropic",
bedrock_config: null,
ollama_config: null,
llamacpp_config: null,
openai_compatible_config: null,
allow_docker_access: false,
sandbox_mode_enabled: true,
@@ -55,7 +60,7 @@ describe("ModelSection — shared auth token toggle", () => {
expect(screen.getByRole("switch", { name: TOGGLE })).toBeInTheDocument();
});
it.each<Backend>(["bedrock", "ollama", "open_ai_compatible"])(
it.each<Backend>(["bedrock", "ollama", "llama_cpp", "open_ai_compatible"])(
"is hidden for the %s backend",
(backend) => {
renderSection({ backend });
@@ -93,3 +98,118 @@ describe("ModelSection — shared auth token toggle", () => {
expect(screen.getByRole("switch", { name: TOGGLE })).toBeDisabled();
});
});
describe("ModelSection — llama.cpp backend", () => {
beforeEach(() => vi.clearAllMocks());
it("is offered as a backend choice", () => {
renderSection();
expect(
screen.getByRole("option", { name: "llama.cpp" }),
).toBeInTheDocument();
});
it("seeds llama-server's default port when the backend is first chosen", () => {
renderSection();
fireEvent.change(screen.getByLabelText("Backend"), {
target: { value: "llama_cpp" },
});
expect(save).toHaveBeenCalledWith({
backend: "llama_cpp",
llamacpp_config: DEFAULT_LLAMACPP_CONFIG,
});
expect(DEFAULT_LLAMACPP_CONFIG.base_url).toContain(":8080");
});
it("does not clobber an existing config when re-selected", () => {
renderSection({
backend: "ollama",
llamacpp_config: {
base_url: "http://gpu-box:9090",
model_id: "mine",
haiku_model_id: null,
},
});
fireEvent.change(screen.getByLabelText("Backend"), {
target: { value: "llama_cpp" },
});
expect(save).toHaveBeenCalledWith({ backend: "llama_cpp" });
});
it("saves the base URL and model on blur", () => {
renderSection({ backend: "llama_cpp" });
const url = screen.getByLabelText("Base URL");
fireEvent.change(url, { target: { value: "http://gpu-box:8080" } });
fireEvent.blur(url);
expect(save).toHaveBeenCalledWith({
llamacpp_config: { ...DEFAULT_LLAMACPP_CONFIG, base_url: "http://gpu-box:8080" },
});
const model = screen.getByLabelText("Model");
fireEvent.change(model, { target: { value: "qwen3.5-coder-30b" } });
fireEvent.blur(model);
expect(save).toHaveBeenCalledWith({
llamacpp_config: { ...DEFAULT_LLAMACPP_CONFIG, model_id: "qwen3.5-coder-30b" },
});
});
});
describe("ModelSection — background (haiku) model override", () => {
beforeEach(() => vi.clearAllMocks());
it("covers exactly the backends that point at a custom endpoint", () => {
expect([...CUSTOM_ENDPOINT_BACKENDS]).toEqual([
"ollama",
"llama_cpp",
"open_ai_compatible",
]);
});
it.each([...CUSTOM_ENDPOINT_BACKENDS])("is offered for the %s backend", (backend) => {
renderSection({ backend });
const field = screen.getByLabelText("Background model");
expect(field).toBeInTheDocument();
// Blank is the documented default — it reuses the main model.
expect(field).toHaveValue("");
expect(field).toHaveAttribute("placeholder", "(same as the model above)");
});
it.each<Backend>(["anthropic", "bedrock"])(
"is not offered for the %s backend, which keeps Claude Code's defaults",
(backend) => {
renderSection({ backend });
expect(screen.queryByLabelText("Background model")).not.toBeInTheDocument();
},
);
it("saves a trimmed override, and clears it back to null when blanked", () => {
renderSection({ backend: "ollama" });
const field = screen.getByLabelText("Background model");
fireEvent.change(field, { target: { value: " qwen3.5:3b " } });
fireEvent.blur(field);
expect(save).toHaveBeenCalledWith({
ollama_config: { ...DEFAULT_OLLAMA_CONFIG, haiku_model_id: "qwen3.5:3b" },
});
fireEvent.change(field, { target: { value: " " } });
fireEvent.blur(field);
expect(save).toHaveBeenCalledWith({
ollama_config: { ...DEFAULT_OLLAMA_CONFIG, haiku_model_id: null },
});
});
it("shows an existing override and explains what it is for", () => {
renderSection({
backend: "llama_cpp",
llamacpp_config: {
base_url: "http://host.docker.internal:8080",
model_id: "big",
haiku_model_id: "small",
},
});
expect(screen.getByLabelText("Background model")).toHaveValue("small");
expect(screen.getByText(/background work/i)).toBeInTheDocument();
});
});
app/src/components/projects/home/config/ModelSection.tsx
+133 -2
View File
@@ -3,6 +3,7 @@ import type {
Backend,
BedrockAuthMethod,
BedrockConfig,
LlamaCppConfig,
OllamaConfig,
OpenAiCompatibleConfig,
Project,
@@ -31,14 +32,28 @@ export const DEFAULT_BEDROCK_CONFIG: BedrockConfig = {
export const DEFAULT_OLLAMA_CONFIG: OllamaConfig = {
base_url: "http://host.docker.internal:11434",
model_id: null,
haiku_model_id: null,
};
/** `llama-server` listens on port 8080 unless `--port` says otherwise. */
export const DEFAULT_LLAMACPP_CONFIG: LlamaCppConfig = {
base_url: "http://host.docker.internal:8080",
model_id: null,
haiku_model_id: null,
};
export const DEFAULT_OPENAI_COMPATIBLE_CONFIG: OpenAiCompatibleConfig = {
base_url: "http://host.docker.internal:4000",
api_key: null,
model_id: null,
haiku_model_id: null,
};
/** Shown under the optional per-backend Haiku override. Kept in one place so
* all three custom-endpoint backends explain it identically. */
const HAIKU_HINT =
"Optional. Claude Code resolves the `haiku` alias to this, and uses it for background work such as conversation titles. Leave blank to reuse the model above — that is what stops background calls failing against a server that only serves one model.";
interface Props {
project: Project;
save: (patch: Partial<Project>) => Promise<boolean>;
@@ -64,6 +79,19 @@ export default function ModelSection({ project, save, disabled }: Props) {
const [ollamaModelId, setOllamaModelId] = useState(
project.ollama_config?.model_id ?? "",
);
const [ollamaHaikuModelId, setOllamaHaikuModelId] = useState(
project.ollama_config?.haiku_model_id ?? "",
);
const [llamaCppBaseUrl, setLlamaCppBaseUrl] = useState(
project.llamacpp_config?.base_url ?? DEFAULT_LLAMACPP_CONFIG.base_url,
);
const [llamaCppModelId, setLlamaCppModelId] = useState(
project.llamacpp_config?.model_id ?? "",
);
const [llamaCppHaikuModelId, setLlamaCppHaikuModelId] = useState(
project.llamacpp_config?.haiku_model_id ?? "",
);
const [oaiBaseUrl, setOaiBaseUrl] = useState(
project.openai_compatible_config?.base_url ??
@@ -75,6 +103,9 @@ export default function ModelSection({ project, save, disabled }: Props) {
const [oaiModelId, setOaiModelId] = useState(
project.openai_compatible_config?.model_id ?? "",
);
const [oaiHaikuModelId, setOaiHaikuModelId] = useState(
project.openai_compatible_config?.haiku_model_id ?? "",
);
useEffect(() => {
const bc = project.bedrock_config ?? DEFAULT_BEDROCK_CONFIG;
@@ -88,12 +119,19 @@ export default function ModelSection({ project, save, disabled }: Props) {
setServiceTier(bc.service_tier ?? "");
setOllamaBaseUrl(project.ollama_config?.base_url ?? DEFAULT_OLLAMA_CONFIG.base_url);
setOllamaModelId(project.ollama_config?.model_id ?? "");
setOllamaHaikuModelId(project.ollama_config?.haiku_model_id ?? "");
setLlamaCppBaseUrl(
project.llamacpp_config?.base_url ?? DEFAULT_LLAMACPP_CONFIG.base_url,
);
setLlamaCppModelId(project.llamacpp_config?.model_id ?? "");
setLlamaCppHaikuModelId(project.llamacpp_config?.haiku_model_id ?? "");
setOaiBaseUrl(
project.openai_compatible_config?.base_url ??
DEFAULT_OPENAI_COMPATIBLE_CONFIG.base_url,
);
setOaiApiKey(project.openai_compatible_config?.api_key ?? "");
setOaiModelId(project.openai_compatible_config?.model_id ?? "");
setOaiHaikuModelId(project.openai_compatible_config?.haiku_model_id ?? "");
}, [project]);
const saveBedrock = (patch: Partial<BedrockConfig>) =>
@@ -104,6 +142,14 @@ export default function ModelSection({ project, save, disabled }: Props) {
ollama_config: { ...(project.ollama_config ?? DEFAULT_OLLAMA_CONFIG), ...patch },
});
const saveLlamaCpp = (patch: Partial<LlamaCppConfig>) =>
save({
llamacpp_config: {
...(project.llamacpp_config ?? DEFAULT_LLAMACPP_CONFIG),
...patch,
},
});
const saveOpenAi = (patch: Partial<OpenAiCompatibleConfig>) =>
save({
openai_compatible_config: {
@@ -122,6 +168,8 @@ export default function ModelSection({ project, save, disabled }: Props) {
patch.bedrock_config = DEFAULT_BEDROCK_CONFIG;
if (mode === "ollama" && !project.ollama_config)
patch.ollama_config = DEFAULT_OLLAMA_CONFIG;
if (mode === "llama_cpp" && !project.llamacpp_config)
patch.llamacpp_config = DEFAULT_LLAMACPP_CONFIG;
if (mode === "open_ai_compatible" && !project.openai_compatible_config)
patch.openai_compatible_config = DEFAULT_OPENAI_COMPATIBLE_CONFIG;
save(patch);
@@ -131,7 +179,7 @@ export default function ModelSection({ project, save, disabled }: Props) {
<ConfigGroup title="Model" description="Which provider serves this project's Claude.">
<Field
label="Backend"
hint="Anthropic connects directly via OAuth (run `claude login` in a terminal). Bedrock routes through AWS. Ollama and OpenAI Compatible point at any compatible endpoint."
hint="Anthropic connects directly via OAuth (run `claude login` in a terminal). Bedrock routes through AWS. Ollama, llama.cpp and OpenAI Compatible point at any endpoint that implements the Anthropic Messages API."
>
{(id) => (
<select
@@ -144,6 +192,7 @@ export default function ModelSection({ project, save, disabled }: Props) {
<option value="anthropic">Anthropic</option>
<option value="bedrock">Bedrock</option>
<option value="ollama">Ollama</option>
<option value="llama_cpp">llama.cpp</option>
<option value="open_ai_compatible">OpenAI Compatible</option>
</select>
)}
@@ -365,6 +414,73 @@ export default function ModelSection({ project, save, disabled }: Props) {
/>
)}
</Field>
<Field label="Background model" hint={HAIKU_HINT}>
{(id) => (
<input
id={id}
value={ollamaHaikuModelId}
onChange={(e) => setOllamaHaikuModelId(e.target.value)}
onBlur={() =>
saveOllama({ haiku_model_id: ollamaHaikuModelId.trim() || null })
}
placeholder="(same as the model above)"
disabled={disabled}
className={monoInputClass}
/>
)}
</Field>
</div>
)}
{project.backend === "llama_cpp" && (
<div className="space-y-4 pt-2 border-t border-[var(--border-color)]">
<Field
label="Base URL"
hint="Your llama-server. It listens on port 8080 by default; use host.docker.internal to reach the host machine."
>
{(id) => (
<input
id={id}
value={llamaCppBaseUrl}
onChange={(e) => setLlamaCppBaseUrl(e.target.value)}
onBlur={() => saveLlamaCpp({ base_url: llamaCppBaseUrl })}
placeholder="http://host.docker.internal:8080"
disabled={disabled}
className={monoInputClass}
/>
)}
</Field>
<Field
label="Model"
hint="The model llama-server was started with. llama-server serves one model, so this is mainly what Claude Code reports — but it is also what the model aliases are pinned to."
>
{(id) => (
<input
id={id}
value={llamaCppModelId}
onChange={(e) => setLlamaCppModelId(e.target.value)}
onBlur={() => saveLlamaCpp({ model_id: llamaCppModelId || null })}
placeholder="qwen3.5-coder-30b"
disabled={disabled}
className={monoInputClass}
/>
)}
</Field>
<Field label="Background model" hint={HAIKU_HINT}>
{(id) => (
<input
id={id}
value={llamaCppHaikuModelId}
onChange={(e) => setLlamaCppHaikuModelId(e.target.value)}
onBlur={() =>
saveLlamaCpp({ haiku_model_id: llamaCppHaikuModelId.trim() || null })
}
placeholder="(same as the model above)"
disabled={disabled}
className={monoInputClass}
/>
)}
</Field>
</div>
)}
@@ -372,7 +488,7 @@ export default function ModelSection({ project, save, disabled }: Props) {
<div className="space-y-4 pt-2 border-t border-[var(--border-color)]">
<Field
label="Base URL"
hint="Any OpenAI API-compatible endpoint — LiteLLM, OpenRouter, vLLM, and so on."
hint="A gateway that implements the Anthropic Messages API (POST /v1/messages) — LiteLLM, for example. An endpoint that only speaks OpenAI /v1/chat/completions will not work."
>
{(id) => (
<input
@@ -413,6 +529,21 @@ export default function ModelSection({ project, save, disabled }: Props) {
/>
)}
</Field>
<Field label="Background model" hint={HAIKU_HINT}>
{(id) => (
<input
id={id}
value={oaiHaikuModelId}
onChange={(e) => setOaiHaikuModelId(e.target.value)}
onBlur={() =>
saveOpenAi({ haiku_model_id: oaiHaikuModelId.trim() || null })
}
placeholder="(same as the model above)"
disabled={disabled}
className={monoInputClass}
/>
)}
</Field>
</div>
)}
</ConfigGroup>