CyberCoveLLC/Triple-C
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases 35 Wiki Activity

Add llama.cpp backend, model gateway, URL relay and browser view

Browse Source 
Four features, plus a latent bug fix.

llama.cpp backend. Claude Code only ever speaks the Anthropic Messages
API — confirmed empirically by pointing it at a logging server, which
received POST /v1/messages?beta=true. llama-server implements that
natively (verified in its README, alongside --port default 8080), so
this is a plain base-URL backend with no translation shim, the same
shape as Ollama. Its --api-key defaults to none, so the auth token is a
placeholder Claude Code requires and llama-server ignores.

Model alias fix. ANTHROPIC_DEFAULT_HAIKU_MODEL is documented as "also
used for background functionality", and Triple-C set none of the alias
vars. So on every custom-endpoint backend, Claude Code resolved `haiku`
to an Anthropic model id and sent it to a local server that does not
have it — background features failed silently. All four
ANTHROPIC_DEFAULT_{OPUS,SONNET,HAIKU,FABLE}_MODEL vars are now pinned to
the backend's configured model, with an optional Haiku override, and
blanked for Anthropic and Bedrock so those keep Claude Code's defaults.
The deprecated ANTHROPIC_SMALL_FAST_MODEL is never emitted. Existing
Ollama and OpenAI-Compatible containers are recreated once so the new
env reaches them; the snapshot is preserved.

Model gateway. Optional LiteLLM sibling container, off by default,
mirroring stt.rs — this is what makes real OpenAI usable, since
api.openai.com has no /v1/messages. Pinned to v1.96.0 by tag and digest:
the 1.82.7/1.82.8 malware was PyPI-only and never affected the official
images, which is precisely why this builds FROM the image rather than
pip-installing, but 1.84.0 is still the floor for proxy CVEs (API-key
SQLi, Host-header auth bypass, MCP auth bypass). Binds 0.0.0.0 because
project containers consume it, and therefore always sets a master_key —
LiteLLM without one accepts any key. The provider key lives in the OS
keychain and is uploaded into a volume, never an image layer or label.

URL relay. A container-side xdg-open/BROWSER shim opens URLs in the
host's browser. Uses an OSC sequence to /dev/tty rather than a printed
sentinel, because the shim usually runs as a grandchild of a process
capturing its children's output. Degrades to printing the URL when no
terminal is attached, so scheduled tasks do not hang. Only http/https,
with control characters rejected before new URL() — which strips
newlines, so java\nscript: would otherwise parse as javascript:. Nothing
auto-opens; the user confirms. The web terminal shows a tap-to-open
banner instead, since that browser may be a phone across a tunnel.

Browser view. A Project Home tab that watches and takes over the browser
Claude drives with Playwright, using Playwright's own dashboard. Zero
image cost — Playwright stays user-installed. It does not reuse the auth
bridge's PortForward, which binds an unauthenticated port: correct for a
throwaway OAuth listener, wrong for mouse and keyboard control of a
browser in a passwordless-sudo container. Instead a token-gated loopback
proxy checks Host, then token or a forbidden-header origin signal,
before a byte reaches the container. Host ports are confined to
47820..=47827 so CSP frame-src can enumerate them rather than widening
to a wildcard, with a test asserting the two agree.

188 frontend tests, 107 Rust tests, both builds clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
shadow-test
2026-08-09 16:55:28 -07:00
co-authored by Claude Opus 5
parent 7d00390e1f
commit cc5f691677
46 changed files with 6194 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/src/components/projects/home/config/ModelSection.tsx
+133 -2
View File
@@ -3,6 +3,7 @@ import type {
Backend,
BedrockAuthMethod,
BedrockConfig,
LlamaCppConfig,
OllamaConfig,
OpenAiCompatibleConfig,
Project,
@@ -31,14 +32,28 @@ export const DEFAULT_BEDROCK_CONFIG: BedrockConfig = {
export const DEFAULT_OLLAMA_CONFIG: OllamaConfig = {
base_url: "http://host.docker.internal:11434",
model_id: null,
haiku_model_id: null,
};
/** `llama-server` listens on port 8080 unless `--port` says otherwise. */
export const DEFAULT_LLAMACPP_CONFIG: LlamaCppConfig = {
base_url: "http://host.docker.internal:8080",
model_id: null,
haiku_model_id: null,
};
export const DEFAULT_OPENAI_COMPATIBLE_CONFIG: OpenAiCompatibleConfig = {
base_url: "http://host.docker.internal:4000",
api_key: null,
model_id: null,
haiku_model_id: null,
};
/** Shown under the optional per-backend Haiku override. Kept in one place so
* all three custom-endpoint backends explain it identically. */
const HAIKU_HINT =
"Optional. Claude Code resolves the `haiku` alias to this, and uses it for background work such as conversation titles. Leave blank to reuse the model above — that is what stops background calls failing against a server that only serves one model.";
interface Props {
project: Project;
save: (patch: Partial<Project>) => Promise<boolean>;
@@ -64,6 +79,19 @@ export default function ModelSection({ project, save, disabled }: Props) {
const [ollamaModelId, setOllamaModelId] = useState(
project.ollama_config?.model_id ?? "",
);
const [ollamaHaikuModelId, setOllamaHaikuModelId] = useState(
project.ollama_config?.haiku_model_id ?? "",
);
const [llamaCppBaseUrl, setLlamaCppBaseUrl] = useState(
project.llamacpp_config?.base_url ?? DEFAULT_LLAMACPP_CONFIG.base_url,
);
const [llamaCppModelId, setLlamaCppModelId] = useState(
project.llamacpp_config?.model_id ?? "",
);
const [llamaCppHaikuModelId, setLlamaCppHaikuModelId] = useState(
project.llamacpp_config?.haiku_model_id ?? "",
);
const [oaiBaseUrl, setOaiBaseUrl] = useState(
project.openai_compatible_config?.base_url ??
@@ -75,6 +103,9 @@ export default function ModelSection({ project, save, disabled }: Props) {
const [oaiModelId, setOaiModelId] = useState(
project.openai_compatible_config?.model_id ?? "",
);
const [oaiHaikuModelId, setOaiHaikuModelId] = useState(
project.openai_compatible_config?.haiku_model_id ?? "",
);
useEffect(() => {
const bc = project.bedrock_config ?? DEFAULT_BEDROCK_CONFIG;
@@ -88,12 +119,19 @@ export default function ModelSection({ project, save, disabled }: Props) {
setServiceTier(bc.service_tier ?? "");
setOllamaBaseUrl(project.ollama_config?.base_url ?? DEFAULT_OLLAMA_CONFIG.base_url);
setOllamaModelId(project.ollama_config?.model_id ?? "");
setOllamaHaikuModelId(project.ollama_config?.haiku_model_id ?? "");
setLlamaCppBaseUrl(
project.llamacpp_config?.base_url ?? DEFAULT_LLAMACPP_CONFIG.base_url,
);
setLlamaCppModelId(project.llamacpp_config?.model_id ?? "");
setLlamaCppHaikuModelId(project.llamacpp_config?.haiku_model_id ?? "");
setOaiBaseUrl(
project.openai_compatible_config?.base_url ??
DEFAULT_OPENAI_COMPATIBLE_CONFIG.base_url,
);
setOaiApiKey(project.openai_compatible_config?.api_key ?? "");
setOaiModelId(project.openai_compatible_config?.model_id ?? "");
setOaiHaikuModelId(project.openai_compatible_config?.haiku_model_id ?? "");
}, [project]);
const saveBedrock = (patch: Partial<BedrockConfig>) =>
@@ -104,6 +142,14 @@ export default function ModelSection({ project, save, disabled }: Props) {
ollama_config: { ...(project.ollama_config ?? DEFAULT_OLLAMA_CONFIG), ...patch },
});
const saveLlamaCpp = (patch: Partial<LlamaCppConfig>) =>
save({
llamacpp_config: {
...(project.llamacpp_config ?? DEFAULT_LLAMACPP_CONFIG),
...patch,
},
});
const saveOpenAi = (patch: Partial<OpenAiCompatibleConfig>) =>
save({
openai_compatible_config: {
@@ -122,6 +168,8 @@ export default function ModelSection({ project, save, disabled }: Props) {
patch.bedrock_config = DEFAULT_BEDROCK_CONFIG;
if (mode === "ollama" && !project.ollama_config)
patch.ollama_config = DEFAULT_OLLAMA_CONFIG;
if (mode === "llama_cpp" && !project.llamacpp_config)
patch.llamacpp_config = DEFAULT_LLAMACPP_CONFIG;
if (mode === "open_ai_compatible" && !project.openai_compatible_config)
patch.openai_compatible_config = DEFAULT_OPENAI_COMPATIBLE_CONFIG;
save(patch);
@@ -131,7 +179,7 @@ export default function ModelSection({ project, save, disabled }: Props) {
<ConfigGroup title="Model" description="Which provider serves this project's Claude.">
<Field
label="Backend"
hint="Anthropic connects directly via OAuth (run `claude login` in a terminal). Bedrock routes through AWS. Ollama and OpenAI Compatible point at any compatible endpoint."
hint="Anthropic connects directly via OAuth (run `claude login` in a terminal). Bedrock routes through AWS. Ollama, llama.cpp and OpenAI Compatible point at any endpoint that implements the Anthropic Messages API."
>
{(id) => (
<select
@@ -144,6 +192,7 @@ export default function ModelSection({ project, save, disabled }: Props) {
<option value="anthropic">Anthropic</option>
<option value="bedrock">Bedrock</option>
<option value="ollama">Ollama</option>
<option value="llama_cpp">llama.cpp</option>
<option value="open_ai_compatible">OpenAI Compatible</option>
</select>
)}
@@ -365,6 +414,73 @@ export default function ModelSection({ project, save, disabled }: Props) {
/>
)}
</Field>
<Field label="Background model" hint={HAIKU_HINT}>
{(id) => (
<input
id={id}
value={ollamaHaikuModelId}
onChange={(e) => setOllamaHaikuModelId(e.target.value)}
onBlur={() =>
saveOllama({ haiku_model_id: ollamaHaikuModelId.trim() || null })
}
placeholder="(same as the model above)"
disabled={disabled}
className={monoInputClass}
/>
)}
</Field>
</div>
)}
{project.backend === "llama_cpp" && (
<div className="space-y-4 pt-2 border-t border-[var(--border-color)]">
<Field
label="Base URL"
hint="Your llama-server. It listens on port 8080 by default; use host.docker.internal to reach the host machine."
>
{(id) => (
<input
id={id}
value={llamaCppBaseUrl}
onChange={(e) => setLlamaCppBaseUrl(e.target.value)}
onBlur={() => saveLlamaCpp({ base_url: llamaCppBaseUrl })}
placeholder="http://host.docker.internal:8080"
disabled={disabled}
className={monoInputClass}
/>
)}
</Field>
<Field
label="Model"
hint="The model llama-server was started with. llama-server serves one model, so this is mainly what Claude Code reports — but it is also what the model aliases are pinned to."
>
{(id) => (
<input
id={id}
value={llamaCppModelId}
onChange={(e) => setLlamaCppModelId(e.target.value)}
onBlur={() => saveLlamaCpp({ model_id: llamaCppModelId || null })}
placeholder="qwen3.5-coder-30b"
disabled={disabled}
className={monoInputClass}
/>
)}
</Field>
<Field label="Background model" hint={HAIKU_HINT}>
{(id) => (
<input
id={id}
value={llamaCppHaikuModelId}
onChange={(e) => setLlamaCppHaikuModelId(e.target.value)}
onBlur={() =>
saveLlamaCpp({ haiku_model_id: llamaCppHaikuModelId.trim() || null })
}
placeholder="(same as the model above)"
disabled={disabled}
className={monoInputClass}
/>
)}
</Field>
</div>
)}
@@ -372,7 +488,7 @@ export default function ModelSection({ project, save, disabled }: Props) {
<div className="space-y-4 pt-2 border-t border-[var(--border-color)]">
<Field
label="Base URL"
hint="Any OpenAI API-compatible endpoint — LiteLLM, OpenRouter, vLLM, and so on."
hint="A gateway that implements the Anthropic Messages API (POST /v1/messages) — LiteLLM, for example. An endpoint that only speaks OpenAI /v1/chat/completions will not work."
>
{(id) => (
<input
@@ -413,6 +529,21 @@ export default function ModelSection({ project, save, disabled }: Props) {
/>
)}
</Field>
<Field label="Background model" hint={HAIKU_HINT}>
{(id) => (
<input
id={id}
value={oaiHaikuModelId}
onChange={(e) => setOaiHaikuModelId(e.target.value)}
onBlur={() =>
saveOpenAi({ haiku_model_id: oaiHaikuModelId.trim() || null })
}
placeholder="(same as the model above)"
disabled={disabled}
className={monoInputClass}
/>
)}
</Field>
</div>
)}
</ConfigGroup>