CyberCoveLLC/Triple-C
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 35 Wiki Activity

Add llama.cpp backend, model gateway, URL relay and browser view

Browse Source 
Four features, plus a latent bug fix.

llama.cpp backend. Claude Code only ever speaks the Anthropic Messages
API — confirmed empirically by pointing it at a logging server, which
received POST /v1/messages?beta=true. llama-server implements that
natively (verified in its README, alongside --port default 8080), so
this is a plain base-URL backend with no translation shim, the same
shape as Ollama. Its --api-key defaults to none, so the auth token is a
placeholder Claude Code requires and llama-server ignores.

Model alias fix. ANTHROPIC_DEFAULT_HAIKU_MODEL is documented as "also
used for background functionality", and Triple-C set none of the alias
vars. So on every custom-endpoint backend, Claude Code resolved `haiku`
to an Anthropic model id and sent it to a local server that does not
have it — background features failed silently. All four
ANTHROPIC_DEFAULT_{OPUS,SONNET,HAIKU,FABLE}_MODEL vars are now pinned to
the backend's configured model, with an optional Haiku override, and
blanked for Anthropic and Bedrock so those keep Claude Code's defaults.
The deprecated ANTHROPIC_SMALL_FAST_MODEL is never emitted. Existing
Ollama and OpenAI-Compatible containers are recreated once so the new
env reaches them; the snapshot is preserved.

Model gateway. Optional LiteLLM sibling container, off by default,
mirroring stt.rs — this is what makes real OpenAI usable, since
api.openai.com has no /v1/messages. Pinned to v1.96.0 by tag and digest:
the 1.82.7/1.82.8 malware was PyPI-only and never affected the official
images, which is precisely why this builds FROM the image rather than
pip-installing, but 1.84.0 is still the floor for proxy CVEs (API-key
SQLi, Host-header auth bypass, MCP auth bypass). Binds 0.0.0.0 because
project containers consume it, and therefore always sets a master_key —
LiteLLM without one accepts any key. The provider key lives in the OS
keychain and is uploaded into a volume, never an image layer or label.

URL relay. A container-side xdg-open/BROWSER shim opens URLs in the
host's browser. Uses an OSC sequence to /dev/tty rather than a printed
sentinel, because the shim usually runs as a grandchild of a process
capturing its children's output. Degrades to printing the URL when no
terminal is attached, so scheduled tasks do not hang. Only http/https,
with control characters rejected before new URL() — which strips
newlines, so java\nscript: would otherwise parse as javascript:. Nothing
auto-opens; the user confirms. The web terminal shows a tap-to-open
banner instead, since that browser may be a phone across a tunnel.

Browser view. A Project Home tab that watches and takes over the browser
Claude drives with Playwright, using Playwright's own dashboard. Zero
image cost — Playwright stays user-installed. It does not reuse the auth
bridge's PortForward, which binds an unauthenticated port: correct for a
throwaway OAuth listener, wrong for mouse and keyboard control of a
browser in a passwordless-sudo container. Instead a token-gated loopback
proxy checks Host, then token or a forbidden-header origin signal,
before a byte reaches the container. Host ports are confined to
47820..=47827 so CSP frame-src can enumerate them rather than widening
to a wildcard, with a test asserting the two agree.

188 frontend tests, 107 Rust tests, both builds clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
shadow-test
2026-08-09 16:55:28 -07:00
co-authored by Claude Opus 5
parent 7d00390e1f
commit cc5f691677
46 changed files with 6194 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/src-tauri/src/docker/container.rs
+449 -3
View File
@@ -8,7 +8,7 @@ use std::collections::HashMap;
use sha2::{Sha256, Digest};
use super::client::get_docker;
use crate::models::{Backend, BedrockAuthMethod, ClaudeCodeSettings, ContainerInfo, EnvVar, GlobalAwsSettings, GlobalOllamaSettings, GlobalOpenAiCompatibleSettings, PortMapping, Project, ProjectPath};
use crate::models::{Backend, BedrockAuthMethod, ClaudeCodeSettings, ContainerInfo, EnvVar, GlobalAwsSettings, GlobalLlamaCppSettings, GlobalOllamaSettings, GlobalOpenAiCompatibleSettings, PortMapping, Project, ProjectPath};
const SCHEDULER_INSTRUCTIONS: &str = r#"## Scheduled Tasks
@@ -194,8 +194,89 @@ const RESERVED_ENV_EXACT: &[&str] = &[
"MISSION_CONTROL_ENABLED",
"TRIPLE_C_PERMISSION_MODE",
CLAUDE_OAUTH_TOKEN_ENV,
// The model-alias vars are already covered by the `ANTHROPIC_` prefix
// above; they are listed explicitly so that a future narrowing of the
// prefix list cannot silently unreserve them, and so `is_reserved_env_key`
// reads as the single, complete statement of what Triple-C owns.
ANTHROPIC_DEFAULT_OPUS_MODEL,
ANTHROPIC_DEFAULT_SONNET_MODEL,
ANTHROPIC_DEFAULT_HAIKU_MODEL,
ANTHROPIC_DEFAULT_FABLE_MODEL,
];
/// Claude Code's model-alias env vars. Each names the concrete model id that
/// one of the `opus` / `sonnet` / `haiku` / `fable` aliases resolves to.
///
/// `ANTHROPIC_DEFAULT_HAIKU_MODEL` is the important one: it is documented as
/// *"Model ID that the `haiku` alias resolves to, also used for background
/// functionality"* — conversation titles, summarisation, and other out-of-band
/// calls. Left unset against a local server, Claude Code sends
/// Anthropic's own Haiku model id to a server that has never heard of it and
/// every background call fails, usually silently.
///
/// (`ANTHROPIC_SMALL_FAST_MODEL` is the deprecated predecessor of the Haiku
/// var and is deliberately *not* used.)
pub const ANTHROPIC_DEFAULT_OPUS_MODEL: &str = "ANTHROPIC_DEFAULT_OPUS_MODEL";
pub const ANTHROPIC_DEFAULT_SONNET_MODEL: &str = "ANTHROPIC_DEFAULT_SONNET_MODEL";
pub const ANTHROPIC_DEFAULT_HAIKU_MODEL: &str = "ANTHROPIC_DEFAULT_HAIKU_MODEL";
pub const ANTHROPIC_DEFAULT_FABLE_MODEL: &str = "ANTHROPIC_DEFAULT_FABLE_MODEL";
/// Resolve the four `ANTHROPIC_DEFAULT_*_MODEL` values for a backend that
/// points Claude Code at a custom endpoint.
///
/// All four aliases fall back to `effective_model` — the backend's configured
/// model id, already resolved per-project → global. That is the right default:
/// a local server almost always serves exactly one model, so every alias must
/// name it or the calls that use an alias (notably the background ones, which
/// use `haiku`) go to a model the server does not have.
///
/// `haiku_override` exists because that is the one alias someone might
/// legitimately want to point elsewhere — at a second, smaller server-side
/// model kept for cheap background work. A blank override falls back to
/// `effective_model` like the others.
///
/// Returns pairs in `(name, value)` form; a blank resolved value emits nothing
/// at all rather than an empty var, so an unconfigured backend is left exactly
/// as Claude Code found it.
pub fn compute_model_aliases(
effective_model: Option<&str>,
haiku_override: Option<&str>,
) -> Vec<(&'static str, String)> {
let base = effective_model.map(str::trim).filter(|s| !s.is_empty());
let haiku = haiku_override
.map(str::trim)
.filter(|s| !s.is_empty())
.or(base);
let mut out: Vec<(&'static str, String)> = Vec::new();
if let Some(m) = base {
out.push((ANTHROPIC_DEFAULT_OPUS_MODEL, m.to_string()));
out.push((ANTHROPIC_DEFAULT_SONNET_MODEL, m.to_string()));
}
if let Some(h) = haiku {
out.push((ANTHROPIC_DEFAULT_HAIKU_MODEL, h.to_string()));
}
if let Some(m) = base {
out.push((ANTHROPIC_DEFAULT_FABLE_MODEL, m.to_string()));
}
out
}
/// The fingerprint contribution of the model aliases, so that changing an
/// alias (or the model it falls back to) forces a container recreation.
/// `container_needs_recreation` is label-based and never diffs env, so an
/// env-only change is invisible without this.
fn model_alias_fingerprint_part(
effective_model: Option<&str>,
haiku_override: Option<&str>,
) -> String {
compute_model_aliases(effective_model, haiku_override)
.into_iter()
.map(|(k, v)| format!("{}={}", k, v))
.collect::<Vec<_>>()
.join(",")
}
/// Whether `key` is an env var name Triple-C reserves for itself.
fn is_reserved_env_key(key: &str) -> bool {
let upper = key.to_uppercase();
@@ -360,7 +441,13 @@ fn compute_bedrock_fingerprint(project: &Project, global_aws: &GlobalAwsSettings
}
/// Compute a fingerprint for the Ollama configuration so we can detect changes.
/// Includes the resolved base_url and model_id (per-project blank → global default).
/// Includes the resolved base_url and model_id (per-project blank → global
/// default) and the resolved model aliases.
///
/// NOTE: adding the alias part changes this hash for every existing Ollama
/// container, so each will be recreated once on the next start. That is exactly
/// what is wanted — recreation is the only way to get the new
/// `ANTHROPIC_DEFAULT_*_MODEL` vars into the container's env.
fn compute_ollama_fingerprint(project: &Project, global_ollama: &GlobalOllamaSettings) -> String {
if let Some(ref ollama) = project.ollama_config {
let effective_url = resolve_with_global(
@@ -371,7 +458,43 @@ fn compute_ollama_fingerprint(project: &Project, global_ollama: &GlobalOllamaSet
ollama.model_id.as_deref(),
global_ollama.default_model_id.as_deref(),
).unwrap_or("").to_string();
let parts = vec![effective_url, effective_model];
let aliases = model_alias_fingerprint_part(
Some(&effective_model),
resolve_with_global(
ollama.haiku_model_id.as_deref(),
global_ollama.default_haiku_model_id.as_deref(),
),
);
let parts = vec![effective_url, effective_model, aliases];
sha256_hex(&parts.join("|"))
} else {
String::new()
}
}
/// Compute a fingerprint for the llama.cpp configuration so we can detect
/// changes. Mirrors [`compute_ollama_fingerprint`].
fn compute_llamacpp_fingerprint(
project: &Project,
global_llamacpp: &GlobalLlamaCppSettings,
) -> String {
if let Some(ref cfg) = project.llamacpp_config {
let effective_url = resolve_with_global(
Some(&cfg.base_url),
global_llamacpp.base_url.as_deref(),
).unwrap_or("").to_string();
let effective_model = resolve_with_global(
cfg.model_id.as_deref(),
global_llamacpp.default_model_id.as_deref(),
).unwrap_or("").to_string();
let aliases = model_alias_fingerprint_part(
Some(&effective_model),
resolve_with_global(
cfg.haiku_model_id.as_deref(),
global_llamacpp.default_haiku_model_id.as_deref(),
),
);
let parts = vec![effective_url, effective_model, aliases];
sha256_hex(&parts.join("|"))
} else {
String::new()
@@ -393,10 +516,18 @@ fn compute_openai_compatible_fingerprint(
config.model_id.as_deref(),
global_openai_compatible.default_model_id.as_deref(),
).unwrap_or("").to_string();
let aliases = model_alias_fingerprint_part(
Some(&effective_model),
resolve_with_global(
config.haiku_model_id.as_deref(),
global_openai_compatible.default_haiku_model_id.as_deref(),
),
);
let parts = vec![
effective_url,
config.api_key.as_deref().unwrap_or("").to_string(),
effective_model,
aliases,
];
sha256_hex(&parts.join("|"))
} else {
@@ -576,6 +707,7 @@ pub async fn create_container(
aws_config_path: Option<&str>,
global_aws: &GlobalAwsSettings,
global_ollama: &GlobalOllamaSettings,
global_llamacpp: &GlobalLlamaCppSettings,
global_openai_compatible: &GlobalOpenAiCompatibleSettings,
global_claude_instructions: Option<&str>,
global_custom_env_vars: &[EnvVar],
@@ -701,6 +833,14 @@ pub async fn create_container(
}
}
// ── Custom-endpoint backends ─────────────────────────────────────────────
// Ollama, llama.cpp and the OpenAI-Compatible gateway all point Claude Code
// at a non-Anthropic server via ANTHROPIC_BASE_URL. Each resolves its model
// id here; the model-alias vars are emitted once below, from
// `alias_model` / `alias_haiku`, so the three backends cannot drift apart.
let mut alias_model: Option<String> = None;
let mut alias_haiku: Option<String> = None;
// Ollama configuration
if project.backend == Backend::Ollama {
if let Some(ref ollama) = project.ollama_config {
@@ -716,7 +856,44 @@ pub async fn create_container(
global_ollama.default_model_id.as_deref(),
) {
env_vars.push(format!("ANTHROPIC_MODEL={}", model));
alias_model = Some(model.to_string());
}
alias_haiku = resolve_with_global(
ollama.haiku_model_id.as_deref(),
global_ollama.default_haiku_model_id.as_deref(),
)
.map(str::to_string);
}
}
// llama.cpp (llama-server) configuration
if project.backend == Backend::LlamaCpp {
if let Some(ref cfg) = project.llamacpp_config {
if let Some(url) = resolve_with_global(
Some(&cfg.base_url),
global_llamacpp.base_url.as_deref(),
) {
env_vars.push(format!("ANTHROPIC_BASE_URL={}", url));
}
// llama-server only enforces an Authorization header when it was
// started with `--api-key` (default: none), so the value here is
// ignored in the common case. Claude Code still refuses to run
// against a custom base URL with no credential at all, so a
// placeholder is always sent — same trick as the Ollama branch
// above, which sends the literal "ollama".
env_vars.push("ANTHROPIC_AUTH_TOKEN=llama.cpp".to_string());
if let Some(model) = resolve_with_global(
cfg.model_id.as_deref(),
global_llamacpp.default_model_id.as_deref(),
) {
env_vars.push(format!("ANTHROPIC_MODEL={}", model));
alias_model = Some(model.to_string());
}
alias_haiku = resolve_with_global(
cfg.haiku_model_id.as_deref(),
global_llamacpp.default_haiku_model_id.as_deref(),
)
.map(str::to_string);
}
}
@@ -737,7 +914,27 @@ pub async fn create_container(
global_openai_compatible.default_model_id.as_deref(),
) {
env_vars.push(format!("ANTHROPIC_MODEL={}", model));
alias_model = Some(model.to_string());
}
alias_haiku = resolve_with_global(
config.haiku_model_id.as_deref(),
global_openai_compatible.default_haiku_model_id.as_deref(),
)
.map(str::to_string);
}
}
// Model aliases — the fix for background Claude Code calls against a local
// server. Only for backends that talk to a custom endpoint: Anthropic and
// Bedrock reach servers that really do host the Anthropic model ids, so
// they keep Claude Code's own defaults. Anything not emitted here is
// blanked by the MANAGED_AUTH_KEYS pass below, so switching *away* from a
// custom endpoint clears the aliases out of the snapshot image too.
if project.backend.uses_custom_endpoint() {
for (key, value) in
compute_model_aliases(alias_model.as_deref(), alias_haiku.as_deref())
{
env_vars.push(format!("{}={}", key, value));
}
}
@@ -778,6 +975,15 @@ pub async fn create_container(
"ANTHROPIC_MODEL",
"DISABLE_PROMPT_CACHING",
"ANTHROPIC_BEDROCK_SERVICE_TIER",
// Switching from a custom-endpoint backend to Anthropic or Bedrock must
// *clear* the aliases, not merely stop setting them: a stale
// ANTHROPIC_DEFAULT_HAIKU_MODEL baked into the snapshot image would
// keep pointing background calls at a model id the new backend has
// never heard of.
ANTHROPIC_DEFAULT_OPUS_MODEL,
ANTHROPIC_DEFAULT_SONNET_MODEL,
ANTHROPIC_DEFAULT_HAIKU_MODEL,
ANTHROPIC_DEFAULT_FABLE_MODEL,
// Revoking the shared token, opting a project out, or switching away
// from the Anthropic backend must *clear* this, not merely stop setting
// it — otherwise the value committed into the snapshot image keeps
@@ -1004,6 +1210,7 @@ pub async fn create_container(
labels.insert("triple-c.paths-fingerprint".to_string(), compute_paths_fingerprint(&project.paths));
labels.insert("triple-c.bedrock-fingerprint".to_string(), compute_bedrock_fingerprint(project, global_aws));
labels.insert("triple-c.ollama-fingerprint".to_string(), compute_ollama_fingerprint(project, global_ollama));
labels.insert("triple-c.llamacpp-fingerprint".to_string(), compute_llamacpp_fingerprint(project, global_llamacpp));
labels.insert("triple-c.openai-compatible-fingerprint".to_string(), compute_openai_compatible_fingerprint(project, global_openai_compatible));
labels.insert("triple-c.ports-fingerprint".to_string(), compute_ports_fingerprint(&project.port_mappings));
labels.insert("triple-c.image".to_string(), image_name.to_string());
@@ -1301,6 +1508,7 @@ pub async fn container_needs_recreation(
project: &Project,
global_aws: &GlobalAwsSettings,
global_ollama: &GlobalOllamaSettings,
global_llamacpp: &GlobalLlamaCppSettings,
global_openai_compatible: &GlobalOpenAiCompatibleSettings,
global_claude_instructions: Option<&str>,
global_custom_env_vars: &[EnvVar],
@@ -1387,6 +1595,17 @@ pub async fn container_needs_recreation(
return Ok(true);
}
// ── llama.cpp config fingerprint ─────────────────────────────────────
// A missing label means the container predates the llama.cpp backend, in
// which case the expected fingerprint is also "" (no llamacpp_config) and
// nothing is recreated needlessly.
let expected_llamacpp_fp = compute_llamacpp_fingerprint(project, global_llamacpp);
let container_llamacpp_fp = get_label("triple-c.llamacpp-fingerprint").unwrap_or_default();
if container_llamacpp_fp != expected_llamacpp_fp {
log::info!("llama.cpp config mismatch");
return Ok(true);
}
// ── OpenAI Compatible config fingerprint ────────────────────────────
let expected_oai_fp = compute_openai_compatible_fingerprint(project, global_openai_compatible);
let container_oai_fp = get_label("triple-c.openai-compatible-fingerprint").unwrap_or_default();
@@ -1635,3 +1854,230 @@ pub async fn list_sibling_containers() -> Result<Vec<ContainerSummary>, String>
Ok(siblings)
}
#[cfg(test)]
mod tests {
use super::*;
const OPUS: &str = ANTHROPIC_DEFAULT_OPUS_MODEL;
const SONNET: &str = ANTHROPIC_DEFAULT_SONNET_MODEL;
const HAIKU: &str = ANTHROPIC_DEFAULT_HAIKU_MODEL;
const FABLE: &str = ANTHROPIC_DEFAULT_FABLE_MODEL;
fn aliases(model: Option<&str>, haiku: Option<&str>) -> Vec<(&'static str, String)> {
compute_model_aliases(model, haiku)
}
#[test]
fn all_four_aliases_fall_back_to_the_configured_model() {
assert_eq!(
aliases(Some("qwen3.5:27b"), None),
vec![
(OPUS, "qwen3.5:27b".to_string()),
(SONNET, "qwen3.5:27b".to_string()),
(HAIKU, "qwen3.5:27b".to_string()),
(FABLE, "qwen3.5:27b".to_string()),
]
);
}
#[test]
fn the_haiku_override_replaces_only_the_haiku_alias() {
let got = aliases(Some("big-model"), Some("small-model"));
assert_eq!(
got,
vec![
(OPUS, "big-model".to_string()),
(SONNET, "big-model".to_string()),
(HAIKU, "small-model".to_string()),
(FABLE, "big-model".to_string()),
]
);
}
#[test]
fn a_blank_or_whitespace_haiku_override_falls_back_to_the_model() {
for override_value in [Some(""), Some(" "), None] {
let got = aliases(Some("m"), override_value);
assert_eq!(
got.iter().find(|(k, _)| *k == HAIKU).map(|(_, v)| v.as_str()),
Some("m"),
"override {:?} should fall back to the model id",
override_value
);
}
}
#[test]
fn values_are_trimmed() {
assert_eq!(
aliases(Some(" m "), Some(" h ")),
vec![
(OPUS, "m".to_string()),
(SONNET, "m".to_string()),
(HAIKU, "h".to_string()),
(FABLE, "m".to_string()),
]
);
}
#[test]
fn no_model_and_no_override_emits_nothing() {
// Nothing to point the aliases at — leave Claude Code's defaults alone
// rather than injecting empty vars.
assert!(aliases(None, None).is_empty());
assert!(aliases(Some(""), Some(" ")).is_empty());
}
#[test]
fn a_haiku_override_alone_still_fixes_background_calls() {
// No model id configured, but the user pointed haiku somewhere: emit
// just that one, because it is the alias background work uses.
assert_eq!(
aliases(None, Some("small-model")),
vec![(HAIKU, "small-model".to_string())]
);
}
#[test]
fn only_custom_endpoint_backends_get_aliases() {
assert!(!Backend::Anthropic.uses_custom_endpoint());
assert!(!Backend::Bedrock.uses_custom_endpoint());
assert!(Backend::Ollama.uses_custom_endpoint());
assert!(Backend::LlamaCpp.uses_custom_endpoint());
assert!(Backend::OpenAiCompatible.uses_custom_endpoint());
}
#[test]
fn every_alias_var_is_reserved_and_managed() {
for key in [OPUS, SONNET, HAIKU, FABLE] {
assert!(is_reserved_env_key(key), "{} must be reserved", key);
assert!(
is_reserved_env_key(&key.to_lowercase()),
"{} must be reserved case-insensitively",
key
);
}
// A user-set alias must never survive into the container env.
let fp = compute_env_fingerprint(&[EnvVar {
key: HAIKU.to_string(),
value: "sneaky".to_string(),
}]);
assert_eq!(fp, "");
}
#[test]
fn the_deprecated_small_fast_model_var_is_never_emitted() {
let rendered: Vec<String> = aliases(Some("m"), Some("h"))
.into_iter()
.map(|(k, _)| k.to_string())
.collect();
assert!(!rendered.iter().any(|k| k == "ANTHROPIC_SMALL_FAST_MODEL"));
}
#[test]
fn the_alias_fingerprint_tracks_both_the_model_and_the_override() {
let base = model_alias_fingerprint_part(Some("m"), None);
assert_eq!(base, model_alias_fingerprint_part(Some("m"), Some("")));
assert_ne!(base, model_alias_fingerprint_part(Some("m2"), None));
assert_ne!(base, model_alias_fingerprint_part(Some("m"), Some("h")));
assert_eq!(model_alias_fingerprint_part(None, None), "");
}
fn project_with_llamacpp(model: Option<&str>, haiku: Option<&str>) -> Project {
let mut p = Project::new("t".to_string(), Vec::new());
p.backend = Backend::LlamaCpp;
p.llamacpp_config = Some(crate::models::LlamaCppConfig {
base_url: "http://host.docker.internal:8080".to_string(),
model_id: model.map(str::to_string),
haiku_model_id: haiku.map(str::to_string),
});
p
}
#[test]
fn llamacpp_fingerprint_changes_when_the_haiku_override_changes() {
let g = GlobalLlamaCppSettings::default();
let a = compute_llamacpp_fingerprint(&project_with_llamacpp(Some("m"), None), &g);
let b = compute_llamacpp_fingerprint(&project_with_llamacpp(Some("m"), Some("h")), &g);
assert_ne!(a, b, "the haiku override must force a container recreation");
// No config at all -> empty, so projects on other backends are not
// flagged for recreation by this fingerprint.
let plain = Project::new("t".to_string(), Vec::new());
assert_eq!(compute_llamacpp_fingerprint(&plain, &g), "");
}
#[test]
fn llamacpp_global_defaults_fill_in_for_blank_per_project_fields() {
let g = GlobalLlamaCppSettings {
base_url: Some("http://elsewhere:8080".to_string()),
default_model_id: Some("global-model".to_string()),
default_haiku_model_id: Some("global-haiku".to_string()),
};
// The per-project base URL is set in the fixture, so only the model and
// haiku fields fall through to the globals. Filling them in from the
// globals must be indistinguishable from setting them per-project.
let with_global = compute_llamacpp_fingerprint(&project_with_llamacpp(None, None), &g);
let explicit = compute_llamacpp_fingerprint(
&project_with_llamacpp(Some("global-model"), Some("global-haiku")),
&GlobalLlamaCppSettings::default(),
);
assert_eq!(with_global, explicit);
// …and changing a global must change the fingerprint, so a global-only
// edit still forces a recreation.
assert_ne!(
with_global,
compute_llamacpp_fingerprint(
&project_with_llamacpp(None, None),
&GlobalLlamaCppSettings {
default_haiku_model_id: Some("other-haiku".to_string()),
..g.clone()
},
)
);
assert_eq!(
resolve_with_global(None, g.default_haiku_model_id.as_deref()),
Some("global-haiku")
);
assert_eq!(
resolve_with_global(Some(" "), g.default_model_id.as_deref()),
Some("global-model")
);
}
#[test]
fn backend_serde_round_trips_llamacpp_and_accepts_legacy_spellings() {
assert_eq!(
serde_json::to_string(&Backend::LlamaCpp).unwrap(),
"\"llama_cpp\""
);
for spelling in ["\"llama_cpp\"", "\"llamacpp\"", "\"llama-cpp\"", "\"llama.cpp\""] {
let parsed: Backend = serde_json::from_str(spelling).unwrap();
assert_eq!(parsed, Backend::LlamaCpp, "failed for {}", spelling);
}
}
#[test]
fn a_project_json_without_llamacpp_config_still_deserialises() {
// `projects.json` written by an older build has no llamacpp_config key.
let json = serde_json::json!({
"id": "p1",
"name": "old",
"paths": [],
"container_id": null,
"status": "stopped",
"backend": "ollama",
"bedrock_config": null,
"ollama_config": { "base_url": "http://x:11434", "model_id": "m" },
"openai_compatible_config": null,
"allow_docker_access": false,
"created_at": "2026-01-01T00:00:00Z",
"updated_at": "2026-01-01T00:00:00Z"
});
let p: Project = serde_json::from_value(json).unwrap();
assert!(p.llamacpp_config.is_none());
// The new per-backend haiku override also defaults cleanly.
assert!(p.ollama_config.unwrap().haiku_model_id.is_none());
}
}
app/src-tauri/src/docker/gateway.rs
+644
View File
@@ -0,0 +1,644 @@
//! Lifecycle for the **model gateway** container — a pinned LiteLLM proxy that
//! Triple-C runs as a sibling of the project containers.
//!
//! Shape mirrors `docker::stt`: an image that is either pulled from a registry
//! or built locally from an embedded Dockerfile, a fixed container name, a
//! named volume, and `get_* / ensure_*_running / stop_* / pull_* / build_*`.
//!
//! Two things differ from STT, both deliberate:
//!
//! * **The port is published on `0.0.0.0`, not `127.0.0.1`.** STT is consumed
//! by the Tauri host process, so loopback is enough. The gateway is consumed
//! by *project containers*, which sit on Docker's default bridge and reach
//! the host through the bridge gateway — a loopback-only bind is invisible to
//! them. See [`gateway_base_url`].
//! * **The rendered config is uploaded into the container over the Docker
//! API** rather than passed as env. It holds the provider API key, and both
//! env vars and labels are readable by anything on the host via
//! `docker inspect`.
use bollard::container::{
Config, CreateContainerOptions, ListContainersOptions, RemoveContainerOptions,
StartContainerOptions, StopContainerOptions, UploadToContainerOptions,
};
use bollard::image::BuildImageOptions;
use bollard::models::{HostConfig, Mount, MountTypeEnum, PortBinding};
use futures_util::StreamExt;
use sha2::{Digest, Sha256};
use std::collections::HashMap;
use std::io::Write;
use super::client::get_docker;
use crate::models::gateway_settings::{GatewaySettings, GatewayStatus};
use crate::storage::secure;
const GATEWAY_CONTAINER_NAME: &str = "triple-c-gateway";
const GATEWAY_CONFIG_VOLUME: &str = "triple-c-gateway-config";
/// Upstream LiteLLM, pinned to an exact release.
///
/// LiteLLM 1.82.7 and 1.82.8 shipped credential-harvesting malware on PyPI, so
/// nothing here may float a tag or resolve `litellm` at build time. v1.96.0 is
/// also above the 1.84.0 floor set by the proxy auth-bypass CVEs — see the long
/// comment in `gateway-container/Dockerfile`, and keep the two in lockstep.
const GATEWAY_REGISTRY_IMAGE: &str = "ghcr.io/berriai/litellm:v1.96.0";
const GATEWAY_LOCAL_IMAGE: &str = "triple-c-gateway:latest";
const GATEWAY_DOCKERFILE: &str = include_str!("../../../../gateway-container/Dockerfile");
const GATEWAY_DEFAULT_CONFIG: &str = include_str!("../../../../gateway-container/config.yaml");
/// Where the generated config lands inside the container. Backed by
/// [`GATEWAY_CONFIG_VOLUME`] so the file with the provider key lives in a
/// Docker-managed volume rather than an image layer.
const GATEWAY_CONFIG_DIR: &str = "/etc/litellm";
const GATEWAY_CONFIG_PATH: &str = "/etc/litellm/config.yaml";
/// Container-side port. Only the *host* port is user-configurable.
const GATEWAY_INTERNAL_PORT: u16 = 4000;
const CONFIG_FINGERPRINT_LABEL: &str = "triple-c.gateway.config-fingerprint";
/// The value a project should use as its base URL (`ANTHROPIC_BASE_URL`).
///
/// Project containers run on Docker's default bridge with no user-defined
/// network and no `--add-host`, so the only address they share with the
/// gateway is the host itself. Publishing the gateway on `0.0.0.0:<port>`
/// makes it reachable from every container network on the machine:
///
/// * Docker Desktop (macOS / Windows / WSL2) resolves `host.docker.internal`
/// from inside containers automatically — that is the portable value and the
/// one already suggested by the existing OpenAI-compatible placeholder text.
/// * On native Linux Docker `host.docker.internal` is not injected, and the
/// equivalent address is the default bridge gateway, normally
/// `http://172.17.0.1:<port>`.
pub fn gateway_base_url(port: u16) -> String {
format!("http://host.docker.internal:{}", port)
}
fn sha256_hex(input: &str) -> String {
let mut hasher = Sha256::new();
hasher.update(input.as_bytes());
format!("{:x}", hasher.finalize())
}
pub async fn get_gateway_status(settings: &GatewaySettings) -> Result<GatewayStatus, String> {
let image_exists = super::image::image_exists(GATEWAY_REGISTRY_IMAGE)
.await
.unwrap_or(false)
|| super::image::image_exists(GATEWAY_LOCAL_IMAGE)
.await
.unwrap_or(false);
let (container_exists, running) = match find_gateway_container().await? {
Some((_, state, _)) => (true, state == "running"),
None => (false, false),
};
Ok(GatewayStatus {
container_exists,
running,
port: settings.port,
image_exists,
model_count: settings.valid_models().len(),
has_api_key: secure::has_gateway_api_key(),
base_url: gateway_base_url(settings.port),
})
}
/// `(id, state, config fingerprint label)` for the gateway container, if any.
async fn find_gateway_container() -> Result<Option<(String, String, String)>, String> {
let docker = get_docker()?;
let filters: HashMap<String, Vec<String>> = HashMap::from([(
"name".to_string(),
vec![format!("/{}", GATEWAY_CONTAINER_NAME)],
)]);
let containers = docker
.list_containers(Some(ListContainersOptions {
all: true,
filters,
..Default::default()
}))
.await
.map_err(|e| format!("Failed to list containers: {}", e))?;
if let Some(container) = containers.first() {
let id = container.id.clone().unwrap_or_default();
let state = container.state.clone().unwrap_or_default();
let fingerprint = container
.labels
.as_ref()
.and_then(|l| l.get(CONFIG_FINGERPRINT_LABEL))
.cloned()
.unwrap_or_default();
return Ok(Some((id, state, fingerprint)));
}
Ok(None)
}
// ─────────────────────────────────────────────────────────────────────────────
// Config generation
// ─────────────────────────────────────────────────────────────────────────────
/// Render a YAML double-quoted scalar.
///
/// Everything that reaches the config comes from user input (model names, base
/// URLs, keys), so nothing may be interpolated raw — a stray `"` or newline
/// would otherwise rewrite the document.
fn yaml_str(value: &str) -> String {
let mut out = String::with_capacity(value.len() + 2);
out.push('"');
for c in value.chars() {
match c {
'"' => out.push_str("\\\""),
'\\' => out.push_str("\\\\"),
'\n' => out.push_str("\\n"),
'\r' => out.push_str("\\r"),
'\t' => out.push_str("\\t"),
c if (c as u32) < 0x20 => out.push_str(&format!("\\x{:02x}", c as u32)),
c => out.push(c),
}
}
out.push('"');
out
}
/// The parts of the config that are safe to hash into a Docker label — i.e.
/// everything except the two secrets, whose changes are tracked by the
/// keychain rotation id instead.
fn config_shape(settings: &GatewaySettings) -> String {
let models: Vec<String> = settings
.valid_models()
.iter()
.map(|m| format!("{}={}", m.name.trim(), m.model_id.trim()))
.collect();
format!(
"provider={};api_base={};port={};models={}",
settings.provider.trim(),
settings.api_base.as_deref().unwrap_or("").trim(),
settings.port,
models.join(",")
)
}
/// Render the LiteLLM config for the current settings.
///
/// `api_key` and `master_key` come from the keychain. The returned string
/// contains both — it goes straight into the Docker upload and must never be
/// logged or surfaced.
fn render_config(settings: &GatewaySettings, api_key: &str, master_key: &str) -> String {
let provider = settings.provider.trim();
let api_base = settings
.api_base
.as_deref()
.map(str::trim)
.filter(|s| !s.is_empty());
let mut out = String::from(
"# Generated by Triple-C — do not edit by hand; it is overwritten on every\n\
# gateway (re)start from Settings → Model Gateway.\n\
model_list:\n",
);
for model in settings.valid_models() {
out.push_str(&format!(" - model_name: {}\n", yaml_str(model.name.trim())));
out.push_str(" litellm_params:\n");
out.push_str(&format!(
" model: {}\n",
yaml_str(&format!("{}/{}", provider, model.model_id.trim()))
));
out.push_str(&format!(" api_key: {}\n", yaml_str(api_key)));
if let Some(base) = api_base {
out.push_str(&format!(" api_base: {}\n", yaml_str(base)));
}
}
out.push_str("general_settings:\n");
out.push_str(&format!(" master_key: {}\n", yaml_str(master_key)));
out.push_str("litellm_settings:\n");
// Claude Code's Anthropic-format requests carry fields some providers
// reject outright; dropping the unsupported ones is what lets the
// translation survive across providers.
out.push_str(" drop_params: true\n");
out
}
/// Upload the rendered config into the container's config volume.
///
/// Runs against a *created but not yet started* container, which is when the
/// volume already exists but LiteLLM has not read anything from it.
async fn upload_config(container_id: &str, config: &str) -> Result<(), String> {
let docker = get_docker()?;
let mut buf = Vec::new();
{
let mut archive = tar::Builder::new(&mut buf);
let mut header = tar::Header::new_gnu();
header.set_size(config.len() as u64);
// World-readable: the upstream image may run LiteLLM as a non-root
// user, and a root-owned 0600 file would simply be unreadable. The
// secret is only exposed to the gateway container itself, which is
// the one process that needs it.
header.set_mode(0o644);
header.set_cksum();
archive
.append_data(&mut header, "config.yaml", config.as_bytes())
.map_err(|e| format!("Failed to build the gateway config archive: {}", e))?;
archive
.finish()
.map_err(|e| format!("Failed to build the gateway config archive: {}", e))?;
}
let _ = buf.flush();
docker
.upload_to_container(
container_id,
Some(UploadToContainerOptions {
path: GATEWAY_CONFIG_DIR,
..Default::default()
}),
buf.into(),
)
.await
.map_err(|e| format!("Failed to upload the gateway config: {}", e))
}
// ─────────────────────────────────────────────────────────────────────────────
// Lifecycle
// ─────────────────────────────────────────────────────────────────────────────
async fn create_gateway_container(
settings: &GatewaySettings,
fingerprint: &str,
) -> Result<String, String> {
let docker = get_docker()?;
// Local build first, then the pinned upstream image — same precedence as
// the STT container.
let image = if super::image::image_exists(GATEWAY_LOCAL_IMAGE)
.await
.unwrap_or(false)
{
GATEWAY_LOCAL_IMAGE.to_string()
} else if super::image::image_exists(GATEWAY_REGISTRY_IMAGE)
.await
.unwrap_or(false)
{
GATEWAY_REGISTRY_IMAGE.to_string()
} else {
return Err(
"Gateway image not found. Please pull or build the image first.".to_string(),
);
};
let mut port_bindings = HashMap::new();
port_bindings.insert(
format!("{}/tcp", GATEWAY_INTERNAL_PORT),
Some(vec![PortBinding {
// Not loopback — project containers reach this through the host.
// See `gateway_base_url`.
host_ip: Some("0.0.0.0".to_string()),
host_port: Some(settings.port.to_string()),
}]),
);
let mut exposed_ports: HashMap<String, HashMap<(), ()>> = HashMap::new();
exposed_ports.insert(format!("{}/tcp", GATEWAY_INTERNAL_PORT), HashMap::new());
let host_config = HostConfig {
port_bindings: Some(port_bindings),
mounts: Some(vec![Mount {
target: Some(GATEWAY_CONFIG_DIR.to_string()),
source: Some(GATEWAY_CONFIG_VOLUME.to_string()),
typ: Some(MountTypeEnum::VOLUME),
..Default::default()
}]),
init: Some(true),
..Default::default()
};
// Non-secret only. Labels are readable by anything on the host.
let mut labels = HashMap::new();
labels.insert(CONFIG_FINGERPRINT_LABEL.to_string(), fingerprint.to_string());
labels.insert(
"triple-c.gateway.port".to_string(),
settings.port.to_string(),
);
labels.insert(
"triple-c.gateway.provider".to_string(),
settings.provider.trim().to_string(),
);
let config = Config {
image: Some(image),
// The upstream entrypoint (`docker/prod_entrypoint.sh`) execs
// `litellm "$@"`. Passed explicitly so the pulled upstream image and
// our locally built one behave identically.
cmd: Some(vec![
"--config".to_string(),
GATEWAY_CONFIG_PATH.to_string(),
"--host".to_string(),
"0.0.0.0".to_string(),
"--port".to_string(),
GATEWAY_INTERNAL_PORT.to_string(),
]),
exposed_ports: Some(exposed_ports),
host_config: Some(host_config),
labels: Some(labels),
..Default::default()
};
let options = CreateContainerOptions {
name: GATEWAY_CONTAINER_NAME,
..Default::default()
};
let response = docker
.create_container(Some(options), config)
.await
.map_err(|e| format!("Failed to create gateway container: {}", e))?;
Ok(response.id)
}
pub async fn ensure_gateway_running(settings: &GatewaySettings) -> Result<GatewayStatus, String> {
let docker = get_docker()?;
if settings.valid_models().is_empty() {
return Err(
"The gateway has no models configured. Add at least one model in Settings."
.to_string(),
);
}
let api_key = secure::get_gateway_api_key()?
.filter(|k| !k.trim().is_empty())
.ok_or_else(|| {
"No provider API key stored for the gateway. Add one in Settings.".to_string()
})?;
let master_key = secure::get_or_create_gateway_master_key()?;
// Rotation id, not a hash of either secret — see `storage::secure`.
let secret_version = secure::get_gateway_secret_version()?.unwrap_or_default();
let fingerprint = sha256_hex(&format!(
"{}|{}",
config_shape(settings),
secret_version
));
if let Some((id, state, existing_fingerprint)) = find_gateway_container().await? {
if existing_fingerprint == fingerprint {
if state == "running" {
return get_gateway_status(settings).await;
}
docker
.start_container(&id, None::<StartContainerOptions<String>>)
.await
.map_err(|e| format!("Failed to start gateway container: {}", e))?;
return get_gateway_status(settings).await;
}
// Config or a secret changed — recreate so the new config is uploaded.
if state == "running" {
docker
.stop_container(&id, None::<StopContainerOptions>)
.await
.map_err(|e| format!("Failed to stop gateway container: {}", e))?;
}
docker
.remove_container(
&id,
Some(RemoveContainerOptions {
force: true,
..Default::default()
}),
)
.await
.map_err(|e| format!("Failed to remove gateway container: {}", e))?;
}
let id = create_gateway_container(settings, &fingerprint).await?;
// Upload before the first start: LiteLLM reads the config once at boot.
let rendered = render_config(settings, &api_key, &master_key);
if let Err(e) = upload_config(&id, &rendered).await {
// Don't leave a half-configured container behind for the next run to
// mistake for a good one.
let _ = docker
.remove_container(
&id,
Some(RemoveContainerOptions {
force: true,
..Default::default()
}),
)
.await;
return Err(e);
}
docker
.start_container(&id, None::<StartContainerOptions<String>>)
.await
.map_err(|e| format!("Failed to start gateway container: {}", e))?;
log::info!(
"Model gateway started on port {} ({} model(s))",
settings.port,
settings.valid_models().len()
);
get_gateway_status(settings).await
}
pub async fn stop_gateway_container() -> Result<(), String> {
let docker = get_docker()?;
if let Some((id, state, _)) = find_gateway_container().await? {
if state == "running" {
docker
.stop_container(&id, None::<StopContainerOptions>)
.await
.map_err(|e| format!("Failed to stop gateway container: {}", e))?;
}
}
Ok(())
}
/// Ask the running gateway whether it is up. LiteLLM takes several seconds to
/// boot, so "container running" and "gateway answering" are not the same thing.
pub async fn check_gateway_health(port: u16) -> Result<bool, String> {
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(5))
.build()
.map_err(|e| format!("Failed to build HTTP client: {}", e))?;
match client
.get(format!("http://127.0.0.1:{}/health/liveliness", port))
.send()
.await
{
Ok(response) => Ok(response.status().is_success()),
Err(e) if e.is_connect() || e.is_timeout() => Ok(false),
Err(e) => Err(format!("Gateway health check failed: {}", e)),
}
}
pub async fn pull_gateway_image<F>(on_progress: F) -> Result<(), String>
where
F: Fn(String) + Send + 'static,
{
super::image::pull_image(GATEWAY_REGISTRY_IMAGE, on_progress).await
}
pub async fn build_gateway_image<F>(on_progress: F) -> Result<(), String>
where
F: Fn(String) + Send + 'static,
{
let docker = get_docker()?;
let tar_bytes = create_gateway_build_context()
.map_err(|e| format!("Failed to create gateway build context: {}", e))?;
let options = BuildImageOptions {
t: GATEWAY_LOCAL_IMAGE,
rm: true,
forcerm: true,
..Default::default()
};
let mut stream = docker.build_image(options, None, Some(tar_bytes.into()));
while let Some(result) = stream.next().await {
match result {
Ok(output) => {
if let Some(stream) = output.stream {
on_progress(stream);
}
if let Some(error) = output.error {
return Err(format!("Build error: {}", error));
}
}
Err(e) => return Err(format!("Build stream error: {}", e)),
}
}
Ok(())
}
fn create_gateway_build_context() -> Result<Vec<u8>, std::io::Error> {
let mut buf = Vec::new();
{
let mut archive = tar::Builder::new(&mut buf);
let mut dockerfile_header = tar::Header::new_gnu();
dockerfile_header.set_size(GATEWAY_DOCKERFILE.len() as u64);
dockerfile_header.set_mode(0o644);
dockerfile_header.set_cksum();
archive.append_data(
&mut dockerfile_header,
"Dockerfile",
GATEWAY_DOCKERFILE.as_bytes(),
)?;
let mut config_header = tar::Header::new_gnu();
config_header.set_size(GATEWAY_DEFAULT_CONFIG.len() as u64);
config_header.set_mode(0o644);
config_header.set_cksum();
archive.append_data(
&mut config_header,
"config.yaml",
GATEWAY_DEFAULT_CONFIG.as_bytes(),
)?;
archive.finish()?;
}
let _ = buf.flush();
Ok(buf)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::models::gateway_settings::GatewayModel;
fn settings() -> GatewaySettings {
GatewaySettings {
enabled: true,
port: 4000,
provider: "openai".to_string(),
api_base: None,
models: vec![
GatewayModel {
name: "gpt-5.1".to_string(),
model_id: "gpt-5.1".to_string(),
},
// Half-filled rows must not reach the YAML.
GatewayModel {
name: " ".to_string(),
model_id: "gpt-4o".to_string(),
},
],
}
}
#[test]
fn valid_models_skips_incomplete_rows() {
assert_eq!(settings().valid_models().len(), 1);
}
#[test]
fn render_config_composes_provider_and_model_id() {
let yaml = render_config(&settings(), "sk-provider", "sk-master");
assert!(yaml.contains("model_name: \"gpt-5.1\""));
assert!(yaml.contains("model: \"openai/gpt-5.1\""));
assert!(yaml.contains("api_key: \"sk-provider\""));
assert!(yaml.contains("master_key: \"sk-master\""));
assert!(yaml.contains("drop_params: true"));
// The skipped row must be absent.
assert!(!yaml.contains("gpt-4o"));
}
#[test]
fn render_config_emits_api_base_only_when_set() {
let mut s = settings();
assert!(!render_config(&s, "k", "m").contains("api_base"));
s.api_base = Some("https://example.test/v1".to_string());
assert!(render_config(&s, "k", "m").contains("api_base: \"https://example.test/v1\""));
// Blank is treated as unset rather than emitted as an empty URL.
s.api_base = Some(" ".to_string());
assert!(!render_config(&s, "k", "m").contains("api_base"));
}
#[test]
fn yaml_str_escapes_injection_attempts() {
let hostile = "a\"\nmaster_key: \"pwned";
let quoted = yaml_str(hostile);
assert!(quoted.starts_with('"') && quoted.ends_with('"'));
// No raw newline can escape the scalar and start a new YAML key.
assert!(!quoted[1..quoted.len() - 1].contains('\n'));
assert!(quoted.contains("\\\""));
}
#[test]
fn config_shape_excludes_secrets_and_tracks_changes() {
let a = config_shape(&settings());
let mut s = settings();
s.models[0].model_id = "gpt-4.1".to_string();
assert_ne!(a, config_shape(&s));
assert!(!a.contains("sk-"));
}
#[test]
fn base_url_points_at_the_host_not_loopback() {
// A project container cannot reach the host's loopback interface.
let url = gateway_base_url(4000);
assert_eq!(url, "http://host.docker.internal:4000");
assert!(!url.contains("127.0.0.1"));
}
}
app/src-tauri/src/docker/mod.rs
+3
View File
@@ -2,9 +2,12 @@ pub mod client;
pub mod container;
pub mod image;
pub mod exec;
pub mod gateway;
pub mod legacy_cleanup;
pub mod stt;
#[allow(unused_imports)]
pub use gateway::*;
#[allow(unused_imports)]
pub use stt::*;
#[allow(unused_imports)]