Bake the browser's runtime libraries into the base image
Build App / compute-version (pull_request) Successful in 4s
Build App / build-macos (pull_request) Successful in 2m28s
Build App / build-windows (pull_request) Successful in 5m13s
Build Container / build-container (pull_request) Successful in 13m11s
Build App / build-linux (pull_request) Successful in 6m53s
Build App / create-tag (pull_request) Skipped
Build App / sync-to-github (pull_request) Skipped
Build App / compute-version (pull_request) Successful in 4s
Build App / build-macos (pull_request) Successful in 2m28s
Build App / build-windows (pull_request) Successful in 5m13s
Build Container / build-container (pull_request) Successful in 13m11s
Build App / build-linux (pull_request) Successful in 6m53s
Build App / create-tag (pull_request) Skipped
Build App / sync-to-github (pull_request) Skipped
`npx playwright install chromium` downloaded ~150 MB of browser that then died with "error while loading shared libraries: libglib-2.0.so.0" — verified, not inferred, against the current image. The image shipped none of Chromium's shared libraries, which is why `apt install google-chrome-stable` looked like the cure: apt was quietly installing the same set as Chrome's own dependencies. Installing them at runtime instead converges on the worst possible state. The libraries land in the container's writable layer, so they are re-paid after every Reset and *lost* on base-image migration, which replays apt from a manifest. The browsers ride in ~/.cache/ms-playwright, inside the home volume, and survive both — leaving a 400 MB browser present with its libraries gone. So the libraries are baked and the browsers are not: each half now lives where it already persists. The layer runs `npx --yes playwright@latest install-deps chromium` rather than a hand-written apt list. Ubuntu 24.04's 64-bit-time_t transition renamed a swathe of these packages (libasound2t64, libatk1.0-0t64, libglib2.0-0t64, …) and a new Chromium dependency would drift straight back into the launch failure this exists to prevent; letting Playwright name its own dependencies is self-maintaining. It sits immediately after Node — npx is its only prerequisite — and well above the shim COPYs, so editing a shim does not re-run it. The `--dry-run` that follows is a build-time assertion, not decoration: on a platform Playwright has no list for, `install-deps` prints a warning and returns having installed **nothing, with exit status 0**. Without the assertion that ships a broken image behind a clean build log. Measured, on a build of this file with the layer applied over an otherwise identical image: +99 packages, +334 MiB unpacked and +119 MiB compressed (2950 → 3284 MiB, 759 → 878 MiB). Two thirds of that is not reachable by trimming — libgbm1, which Chromium needs, pulls mesa-libgallium, which pulls libllvm20. A chromium-only apt list measures 247 MiB against install-deps' 341 MiB; the ~94 MiB difference is xvfb and the CJK/emoji fonts, kept because the base ships no fonts at all and every page this feature exists to display would otherwise render as tofu. Verified on real builds, both architectures: a `--platform linux/arm64` build of this file installs the same 99 packages and passes the same assertion. On the new amd64 image, `playwright install chromium` with no `--with-deps` and no `install-deps` launches headless Chromium 151.0.7922.34 and loads a page; on the old image the identical script fails on libglib-2.0.so.0. `install.rs` no longer runs `install-deps` unconditionally — that would be a minutes-long apt run for nothing on a current image. It asks `install-deps --dry-run` first and skips the install when everything is present, saying which of the two happened on the progress stream. The check is Playwright's rather than a probe of our own for library names, so check and fix cannot disagree about what the dependency set is. Note that `--dry-run` exits 0 both when everything is installed and when Playwright has no list for the platform, so the verdict is read from its output. Containers on older images stay the normal case until people migrate, and they still work: on such an image the simulation cannot even resolve the package names (the index is cleaned in every base image), which reports as "couldn't tell" and installs — the right answer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KSP2KNPhuWKQ4DL5TZEn3k
This commit is contained in:
@@ -11,10 +11,14 @@
|
||||
//! image leaves npm's prefix at `/usr`, so the unprivileged form fails with
|
||||
//! `EACCES` first.
|
||||
//! * `playwright install chromium` downloads a browser that then cannot start,
|
||||
//! because the base image ships **none** of Chromium's shared libraries
|
||||
//! because the image shipped **none** of Chromium's shared libraries
|
||||
//! (`libnss3`, `libgbm1`, `libatk*`, `libasound2`, `libcups2`, …). The
|
||||
//! download succeeds, the launch fails, and the error reads like a Playwright
|
||||
//! bug.
|
||||
//! bug. Current base images bake those libraries in (see `container/
|
||||
//! Dockerfile`), so this step is now usually a no-op — but a project stays on
|
||||
//! the base image it was first built from until someone migrates it, so the
|
||||
//! old case is the *normal* case and has to keep working. Hence: check, then
|
||||
//! install only if needed, and say which happened.
|
||||
//! * Getting from there to a working pane took a long tail of further commands.
|
||||
//!
|
||||
//! ## Where the packages go, and why it is `/workspace`
|
||||
@@ -86,6 +90,10 @@ pub const INSTALL_DIR: &str = "/workspace";
|
||||
const NPM_TIMEOUT: Duration = Duration::from_secs(10 * 60);
|
||||
/// `apt-get update` plus a dozen library packages, or Google's apt repository.
|
||||
const DEPS_TIMEOUT: Duration = Duration::from_secs(20 * 60);
|
||||
/// `apt-get install -s` over ~100 already-installed packages. Local work; a
|
||||
/// container that cannot answer this in two minutes gets the libraries
|
||||
/// installed rather than a hang.
|
||||
const DEPS_CHECK_TIMEOUT: Duration = Duration::from_secs(2 * 60);
|
||||
/// The browser download itself, on a bad connection.
|
||||
const BROWSER_TIMEOUT: Duration = Duration::from_secs(45 * 60);
|
||||
/// Starting a headless browser, and one page load.
|
||||
@@ -148,8 +156,9 @@ impl BrowserTarget {
|
||||
pub fn download_note(self) -> &'static str {
|
||||
match self {
|
||||
Self::Chromium => {
|
||||
"Playwright's Chromium build plus the system libraries it needs — several \
|
||||
hundred MB in total, a few minutes on a normal connection"
|
||||
"Playwright's Chromium build — a few hundred MB, a few minutes on a normal \
|
||||
connection. The system libraries it needs are already in current base images; \
|
||||
on an older container they are installed first"
|
||||
}
|
||||
Self::Chrome => {
|
||||
"Google Chrome from Google's apt repository, with its dependencies — roughly \
|
||||
@@ -302,47 +311,83 @@ pub async fn install_browser(
|
||||
// cause of the "Chromium downloads and then dies" reports, so it gets its
|
||||
// own progress line rather than being folded into the download.
|
||||
//
|
||||
// This is what `playwright install --with-deps` does internally. Running
|
||||
// `install-deps` directly *as root* is the same apt work without depending
|
||||
// on Playwright's own privilege escalation: read from the shipped source, it
|
||||
// shells out to `sudo -- sh -c "apt-get update && apt-get install …"` when
|
||||
// it is not root, which would work here (`claude` has passwordless sudo) but
|
||||
// puts an extra failure mode between the user and the answer.
|
||||
// Current base images bake these in, so on an up-to-date container there is
|
||||
// nothing to do here. That is *not* a reason to drop the step: a project
|
||||
// keeps the base image it was first built from until it is migrated, so
|
||||
// containers without the libraries are the common case for a long while
|
||||
// yet. So: ask first, skip loudly, install only when the answer is no.
|
||||
//
|
||||
// The question is put to Playwright rather than answered by probing for
|
||||
// library names ourselves. `install-deps --dry-run` simulates the very
|
||||
// `apt-get install` that `install-deps` would run and exits non-zero if
|
||||
// anything is missing, which means the check and the fix can never disagree
|
||||
// about what "the libraries" means — including after a Playwright release
|
||||
// adds one.
|
||||
//
|
||||
// Installing runs `install-deps` directly *as root*: that is what `playwright
|
||||
// install --with-deps` does internally, minus Playwright's own privilege
|
||||
// escalation (read from the shipped source, it shells out to
|
||||
// `sudo -- sh -c "apt-get update && apt-get install …"` when it is not root,
|
||||
// which would work here — `claude` has passwordless sudo — but puts an extra
|
||||
// failure mode between the user and the answer).
|
||||
//
|
||||
// For the Chrome channel apt installs `google-chrome-stable`, whose own
|
||||
// dependencies cover the same libraries — but running `install-deps` first
|
||||
// costs little and makes the two paths behave identically.
|
||||
// dependencies cover the same libraries — but going through the same check
|
||||
// first makes the two paths behave identically.
|
||||
emit_progress(
|
||||
app,
|
||||
project_id,
|
||||
"Step 1/3 — installing browser system libraries with apt (needs root; a minute or two)…",
|
||||
"Step 1/3 — checking whether this container already has the browser system libraries…",
|
||||
);
|
||||
let deps = run_step(
|
||||
app,
|
||||
project_id,
|
||||
container_id,
|
||||
"root",
|
||||
"/tmp",
|
||||
vec![
|
||||
"node".to_string(),
|
||||
cli.clone(),
|
||||
"install-deps".to_string(),
|
||||
target.cli_name().to_string(),
|
||||
],
|
||||
DEPS_TIMEOUT,
|
||||
)
|
||||
.await?;
|
||||
log.push_str(&deps.log);
|
||||
if deps.exit_code != 0 {
|
||||
// Not fatal on its own — the libraries may already be present — but it
|
||||
// must never pass silently, because the failure it causes surfaces much
|
||||
// later and looks like something else.
|
||||
warning = Some(format!(
|
||||
"Installing the browser's system libraries failed (exit {}). The browser may install \
|
||||
and then refuse to start. apt said:\n{}",
|
||||
deps.exit_code,
|
||||
deps.log_or("nothing")
|
||||
));
|
||||
let state = check_libraries(container_id, &cli, target).await;
|
||||
match &state {
|
||||
LibraryState::Present => {
|
||||
emit_progress(
|
||||
app,
|
||||
project_id,
|
||||
"Step 1/3 — already there: this image ships the browser system libraries. \
|
||||
Skipping the apt install.",
|
||||
);
|
||||
push_section(&mut log, "Browser system libraries: already installed, apt skipped.");
|
||||
}
|
||||
LibraryState::Missing(_) | LibraryState::Unknown(_) => {
|
||||
emit_progress(
|
||||
app,
|
||||
project_id,
|
||||
&format!(
|
||||
"Step 1/3 — {} Installing browser system libraries with apt (needs root; a \
|
||||
minute or two)…",
|
||||
state.detail()
|
||||
),
|
||||
);
|
||||
let deps = run_step(
|
||||
app,
|
||||
project_id,
|
||||
container_id,
|
||||
"root",
|
||||
"/tmp",
|
||||
vec![
|
||||
"node".to_string(),
|
||||
cli.clone(),
|
||||
"install-deps".to_string(),
|
||||
target.cli_name().to_string(),
|
||||
],
|
||||
DEPS_TIMEOUT,
|
||||
)
|
||||
.await?;
|
||||
push_section(&mut log, &deps.log);
|
||||
if deps.exit_code != 0 {
|
||||
// Not fatal on its own — some of the libraries may already be
|
||||
// present — but it must never pass silently, because the failure
|
||||
// it causes surfaces much later and looks like something else.
|
||||
warning = Some(format!(
|
||||
"Installing the browser's system libraries failed (exit {}). The browser may \
|
||||
install and then refuse to start. apt said:\n{}",
|
||||
deps.exit_code,
|
||||
deps.log_or("nothing")
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Step 2 — the download the user was warned about. Chromium is fetched as
|
||||
@@ -436,6 +481,102 @@ pub async fn install_browser(
|
||||
})
|
||||
}
|
||||
|
||||
/// Phrases `install-deps --dry-run` prints. Matching on Playwright's own words
|
||||
/// is load-bearing: the exit code alone cannot tell "everything is installed"
|
||||
/// (0) apart from "this platform isn't in my table, so I did nothing" (also 0).
|
||||
const DEPS_OK_MARKER: &str = "All system dependencies are installed";
|
||||
const DEPS_MISSING_MARKER: &str = "Missing system dependencies";
|
||||
const DEPS_UNKNOWN_PLATFORM_MARKER: &str = "Cannot install dependencies for";
|
||||
|
||||
/// Whether this container already has the libraries a browser links against.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
enum LibraryState {
|
||||
/// Playwright confirms every package it would install is present. Current
|
||||
/// base images bake them, so this is the answer on an up-to-date container.
|
||||
Present,
|
||||
/// Playwright named packages that are absent.
|
||||
Missing(String),
|
||||
/// The check could not answer. Always installs — an unnecessary apt run
|
||||
/// costs a minute, a skipped one costs a browser that will not start.
|
||||
Unknown(String),
|
||||
}
|
||||
|
||||
impl LibraryState {
|
||||
/// What the user sees on the progress line, and why.
|
||||
fn detail(&self) -> &str {
|
||||
match self {
|
||||
Self::Present => "",
|
||||
Self::Missing(d) | Self::Unknown(d) => d,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Ask Playwright whether the libraries are already installed.
|
||||
///
|
||||
/// `--dry-run` simulates the same `apt-get install` that `install-deps` would
|
||||
/// perform and exits non-zero if any package is missing, so the check can never
|
||||
/// disagree with the fix about what the dependency set is — including after a
|
||||
/// Playwright release changes it.
|
||||
///
|
||||
/// Note that the simulation works on an image whose `/var/lib/apt/lists` has
|
||||
/// been cleaned (every base image's has): apt knows installed packages from
|
||||
/// dpkg's status file. A *missing* package on such an image is simply not in
|
||||
/// any index, so apt fails, Playwright reports the failure, and this returns
|
||||
/// [`LibraryState::Unknown`] — which installs, which is the right answer.
|
||||
async fn check_libraries(container_id: &str, cli: &str, target: BrowserTarget) -> LibraryState {
|
||||
let run = exec_oneshot_as(
|
||||
container_id,
|
||||
"root",
|
||||
vec![
|
||||
"node".to_string(),
|
||||
cli.to_string(),
|
||||
"install-deps".to_string(),
|
||||
"--dry-run".to_string(),
|
||||
target.cli_name().to_string(),
|
||||
],
|
||||
vec![],
|
||||
);
|
||||
match tokio::time::timeout(DEPS_CHECK_TIMEOUT, run).await {
|
||||
Ok(Ok((output, code))) => classify_library_check(&output, code),
|
||||
Ok(Err(e)) => LibraryState::Unknown(format!(
|
||||
"Couldn't ask Playwright whether they're already there ({}), so installing them to be \
|
||||
sure.",
|
||||
e
|
||||
)),
|
||||
Err(_) => LibraryState::Unknown(
|
||||
"The check for them didn't finish in time, so installing them to be sure.".to_string(),
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
/// Read the verdict out of `install-deps --dry-run`'s output.
|
||||
fn classify_library_check(output: &str, exit_code: i64) -> LibraryState {
|
||||
// Checked before the success marker, not after: this branch also exits 0.
|
||||
if output.contains(DEPS_UNKNOWN_PLATFORM_MARKER) {
|
||||
return LibraryState::Unknown(
|
||||
"Playwright doesn't have a dependency list for this container's platform, so it \
|
||||
can't say — installing them to be sure."
|
||||
.to_string(),
|
||||
);
|
||||
}
|
||||
if exit_code == 0 && output.contains(DEPS_OK_MARKER) {
|
||||
return LibraryState::Present;
|
||||
}
|
||||
if let Some(idx) = output.find(DEPS_MISSING_MARKER) {
|
||||
let summary = output[idx..]
|
||||
.lines()
|
||||
.next()
|
||||
.unwrap_or(DEPS_MISSING_MARKER)
|
||||
.trim();
|
||||
return LibraryState::Missing(format!("Playwright reports {}", summary.to_lowercase()));
|
||||
}
|
||||
LibraryState::Unknown(format!(
|
||||
"Couldn't tell whether they're already there (the check exited {}), so installing them to \
|
||||
be sure.",
|
||||
exit_code
|
||||
))
|
||||
}
|
||||
|
||||
/// The result of actually starting a browser.
|
||||
#[derive(Debug, Clone)]
|
||||
struct LaunchVerdict {
|
||||
@@ -759,6 +900,55 @@ mod tests {
|
||||
assert!(err.contains("chrome"), "{}", err);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn baked_in_libraries_are_detected_and_the_apt_step_is_skipped() {
|
||||
// What a container built from a current base image says. The whole
|
||||
// point of baking them: this must not re-run apt.
|
||||
assert_eq!(
|
||||
classify_library_check("All system dependencies are installed.\n", 0),
|
||||
LibraryState::Present
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_older_image_without_them_is_detected_and_named() {
|
||||
let v = classify_library_check(
|
||||
"Missing system dependencies (12):\n libnss3\n libgbm1\n",
|
||||
1,
|
||||
);
|
||||
match v {
|
||||
LibraryState::Missing(d) => assert!(d.contains("(12)"), "{}", d),
|
||||
other => panic!("expected Missing, got {:?}", other),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unrecognised_platform_installs_rather_than_reporting_success() {
|
||||
// Playwright prints this and exits **0** having installed nothing, so a
|
||||
// check that trusted the exit code would skip the apt step on exactly
|
||||
// the container that needs it.
|
||||
let v = classify_library_check(
|
||||
"Cannot install dependencies for ubuntu24.04-riscv64 with Playwright 1.62.1!\n",
|
||||
0,
|
||||
);
|
||||
assert!(matches!(v, LibraryState::Unknown(_)), "{:?}", v);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_check_that_could_not_run_installs_rather_than_guessing() {
|
||||
// e.g. apt cannot resolve a package because the image's package index
|
||||
// was cleaned and the package is genuinely absent.
|
||||
let v = classify_library_check("E: Unable to locate package libgbm1\n", 100);
|
||||
match v {
|
||||
LibraryState::Unknown(d) => assert!(d.contains("100"), "{}", d),
|
||||
other => panic!("expected Unknown, got {:?}", other),
|
||||
}
|
||||
// Present contributes nothing to the progress line; the other two must
|
||||
// explain themselves, because the reason is shown to the user.
|
||||
assert_eq!(LibraryState::Present.detail(), "");
|
||||
assert!(!LibraryState::Unknown("why".into()).detail().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_successful_launch_and_page_load_is_reported_as_working() {
|
||||
let v = parse_launch_output(concat!(
|
||||
|
||||
@@ -310,8 +310,11 @@ function missingParts(d: PlaywrightDetection | null): string[] {
|
||||
* The old pane printed npm commands here and left the rest to the user. The
|
||||
* result, verified with a real one: an `@playwright/mcp` install that could
|
||||
* never satisfy this pane, a global install that hit EACCES, a Chromium that
|
||||
* downloaded and then would not start because the image ships none of its
|
||||
* shared libraries, and a long tail of commands after that.
|
||||
* downloaded and then would not start because the image shipped none of its
|
||||
* shared libraries, and a long tail of commands after that. Current base images
|
||||
* bake those libraries in, so that last one is fixed at the source — but a
|
||||
* project keeps its original base image until it is migrated, so the install
|
||||
* action still handles a container that lacks them.
|
||||
*/
|
||||
function Setup({
|
||||
detection,
|
||||
@@ -386,11 +389,13 @@ function Setup({
|
||||
title="2. A browser to drive"
|
||||
detail={
|
||||
<>
|
||||
Both install the system libraries first — the base image ships none of them,
|
||||
which is why a browser can download successfully and then refuse to start —
|
||||
and both end by actually launching the browser to prove it works. Browsers
|
||||
land in <Code>~/.cache/ms-playwright</Code>, which is on the home volume, so
|
||||
they survive container recreation and are only lost on a project Reset.
|
||||
Both check the system libraries a browser links against first. Current base
|
||||
images ship them, so that step is normally skipped; a container built from an
|
||||
older image gets them installed with apt, which is the difference between a
|
||||
browser that downloads successfully and one that also starts. Both end by
|
||||
actually launching the browser to prove it works. Browsers land in{" "}
|
||||
<Code>~/.cache/ms-playwright</Code>, which is on the home volume, so they
|
||||
survive container recreation and are only lost on a project Reset.
|
||||
</>
|
||||
}
|
||||
done={browsers.length > 0 || chrome !== null}
|
||||
|
||||
Reference in New Issue
Block a user