Skip to content

D3 · Pinned toolchain, bootstrap and doctor

Done 2026-10-27 · one pinned list, one install script, one doctor · doctor passes in a cloud session (sind and kind skipped: cgroup v1) and in CI with every layer · cold bootstrap 25 s here, 55 s in CI · 4 questions for Thomas

D3 figure: dev/versions.env, with two hash-locked dependency files, feeds dev/bootstrap.sh --ci, which installs the tools in four parallel jobs into ~/.local/share/spectio (plus image pulls with --ci) and runs in a cloud session, a CI runner and on a laptop. Cold runs took 25 s in a cloud session and 56 s on a CI runner, against a 5-minute budget. dev/doctor.sh then checks six layers: in the cloud session sind and kind are skipped because of cgroup v1 and the other four pass; on the CI runner all six pass, and in strict mode a skip fails. The first failure exits 1 and names the tool. D3 figure: dev/versions.env, with two hash-locked dependency files, feeds dev/bootstrap.sh --ci, which installs the tools in four parallel jobs into ~/.local/share/spectio (plus image pulls with --ci) and runs in a cloud session, a CI runner and on a laptop. Cold runs took 25 s in a cloud session and 56 s on a CI runner, against a 5-minute budget. dev/doctor.sh then checks six layers: in the cloud session sind and kind are skipped because of cgroup v1 and the other four pass; on the CI runner all six pass, and in strict mode a skip fails. The first failure exits 1 and names the tool.

The timings and ticks come from a cloud session and from GitHub Actions run 40523940610, both on 2026-10-27. The later run 40526621317 gave the same result (bootstrap 55 s). The PNGs were rendered with the local Virgil font, which is not committed (inbox #1).

What changed

  • dev/versions.env: one exact pin per tool, each with its source. Every binary and tarball has a sha256, and every image the doctor runs has a digest. It also holds the on-demand pins (mermaid-cli, mkdocs, mkdocs-material, depictio), which bootstrap never installs. The values are listed in §1 and in the file.
  • Two hash locks:
    • dev/snakemake.lock.txt: Snakemake 9.27.1, the SLURM and Kubernetes executor plugins and their 72 packages, each with --hash=sha256:;
    • dev/npm/package.json and package-lock.json: Playwright 1.63.0 with integrity hashes.
  • dev/bootstrap.sh [--ci]: installs the toolchain in four parallel jobs, each with its own log, and is idempotent. Downloads are checked before they are moved into place. With --ci it also starts dockerd when nothing else will, and pre-pulls the doctor's images by digest, from mirror.gcr.io first.

  • dev/doctor.sh:

    • checks each tool's version against the pin;
    • runs one smoke test per layer: docker, sind, kind, nextflow, snakemake and playwright;
    • exits 1 on the first failure, printing doctor: FAILED: <tool>.

    sind and kind are skipped, with the reason, where Docker runs on cgroup v1. SPECTIO_DOCTOR_STRICT=1 turns a skip into a failure. - dev/lib.sh: the paths and the Docker helpers shared by both scripts. - .github/workflows/dev-env.yml: bootstrap timed, then the doctor in strict mode on ubuntu-24.04; the output goes to the job summary. It runs on pushes that touch dev/, weekly, and by hand. - Spec dev-environment-and-testing.md: - §1: the table, a new "Bootstrap" paragraph and the rewritten "Checks"; - §2: the cloud environment settings, with the network list, environment variables and setup script; - §4.2: answered; - §7.2: a note that versions.env now exists; - §11: the dev-env.yml row. - Other files: - decision log, "D3: pinned toolchain, bootstrap and doctor"; - review record; - inbox entries #41 to #44.

Why

The dev spec asked for one bootstrap script that runs in the cloud setup script, in CI and on a laptop, and for a doctor that proves each layer works. Spike D2 measured what a cloud session can run. Its findings set the design:

  • sind and kind need cgroup v2, so they are proven in CI;
  • dockerd is not running at session start;
  • Docker Hub rate-limits anonymous pulls;
  • releases/latest URLs are refused.

Prompt: next-phase.md · D3. The step's own choices are in the decision log.

Evidence

In this cloud session (Docker on cgroup v1). This is the exit check, run after the last code change (51c759a):

$ bash dev/doctor.sh
spectio doctor: <repo>, pins from dev/versions.env
--- versions
ok    docker            29.9.2 (cgroup v1)
ok    java              21.0.12.1
ok    uv                0.11.32
ok    python            3.13.14
ok    sqlite            3.53.1
ok    node              24.22.1
ok    playwright        1.63.0
ok    nextflow          26.10.4
ok    nextflow-25.10.9  25.10.9
ok    nextflow-25.04.8  25.04.8
ok    snakemake         9.27.1
ok    smk-slurm         2.8.0
ok    smk-kubernetes    0.5.1
ok    sind              0.15.0
ok    kind              0.33.0
ok    kubectl           1.37.1
ok    helm              4.3.1
--- smoke tests
ok    docker            docker run hello-world (0s)
skip  sind              Docker on cgroup v1; sind and kind need cgroup v2 (spike D2), so they run in CI
skip  kind              Docker on cgroup v1; sind and kind need cgroup v2 (spike D2), so they run in CI
ok    nextflow          synthetic pipeline, test,local (14s)
ok    snakemake         dry run of the synthetic Snakefile (2s)
ok    playwright        chromium 153.0.8010.12 opened about:blank (0s)
doctor: all layers passed (2 skipped)
$ echo $?
0

Bootstrap timings in this session:

Run Time
first cold --ci run 25 s (sind and kind images not pulled on cgroup v1)
cold again, with uv's and npm's caches warm 8 s
second run, nothing to do 1 s, bootstrap: done in 1s

Deliberately broken layers (exit 1, naming the tool):

$ PLAYWRIGHT_BROWSERS_PATH=/nonexistent bash dev/doctor.sh    # no Chromium
FAIL  playwright        Chromium did not open about:blank
doctor: FAILED: playwright                                     # exit 1
$ SPECTIO_DOCTOR_STRICT=1 bash dev/doctor.sh                   # a skip is a failure
FAIL  sind              skipped, and SPECTIO_DOCTOR_STRICT=1: Docker on cgroup v1; sind and kind need cgroup v2 (spike D2), so they run in CI
doctor: FAILED: sind                                           # exit 1

A tampered lock is refused. Two wrong hashes for annotated-types in a copy of dev/snakemake.lock.txt:

  ╰─▶ Hash mismatch for `annotated-types==0.8.0`
      Expected:
        sha256:0000000000000000000000000000000000000000000000000000000000000000
        sha256:1111111111111111111111111111111111111111111111111111111111111111
      Computed:
        sha256:f072f4d804ea359e4eaf198b1af7a8b0943881a87f31bb764f8bf219bb9419e0
uv pip sync exit=1

In GitHub Actions, run 40526621317 on 51c759a (ubuntu-24.04, cgroup v2, strict). It passed.

bootstrap: done in 55s. Every job reported ok: the four installs and the five images.

spectio doctor: /home/runner/work/spectio/spectio, pins from dev/versions.env
--- versions
ok    docker            28.0.4 (cgroup v2)
ok    java              21.0.13
ok    uv                0.11.32
ok    python            3.13.14
ok    sqlite            3.53.1
ok    node              24.22.1
ok    playwright        1.63.0
ok    nextflow          26.10.4
ok    nextflow-25.10.9  25.10.9
ok    nextflow-25.04.8  25.04.8
ok    snakemake         9.27.1
ok    smk-slurm         2.8.0
ok    smk-kubernetes    0.5.1
ok    sind              0.15.0
ok    kind              0.33.0
ok    kubectl           1.37.1
ok    helm              4.3.1
--- smoke tests
ok    docker            docker run hello-world (3s)
ok    sind              cluster up, sbatch --wait --wrap hostname ran on worker-0 (15s)
ok    kind              cluster up, Job completed (39s)
ok    nextflow          synthetic pipeline, test,local (25s)
ok    snakemake         dry run of the synthetic Snakefile (2s)
ok    playwright        chromium 153.0.8010.12 opened about:blank (2s)
doctor: all layers passed (0 skipped)

The first green run before the review fixes was 40523940610, with bootstrap at 56 s.

Static checks: shellcheck -x dev/bootstrap.sh dev/doctor.sh dev/lib.sh is clean.

review-code:

CODE  step=D3  review=approve b/M/m=0/0/5  security=approve b/M/m=0/1/4  rounds=2  simplify=changed -1/+0  tests=pass
  • code-reviewer: approve (0/0/4), then approve (0/0/1).
  • security-reviewer: changes (0/1/3), then approve (0/0/1). The major, SEC-D3-01, was that the dependency trees were not locked; both are now hash-locked.
  • code-simplifier: removed one orphan export; tests pass.
  • Findings and their status: reviews/code/D3.md.

Figures and media

  • The dev-toolchain sketch, light and dark, at the top of this page: python3 tools/diagrams/dev_toolchain.py.
  • The dev-doctor cast, a real run of the scripts in this session: python3 tools/media/cast.py tools/media/casts/dev-doctor.yaml. It shows three steps:
    1. a second bootstrap is a no-op;
    2. the doctor passes, with sind and kind skipped;
    3. a broken Playwright layer exits 1 naming the tool.

Findings and open questions

  • For Thomas:
    • #43: enter the §2 settings in the cloud environment (network list, environment variables, setup script). Until then each session runs bash dev/bootstrap.sh --ci itself.
    • #41: switch the Codespaces setup.sh to dev/bootstrap.sh.
    • #42: move the synthetic matrix and the required check names to Nextflow 26.10.4.
    • #44 (assumed): Linux x86_64 only, and Node 24 now ahead of the image's Node 22 on PATH.
  • Not hashed by us (§1): the uv-managed CPython [to be validated] and Playwright's Chromium where it has to be downloaded (CI).
  • Accepted, not changed: the doctor starts checked images by tag, not by digest (SEC-D3-03), so that mirror pulls keep working.
  • For the D2 re-probe:
    • narrow the wildcard hosts of the §2 list (SEC-D3-04);
    • check where Nextflow plugin downloads redirect;
    • check whether the repository is cloned before the setup script runs.
  • D5's request, partly done: uv, mkdocs, mkdocs-material and Playwright are now pinned in versions.env. The dev-container image and its features are not; #41 covers them.