Skip to content

P3 · M3 Served safely

Running since 2026-10-30 · milestone M3 of step P3, Tasks 11–16 of P1 and G6's start-text part (P2 §3) · Task 11 done on autopilot/P3-t11 (spectio serve on 127.0.0.1, boot and index rebuild), local-only order of inbox #80 (a), its PR into autopilot/integration opening with this page · its review-code: approve after 3 rounds · Tasks 12–16 to do

P3 Task 11: spectio serve, one process on 127.0.0.1, the app, the runner loop and the index; every refusal exits 2 before anything is left behind. Top row, the start in five steps: 1, loopback only: no --host flag, [server] host exists only to be refused by name, and the runs root must be yours and not writable by others (dev-env req. 10); refused: a host other than 127.0.0.1, a runs root not yours or loose. 2, one server per node: serve.lock in the node-local database directory, held with flock until the process ends; a free lock means no server here, so a stale state file is a dead server's; refused: another spectio serve, naming its pid. 3, the recorded port, bound before the boot, default 8080 or --port at init; refused: a taken port, naming its owner's uid read from /proc/net/tcp; Spectio never picks another. 4, the state file server.json in the data directory, created with O_EXCL, holding host, port, pid, db dir and start time; this node's is a dead server's and is replaced; removed at a clean stop if it is still ours; refused: another node's, unless spectio serve --force. 5, boot then serve: the database, the tenancy rows, the index rebuilt, one runner tick that adopts heads and replays spools, then the runner thread and uvicorn on the bound socket serving GET / as a plain page; no data route yet, sign-in comes with Task 12. Below, inside the boot, rebuild_index adds the run directories the index lacks, oldest first: a, walk under req. 10, a new run is read, one already in the index is passed over, a name that is not a run ID or a symlinked, loose or foreign directory is skipped and counted; b, caps of 1000 runs and 64 MiB of manifests per boot, the rest waits for the next boot; c, the manifest .spectio/run.json, strict, extra fields refused, its run ID must be the directory's name, paths derived, never read; d, the launch directory decides the state: launch-failed.json gives failed with its class, else launch_error, and with no message 'the launch failed, index rebuilt' (fixed, CR-P3T11-06); no head.pid gives cancelled under a cancel marker, else failed head_lost; a head.pid gives queued, or cancelling under the marker, and the launch keeps pid.start so the runner adopts the head and replays its spool from 0. One transaction per run with its launch: an interrupted boot leaves whole runs only, and the next boot adds the rest. The workflow: a location starting with a slash is local, any other is git, one workflows row per location and engine (fixed, CR-P3T11-04). At the bottom, the start text in local mode, and the two executors: local starts the runner thread with the server, Ctrl-C or SIGTERM is a clean stop undone in reverse order; slurm adds an ssh -N -L tunnel line and starts no runner loop until the tmux launcher, P1 Task 10 P3 Task 11: spectio serve, one process on 127.0.0.1, the app, the runner loop and the index; every refusal exits 2 before anything is left behind. Top row, the start in five steps: 1, loopback only: no --host flag, [server] host exists only to be refused by name, and the runs root must be yours and not writable by others (dev-env req. 10); refused: a host other than 127.0.0.1, a runs root not yours or loose. 2, one server per node: serve.lock in the node-local database directory, held with flock until the process ends; a free lock means no server here, so a stale state file is a dead server's; refused: another spectio serve, naming its pid. 3, the recorded port, bound before the boot, default 8080 or --port at init; refused: a taken port, naming its owner's uid read from /proc/net/tcp; Spectio never picks another. 4, the state file server.json in the data directory, created with O_EXCL, holding host, port, pid, db dir and start time; this node's is a dead server's and is replaced; removed at a clean stop if it is still ours; refused: another node's, unless spectio serve --force. 5, boot then serve: the database, the tenancy rows, the index rebuilt, one runner tick that adopts heads and replays spools, then the runner thread and uvicorn on the bound socket serving GET / as a plain page; no data route yet, sign-in comes with Task 12. Below, inside the boot, rebuild_index adds the run directories the index lacks, oldest first: a, walk under req. 10, a new run is read, one already in the index is passed over, a name that is not a run ID or a symlinked, loose or foreign directory is skipped and counted; b, caps of 1000 runs and 64 MiB of manifests per boot, the rest waits for the next boot; c, the manifest .spectio/run.json, strict, extra fields refused, its run ID must be the directory's name, paths derived, never read; d, the launch directory decides the state: launch-failed.json gives failed with its class, else launch_error, and with no message 'the launch failed, index rebuilt' (fixed, CR-P3T11-06); no head.pid gives cancelled under a cancel marker, else failed head_lost; a head.pid gives queued, or cancelling under the marker, and the launch keeps pid.start so the runner adopts the head and replays its spool from 0. One transaction per run with its launch: an interrupted boot leaves whole runs only, and the next boot adds the rest. The workflow: a location starting with a slash is local, any other is git, one workflows row per location and engine (fixed, CR-P3T11-04). At the bottom, the start text in local mode, and the two executors: local starts the runner thread with the server, Ctrl-C or SIGTERM is a clean stop undone in reverse order; slurm adds an ssh -N -L tunnel line and starts no runner loop until the tmux launcher, P1 Task 10

One Progress page per milestone, opened with its first PR and closed with its done proof (P2 §0). This page covers M3, "Served safely" (P2 §3). M2 is on P3-M2.

Tasks

Task P1 Depends on Branch Status
11 · spectio serve, bind (§15.3 req. 1) and boot Task 11, G6's test_serve_local_start_text 3, 8, 9, 10 (10 is not built yet: inbox #85) autopilot/P3-t11 done; PR #93 into autopilot/integration
12 · Token link, secrets and sessions (req. 2) Task 12 11 to do
13 · Session cookie (req. 3) Task 13 12 to do
14 · Host allow-list (req. 4) Task 14 11 to do
15 · File modes (req. 5) Task 15 12 to do
16 · Writes need Origin and JSON (req. 7) Task 16 12, 14 to do
  • Task 11 (its exit check, P1 Task 11 Step 4: 31 passed, see Evidence)
  • Tasks 12–16, and G6's test_serve_local_same_requirements (it needs the ticket of Task 12)
  • M3 done proof: uv run pytest packages/spectio-server/tests/security packages/spectio-server/tests/test_serve.py -v all green (P2 §3 item 3)

What changed

Task 11 · spectio serve, bind and boot

Branch autopilot/P3-t11, base c16ea57 (Task 9 merged), head fd2ae94 before the figure and this page; 5 commits, 17 files, +1524 −6 (+1408 −4 in packages/). The bind, the state file, where data lives and the rebuild are owned by dev-env §15.3; the process and the boot order by architecture §7.1 and §7.2; the head's detach by schedulers §3.1.1. This list names the files.

  • The server (packages/spectio-server/src/spectio_server/; new dependencies fastapi>=0.115,<1 and uvicorn>=0.32,<1, locked at fastapi 0.162.0, starlette 1.10.0, uvicorn 0.54.0, h11 0.16.0):

    • serve.py (308 lines): started(settings, force) makes the start in the order of the figure. Each refusal is a ServeError, exit 2:

      1. a bind address other than 127.0.0.1, and a runs root that is not the user's or is writable by others (req. 10);
      2. <db dir>/serve.lock, an flock held until the process ends, refused with the holder's PID;
      3. the recorded port, bound on 127.0.0.1 before the boot; a taken port is refused with its owner's uid from /proc/net/tcp, and no other port is tried (req. 9);
      4. <data dir>/server.json, written with O_EXCL (P1 Assumption 9): host, port, pid, db_dir, started_at.

      Then boot(): the database, the tenancy rows (ensure_tenancy), the rebuild, one runner tick, the runner thread and uvicorn with proxy headers off. serve() prints the start text and serves until Ctrl-C or SIGTERM. Everything is undone in reverse order, and server.json is removed if it is still this server's. - rebuild.py (252 lines): rebuild_index(writer, runs_root) -> Rebuilt(added, skipped, deferred). It adds the run directories the index lacks, oldest first, one transaction per run with its launch. It walks each directory under req. 10, parses run.json strictly, and derives the state from the launch directory. Caps: 1000 runs and 64 MiB of manifests per boot. - sink.py (135 lines): StoreSink, the server's RunnerSink (P1 Task 9) through the one writer. Commands are leased for 60 s and delivered again after; launch directories come from the runs root and the run ID, never from a row. - app.py (29 lines): create_app(settings), one route, GET /, a plain page with the version; docs_url, redoc_url and openapi_url off. - bootstrap.py: _tenancy becomes the public ensure_tenancy, used by spectio init and by the boot. - The command line (packages/spectio): - cli/serve.py (33 lines) and cli/main.py: spectio serve [--force]. No --host and no --detach. A bad setting, a missing server package or a ServeError exits 2. - config.py: LOOPBACK = "127.0.0.1", and [server] host with SPECTIO_SERVER_HOST. They exist only so that another value is refused by name; render_toml never writes [server]. - Import contracts (pyproject.toml): one more ignored edge, spectio.cli.serve -> spectio_server.serve, in the two contracts of architecture §6. Still 4 contracts. - Tests: tests/security/test_req1_bind.py (3 tests, 10 cases) and tests/test_serve.py (21 tests), with tests/conftest.py (local-mode settings in a private HOME). Two tests chown to another uid and run only as root. - Specs (P1 Process: the owning spec changes in the same PR): - dev-env §15.3: three dated notes, on the state file (One login node per server), on the rebuild (Where data lives), and on req. 1 with the placeholder app until Task 12. - schedulers §3.1.1, Detach: a dated note, systemd-run --user --scope is not built in 0.0.1. - Each is marked (assumed — pending Thomas, inbox #85). No section number changed. - Decision log: 2026-10-30 — spectio serve in the local-only order. - Inbox: #85 new (commit ebaa6e4). - Review record: docs/reviews/code/P3-t11.md. - Docs of this task: this page, the p3-t11-serve figure (tools/diagrams/p3_t11_serve.py, by diagram-artist, commit 9f67154), and its entry in docs/assets/media.yaml.

Differences from the plan's interface (Task 11)

P1 Task 11 gives the interface. The code differs as follows; inbox #85 holds the rules. P1's text changes in the next /apply-findings pass.

Plan Code Why
serve(settings) -> None serve(settings, *, force=False); also started(settings, *, force), a context manager that yields the bound server, and boot(settings, db_dir) -> Booted --force (P1 Assumption 9); the tests start and stop the server in a thread
rebuild_index(writer, runs_root) -> int -> Rebuilt(added, skipped, deferred), with keyword caps max_runs, max_bytes the req. 10 tests count what is skipped and what waits for the next boot
rebuild when the database was just created; reconcile at every boot; a completion mark in the last transaction every boot adds the missing run directories (all of them for a new database), one transaction per run with its launch; no mark inbox #85 rule 5: whole runs only, so the next boot finishes an interrupted one
adopt the heads, replay the spools, start the runner loop as planned on a local environment; on a SLURM environment the runner loop is not started inbox #85 rule 1: Task 10 (tmux) is not built
server key in the node-local directory; start text with the token link no server key, no link; the start text gives http://127.0.0.1:<port>/ and "Sign-in comes with P1 Task 12" inbox #85 rule 3: Task 12 is the key's only user
spectio serve --detach; test_detach_starts_tmux_session not built inbox #85 rule 2: the session spectio-server is Task 10's
state file ~/.spectio/server.json <data dir>/server.json; one that names this node is replaced inbox #85 rule 8: the lock was free, so its server is dead
test_rebuild_skips_symlinked_foreign_or_loose_dirs, test_runs_root_loose_refused the "other uid" cases split out into test_rebuild_skips_foreign_dir and test_runs_root_foreign_refused, skipped unless root CR-P3T11-01 (r1): chown to another uid fails in CI
G6: test_serve_local_start_text, test_serve_local_same_requirements the first written; the second waits for Task 12 the second checks that local mode needs the ticket
tests: 13 24 tests, 31 cases one test per refusal and per review fix

Why

Prompt: next-phase.md · P3. Step: P3 in docs/playbooks/pipeline.yaml (wave W5, kind: build). P2 §3 starts Task 11 after Tasks 3, 8, 9 and 10. In the order of inbox #80 (a), assumed, it runs before Task 10, for a visible local slice sooner.

  • One process for everything. The UI, the API, the runner and SQLite run in one spectio serve per user (architecture §7.1). Task 11 builds that process and its start, so the runner loop of Task 9 runs for the first time from a command.
  • The port is the only door. On a shared login node every Unix user can reach a loopback port. 0.0.1 binds 127.0.0.1 only (req. 1). The real guards come next: the session (Task 12) and the Host allow-list (Task 14). Until then the app serves no data (SEC-P3T11-04 (r1), SEC-P3T11-05 (r2)).
  • The database can vanish. It lives in a node-local temporary directory (dev-env §15.3, Where data lives), so a purge or a start on another node loses it. The run directories are the source of truth, and the boot rebuilds the index from them. The run directories sit on a shared disk, so each one is read as untrusted input (req. 10).
  • One server per user and node, found from any node. serve.lock stops a second server on the same node. server.json in the shared home tells another node, and later spectio open, where the server runs (P1 Assumption 9).

Evidence

Task 11 · spectio serve, bind and boot

Files changed by Task 11, against its base:

$ git log --oneline c16ea57..fd2ae94
fd2ae94 refactor(P3-t11): simplify ENGINES, derived from the Engine literal (CR-P3T11-05)
b9b94f2 fix(server): rebuild infers workflow source from location; default launch failure message
62bb506 test(server): root-only guard on the foreign-uid serve tests (CR-P3T11-01)
ebaa6e4 docs(inbox): #85, what spectio serve assumes in the local-only order
d0f716b feat(server): spectio serve bound to loopback
$ git diff --stat c16ea57 fd2ae94 | tail -1
 17 files changed, 1524 insertions(+), 6 deletions(-)

d0f716b carries Release-As: 0.0.1.

The exit check, re-run by doc-writer on fd2ae94 (P1 Task 11 Step 4; uv's UV_NATIVE_TLS warning and the percentages cut):

$ uv run --locked pytest packages/spectio-server/tests/security/test_req1_bind.py packages/spectio-server/tests/test_serve.py -v
collecting ... collected 31 items

packages/spectio-server/tests/security/test_req1_bind.py::test_req1_binds_loopback_only PASSED
packages/spectio-server/tests/security/test_req1_bind.py::test_req1_refuses_other_bind[toml-0.0.0.0] PASSED
packages/spectio-server/tests/security/test_req1_bind.py::test_req1_refuses_other_bind[toml-::] PASSED
packages/spectio-server/tests/security/test_req1_bind.py::test_req1_refuses_other_bind[toml-localhost] PASSED
packages/spectio-server/tests/security/test_req1_bind.py::test_req1_refuses_other_bind[toml-192.0.2.1] PASSED
packages/spectio-server/tests/security/test_req1_bind.py::test_req1_refuses_other_bind[env-0.0.0.0] PASSED
packages/spectio-server/tests/security/test_req1_bind.py::test_req1_refuses_other_bind[env-::] PASSED
packages/spectio-server/tests/security/test_req1_bind.py::test_req1_refuses_other_bind[env-localhost] PASSED
packages/spectio-server/tests/security/test_req1_bind.py::test_req1_refuses_other_bind[env-192.0.2.1] PASSED
packages/spectio-server/tests/security/test_req1_bind.py::test_req1_has_no_host_flag PASSED
packages/spectio-server/tests/test_serve.py::test_rebuild_from_run_dirs PASSED
packages/spectio-server/tests/test_serve.py::test_boot_reconciles_missing_runs PASSED
packages/spectio-server/tests/test_serve.py::test_rebuild_skips_symlinked_or_loose_dirs PASSED
packages/spectio-server/tests/test_serve.py::test_rebuild_skips_foreign_dir PASSED
packages/spectio-server/tests/test_serve.py::test_rebuild_run_id_must_match_dir PASSED
packages/spectio-server/tests/test_serve.py::test_rebuild_ignores_manifest_session_and_paths PASSED
packages/spectio-server/tests/test_serve.py::test_rebuild_caps PASSED
packages/spectio-server/tests/test_serve.py::test_runs_root_loose_refused PASSED
packages/spectio-server/tests/test_serve.py::test_runs_root_foreign_refused PASSED
packages/spectio-server/tests/test_serve.py::test_second_serve_refused PASSED
packages/spectio-server/tests/test_serve.py::test_state_file_names_other_node PASSED
packages/spectio-server/tests/test_serve.py::test_stale_state_file_of_this_node_replaced PASSED
packages/spectio-server/tests/test_serve.py::test_port_taken_refuses PASSED
packages/spectio-server/tests/test_serve.py::test_cli_refusal_exits_2 PASSED
packages/spectio-server/tests/test_serve.py::test_serve_local_start_text PASSED
packages/spectio-server/tests/test_serve.py::test_serve_slurm_start_text_and_no_runner PASSED
packages/spectio-server/tests/test_serve.py::test_rebuild_one_engine_per_location PASSED
packages/spectio-server/tests/test_serve.py::test_rebuild_failed_launch_without_message PASSED
packages/spectio-server/tests/test_serve.py::test_sink_leases_once_until_the_lease_ends PASSED
packages/spectio-server/tests/test_serve.py::test_sigterm_stops_cleanly PASSED
packages/spectio-server/tests/test_serve.py::test_missing_data_dir_refused PASSED

============================== 31 passed in 6.20s ==============================

This session runs as uid 0, so test_rebuild_skips_foreign_dir and test_runs_root_foreign_refused ran. In CI, as a non-root user, they are skipped (CR-P3T11-01 (r1); the gap is FU-P3-16).

test_rebuild_from_run_dirs is the plan's rebuild test. Its five runs are two finished ones (the T2 Nextflow and Snakemake spools), one live head (a sleep in its own session), one cancelled run and one failed launch. It boots, deletes the database, and boots again. Both times it gets the T2 goldens (65 and 67 succeeded tasks), the live head adopted and never signalled, cancelled, failed with engine_missing and its message, and no duplicate event_id.

A real spectio serve, by doc-writer on fd2ae94, from the worktree's .venv, in a fresh 0700 demo home (shown as ~) with its own TMPDIR. The node is the cloud session's vm, running as root (uid 0). The script (serve-demo.sh, kept in the session's scratch directory) replaces the PIDs and the start time with <pid> and <time>; the rest is verbatim:

$ spectio init --local --port 18080
warning: nextflow not found on the login shell's PATH: give it with --nextflow
warning: snakemake not found on the login shell's PATH: give it with --snakemake
Wrote ~/.spectio/spectio.toml; database in $TMPDIR/spectio-0; runs under ~/spectio-runs; port 18080
warning: nf-spectio not unpacked: …/packages/spectio/src/spectio/bridges/nf-spectio-0.0.1 is missing: build the bridges or set SPECTIO_BRIDGES_DIR
exit 0
$ spectio serve &
Spectio 0.0.1 is running on vm, port 18080.
Open this page in a browser on this machine:  http://127.0.0.1:18080/
Sign-in comes with P1 Task 12: until then this build is local-only.
INFO:     127.0.0.1:49268 - "GET / HTTP/1.1" 200 OK
$ curl -s http://127.0.0.1:18080/
Spectio 0.0.1 is running.
Sign-in comes with P1 Task 12 and the screens with Task 19: this build is local-only.
$ curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:18080/docs
404
$ awk '$2 ~ /:46A0$/ && $4 == "0A" {print $2}' /proc/net/tcp /proc/net/tcp6   # LISTEN on port 18080
awk: cannot open "/proc/net/tcp6" (No such file or directory)
0100007F:46A0
$ stat -c "%a %n" ~/.spectio/server.json $TMPDIR/spectio-$(id -u) $TMPDIR/spectio-$(id -u)/serve.lock
600 ~/.spectio/server.json
700 $TMPDIR/spectio-0
600 $TMPDIR/spectio-0/serve.lock
$ cat ~/.spectio/server.json
{"host": "vm", "port": 18080, "pid": <pid>, "db_dir": "$TMPDIR/spectio-0", "started_at": "<time>"}
$ spectio serve        # a second one, same node
spectio serve: another spectio serve (pid <pid>) runs on this node: it holds $TMPDIR/spectio-0/serve.lock
exit 2
$ SPECTIO_SERVER_HOST=0.0.0.0 spectio serve
spectio serve: 0.0.1 binds to 127.0.0.1 only: remove host = '0.0.0.0' from [server] in spectio.toml, or unset SPECTIO_SERVER_HOST
exit 2
$ kill -TERM <pid of the first serve>; wait
exit 0
$ ls ~/.spectio
spectio.toml
$ cat serve.err
INFO alembic.runtime.migration: Context impl SQLiteImpl.
INFO alembic.runtime.migration: Will assume non-transactional DDL.
INFO alembic.runtime.migration: Context impl SQLiteImpl.
INFO alembic.runtime.migration: Will assume non-transactional DDL.
INFO spectio_server.serve: index reconciled: 0 runs added
INFO:     Started server process [<pid>]
INFO:     Shutting down
INFO:     Finished server process [<pid>]

Five things to read from it:

  • The one listener is 0100007F:46A0, that is 127.0.0.1:18080. This machine has no IPv6 table, so nothing listens on :: either.
  • The generated docs answer 404.
  • server.json records the database directory the start used (this closes SEC-P3T08-07 of Task 8's review).
  • The SIGTERM stop exits 0 and removes server.json.
  • spectio init had already created the database, so this first serve "reconciled" the index instead of rebuilding it. The runs root was empty, so it added 0 runs.

Default gate, re-run by doc-writer on fd2ae94 (Python 3.13.14):

$ uv run --locked pytest -q | tail -1
333 passed, 15 deselected in 45.17s
$ uv run --locked ruff check .
All checks passed!
$ uv run --locked ruff format --check .
57 files already formatted
$ uv run --locked mypy packages
Success: no issues found in 56 source files
$ uv run --locked lint-imports | tail -1
Contracts: 4 kept, 0 broken.
$ uv lock --check
Resolved 43 packages in 2ms

The 15 deselected are the engines tests, as after Task 9, which left 302 passed.

Test-first, from the review record: both round-2 fixes (b9b94f2) were written as tests first, and both tests failed before the fix. The git URL got a second workflows row; the message was stored as 'None'. The simplifier's change (fd2ae94, −2 +2) kept the gate green: 333 passed (record).

Review (record). Finding IDs repeat across rounds, so each carries its round:

  • Round 1, on ebaa6e4: code-reviewer changes. The major, CR-P3T11-01 (r1): two tests chown to another uid, which fails for CI's non-root user. Fixed in 62bb506. security-reviewer approved with 8 minors.
  • Round 2, on 62bb506: both approve. code-reviewer gave 5 minors and 1 simplify, security-reviewer 5 minors. CR-P3T11-04 (r2) (a git workflow with no revision taken as local) and CR-P3T11-06 (r2) (the message 'None') were fixed in b9b94f2.
  • Round 3, on b9b94f2: both approve. security-reviewer gave one minor: rebuilt git locations are not checked against the import rule (SEC-P3T11-01 (r3)).
  • code-simplifier, after round 3: one change, fd2ae94 (CR-P3T11-05 (r2), ENGINES = get_args(Engine)).
  • Open: see Findings and open questions.
CODE  step=P3-t11  review=approve b/M/m=0/1/8  security=approve b/M/m=0/0/14  rounds=3  simplify=changed -2/+2  tests=pass

The counts add up the three rounds per reviewer; CR-P3T11-05 (r2), a simplify, is not counted as a minor.

Exit check. P1 Task 11's Step 4 passes: 31 passed, above.

Docs gate, re-run after the spectio-serve cast was added, with the figure and the cast in place:

$ python3 tools/docs/check.py
build: progress hook output checked on 3 pages
mermaid: 5 blocks checked
diagrams: 42 scripts re-run, 84 SVGs compared
media: 196 outputs in the manifest
progress: 52 pages checked
docs gate: 0 failure(s)
$ uvx --with mkdocs-material mkdocs build --strict -d /tmp/spectio-site
INFO    -  Documentation built in 8.62 seconds
$ echo $?
0

Figures and media

  • p3-t11-serve sketch, light and dark, at the top of this page. It is drawn by diagram-artist with python3 tools/diagrams/p3_t11_serve.py, from serve.py, rebuild.py, app.py, config.py and cli/main.py at fd2ae94. It shows:
    • the start in five numbered steps (loopback only, serve.lock, the recorded port, server.json, boot then serve), each with what it refuses;
    • inside the boot, rebuild_index in four lettered steps: the walk under req. 10, the per-boot caps, the strict manifest, and the state that the launch directory decides;
    • the two fixes of round 2, marked where they apply (CR-P3T11-04, CR-P3T11-06);
    • the start text in local mode, and what changes on a SLURM environment.

  • The spectio-serve cast, the CLI flow of Task 11, as documentation-system §3.1 asks of a build step. It runs the real CLI of fd2ae94 from the checkout's .venv, under a demo host name in its own UTS namespace: unshare --uts sh -c 'hostname demo-hpc-login1 && python3 tools/media/cast.py tools/media/casts/spectio-serve.yaml'. The setup:

    • the cast's temporary directory is a demo HOME, shown as ~, with a demo TMPDIR inside it, shown as $TMPDIR;
    • the node is demo-hpc-login1, not the session's vm;
    • nextflow and snakemake are the two fake scripts of the spectio-init-doctor cast;
    • init's warning that the nf-spectio bridge is not unpacked is dropped (the bridges are not built, and serve does not use them);
    • the first serve runs in the background with its output in two files, because the cast captures each command's output.

    It shows four steps: 1. spectio init --local --port 18080; 2. spectio serve: the start text naming the node, the port and the page; GET /; then server.json; 3. a second spectio serve on the same node, refused with the PID of the first, exit 2; 4. kill -TERM with the PID from server.json: server.json is removed, and the log ends with uvicorn's clean shutdown.

  • No screenshot. The only page is plain text with no data. The first screen is P1 Task 19's.

Findings and open questions

Deferred from Task 11's review (record), each with its owner:

Id What Goes to
SEC-P3T11-04 (r1), SEC-P3T11-05 (r2) No session (req. 2) and no Host allow-list (req. 4) yet. Every Unix user of a login node can reach the loopback port; only GET / with the version is served P1 Tasks 12 and 14. Task 12 must be merged before any data route (Tasks 17–18) is served on a shared login node (inbox #85)
SEC-P3T11-03 (r1) Idle connections to the port are held with no time limit before the first response; the runner shares the process and its descriptors P1 Task 12 [to be validated] against uvicorn's timeouts
SEC-P3T11-02 (r1, r2), SEC-P3T11-08 (r1), SEC-P3T11-01 (r1), CR-P3T11-03 (r1) server.json is first written in place, not under a temporary name; a 0755 data directory is accepted there; serve.lock is written before the parents of db_dir are checked; a server.json of this node is replaced without checking its PID, so two servers with different db_dir and ports can both start P1 Task 15 (file modes, req. 5): temporary name, fsync and rename; the 0700 rule; the parent check before the lock; on this node, refuse while the recorded PID is a live spectio serve
CR-P3T11-04 (r1) On a SLURM environment the rebuild stores the local native ID <pid>.<start> P1 Task 10 (the tmux native ID; reconciled rows are not visited again)
SEC-P3T11-07 (r1) No test pins that a rebuilt queued run is never launched from the manifest on disk P1 Task 17 (the launch path)
SEC-P3T11-01 (r3) A rebuilt non-absolute location becomes a git workflow without the import rule of api §5 (ext::, file://, credentials in the URL) P1 Task 17: G5's validator, applied to rebuilt rows too
SEC-P3T11-05 (r1), SEC-P3T11-03 (r2) The dependency floors allow h11 before 0.16.0 (CVE-2025-43859) and starlette before the 2025 DoS fixes [to be validated against the advisories]; uv.lock pins safe versions P1 Task 21 (the published wheel's metadata)
CR-P3T11-02 (r1), SEC-P3T11-04 (r2), SEC-P3T11-06 (r1) Refused entries do not count toward the per-boot caps; the byte count is not the bytes read; the first tick replays up to 2 MiB per launch with no per-boot cap; a bad manifest's ValidationError is logged with input values follow-up FU-P3-15
CR-P3T11-02 (r2), SEC-P3T11-01 (r2), CR-P3T11-07 (r2) CI runs no "another user's directory" refusal (both foreign-uid tests need root); serve.py imports the private _require_private_dir follow-up FU-P3-16
CR-P3T11-03 (r2) No tunnel line in the local-mode start text not changed: G6's test_serve_local_start_text asks for none

Open points from Task 11:

  1. Built in the local-only order (assumed — standing rule, inbox #85). Until Task 10:

    • on a SLURM environment, spectio serve starts, rebuilds the index and serves, but no runner loop runs; the log and the start text say so;
    • there is no --detach.

    Until Task 12, there is no server key, no token link and no data route. 2. The rebuild differs from dev-env §15.3 in one place. Found by doc-writer reading the spec against rebuild.py; the reviewers did not raise it. - The spec (Where data lives) says: "A manifest with no PID file and no marker was never launched: the run ends failed, class launch_error, 'interrupted before launch'". - The code gives class head_lost, "No head ran for this run (index rebuilt)." - api §7 defines head_lost as a head that exited without an engine terminal event, and launch_error as a launch that failed before the engine ran. The spec's class fits better, so the spec is kept. - Owner: follow-up FU-P3-15, at the next change of rebuild.py. 3. Two rebuild rules of dev-env §15.3 are not built yet. - The pointers of ~/.spectio/observed/ (runs started from a terminal) are P1 Task 25's, which changes rebuild.py. - "On another login node" for a head.pid that names another node is not shown yet. The local launcher reports such a head as unknown with its node, and never signals it (Task 9). The run detail that shows it comes with M4. 4. For the next /apply-findings pass: P1 Task 11's interface, in the table Differences from the plan's interface (Task 11). 5. The squash message must keep Release-As: 0.0.1 (dev-env §15.2); d0f716b carries it.

Process:

  • Follow-up FU-P3-15: rebuild hardening, at the next change of rebuild.py (P1 Task 25 at the latest). It covers:
    • refused entries count toward a per-boot cap, and the byte cap counts the bytes read;
    • the first tick's spool replay gets a per-boot byte cap (req. 10);
    • a bad manifest is logged without its input values;
    • a run with no head and no marker ends failed, class launch_error, as dev-env §15.3 says (open point 2).
  • Follow-up FU-P3-16: test and module hygiene, at the next change of the serve tests (P1 Task 12 at the latest, which adds the security conftest.py). It covers:
    • the foreign-uid cases run without root, through the repository's monkeypatch of the uid, so CI checks the uid comparison of open_private and read_small;
    • _require_private_dir gets a public name in spectio_server.store.db.
  • Still pending Thomas: inbox #64 (which directory is node-local at EMBL), #80 and #85.
  • Doc review: the verdict goes to the journal when the PR opens. The front matter's review stays null until the milestone page closes with Task 16.