test/run: phase-2 byte-id gates run parallel; rule-14 NNN constraint retired (test-perf T3b)
With intermediates following the output stem, concurrent wwstage-driver builds no longer share paths - 950/990-997 join the xargs -P group. Measured: 202.7s -> 153.8s full-test wall (x2 runs race-clean); the serial tail (57.7% of wall) is gone, remainder is CPU-bound. CLAUDE.md rule 14 documents the new reality: NNN range no longer a correctness constraint, still keys phase membership / skip set / cache set.
This commit is contained in:
@@ -11,4 +11,4 @@
|
||||
11. Split commits when they bundle unrelated concerns. Bisect-cleanliness is the default. Multi-fix commits need a body paragraph explaining why they couldn't split.
|
||||
12. Simple data, simple algorithms. Sea-of-stars style. Mirror Hare's structural choices over clever alternatives.
|
||||
13. No hardcoded size literals in size-computation contexts. Always route through the type table (tinfo.size / Type.size / ty_*->size / size(T) / primtypesize / tyslicesize). Per Drew's framing of Hare's discipline. `make sizelint` enforces and runs as a dep of `make test`. Exemptions documented inline with `// sizelint-ok: <reason>` (ww) or `/* sizelint-ok: <reason> */` (C). Optional local enforcement: `ln -s ../../tools/sizelint .git/hooks/pre-commit`.
|
||||
14. Test targets. Inner-loop dev runs `make test-unit` (~3.8s, skips wwstage-driver tests: 950, 990–997). Pre-commit runs `make test-commit` (content-keyed tier: the 10 pure gates — 950, 989_lib_byteid, 990–997 — report `cached` when the input key matches the last green full run; ~25s on a hit, full gate run on any miss; key = stage-binary md5s + tracked selfhost/lib/cmd/rt/test/examples sources + harness + Makefile; cache in out/.testcache, wiped by make clean; mtime keys and partial keys are vetoed). Pre-push runs full `make test` (~3:40 measured at f940072, all 353 incl. byte-id gates) — the bootstrap-correctness gate, never cached; never push without it green. test-unit is a fast confidence check on unit-level code (stages, stdlib, smoke); a green test-unit doesn't prove bootstrap, only that the unit set is intact. Don't add new wwstage-driver tests outside the 950/990–997 range (the test/run skip set is keyed by NNN prefix). Phase split background: ww_ww writes intermediates next to every traversed source (filed bug), so concurrent wwstage-driver tests would race on `selfhost/cmd/<tool>/main.{s,combined.ww,o}` fixtures.
|
||||
14. Test targets. Inner-loop dev runs `make test-unit` (~3.8s, skips wwstage-driver tests: 950, 990–997). Pre-commit runs `make test-commit` (content-keyed tier: the 10 pure gates — 950, 989_lib_byteid, 990–997 — report `cached` when the input key matches the last green full run; ~25s on a hit, full gate run on any miss; key = stage-binary md5s + tracked selfhost/lib/cmd/rt/test/examples sources + harness + Makefile; cache in out/.testcache, wiped by make clean; mtime keys and partial keys are vetoed). Pre-push runs full `make test` (~3:40 measured at f940072, all 353 incl. byte-id gates) — the bootstrap-correctness gate, never cached; never push without it green. test-unit is a fast confidence check on unit-level code (stages, stdlib, smoke); a green test-unit doesn't prove bootstrap, only that the unit set is intact. New wwstage-driver tests are no longer confined to the 950/990–997 range for correctness: T3 (test-perf arc) made every build's intermediates follow its OUTPUT (`ww build -o <stem>` / `ww run`'s per-pid temp) instead of landing next to every traversed source, so concurrent driver builds no longer race on `selfhost/cmd/<tool>/main.{s,combined.ww,o}` fixtures — the phase-2 byte-id group (incl. 993/995/950) now runs fully parallel. The 950/990–997 NNN prefixes still drive test/run's phase-2 membership, the `make test-unit` skip set, and the `make test-commit` cache set, so a heavy 43k self-compile gate should still use that range to be skipped by test-unit and cached by test-commit; a light wwstage-driver test may live at any NNN (just `-o`-redirect its intermediates off the source tree).
|
||||
|
||||
44
test/run
44
test/run
@@ -132,12 +132,12 @@ fi
|
||||
# so the collector can report wall shares onto ken's three poles.
|
||||
run_start=$(date +%s.%N)
|
||||
|
||||
# Phase 2 runs the wwstage byte-id gates (990-997) + 950_selfcheck.
|
||||
# ww_ww writes intermediates next to every traversed source (task #15),
|
||||
# so 991/992/994's reads of selfhost/cmd/<tool>/main.{s,o,combined.ww}
|
||||
# race against any concurrent driver-using test that emits a sibling
|
||||
# .combined.ww (task #16). Phase 1 stays parallel for unit coverage; the
|
||||
# wwstage-driver tests stay out of phase 1 to keep those reads honest.
|
||||
# Phase 2 runs the wwstage byte-id / self-compile gates (990-997) +
|
||||
# 950_selfcheck. They're kept out of phase 1 only so the heavy 43k
|
||||
# self-compiles don't contend with the phase-1 fixture swarm; within
|
||||
# phase 2 they all run in parallel (post-T3 each build's intermediates
|
||||
# follow its output via `-o`/temp, so no member writes a path another
|
||||
# reads — see the phase-2 block below).
|
||||
for t in test/wcc/*.c; do
|
||||
[ -f "$t" ] || continue
|
||||
case ${t##*/} in
|
||||
@@ -170,34 +170,22 @@ if [ "${UNIT:-0}" != "1" ]; then
|
||||
done
|
||||
else
|
||||
p2par_start=$(date +%s.%N)
|
||||
# Phase 2 is partitioned (#3): the pure-reader byte-id gates run in
|
||||
# parallel; the source-tree writers run in a serial tail afterward.
|
||||
#
|
||||
# PARALLEL GROUP — 990/991/992/994/996/997. Each reads only committed
|
||||
# or make-all-produced files and writes only /tmp or a disjoint tracked
|
||||
# stem (990→out smoke.*, 996→examples/mandelbrot/*, 997→/tmp fixture
|
||||
# copy). No group member writes a stem another member reads (per-gate FS
|
||||
# footprint audited race-free, #3), so they fan out via the same
|
||||
# xargs -P "$JOBS" machinery as phase 1.
|
||||
# Phase 2 — the wwstage byte-id / self-compile gates, all parallel.
|
||||
# Pre-T3 the source-tree writers (993/995/950) ran in a serial tail
|
||||
# because ww_ww wrote intermediates next to every traversed source, so
|
||||
# they raced each other AND the 990/991/992/994 readers. T3 made every
|
||||
# build's .combined.ww/.s/.o follow its OUTPUT (`-o`/per-pid temp): 993
|
||||
# → its dc/dw workdirs, 995 → /tmp/wwsr_<pid>_<tool>, 950 → /tmp only.
|
||||
# No member now writes a stem another reads (the readers see only the
|
||||
# stable make-all regen), so the whole group fans out via xargs -P.
|
||||
for t in test/wcc/990_*.c test/wcc/991_*.c test/wcc/992_*.c \
|
||||
test/wcc/994_*.c test/wcc/996_*.c test/wcc/997_*.c; do
|
||||
test/wcc/993_*.c test/wcc/994_*.c test/wcc/995_*.c \
|
||||
test/wcc/996_*.c test/wcc/997_*.c test/wcc/950_*.c; do
|
||||
[ -f "$t" ] || continue
|
||||
name=${t##*/}; name=${name%.c}
|
||||
printf '%s/%s\0%s\0' "$RESULTS" "$name" "$t"
|
||||
done | xargs -0 -n2 -P "$JOBS" "$0" --one || true
|
||||
p2par_end=$(date +%s.%N)
|
||||
|
||||
p2ser_start=$(date +%s.%N)
|
||||
# SERIAL TAIL — 993/995/950, the only source-tree writers. 993 & 995
|
||||
# both write cmd/wwdump/main.* (995 writes all 5 cmd/*/main.*), so they
|
||||
# run sequentially w.r.t. each other AND after the parallel group, so
|
||||
# the group's readers see stable make-all output, never a torn write.
|
||||
for t in test/wcc/993_*.c test/wcc/995_*.c test/wcc/950_*.c; do
|
||||
[ -f "$t" ] || continue
|
||||
name=${t##*/}; name=${name%.c}
|
||||
"$0" --one "$RESULTS/$name" "$t" || true
|
||||
done
|
||||
p2ser_end=$(date +%s.%N)
|
||||
fi
|
||||
|
||||
# combined.ww freshness gate (#110): 990/995 above unconditionally
|
||||
|
||||
Reference in New Issue
Block a user