make: content-keyed test-commit tier caches the 10 pure gates (test-perf T1)
The 990-997/950/989_lib_byteid gates are pure functions of (stage binaries, tracked sources, fixtures, harness, Makefile) and eat 87% of full-test wall (measured, .ai/testperf-baseline.md). test-commit hashes exactly those inputs; a match with the last green full run reports the gates as 'cached' (counted separately, never silently) and runs everything else - 25s vs 212s on a hit, full run on any miss. A red run never records the key; make test itself is never cached and stays the pre-push bar (rule 14 updated). Key gap caught in review: examples/ feeds 996_dyn_ww - included.
This commit is contained in:
@@ -11,4 +11,4 @@
|
|||||||
11. Split commits when they bundle unrelated concerns. Bisect-cleanliness is the default. Multi-fix commits need a body paragraph explaining why they couldn't split.
|
11. Split commits when they bundle unrelated concerns. Bisect-cleanliness is the default. Multi-fix commits need a body paragraph explaining why they couldn't split.
|
||||||
12. Simple data, simple algorithms. Sea-of-stars style. Mirror Hare's structural choices over clever alternatives.
|
12. Simple data, simple algorithms. Sea-of-stars style. Mirror Hare's structural choices over clever alternatives.
|
||||||
13. No hardcoded size literals in size-computation contexts. Always route through the type table (tinfo.size / Type.size / ty_*->size / size(T) / primtypesize / tyslicesize). Per Drew's framing of Hare's discipline. `make sizelint` enforces and runs as a dep of `make test`. Exemptions documented inline with `// sizelint-ok: <reason>` (ww) or `/* sizelint-ok: <reason> */` (C). Optional local enforcement: `ln -s ../../tools/sizelint .git/hooks/pre-commit`.
|
13. No hardcoded size literals in size-computation contexts. Always route through the type table (tinfo.size / Type.size / ty_*->size / size(T) / primtypesize / tyslicesize). Per Drew's framing of Hare's discipline. `make sizelint` enforces and runs as a dep of `make test`. Exemptions documented inline with `// sizelint-ok: <reason>` (ww) or `/* sizelint-ok: <reason> */` (C). Optional local enforcement: `ln -s ../../tools/sizelint .git/hooks/pre-commit`.
|
||||||
14. Test targets. Inner-loop dev runs `make test-unit` (~3.8s, skips wwstage-driver tests: 950, 990–997). Pre-commit / pre-push runs full `make test` (~8:30, all 132 incl. byte-id gates). The `test` target is the bootstrap-correctness gate — never push without it green. test-unit is a fast confidence check on unit-level code (stages, stdlib, smoke); a green test-unit doesn't prove bootstrap, only that the unit set is intact. Don't add new wwstage-driver tests outside the 950/990–997 range (the test/run skip set is keyed by NNN prefix). Phase split background: ww_ww writes intermediates next to every traversed source (filed bug), so concurrent wwstage-driver tests would race on `selfhost/cmd/<tool>/main.{s,combined.ww,o}` fixtures.
|
14. Test targets. Inner-loop dev runs `make test-unit` (~3.8s, skips wwstage-driver tests: 950, 990–997). Pre-commit runs `make test-commit` (content-keyed tier: the 10 pure gates — 950, 989_lib_byteid, 990–997 — report `cached` when the input key matches the last green full run; ~25s on a hit, full gate run on any miss; key = stage-binary md5s + tracked selfhost/lib/cmd/rt/test/examples sources + harness + Makefile; cache in out/.testcache, wiped by make clean; mtime keys and partial keys are vetoed). Pre-push runs full `make test` (~3:40 measured at f940072, all 353 incl. byte-id gates) — the bootstrap-correctness gate, never cached; never push without it green. test-unit is a fast confidence check on unit-level code (stages, stdlib, smoke); a green test-unit doesn't prove bootstrap, only that the unit set is intact. Don't add new wwstage-driver tests outside the 950/990–997 range (the test/run skip set is keyed by NNN prefix). Phase split background: ww_ww writes intermediates next to every traversed source (filed bug), so concurrent wwstage-driver tests would race on `selfhost/cmd/<tool>/main.{s,combined.ww,o}` fixtures.
|
||||||
|
|||||||
12
Makefile
12
Makefile
@@ -2556,6 +2556,16 @@ test: all sizelint peellint $(TESTS)
|
|||||||
test-unit: all sizelint peellint $(TESTS)
|
test-unit: all sizelint peellint $(TESTS)
|
||||||
@WW=$(BIN)/ww BIN=$(BIN) UNIT=1 sh test/run
|
@WW=$(BIN)/ww BIN=$(BIN) UNIT=1 sh test/run
|
||||||
|
|
||||||
|
# ---- test-commit: per-commit default (task #10 T1) ---------------------
|
||||||
|
# Full suite, but content-key-SKIPS the expensive wwstage byte-id /
|
||||||
|
# self-compile gates (950/989_lib_byteid/990-997) when their input set is
|
||||||
|
# byte-identical to the last green run — reported as `cached`, never a
|
||||||
|
# silent skip. NOT a substitute for the push bar: `make test` stays the
|
||||||
|
# no-skip bootstrap gate. Cache lives in out/.testcache (gitignored, wiped
|
||||||
|
# by `make clean`); a green `make test` primes it. See test/run.
|
||||||
|
test-commit: all sizelint peellint $(TESTS)
|
||||||
|
@WW=$(BIN)/ww BIN=$(BIN) COMMIT=1 sh test/run
|
||||||
|
|
||||||
# ---- smoke: inner-loop cross-stage byte-id check on a SMALL input ------
|
# ---- smoke: inner-loop cross-stage byte-id check on a SMALL input ------
|
||||||
# Compiles one self-contained fixture through cstage w6c AND wwstage
|
# Compiles one self-contained fixture through cstage w6c AND wwstage
|
||||||
# w6c_ww and diffs the emitted .s. Catches a cs!=ww emission divergence
|
# w6c_ww and diffs the emitted .s. Catches a cs!=ww emission divergence
|
||||||
@@ -2723,4 +2733,4 @@ nocc:
|
|||||||
@echo
|
@echo
|
||||||
@echo "NOCC OK: $(STAGE0)/* reproduces itself from source. cc not invoked."
|
@echo "NOCC OK: $(STAGE0)/* reproduces itself from source. cc not invoked."
|
||||||
|
|
||||||
.PHONY: all cstage wwstage test test-unit smoke sizelint peellint install clean bootstrap nocc bootstrap-snapshot
|
.PHONY: all cstage wwstage test test-unit test-commit smoke sizelint peellint install clean bootstrap nocc bootstrap-snapshot
|
||||||
|
|||||||
97
test/run
97
test/run
@@ -64,6 +64,70 @@ trap 'rm -rf "$RESULTS"' EXIT
|
|||||||
|
|
||||||
JOBS=${JOBS:-$(nproc 2>/dev/null || echo 4)}
|
JOBS=${JOBS:-$(nproc 2>/dev/null || echo 4)}
|
||||||
|
|
||||||
|
# ---- content-keyed gate cache (task #10 T1; .ai/rob-testperf-go.md L1) -------
|
||||||
|
# `make test-commit` (COMMIT=1) reports the expensive wwstage byte-id /
|
||||||
|
# self-compile gates as `cached` when their full input set is byte-identical to
|
||||||
|
# the last green run. Those gate outcomes are a PURE function of (stage binaries
|
||||||
|
# + selfhost/lib/cmd/rt sources + test fixtures + this harness + the Makefile);
|
||||||
|
# if that content hash equals the recorded last-green key the result is provably
|
||||||
|
# unchanged, so `cached` is sound — not a guess. `make test` NEVER reads the
|
||||||
|
# cache (it stays the no-skip pre-push bar) but a green run of the gates DOES
|
||||||
|
# record the key, as the natural primer. A red run never records: the write is
|
||||||
|
# guarded on fail=0 (and the fail>0 path exits before it). Cache lives under
|
||||||
|
# out/.testcache (out/ is gitignored); `make clean` wipes out/ and thus the
|
||||||
|
# cache, which is correct — the key is keyed on the just-built binaries, so a
|
||||||
|
# from-scratch rebuild must start from a clean cache.
|
||||||
|
OUT_DIR=$(dirname "$BIN")
|
||||||
|
CACHE_DIR=$OUT_DIR/.testcache
|
||||||
|
CACHE_FILE=$CACHE_DIR/last-green
|
||||||
|
|
||||||
|
# The exact cached set, confirmed from the Makefile/test/run: the UNIT-guarded
|
||||||
|
# phase-2 gates (950 + 990-997) plus the single phase-1 lib byte-id gate
|
||||||
|
# (989_lib_byteid). The other 989_*/998/999 files are cheap functional _run
|
||||||
|
# tests, NOT byte-id gates, and stay always-on. Every member is a deterministic
|
||||||
|
# pure function of the hashed inputs (no /tmp/env/clock/network reads); a future
|
||||||
|
# gate that reads outside that set must be kept OUT of this list (Go marks such
|
||||||
|
# tests uncacheable for the same reason).
|
||||||
|
is_cached_gate() {
|
||||||
|
case $1 in
|
||||||
|
950_selfcheck.c|989_lib_byteid.c|\
|
||||||
|
990_*|991_*|992_*|993_*|994_*|995_*|996_*|997_*) return 0 ;;
|
||||||
|
esac
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
emit_cached() {
|
||||||
|
printf 'cached %s\n' "$1" > "$RESULTS/$1.status"
|
||||||
|
: > "$RESULTS/$1.cached"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Content key over EVERYTHING that determines the cached gates' outcomes: the
|
||||||
|
# stage binary md5s first (fixed order), then tracked source/fixture/harness
|
||||||
|
# content (git ls-files is sorted → deterministic; md5sum hashes working-tree
|
||||||
|
# bytes, so uncommitted edits flip the key too). Any changed byte flips the key
|
||||||
|
# → the gates RUN. A missed input would be a false green (rob's veto), so the
|
||||||
|
# set is a deliberate conservative superset (wwstage binary md5s overlap the
|
||||||
|
# selfhost/lib source hashes, rt/ is hashed though it only reaches the gates via
|
||||||
|
# libwwrt); an extra input only ever costs a needless honest re-run. examples/
|
||||||
|
# is hashed because 996_dyn_ww links examples/mandelbrot/mandelbrot.{ww,o} — a
|
||||||
|
# real gate input outside the source tree.
|
||||||
|
testcache_key() {
|
||||||
|
for b in ww w6c w6a w6l wwdump ww_ww w6c_ww w6a_ww w6l_ww wwdump_ww; do
|
||||||
|
[ -f "$BIN/$b" ] && md5sum "$BIN/$b"
|
||||||
|
done
|
||||||
|
git ls-files selfhost lib cmd rt examples test/wcc test/run Makefile | xargs md5sum
|
||||||
|
}
|
||||||
|
|
||||||
|
KEY=""
|
||||||
|
if [ "${UNIT:-0}" != "1" ]; then
|
||||||
|
KEY=$(testcache_key 2>/dev/null | md5sum 2>/dev/null | cut -d' ' -f1) || KEY=""
|
||||||
|
fi
|
||||||
|
CACHE_HIT=0
|
||||||
|
if [ "${COMMIT:-0}" = "1" ] && [ -n "$KEY" ] && [ -f "$CACHE_FILE" ] \
|
||||||
|
&& [ "$(cat "$CACHE_FILE" 2>/dev/null)" = "$KEY" ]; then
|
||||||
|
CACHE_HIT=1
|
||||||
|
fi
|
||||||
|
|
||||||
# Phase wall-clock stamps (task #10 baseline). Each phase records start/end
|
# Phase wall-clock stamps (task #10 baseline). Each phase records start/end
|
||||||
# so the collector can report wall shares onto ken's three poles.
|
# so the collector can report wall shares onto ken's three poles.
|
||||||
run_start=$(date +%s.%N)
|
run_start=$(date +%s.%N)
|
||||||
@@ -80,6 +144,12 @@ for t in test/wcc/*.c; do
|
|||||||
950_*|990_*|991_*|992_*|993_*|994_*|995_*|996_*|997_*) continue ;;
|
950_*|990_*|991_*|992_*|993_*|994_*|995_*|996_*|997_*) continue ;;
|
||||||
esac
|
esac
|
||||||
name=${t##*/}; name=${name%.c}
|
name=${t##*/}; name=${name%.c}
|
||||||
|
# The only cached gate that lives in phase 1 is 989_lib_byteid; on a
|
||||||
|
# content-key hit report it cached instead of re-running it.
|
||||||
|
if [ "$CACHE_HIT" = 1 ] && is_cached_gate "${t##*/}"; then
|
||||||
|
emit_cached "$name"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
printf '%s/%s\0%s\0' "$RESULTS" "$name" "$t"
|
printf '%s/%s\0%s\0' "$RESULTS" "$name" "$t"
|
||||||
done | xargs -0 -n2 -P "$JOBS" "$0" --one || true
|
done | xargs -0 -n2 -P "$JOBS" "$0" --one || true
|
||||||
p1_end=$(date +%s.%N)
|
p1_end=$(date +%s.%N)
|
||||||
@@ -88,6 +158,17 @@ p1_end=$(date +%s.%N)
|
|||||||
# byte-id gates (990-997) and 950_selfcheck entirely. Pre-push uses
|
# byte-id gates (990-997) and 950_selfcheck entirely. Pre-push uses
|
||||||
# `make test` for the full suite.
|
# `make test` for the full suite.
|
||||||
if [ "${UNIT:-0}" != "1" ]; then
|
if [ "${UNIT:-0}" != "1" ]; then
|
||||||
|
if [ "$CACHE_HIT" = 1 ]; then
|
||||||
|
# Content key matched last-green → the wwstage byte-id / self-compile
|
||||||
|
# gates are provably unchanged. Report them cached; do NOT re-run.
|
||||||
|
for t in test/wcc/990_*.c test/wcc/991_*.c test/wcc/992_*.c \
|
||||||
|
test/wcc/993_*.c test/wcc/994_*.c test/wcc/995_*.c \
|
||||||
|
test/wcc/996_*.c test/wcc/997_*.c test/wcc/950_*.c; do
|
||||||
|
[ -f "$t" ] || continue
|
||||||
|
name=${t##*/}; name=${name%.c}
|
||||||
|
emit_cached "$name"
|
||||||
|
done
|
||||||
|
else
|
||||||
p2par_start=$(date +%s.%N)
|
p2par_start=$(date +%s.%N)
|
||||||
# Phase 2 is partitioned (#3): the pure-reader byte-id gates run in
|
# Phase 2 is partitioned (#3): the pure-reader byte-id gates run in
|
||||||
# parallel; the source-tree writers run in a serial tail afterward.
|
# parallel; the source-tree writers run in a serial tail afterward.
|
||||||
@@ -117,6 +198,7 @@ if [ "${UNIT:-0}" != "1" ]; then
|
|||||||
"$0" --one "$RESULTS/$name" "$t" || true
|
"$0" --one "$RESULTS/$name" "$t" || true
|
||||||
done
|
done
|
||||||
p2ser_end=$(date +%s.%N)
|
p2ser_end=$(date +%s.%N)
|
||||||
|
fi
|
||||||
|
|
||||||
# combined.ww freshness gate (#110): 990/995 above unconditionally
|
# combined.ww freshness gate (#110): 990/995 above unconditionally
|
||||||
# regenerate every tracked combined.ww amalgamation, so a diff vs HEAD
|
# regenerate every tracked combined.ww amalgamation, so a diff vs HEAD
|
||||||
@@ -140,11 +222,16 @@ fi
|
|||||||
|
|
||||||
fail=0
|
fail=0
|
||||||
ran=0
|
ran=0
|
||||||
|
cached=0
|
||||||
for s in "$RESULTS"/*.status; do
|
for s in "$RESULTS"/*.status; do
|
||||||
[ -f "$s" ] || continue
|
[ -f "$s" ] || continue
|
||||||
cat "$s"
|
cat "$s"
|
||||||
ran=$((ran + 1))
|
|
||||||
prefix=${s%.status}
|
prefix=${s%.status}
|
||||||
|
if [ -f "$prefix.cached" ]; then
|
||||||
|
cached=$((cached + 1))
|
||||||
|
else
|
||||||
|
ran=$((ran + 1))
|
||||||
|
fi
|
||||||
[ -f "$prefix.fail" ] && fail=$((fail + 1))
|
[ -f "$prefix.fail" ] && fail=$((fail + 1))
|
||||||
done
|
done
|
||||||
|
|
||||||
@@ -176,7 +263,7 @@ awk -v rs="$run_start" -v p1e="$p1_end" \
|
|||||||
printf "total\t%.3f\n", re - rs
|
printf "total\t%.3f\n", re - rs
|
||||||
}'
|
}'
|
||||||
|
|
||||||
if [ $ran -eq 0 ]; then
|
if [ $ran -eq 0 ] && [ $cached -eq 0 ]; then
|
||||||
echo "no tests were run"
|
echo "no tests were run"
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
@@ -184,4 +271,10 @@ if [ $fail -gt 0 ]; then
|
|||||||
echo "$fail test(s) failed"
|
echo "$fail test(s) failed"
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
# Green and the gates actually RAN (not a cache hit, not test-unit): record the
|
||||||
|
# content key as the last-green primer. A red run exits above, never here.
|
||||||
|
if [ "${UNIT:-0}" != "1" ] && [ "$CACHE_HIT" != 1 ] && [ -n "$KEY" ]; then
|
||||||
|
mkdir -p "$CACHE_DIR" && printf '%s\n' "$KEY" > "$CACHE_FILE"
|
||||||
|
fi
|
||||||
|
[ $cached -gt 0 ] && echo "$cached gate(s) cached (content key matched last green)"
|
||||||
echo "all $ran tests passed"
|
echo "all $ran tests passed"
|
||||||
|
|||||||
Reference in New Issue
Block a user