test/run: missing test binary is a loud FAIL, not a silent SKIP

A test/wcc/NNN_*.c with no Makefile-wired binary used to write a SKIP
status, count toward "all N tests passed", and leave MAKE_TEST_EXIT=0.
Two live instances: 953_arrlit_slice_run skipped under a green gate for
weeks (committed unwired at bf1037d, wired in the previous commit), and
a 938 hit the same hole mid-gate. The runner now treats a missing binary
as FAIL naming the file and the target to add, touches the .fail marker,
and so flips the summary + exit code. Closed by construction: make test
builds every $(TESTS) target before the runner walks test/wcc/*.c, so
the missing-binary path is reachable only by an unwired file — there is
no legitimate missing-binary SKIP to preserve. Intentional skips keep
their existing visible forms (UNIT-mode non-enumeration; in-test per-row
skip messages), neither of which folds into the pass count.

Count math: the summary N is unchanged (a SKIP already incremented ran);
what changes is honesty — pre-fix "all N passed" could include silent
skips, post-fix every counted test actually executed. A bare sh test/run
without the make-built binaries now fails loud instead of green-skipping
the whole suite.
This commit is contained in:
2026-06-05 03:01:32 +09:00
parent ec7e8af6e9
commit f272068940

View File

@@ -20,7 +20,16 @@ if [ "$1" = "--one" ]; then
short=${name#[0-9][0-9][0-9]_}
bin=$BIN/test_$short
if [ ! -x "$bin" ]; then
printf 'SKIP %s (no binary %s)\n' "$name" "$bin" > "$prefix.status"
# A missing binary means an unwired test/wcc file: `make test`
# builds every $(TESTS) target before this runs, so the only way
# to get here is a .c with no Makefile rule. That used to SKIP
# and still count toward "all N tests passed" — 953_arrlit_slice
# sat dark for weeks under a green gate. Fail loud instead.
{
printf 'FAIL %s (no binary %s)\n' "$name" "$bin"
printf 'unwired test: add $(BIN)/test_%s to TESTS + a build rule in Makefile\n' "$short"
} > "$prefix.status"
: > "$prefix.fail"
exit 0
fi
if "$bin" > "$prefix.out" 2> "$prefix.err"; then