test/run: missing test binary is a loud FAIL, not a silent SKIP
A test/wcc/NNN_*.c with no Makefile-wired binary used to write a SKIP
status, count toward "all N tests passed", and leave MAKE_TEST_EXIT=0.
Two live instances: 953_arrlit_slice_run skipped under a green gate for
weeks (committed unwired at bf1037d, wired in the previous commit), and
a 938 hit the same hole mid-gate. The runner now treats a missing binary
as FAIL naming the file and the target to add, touches the .fail marker,
and so flips the summary + exit code. Closed by construction: make test
builds every $(TESTS) target before the runner walks test/wcc/*.c, so
the missing-binary path is reachable only by an unwired file — there is
no legitimate missing-binary SKIP to preserve. Intentional skips keep
their existing visible forms (UNIT-mode non-enumeration; in-test per-row
skip messages), neither of which folds into the pass count.
Count math: the summary N is unchanged (a SKIP already incremented ran);
what changes is honesty — pre-fix "all N passed" could include silent
skips, post-fix every counted test actually executed. A bare sh test/run
without the make-built binaries now fails loud instead of green-skipping
the whole suite.
This commit is contained in:
11
test/run
11
test/run
@@ -20,7 +20,16 @@ if [ "$1" = "--one" ]; then
|
||||
short=${name#[0-9][0-9][0-9]_}
|
||||
bin=$BIN/test_$short
|
||||
if [ ! -x "$bin" ]; then
|
||||
printf 'SKIP %s (no binary %s)\n' "$name" "$bin" > "$prefix.status"
|
||||
# A missing binary means an unwired test/wcc file: `make test`
|
||||
# builds every $(TESTS) target before this runs, so the only way
|
||||
# to get here is a .c with no Makefile rule. That used to SKIP
|
||||
# and still count toward "all N tests passed" — 953_arrlit_slice
|
||||
# sat dark for weeks under a green gate. Fail loud instead.
|
||||
{
|
||||
printf 'FAIL %s (no binary %s)\n' "$name" "$bin"
|
||||
printf 'unwired test: add $(BIN)/test_%s to TESTS + a build rule in Makefile\n' "$short"
|
||||
} > "$prefix.status"
|
||||
: > "$prefix.fail"
|
||||
exit 0
|
||||
fi
|
||||
if "$bin" > "$prefix.out" 2> "$prefix.err"; then
|
||||
|
||||
Reference in New Issue
Block a user