From 91fce80b37a581059365e64c264f80ddc252ee51 Mon Sep 17 00:00:00 2001 From: Hojun-Cho Date: Sat, 22 Aug 2026 12:59:08 +0900 Subject: [PATCH] test: accept empty skip reasons --- docs/build-system.md | 225 ++++++++++++++++++++++ docs/spec.md | 16 ++ docs/test-system-v2.md | 15 ++ lib/test/run.ww | 4 +- test/package/package_test.ww | 352 +++++++++++++++++++++++++++++++++++ 5 files changed, 610 insertions(+), 2 deletions(-) diff --git a/docs/build-system.md b/docs/build-system.md index ec0bc8b0..e7b7d475 100644 --- a/docs/build-system.md +++ b/docs/build-system.md @@ -9320,6 +9320,231 @@ fixed point and round-tripped all five WWstage tools byte-identically; the platform gate produced a byte-identical dynamic-link result. No production code changed after the successful ordered sequence began. +### 11.47 Implemented empty-reason test skip classification + +Pinned Go's applicable semantic rule is that a test may skip without supplying +a message, remains a successful skipped test, and does not prevent the next +selected test from running. Mapping that rule to WW's already representable +`test.skip("")` call is **behavior derived from the pinned implementation**. + +WW realizes that rule by writing a structurally valid `TST_SKIPPED` control +frame with a zero-length payload and accepting it when the child otherwise exits +normally with status zero. Presentation remains +`qualified.name ... SKIP: ` followed immediately by newline. Skipped +accounting increases once; failure and harness-error accounting do not. +Nonempty reasons keep their existing behavior. A 4,094-byte reason remains an +invalid control result, and skip outside an active test still aborts. Those +WW-specific frame, presentation, bound, accounting, and outside-active details +are **directly measured WW behavior** in the post-change proof below; they are +not attributed to Go's implementation. + +#### Pinned evidence and fact classification + +The sole semantic authority is official Go 1.26.5 at +`c19862e5f8415b4f24b189d065ed739517c548ba`: + +- [`(*T).Skip` and `(*T).SkipNow` in `testing/testing.go`, lines + 1231–1259](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/testing.go#L1231-L1259) + show `Skip` accepting a variadic argument list and logging + `fmt.Sprintln(args...)` before calling zero-argument `SkipNow`; `SkipNow` + marks and stops that test and explicitly permits the next test to continue; +- [the package documentation in `testing/testing.go`, lines + 273–293](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/testing.go#L273-L293) + includes a zero-argument `t.Skip()`; and +- official assertions in + [`testing/sub_test.go`, lines + 182–190](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/sub_test.go#L182-L190), + [237–240](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/sub_test.go#L237-L240), + and + [341–346](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/sub_test.go#L341-L346) + mark message-less `SkipNow` and zero-argument `t.Skip()` cases successful. + +Those methods, documentation, and official assertions are **behavior directly +implemented or asserted by pinned Go**: a message-less call is admitted and +successful, is classified skipped, and permits the next selected test to run. +That a suite containing this skip and otherwise passing selected tests has an +overall successful result is **behavior derived from the pinned +implementation**. The rule honestly applies within WW's local, dotted-import, +manifest-free model: +WW's public `test.skip` already takes a `str`, and that type already represents +the applicable empty-reason case without adding Go syntax, modules, manifests, +package-path conventions, regex filters, or a new build language. + +#### Fresh four-axis audit and direct pre-fix measurements + +The fresh audit classified multiple named source files, explicit +`*_test.ww` build operands, shared top-level test-process state and abort +boundaries, bare effective import bindings, and lexical import shadowing as +applicable differences that remain unselected. Leading-underscore explicit +source behavior retains a bounded applicability question, mixed declarations +are aligned, and grouped, quoted, dot, and generalized imports are inapplicable +to WW's import model. The exact explicitly named `*_test.go` build case lacks an +official pinned testdata assertion even though pinned source derives its +behavior, so it was not evidence-complete for this session. The empty-reason +skip was selected because its complete observable meaning has one bounded +shared-runtime owner and does not partially implement or redefine those broader +differences. + +Before this slice, the following observations were **directly measured WW +behavior**. The fixed-path probe +`/tmp/ww-empty-skip-pre.Wkh1CW/pkg` defined production `marker()`, then test +`first`, which called `test.skip("")`, and passing test `second`. Both Cstage +and WWstage exited 1, wrote empty stderr, and emitted byte-identical 255-byte +stdout with SHA-256 +`ed5015be2946b902cd42819d36ad102af34c229732b477441dc4b285ed612be3`: + +```text +emptyskip.first ... HARNESS (malformed or contradictory result) +emptyskip.second ... ok +1 passed, 0 failed, 0 skipped, 1 harness errors +2 discovered, 2 selected, 2 started, 2 completed +FAIL /tmp/ww-empty-skip-pre.Wkh1CW/pkg [emptyskip] (test exit 1) +FAIL +``` + +Filtering to `first` failed with one harness error in both stages; filtering to +`second` succeeded; list mode succeeded and printed both names without +executing either body. `test -c -o` succeeded with empty streams and produced +stage-byte-identical 112,861-byte executables with SHA-256 +`2d8f3a1adbd2f158ac605b4cecf9966a2b700ff0cc7f31aaa431bad3bc89e844`. +Direct execution of those binaries failed identically, with stdout SHA-256 +`b83ef1368512dd3d2ee46c5e6d1c08369b40eb4b849eae70303aebaef80ee650` +and empty stderr. These are pre-fix measurements only. + +For that fixed fixture, the directly measured pre-fix four-axis result was: + +- **Go-like build:** both stages selected and built the production/test inputs + sufficiently to publish byte-identical retained test executables; this probe + did not independently trace graph/action topology; +- **Go-like test:** empty-reason classification, result presentation, + accounting, package status, and direct retained execution were different, + while filtering, list non-execution, and later-test execution were aligned; +- **Go-like package:** the production source and same-package test source formed + one runnable package result; this probe made no external/test-only-role claim; + and +- **Go-like import:** `import test` resolved and linked in both stages; this + probe made no separate qualifier, unused-import, or graph-provenance claim. + +The broader unchanged role, identity, graph, persistence, and lifecycle items +below are implementation boundaries, not additional facts attributed to this +pre-fix fixture. + +#### Ownership and complete four-axis contract + +The true semantic owner is the shared in-binary test control-frame producer and +interpreter in `lib/test/run.ww`. Both stages link that one WW runtime; there is +no duplicate C/WW implementation. The producer rejects only a reason larger +than the control-frame payload limit and writes the empty `TST_SKIPPED` payload +normally. After the existing header and exact-length checks, the interpreter +accepts skipped code with payload length zero as well as positive length, while +retaining the requirement for normal status-zero child termination. + +- **Go-like build:** ordinary source loading, graph/action construction and + scheduling, compiler, assembler, archiver, linker, naming, and output + publication semantics do not change. The source-content change invalidates + and relinks affected products through existing dependency rules. Ordinary + products that do not import `test` stay outside the changed source; an + ordinary product that explicitly imports `test` may rebuild with changed + artifact bytes, while its outside-active runtime abort remains unchanged. +- **Go-like test:** every shared-runtime descriptor accepts the empty reason as + one successful skip. This covers raw single-file, same-package, + external-test, honest test-only, filtered, coordinator-run retained, and + later direct retained-binary execution, including an active-test call reached + through production package code. Production source still contributes no test + descriptor. The result has no diagnostic or harness error, uses the existing + blank-after-colon skip line, increments only skipped accounting, and permits + later selected tests to continue. +- **Go-like package:** source roles, descriptor order, initialization, declared + names, and canonical package and variant identities do not change. A runtime + reason is result data, never identity. +- **Go-like import:** `import test`, exact dotted dependency resolution, graph + edges, initialization, qualifier binding, and unused-import behavior do not + change. No reason byte becomes import, graph, action, symbol, `.wwi`, + artifact, publication, or persistence identity. + +#### Lifecycle, parity, proof, and formats + +Filename and source eligibility, selected-file order, and test-role +classification are unchanged. An empty reason has no loader representation. +Graph nodes, edges, actions, ordering, product scheduling, generated +descriptors, compiler/assembler/archiver/linker operation, and private/public +output naming remain unchanged. Existing source-content invalidation rebuilds +affected runtime/test actions; there is no test-result cache. + +At runtime, only the valid zero-length skip classification changes. Passes, +nonempty skips, assertion failures, signals, expected abort, premature clean +exit, ordinary abort, timeout ownership, process groups, descendant cleanup, +and the existing per-test process boundary retain their current owners and +behavior. An empty skip emits no diagnostic. Oversized reasons remain harness +errors, skip outside an active test still aborts, and all build, producer, +runtime, publication, and cleanup diagnostic channels and precedence remain +unchanged. + +Affected private and retained test executables change because their shared +runtime changes; comparable Cstage and WWstage executables must remain +byte-identical. `-c`, running `-o`, guarded installation, destination modes and +names, exact null discard, private execution, and publication order are +unchanged. A successful empty skip reaches the existing success-publication +path. Producer, other runtime, and publication failures retain their existing +failure classification, cold no-partial-publication guarantee, and warm +preservation of committed generations and prior public bytes. + +Persistence keys and schemas do not change. Existing content invalidation, +warm action reuse, relink, commit, rollback, parallel product isolation, +capture ownership, cancellation, interruption escalation, owned-child cleanup, +and transaction/scratch cleanup remain unchanged. No active `.new`, +`.install`, `.wwtxn.*`, adjacent `.sepwork`, capture, result, scratch, or +tool-stage transaction residue may remain outside an explicitly retained or +recipe-owned boundary. + +After the change, both stages directly ran the fixed-path two-test probe with +status 0 and empty stderr. They emitted the same empty-reason skip line, ran the +later passing test, reported one pass, one skip, zero harness errors, complete +2/2 accounting, and the ordinary package `ok` result. Empty-only and pass-only +filters each succeeded, and list mode still printed both names without running +them. Both `test -c -o` invocations had empty streams and produced +byte-identical 112,861-byte mode-0755 executables with SHA-256 +`257d05a99e920875e9d131ce18e7f11592179802fc6f864fc136ebf618d4a88c`. +Both retained executables then ran directly with status 0, identical empty +stderr, and identical skip/pass/accounting output. + +The WW-native `empty_skip_reason_is_a_successful_skip` observer passed after +rebuilding `out/bin/test_package`. Its dynamic sources directly prove both-stage +status and stdout/stderr parity for raw single-file, same-package, external, +and honest test-only descriptors; an active-test call through production code; +later-test continuation; a nonempty-skip control; empty-only and pass-only +filters; list nonexecution; cold and warm work; compile-only and running +retention; byte-identical retained and ordinary `import test` executables; +direct retained execution; unchanged outside-active abort; the unchanged +4,094-byte oversized-reason harness error; and transaction/residue cleanup. +The complete focused `make -j1 JOBS=1 test-package` owner then passed all 53 +observers with zero failures, skips, or harness errors. All ordered full gates +then passed serially after the final executable production and proof changes: + +1. `make -j4 JOBS=4 test` +2. `make -j4 JOBS=4 test-commit` +3. `make -j4 JOBS=4 test-byteid` +4. `make -j1 JOBS=1 test-bootstrap` +5. `make -j1 JOBS=1 test-platform` +6. `make -j1 JOBS=1 test-all` + +Every command exited zero. The byte-identity gates compared 161 language files +and 1,421 data fixtures with zero pinned-divergent fixtures; bootstrap retained +the `ww2 == ww3 == ww4` fixed point and five-tool WWstage round-trip byte +identity; and the platform dynamic-link artifact remained Cstage/WWstage +byte-identical at 8,464 bytes. + +A final read-only review produced two wording-only corrections: the pinned +`Skip`/`SkipNow` description was split precisely, and the warm observer comment +was limited to persisted-work execution rather than claiming unmeasured action +reuse. With production and executable proof logic unchanged, the exact final +observer source then passed `make -j1 JOBS=1 test-package` again: all 53 tests +passed with zero failures, skips, or harness errors. + +No format bump. Build workdir format remains `18`, test workdir format remains +`19`, semantic storage format remains `3`, and no test-result cache is +introduced. + ## 12. Candidate architectures and hard-gate decision Five candidates were developed as coherent systems, not as feature bins. diff --git a/docs/spec.md b/docs/spec.md index c4379f75..0ce51581 100644 --- a/docs/spec.md +++ b/docs/spec.md @@ -930,6 +930,22 @@ and accounting in its product-local capture; a retained binary invoked directly keeps the warning on standard error, its accounting on standard output, and has no coordinator package result. +Calling `test.skip(reason)` from an active test records a successful skip and +stops that test body. The empty string is a valid reason: `test.skip("")` is +not a malformed harness result. Its ordinary result line is exactly the +qualified test name, ` ... SKIP: `, and a newline, with no reason bytes after +the space. It increments skipped accounting once, increments neither failed nor +harness-error accounting, and permits later selected tests to run. The same +classification applies to raw single-file, same-package, external-package, and +honest test-only descriptors; an active-test call reached through production +package code; filtered execution; a coordinator-run retained binary; and later +direct execution of a retained binary. Production files do not acquire a test +descriptor. A nonempty reason keeps the same presentation with its bytes after +`SKIP: `. An oversized reason remains an invalid control result, and +`test.skip` outside an active test still aborts. This result rule does not +change the current per-test process boundary, list mode, filtering language, +package initialization, or package/import/action identity. + After those ordered results, an ordinary explicit `ww test` request that records an attributable setup, build, or execution failure emits exactly one standalone `FAIL\n` on standard output. The line is command-owned: it follows even a later diff --git a/docs/test-system-v2.md b/docs/test-system-v2.md index 9d28a39e..0f3128a7 100644 --- a/docs/test-system-v2.md +++ b/docs/test-system-v2.md @@ -235,6 +235,21 @@ work never caches a test result. A directly invoked retained binary exposes the warning on stderr and accounting on stdout because no coordinator combines its descriptors or emits a package result. +The shared language runtime accepts `test.skip("")` as a successful skip. Its +zero-length reason is a structurally valid skipped control result, so a normal +zero child exit produces the existing `qualified.name ... SKIP: ` line followed +immediately by newline, increments skipped accounting, and contributes neither +a failure nor a harness error. Later selected tests continue. This one runtime +classification covers raw single-file, same-package, external-test, and honest +test-only descriptors, an active-test call reached through production package +code, filtered execution, coordinator-run retained binaries, and later direct +retained-binary execution. Production files still define no test-body +descriptor. Nonempty skips are unchanged; an oversized reason is still an +invalid harness result, and skip outside an active test still aborts. List mode +still executes no body. This rule does not merge per-test children or change +package/import initialization, product scheduling, timeout and process-group +ownership, publication, persistence, or cleanup. + Directory test binaries are always linked under the coordinator's temporary product root. The retained-output option has the exact registered name `o` and accepts `-o VALUE`, `--o VALUE`, `-o=VALUE`, and `--o=VALUE`. An equals form diff --git a/lib/test/run.ww b/lib/test/run.ww index 79e82c91..af1d3cbb 100644 --- a/lib/test/run.ww +++ b/lib/test/run.ww @@ -140,7 +140,7 @@ export fn run(tests: [](str, *fn() void)) i32 = { // exit status alone cannot say whether user code returned, skipped, or died. export fn skip(reason: str) never = { if (!tstactive) { abort("test.skip called outside a test"); }; - if (reason.len == 0 || reason.len > TST_CONTROL_MAX - TST_FRAME_HEADER) { + if (reason.len > TST_CONTROL_MAX - TST_FRAME_HEADER) { tstwriteframe(0u8, ""); os.exit(0); }; @@ -458,7 +458,7 @@ fn tstinterpret(result: *tstresult) void = { }; return; }; - if (code == TST_SKIPPED && payload > 0) { + if (code == TST_SKIPPED) { if (result.exited && result.detail == 0) { result.kind = TST_SKIP; } else { diff --git a/test/package/package_test.ww b/test/package/package_test.ww index e1b02719..7b72a894 100644 --- a/test/package/package_test.ww +++ b/test/package/package_test.ww @@ -18689,3 +18689,355 @@ fn runtimepath(relative: str) str = { && !directoryhasfragment(root, ".install")); clean(root); }; + +// A skip is an outcome even when it carries no explanatory bytes. Keep that +// boundary identical through every directory-package test descriptor and the +// retained-binary route; an empty payload must not become a harness error. +@test fn empty_skip_reason_is_a_successful_skip() void = { + let root: str = fresh(); + let mixed: str = strings.concat(root, "/mixed"); + let only: str = strings.concat(root, "/only"); + let raw: str = strings.concat(root, "/raw_test.ww"); + let oversized: str = strings.concat(root, "/oversized"); + let outside: str = strings.concat(root, "/outside"); + mkdirall(mixed); + mkdirall(only); + mkdirall(oversized); + mkdirall(outside); + writefile(strings.concat(mixed, "/prod.ww"), strings.concat( + "package skipmix;\n", + "import test;\n", + "export fn value() i32 = { return 7; };\n", + "export fn skip_from_production() never = { test.skip(\"\"); };\n")); + writefile(strings.concat(mixed, "/same_test.ww"), strings.concat( + "package skipmix;\n", + "import test;\n", + "@test fn aa_empty_production() void = { skip_from_production(); };\n", + "@test fn aa_empty_same() void = { test.skip(\"\"); };\n", + "@test fn ab_same_peer() void = { assert(value() == 7); };\n", + "@test fn ac_named_same() void = { test.skip(\"named reason\"); };\n")); + writefile(strings.concat(mixed, "/external_test.ww"), strings.concat( + "package skipmix_test;\n", + "import skipmix;\n", + "import test;\n", + "@test fn ad_empty_external() void = {\n", + " assert(skipmix.value() == 7); test.skip(\"\");\n", + "};\n", + "@test fn ae_external_peer() void = {\n", + " assert(skipmix.value() == 7);\n", + "};\n")); + writefile(strings.concat(only, "/only_test.ww"), strings.concat( + "package skiponly;\n", + "import test;\n", + "@test fn aa_empty_test_only() void = { test.skip(\"\"); };\n", + "@test fn ab_test_only_peer() void = { assert(true); };\n")); + writefile(raw, strings.concat( + "package rawskip;\n", + "import test;\n", + "@test fn aa_empty_raw() void = { test.skip(\"\"); };\n", + "@test fn ab_raw_peer() void = { assert(true); };\n")); + writefile(strings.concat(oversized, "/oversized_test.ww"), strings.concat( + "package skipoversized;\n", + "import strings;\n", + "import test;\n", + "@test fn too_large() void = {\n", + " let reason: []u8 = alloc([], 4094u64)!; reason.len = 4094;\n", + " let i: i32 = 0; for (i < reason.len) { reason[i] = 'x'; i += 1; };\n", + " test.skip(strings.frombytes(reason));\n", + "};\n")); + writefile(strings.concat(outside, "/main.ww"), strings.concat( + "package main;\n", + "import test;\n", + "fn main() void = { test.skip(\"\"); };\n")); + + let stages: []str = ["ww", "ww_ww"]; + let tags: []str = ["c", "ww"]; + let works: []str = [strings.concat(root, "/work-c"), + strings.concat(root, "/work-ww")]; + let oversizedworks: []str = [strings.concat(root, "/oversized-work-c"), + strings.concat(root, "/oversized-work-ww")]; + let outsideworks: []str = [strings.concat(root, "/outside-work-c"), + strings.concat(root, "/outside-work-ww")]; + let outsidebins: []str = [strings.concat(root, "/outside-c"), + strings.concat(root, "/outside-ww")]; + let retained: []str = [strings.concat(root, "/retained-c.test"), + strings.concat(root, "/retained-ww.test")]; + let runningretained: []str = [strings.concat(root, "/running-c.test"), + strings.concat(root, "/running-ww.test")]; + let ordinaryout: []str = ["", ""]; + let ordinaryerr: []str = ["", ""]; + let emptyout: []str = ["", ""]; + let emptyerr: []str = ["", ""]; + let passout: []str = ["", ""]; + let passerr: []str = ["", ""]; + let listout: []str = ["", ""]; + let listerr: []str = ["", ""]; + let rawout: []str = ["", ""]; + let rawerr: []str = ["", ""]; + let oversizedout: []str = ["", ""]; + let oversizederr: []str = ["", ""]; + let runningout: []str = ["", ""]; + let runningerr: []str = ["", ""]; + let compileout: []str = ["", ""]; + let compileerr: []str = ["", ""]; + let directout: []str = ["", ""]; + let directerr: []str = ["", ""]; + let outsideout: []str = ["", ""]; + let outsideerr: []str = ["", ""]; + let out: commandout; + let si: i32 = 0; + for (si < stages.len) { + mkdirall(outsideworks[si]); + // An ordinary product may import the shared test library. Its artifact + // therefore participates in normal content invalidation, while invoking + // skip without an active test retains the preexisting abort boundary. + let outsidebuildav: []str = [driver(stages[si]), "build", "-w", + outsideworks[si], "-o", outsidebins[si], outside]; + runcommand(root, strings.concat("empty-skip-outside-build-", tags[si]), + outsidebuildav, (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stdout.len == 0 && out.stderr.len == 0 + && os.exists(outsidebins[si])); + let outsiderunav: []str = [outsidebins[si]]; + runcommand(root, strings.concat("empty-skip-outside-run-", tags[si]), + outsiderunav, (30i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 1); + assert(out.stdout.len == 0 + && same(out.stderr, "test.skip called outside a test")); + outsideout[si] = strings.dup(out.stdout); + outsideerr[si] = strings.dup(out.stderr); + assert(!directoryhasnew(outsideworks[si]) + && !directoryhasfragment(outsideworks[si], ".wwtxn.") + && !directoryhasfragment(outsideworks[si], ".install")); + + let ordinaryav: []str = [driver(stages[si]), "test", "-w", + works[si], "-I", root, mixed, only]; + runcommand(root, strings.concat("empty-skip-ordinary-cold-", tags[si]), + ordinaryav, (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stderr.len == 0); + assert(occurrences(out.stdout, + "skipmix.aa_empty_production ... SKIP: \n") == 1); + assert(occurrences(out.stdout, + "skipmix.aa_empty_same ... SKIP: \n") == 1); + assert(occurrences(out.stdout, + "skipmix.ad_empty_external ... SKIP: \n") == 1); + assert(occurrences(out.stdout, + "skiponly.aa_empty_test_only ... SKIP: \n") == 1); + assert(occurrences(out.stdout, + "skipmix.ac_named_same ... SKIP: named reason\n") == 1); + assert(occurrences(out.stdout, + "skipmix.ab_same_peer ... ok\n") == 1); + assert(occurrences(out.stdout, + "skipmix.ae_external_peer ... ok\n") == 1); + assert(occurrences(out.stdout, + "skiponly.ab_test_only_peer ... ok\n") == 1); + assert(pos(out.stdout, "skipmix.aa_empty_same ... SKIP: \n") + < pos(out.stdout, "skipmix.ab_same_peer ... ok\n")); + assert(occurrences(out.stdout, + "2 passed, 0 failed, 4 skipped, 0 harness errors\n") == 1); + assert(occurrences(out.stdout, + "6 discovered, 6 selected, 6 started, 6 completed\n") == 1); + assert(occurrences(out.stdout, + "1 passed, 0 failed, 1 skipped, 0 harness errors\n") == 1); + assert(occurrences(out.stdout, + "2 discovered, 2 selected, 2 started, 2 completed\n") == 1); + assert(occurrences(out.stdout, strings.concat("ok ", mixed, + " [skipmix]\n")) == 1); + assert(occurrences(out.stdout, strings.concat("ok ", only, + " [skiponly]\n")) == 1); + assert(!has(out.stdout, "HARNESS")); + ordinaryout[si] = strings.dup(out.stdout); + ordinaryerr[si] = strings.dup(out.stderr); + + // Exact repetition against persisted work preserves the externally + // visible result and leaves no transactional residue. + runcommand(root, strings.concat("empty-skip-ordinary-warm-", tags[si]), + ordinaryav, (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(same(out.stdout, ordinaryout[si])); + assert(same(out.stderr, ordinaryerr[si])); + assert(!directoryhasnew(works[si]) + && !directoryhasfragment(works[si], ".wwtxn.") + && !directoryhasfragment(works[si], ".install")); + + let emptyav: []str = [driver(stages[si]), "test", "-w", + works[si], "-I", root, "-run", "aa_empty_same", mixed]; + runcommand(root, strings.concat("empty-skip-filter-empty-", tags[si]), + emptyav, (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stderr.len == 0 + && occurrences(out.stdout, + "skipmix.aa_empty_same ... SKIP: \n") == 1 + && occurrences(out.stdout, + "0 passed, 0 failed, 1 skipped, 0 harness errors\n") == 1 + && occurrences(out.stdout, + "6 discovered, 1 selected, 1 started, 1 completed\n") == 1 + && !has(out.stdout, "ab_same_peer ...") + && !has(out.stdout, "HARNESS")); + emptyout[si] = strings.dup(out.stdout); + emptyerr[si] = strings.dup(out.stderr); + + let passav: []str = [driver(stages[si]), "test", "-w", + works[si], "-I", root, "-run", "ab_same_peer", mixed]; + runcommand(root, strings.concat("empty-skip-filter-pass-", tags[si]), + passav, (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stderr.len == 0 + && occurrences(out.stdout, "skipmix.ab_same_peer ... ok\n") == 1 + && occurrences(out.stdout, + "1 passed, 0 failed, 0 skipped, 0 harness errors\n") == 1 + && occurrences(out.stdout, + "6 discovered, 1 selected, 1 started, 1 completed\n") == 1 + && !has(out.stdout, "SKIP:") && !has(out.stdout, "HARNESS")); + passout[si] = strings.dup(out.stdout); + passerr[si] = strings.dup(out.stderr); + + let listav: []str = [driver(stages[si]), "test", "-w", works[si], + "-I", root, "-list", mixed, only]; + runcommand(root, strings.concat("empty-skip-list-", tags[si]), listav, + (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stderr.len == 0); + assert(occurrences(out.stdout, "skipmix.aa_empty_same\n") == 1 + && occurrences(out.stdout, "skipmix.ad_empty_external\n") == 1 + && occurrences(out.stdout, "skiponly.aa_empty_test_only\n") == 1 + && !has(out.stdout, " ... ") && !has(out.stdout, "SKIP:") + && !has(out.stdout, " discovered, ") && !has(out.stdout, "HARNESS")); + listout[si] = strings.dup(out.stdout); + listerr[si] = strings.dup(out.stderr); + + // The raw single-file compatibility route wraps the same runtime and + // therefore accepts the same zero-byte skip payload before continuing. + let rawav: []str = [driver(stages[si]), "test", raw]; + runcommand(root, strings.concat("empty-skip-raw-", tags[si]), rawav, + (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stderr.len == 0 + && occurrences(out.stdout, + "aa_empty_raw ... SKIP: \n") == 1 + && occurrences(out.stdout, "ab_raw_peer ... ok\n") == 1 + && pos(out.stdout, "aa_empty_raw ... SKIP: \n") + < pos(out.stdout, "ab_raw_peer ... ok\n") + && occurrences(out.stdout, + "1 passed, 0 failed, 1 skipped, 0 harness errors\n") == 1 + && occurrences(out.stdout, + "2 discovered, 2 selected, 2 started, 2 completed\n") == 1 + && !has(out.stdout, "HARNESS")); + rawout[si] = strings.dup(out.stdout); + rawerr[si] = strings.dup(out.stderr); + + // The empty-payload correction does not broaden the framed protocol: + // an oversized reason remains an isolated harness error. + let oversizedav: []str = [driver(stages[si]), "test", "-w", + oversizedworks[si], oversized]; + runcommand(root, strings.concat("empty-skip-oversized-", tags[si]), + oversizedav, (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 1); + assert(out.stderr.len == 0 + && occurrences(out.stdout, strings.concat( + "skipoversized.too_large ... ", + "HARNESS (malformed or contradictory result)\n")) == 1 + && occurrences(out.stdout, + "0 passed, 0 failed, 0 skipped, 1 harness errors\n") == 1 + && occurrences(out.stdout, + "1 discovered, 1 selected, 1 started, 1 completed\n") == 1 + && !has(out.stdout, "SKIP:")); + oversizedout[si] = strings.dup(out.stdout); + oversizederr[si] = strings.dup(out.stderr); + assert(!directoryhasnew(oversizedworks[si]) + && !directoryhasfragment(oversizedworks[si], ".wwtxn.") + && !directoryhasfragment(oversizedworks[si], ".install")); + + // Running retention consumes the same result before guarded publication. + let runningav: []str = [driver(stages[si]), "test", "-w", + works[si], "-o", runningretained[si], "-I", root, mixed]; + runcommand(root, strings.concat("empty-skip-running-retained-", tags[si]), + runningav, (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stderr.len == 0 + && occurrences(out.stdout, + "skipmix.aa_empty_production ... SKIP: \n") == 1 + && occurrences(out.stdout, + "skipmix.aa_empty_same ... SKIP: \n") == 1 + && occurrences(out.stdout, + "skipmix.ad_empty_external ... SKIP: \n") == 1 + && occurrences(out.stdout, + "skipmix.ac_named_same ... SKIP: named reason\n") == 1 + && occurrences(out.stdout, + "2 passed, 0 failed, 4 skipped, 0 harness errors\n") == 1 + && occurrences(out.stdout, + "6 discovered, 6 selected, 6 started, 6 completed\n") == 1 + && !has(out.stdout, "HARNESS") + && os.exists(runningretained[si]) + && !os.exists(strings.concat(runningretained[si], ".new")) + && !os.exists(strings.concat(runningretained[si], ".sepwork"))); + runningout[si] = strings.dup(out.stdout); + runningerr[si] = strings.dup(out.stderr); + + let compileav: []str = [driver(stages[si]), "test", "-c", "-w", + works[si], "-o", retained[si], "-I", root, mixed]; + runcommand(root, strings.concat("empty-skip-compile-", tags[si]), + compileav, (120i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stdout.len == 0 && out.stderr.len == 0 + && os.exists(retained[si]) + && !os.exists(strings.concat(retained[si], ".new")) + && !os.exists(strings.concat(retained[si], ".sepwork"))); + compileout[si] = strings.dup(out.stdout); + compileerr[si] = strings.dup(out.stderr); + + let directav: []str = [retained[si], "-package=skipmix"]; + runcommand(root, strings.concat("empty-skip-direct-", tags[si]), + directav, (30i64 * (time.second: i64)): time.duration, &out); + expectexit(&out, 0); + assert(out.stderr.len == 0 + && occurrences(out.stdout, + "skipmix.aa_empty_production ... SKIP: \n") == 1 + && occurrences(out.stdout, + "skipmix.aa_empty_same ... SKIP: \n") == 1 + && occurrences(out.stdout, + "skipmix.ad_empty_external ... SKIP: \n") == 1 + && occurrences(out.stdout, + "skipmix.ac_named_same ... SKIP: named reason\n") == 1 + && occurrences(out.stdout, + "2 passed, 0 failed, 4 skipped, 0 harness errors\n") == 1 + && occurrences(out.stdout, + "6 discovered, 6 selected, 6 started, 6 completed\n") == 1 + && !has(out.stdout, "HARNESS")); + directout[si] = strings.dup(out.stdout); + directerr[si] = strings.dup(out.stderr); + assert(!directoryhasnew(works[si]) + && !directoryhasfragment(works[si], ".wwtxn.") + && !directoryhasfragment(works[si], ".install")); + si += 1; + }; + + assert(same(ordinaryout[0], ordinaryout[1]) + && same(ordinaryerr[0], ordinaryerr[1])); + assert(same(emptyout[0], emptyout[1]) + && same(emptyerr[0], emptyerr[1])); + assert(same(passout[0], passout[1]) + && same(passerr[0], passerr[1])); + assert(same(listout[0], listout[1]) + && same(listerr[0], listerr[1])); + assert(same(rawout[0], rawout[1]) + && same(rawerr[0], rawerr[1])); + assert(same(oversizedout[0], oversizedout[1]) + && same(oversizederr[0], oversizederr[1])); + assert(same(runningout[0], runningout[1]) + && same(runningerr[0], runningerr[1])); + assert(same(readfile(runningretained[0]), readfile(runningretained[1]))); + assert(same(compileout[0], compileout[1]) + && same(compileerr[0], compileerr[1])); + assert(same(readfile(retained[0]), readfile(retained[1]))); + assert(same(directout[0], directout[1]) + && same(directerr[0], directerr[1])); + assert(same(outsideout[0], outsideout[1]) + && same(outsideerr[0], outsideerr[1])); + assert(same(readfile(outsidebins[0]), readfile(outsidebins[1]))); + assert(!directoryhasnew(root) + && !directoryhasfragment(root, ".wwtxn.") + && !directoryhasfragment(root, ".install")); + clean(root); +};