test: accept empty skip reasons
This commit is contained in:
@@ -9320,6 +9320,231 @@ fixed point and round-tripped all five WWstage tools byte-identically; the
|
||||
platform gate produced a byte-identical dynamic-link result. No production
|
||||
code changed after the successful ordered sequence began.
|
||||
|
||||
### 11.47 Implemented empty-reason test skip classification
|
||||
|
||||
Pinned Go's applicable semantic rule is that a test may skip without supplying
|
||||
a message, remains a successful skipped test, and does not prevent the next
|
||||
selected test from running. Mapping that rule to WW's already representable
|
||||
`test.skip("")` call is **behavior derived from the pinned implementation**.
|
||||
|
||||
WW realizes that rule by writing a structurally valid `TST_SKIPPED` control
|
||||
frame with a zero-length payload and accepting it when the child otherwise exits
|
||||
normally with status zero. Presentation remains
|
||||
`qualified.name ... SKIP: ` followed immediately by newline. Skipped
|
||||
accounting increases once; failure and harness-error accounting do not.
|
||||
Nonempty reasons keep their existing behavior. A 4,094-byte reason remains an
|
||||
invalid control result, and skip outside an active test still aborts. Those
|
||||
WW-specific frame, presentation, bound, accounting, and outside-active details
|
||||
are **directly measured WW behavior** in the post-change proof below; they are
|
||||
not attributed to Go's implementation.
|
||||
|
||||
#### Pinned evidence and fact classification
|
||||
|
||||
The sole semantic authority is official Go 1.26.5 at
|
||||
`c19862e5f8415b4f24b189d065ed739517c548ba`:
|
||||
|
||||
- [`(*T).Skip` and `(*T).SkipNow` in `testing/testing.go`, lines
|
||||
1231–1259](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/testing.go#L1231-L1259)
|
||||
show `Skip` accepting a variadic argument list and logging
|
||||
`fmt.Sprintln(args...)` before calling zero-argument `SkipNow`; `SkipNow`
|
||||
marks and stops that test and explicitly permits the next test to continue;
|
||||
- [the package documentation in `testing/testing.go`, lines
|
||||
273–293](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/testing.go#L273-L293)
|
||||
includes a zero-argument `t.Skip()`; and
|
||||
- official assertions in
|
||||
[`testing/sub_test.go`, lines
|
||||
182–190](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/sub_test.go#L182-L190),
|
||||
[237–240](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/sub_test.go#L237-L240),
|
||||
and
|
||||
[341–346](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/sub_test.go#L341-L346)
|
||||
mark message-less `SkipNow` and zero-argument `t.Skip()` cases successful.
|
||||
|
||||
Those methods, documentation, and official assertions are **behavior directly
|
||||
implemented or asserted by pinned Go**: a message-less call is admitted and
|
||||
successful, is classified skipped, and permits the next selected test to run.
|
||||
That a suite containing this skip and otherwise passing selected tests has an
|
||||
overall successful result is **behavior derived from the pinned
|
||||
implementation**. The rule honestly applies within WW's local, dotted-import,
|
||||
manifest-free model:
|
||||
WW's public `test.skip` already takes a `str`, and that type already represents
|
||||
the applicable empty-reason case without adding Go syntax, modules, manifests,
|
||||
package-path conventions, regex filters, or a new build language.
|
||||
|
||||
#### Fresh four-axis audit and direct pre-fix measurements
|
||||
|
||||
The fresh audit classified multiple named source files, explicit
|
||||
`*_test.ww` build operands, shared top-level test-process state and abort
|
||||
boundaries, bare effective import bindings, and lexical import shadowing as
|
||||
applicable differences that remain unselected. Leading-underscore explicit
|
||||
source behavior retains a bounded applicability question, mixed declarations
|
||||
are aligned, and grouped, quoted, dot, and generalized imports are inapplicable
|
||||
to WW's import model. The exact explicitly named `*_test.go` build case lacks an
|
||||
official pinned testdata assertion even though pinned source derives its
|
||||
behavior, so it was not evidence-complete for this session. The empty-reason
|
||||
skip was selected because its complete observable meaning has one bounded
|
||||
shared-runtime owner and does not partially implement or redefine those broader
|
||||
differences.
|
||||
|
||||
Before this slice, the following observations were **directly measured WW
|
||||
behavior**. The fixed-path probe
|
||||
`/tmp/ww-empty-skip-pre.Wkh1CW/pkg` defined production `marker()`, then test
|
||||
`first`, which called `test.skip("")`, and passing test `second`. Both Cstage
|
||||
and WWstage exited 1, wrote empty stderr, and emitted byte-identical 255-byte
|
||||
stdout with SHA-256
|
||||
`ed5015be2946b902cd42819d36ad102af34c229732b477441dc4b285ed612be3`:
|
||||
|
||||
```text
|
||||
emptyskip.first ... HARNESS (malformed or contradictory result)
|
||||
emptyskip.second ... ok
|
||||
1 passed, 0 failed, 0 skipped, 1 harness errors
|
||||
2 discovered, 2 selected, 2 started, 2 completed
|
||||
FAIL /tmp/ww-empty-skip-pre.Wkh1CW/pkg [emptyskip] (test exit 1)
|
||||
FAIL
|
||||
```
|
||||
|
||||
Filtering to `first` failed with one harness error in both stages; filtering to
|
||||
`second` succeeded; list mode succeeded and printed both names without
|
||||
executing either body. `test -c -o` succeeded with empty streams and produced
|
||||
stage-byte-identical 112,861-byte executables with SHA-256
|
||||
`2d8f3a1adbd2f158ac605b4cecf9966a2b700ff0cc7f31aaa431bad3bc89e844`.
|
||||
Direct execution of those binaries failed identically, with stdout SHA-256
|
||||
`b83ef1368512dd3d2ee46c5e6d1c08369b40eb4b849eae70303aebaef80ee650`
|
||||
and empty stderr. These are pre-fix measurements only.
|
||||
|
||||
For that fixed fixture, the directly measured pre-fix four-axis result was:
|
||||
|
||||
- **Go-like build:** both stages selected and built the production/test inputs
|
||||
sufficiently to publish byte-identical retained test executables; this probe
|
||||
did not independently trace graph/action topology;
|
||||
- **Go-like test:** empty-reason classification, result presentation,
|
||||
accounting, package status, and direct retained execution were different,
|
||||
while filtering, list non-execution, and later-test execution were aligned;
|
||||
- **Go-like package:** the production source and same-package test source formed
|
||||
one runnable package result; this probe made no external/test-only-role claim;
|
||||
and
|
||||
- **Go-like import:** `import test` resolved and linked in both stages; this
|
||||
probe made no separate qualifier, unused-import, or graph-provenance claim.
|
||||
|
||||
The broader unchanged role, identity, graph, persistence, and lifecycle items
|
||||
below are implementation boundaries, not additional facts attributed to this
|
||||
pre-fix fixture.
|
||||
|
||||
#### Ownership and complete four-axis contract
|
||||
|
||||
The true semantic owner is the shared in-binary test control-frame producer and
|
||||
interpreter in `lib/test/run.ww`. Both stages link that one WW runtime; there is
|
||||
no duplicate C/WW implementation. The producer rejects only a reason larger
|
||||
than the control-frame payload limit and writes the empty `TST_SKIPPED` payload
|
||||
normally. After the existing header and exact-length checks, the interpreter
|
||||
accepts skipped code with payload length zero as well as positive length, while
|
||||
retaining the requirement for normal status-zero child termination.
|
||||
|
||||
- **Go-like build:** ordinary source loading, graph/action construction and
|
||||
scheduling, compiler, assembler, archiver, linker, naming, and output
|
||||
publication semantics do not change. The source-content change invalidates
|
||||
and relinks affected products through existing dependency rules. Ordinary
|
||||
products that do not import `test` stay outside the changed source; an
|
||||
ordinary product that explicitly imports `test` may rebuild with changed
|
||||
artifact bytes, while its outside-active runtime abort remains unchanged.
|
||||
- **Go-like test:** every shared-runtime descriptor accepts the empty reason as
|
||||
one successful skip. This covers raw single-file, same-package,
|
||||
external-test, honest test-only, filtered, coordinator-run retained, and
|
||||
later direct retained-binary execution, including an active-test call reached
|
||||
through production package code. Production source still contributes no test
|
||||
descriptor. The result has no diagnostic or harness error, uses the existing
|
||||
blank-after-colon skip line, increments only skipped accounting, and permits
|
||||
later selected tests to continue.
|
||||
- **Go-like package:** source roles, descriptor order, initialization, declared
|
||||
names, and canonical package and variant identities do not change. A runtime
|
||||
reason is result data, never identity.
|
||||
- **Go-like import:** `import test`, exact dotted dependency resolution, graph
|
||||
edges, initialization, qualifier binding, and unused-import behavior do not
|
||||
change. No reason byte becomes import, graph, action, symbol, `.wwi`,
|
||||
artifact, publication, or persistence identity.
|
||||
|
||||
#### Lifecycle, parity, proof, and formats
|
||||
|
||||
Filename and source eligibility, selected-file order, and test-role
|
||||
classification are unchanged. An empty reason has no loader representation.
|
||||
Graph nodes, edges, actions, ordering, product scheduling, generated
|
||||
descriptors, compiler/assembler/archiver/linker operation, and private/public
|
||||
output naming remain unchanged. Existing source-content invalidation rebuilds
|
||||
affected runtime/test actions; there is no test-result cache.
|
||||
|
||||
At runtime, only the valid zero-length skip classification changes. Passes,
|
||||
nonempty skips, assertion failures, signals, expected abort, premature clean
|
||||
exit, ordinary abort, timeout ownership, process groups, descendant cleanup,
|
||||
and the existing per-test process boundary retain their current owners and
|
||||
behavior. An empty skip emits no diagnostic. Oversized reasons remain harness
|
||||
errors, skip outside an active test still aborts, and all build, producer,
|
||||
runtime, publication, and cleanup diagnostic channels and precedence remain
|
||||
unchanged.
|
||||
|
||||
Affected private and retained test executables change because their shared
|
||||
runtime changes; comparable Cstage and WWstage executables must remain
|
||||
byte-identical. `-c`, running `-o`, guarded installation, destination modes and
|
||||
names, exact null discard, private execution, and publication order are
|
||||
unchanged. A successful empty skip reaches the existing success-publication
|
||||
path. Producer, other runtime, and publication failures retain their existing
|
||||
failure classification, cold no-partial-publication guarantee, and warm
|
||||
preservation of committed generations and prior public bytes.
|
||||
|
||||
Persistence keys and schemas do not change. Existing content invalidation,
|
||||
warm action reuse, relink, commit, rollback, parallel product isolation,
|
||||
capture ownership, cancellation, interruption escalation, owned-child cleanup,
|
||||
and transaction/scratch cleanup remain unchanged. No active `.new`,
|
||||
`.install`, `.wwtxn.*`, adjacent `.sepwork`, capture, result, scratch, or
|
||||
tool-stage transaction residue may remain outside an explicitly retained or
|
||||
recipe-owned boundary.
|
||||
|
||||
After the change, both stages directly ran the fixed-path two-test probe with
|
||||
status 0 and empty stderr. They emitted the same empty-reason skip line, ran the
|
||||
later passing test, reported one pass, one skip, zero harness errors, complete
|
||||
2/2 accounting, and the ordinary package `ok` result. Empty-only and pass-only
|
||||
filters each succeeded, and list mode still printed both names without running
|
||||
them. Both `test -c -o` invocations had empty streams and produced
|
||||
byte-identical 112,861-byte mode-0755 executables with SHA-256
|
||||
`257d05a99e920875e9d131ce18e7f11592179802fc6f864fc136ebf618d4a88c`.
|
||||
Both retained executables then ran directly with status 0, identical empty
|
||||
stderr, and identical skip/pass/accounting output.
|
||||
|
||||
The WW-native `empty_skip_reason_is_a_successful_skip` observer passed after
|
||||
rebuilding `out/bin/test_package`. Its dynamic sources directly prove both-stage
|
||||
status and stdout/stderr parity for raw single-file, same-package, external,
|
||||
and honest test-only descriptors; an active-test call through production code;
|
||||
later-test continuation; a nonempty-skip control; empty-only and pass-only
|
||||
filters; list nonexecution; cold and warm work; compile-only and running
|
||||
retention; byte-identical retained and ordinary `import test` executables;
|
||||
direct retained execution; unchanged outside-active abort; the unchanged
|
||||
4,094-byte oversized-reason harness error; and transaction/residue cleanup.
|
||||
The complete focused `make -j1 JOBS=1 test-package` owner then passed all 53
|
||||
observers with zero failures, skips, or harness errors. All ordered full gates
|
||||
then passed serially after the final executable production and proof changes:
|
||||
|
||||
1. `make -j4 JOBS=4 test`
|
||||
2. `make -j4 JOBS=4 test-commit`
|
||||
3. `make -j4 JOBS=4 test-byteid`
|
||||
4. `make -j1 JOBS=1 test-bootstrap`
|
||||
5. `make -j1 JOBS=1 test-platform`
|
||||
6. `make -j1 JOBS=1 test-all`
|
||||
|
||||
Every command exited zero. The byte-identity gates compared 161 language files
|
||||
and 1,421 data fixtures with zero pinned-divergent fixtures; bootstrap retained
|
||||
the `ww2 == ww3 == ww4` fixed point and five-tool WWstage round-trip byte
|
||||
identity; and the platform dynamic-link artifact remained Cstage/WWstage
|
||||
byte-identical at 8,464 bytes.
|
||||
|
||||
A final read-only review produced two wording-only corrections: the pinned
|
||||
`Skip`/`SkipNow` description was split precisely, and the warm observer comment
|
||||
was limited to persisted-work execution rather than claiming unmeasured action
|
||||
reuse. With production and executable proof logic unchanged, the exact final
|
||||
observer source then passed `make -j1 JOBS=1 test-package` again: all 53 tests
|
||||
passed with zero failures, skips, or harness errors.
|
||||
|
||||
No format bump. Build workdir format remains `18`, test workdir format remains
|
||||
`19`, semantic storage format remains `3`, and no test-result cache is
|
||||
introduced.
|
||||
|
||||
## 12. Candidate architectures and hard-gate decision
|
||||
|
||||
Five candidates were developed as coherent systems, not as feature bins.
|
||||
|
||||
16
docs/spec.md
16
docs/spec.md
@@ -930,6 +930,22 @@ and accounting in its product-local capture; a retained binary invoked directly
|
||||
keeps the warning on standard error, its accounting on standard output, and has
|
||||
no coordinator package result.
|
||||
|
||||
Calling `test.skip(reason)` from an active test records a successful skip and
|
||||
stops that test body. The empty string is a valid reason: `test.skip("")` is
|
||||
not a malformed harness result. Its ordinary result line is exactly the
|
||||
qualified test name, ` ... SKIP: `, and a newline, with no reason bytes after
|
||||
the space. It increments skipped accounting once, increments neither failed nor
|
||||
harness-error accounting, and permits later selected tests to run. The same
|
||||
classification applies to raw single-file, same-package, external-package, and
|
||||
honest test-only descriptors; an active-test call reached through production
|
||||
package code; filtered execution; a coordinator-run retained binary; and later
|
||||
direct execution of a retained binary. Production files do not acquire a test
|
||||
descriptor. A nonempty reason keeps the same presentation with its bytes after
|
||||
`SKIP: `. An oversized reason remains an invalid control result, and
|
||||
`test.skip` outside an active test still aborts. This result rule does not
|
||||
change the current per-test process boundary, list mode, filtering language,
|
||||
package initialization, or package/import/action identity.
|
||||
|
||||
After those ordered results, an ordinary explicit `ww test` request that records
|
||||
an attributable setup, build, or execution failure emits exactly one standalone
|
||||
`FAIL\n` on standard output. The line is command-owned: it follows even a later
|
||||
|
||||
@@ -235,6 +235,21 @@ work never caches a test result. A directly invoked retained binary exposes the
|
||||
warning on stderr and accounting on stdout because no coordinator combines its
|
||||
descriptors or emits a package result.
|
||||
|
||||
The shared language runtime accepts `test.skip("")` as a successful skip. Its
|
||||
zero-length reason is a structurally valid skipped control result, so a normal
|
||||
zero child exit produces the existing `qualified.name ... SKIP: ` line followed
|
||||
immediately by newline, increments skipped accounting, and contributes neither
|
||||
a failure nor a harness error. Later selected tests continue. This one runtime
|
||||
classification covers raw single-file, same-package, external-test, and honest
|
||||
test-only descriptors, an active-test call reached through production package
|
||||
code, filtered execution, coordinator-run retained binaries, and later direct
|
||||
retained-binary execution. Production files still define no test-body
|
||||
descriptor. Nonempty skips are unchanged; an oversized reason is still an
|
||||
invalid harness result, and skip outside an active test still aborts. List mode
|
||||
still executes no body. This rule does not merge per-test children or change
|
||||
package/import initialization, product scheduling, timeout and process-group
|
||||
ownership, publication, persistence, or cleanup.
|
||||
|
||||
Directory test binaries are always linked under the coordinator's temporary
|
||||
product root. The retained-output option has the exact registered name `o` and
|
||||
accepts `-o VALUE`, `--o VALUE`, `-o=VALUE`, and `--o=VALUE`. An equals form
|
||||
|
||||
@@ -140,7 +140,7 @@ export fn run(tests: [](str, *fn() void)) i32 = {
|
||||
// exit status alone cannot say whether user code returned, skipped, or died.
|
||||
export fn skip(reason: str) never = {
|
||||
if (!tstactive) { abort("test.skip called outside a test"); };
|
||||
if (reason.len == 0 || reason.len > TST_CONTROL_MAX - TST_FRAME_HEADER) {
|
||||
if (reason.len > TST_CONTROL_MAX - TST_FRAME_HEADER) {
|
||||
tstwriteframe(0u8, "");
|
||||
os.exit(0);
|
||||
};
|
||||
@@ -458,7 +458,7 @@ fn tstinterpret(result: *tstresult) void = {
|
||||
};
|
||||
return;
|
||||
};
|
||||
if (code == TST_SKIPPED && payload > 0) {
|
||||
if (code == TST_SKIPPED) {
|
||||
if (result.exited && result.detail == 0) {
|
||||
result.kind = TST_SKIP;
|
||||
} else {
|
||||
|
||||
@@ -18689,3 +18689,355 @@ fn runtimepath(relative: str) str = {
|
||||
&& !directoryhasfragment(root, ".install"));
|
||||
clean(root);
|
||||
};
|
||||
|
||||
// A skip is an outcome even when it carries no explanatory bytes. Keep that
|
||||
// boundary identical through every directory-package test descriptor and the
|
||||
// retained-binary route; an empty payload must not become a harness error.
|
||||
@test fn empty_skip_reason_is_a_successful_skip() void = {
|
||||
let root: str = fresh();
|
||||
let mixed: str = strings.concat(root, "/mixed");
|
||||
let only: str = strings.concat(root, "/only");
|
||||
let raw: str = strings.concat(root, "/raw_test.ww");
|
||||
let oversized: str = strings.concat(root, "/oversized");
|
||||
let outside: str = strings.concat(root, "/outside");
|
||||
mkdirall(mixed);
|
||||
mkdirall(only);
|
||||
mkdirall(oversized);
|
||||
mkdirall(outside);
|
||||
writefile(strings.concat(mixed, "/prod.ww"), strings.concat(
|
||||
"package skipmix;\n",
|
||||
"import test;\n",
|
||||
"export fn value() i32 = { return 7; };\n",
|
||||
"export fn skip_from_production() never = { test.skip(\"\"); };\n"));
|
||||
writefile(strings.concat(mixed, "/same_test.ww"), strings.concat(
|
||||
"package skipmix;\n",
|
||||
"import test;\n",
|
||||
"@test fn aa_empty_production() void = { skip_from_production(); };\n",
|
||||
"@test fn aa_empty_same() void = { test.skip(\"\"); };\n",
|
||||
"@test fn ab_same_peer() void = { assert(value() == 7); };\n",
|
||||
"@test fn ac_named_same() void = { test.skip(\"named reason\"); };\n"));
|
||||
writefile(strings.concat(mixed, "/external_test.ww"), strings.concat(
|
||||
"package skipmix_test;\n",
|
||||
"import skipmix;\n",
|
||||
"import test;\n",
|
||||
"@test fn ad_empty_external() void = {\n",
|
||||
" assert(skipmix.value() == 7); test.skip(\"\");\n",
|
||||
"};\n",
|
||||
"@test fn ae_external_peer() void = {\n",
|
||||
" assert(skipmix.value() == 7);\n",
|
||||
"};\n"));
|
||||
writefile(strings.concat(only, "/only_test.ww"), strings.concat(
|
||||
"package skiponly;\n",
|
||||
"import test;\n",
|
||||
"@test fn aa_empty_test_only() void = { test.skip(\"\"); };\n",
|
||||
"@test fn ab_test_only_peer() void = { assert(true); };\n"));
|
||||
writefile(raw, strings.concat(
|
||||
"package rawskip;\n",
|
||||
"import test;\n",
|
||||
"@test fn aa_empty_raw() void = { test.skip(\"\"); };\n",
|
||||
"@test fn ab_raw_peer() void = { assert(true); };\n"));
|
||||
writefile(strings.concat(oversized, "/oversized_test.ww"), strings.concat(
|
||||
"package skipoversized;\n",
|
||||
"import strings;\n",
|
||||
"import test;\n",
|
||||
"@test fn too_large() void = {\n",
|
||||
" let reason: []u8 = alloc([], 4094u64)!; reason.len = 4094;\n",
|
||||
" let i: i32 = 0; for (i < reason.len) { reason[i] = 'x'; i += 1; };\n",
|
||||
" test.skip(strings.frombytes(reason));\n",
|
||||
"};\n"));
|
||||
writefile(strings.concat(outside, "/main.ww"), strings.concat(
|
||||
"package main;\n",
|
||||
"import test;\n",
|
||||
"fn main() void = { test.skip(\"\"); };\n"));
|
||||
|
||||
let stages: []str = ["ww", "ww_ww"];
|
||||
let tags: []str = ["c", "ww"];
|
||||
let works: []str = [strings.concat(root, "/work-c"),
|
||||
strings.concat(root, "/work-ww")];
|
||||
let oversizedworks: []str = [strings.concat(root, "/oversized-work-c"),
|
||||
strings.concat(root, "/oversized-work-ww")];
|
||||
let outsideworks: []str = [strings.concat(root, "/outside-work-c"),
|
||||
strings.concat(root, "/outside-work-ww")];
|
||||
let outsidebins: []str = [strings.concat(root, "/outside-c"),
|
||||
strings.concat(root, "/outside-ww")];
|
||||
let retained: []str = [strings.concat(root, "/retained-c.test"),
|
||||
strings.concat(root, "/retained-ww.test")];
|
||||
let runningretained: []str = [strings.concat(root, "/running-c.test"),
|
||||
strings.concat(root, "/running-ww.test")];
|
||||
let ordinaryout: []str = ["", ""];
|
||||
let ordinaryerr: []str = ["", ""];
|
||||
let emptyout: []str = ["", ""];
|
||||
let emptyerr: []str = ["", ""];
|
||||
let passout: []str = ["", ""];
|
||||
let passerr: []str = ["", ""];
|
||||
let listout: []str = ["", ""];
|
||||
let listerr: []str = ["", ""];
|
||||
let rawout: []str = ["", ""];
|
||||
let rawerr: []str = ["", ""];
|
||||
let oversizedout: []str = ["", ""];
|
||||
let oversizederr: []str = ["", ""];
|
||||
let runningout: []str = ["", ""];
|
||||
let runningerr: []str = ["", ""];
|
||||
let compileout: []str = ["", ""];
|
||||
let compileerr: []str = ["", ""];
|
||||
let directout: []str = ["", ""];
|
||||
let directerr: []str = ["", ""];
|
||||
let outsideout: []str = ["", ""];
|
||||
let outsideerr: []str = ["", ""];
|
||||
let out: commandout;
|
||||
let si: i32 = 0;
|
||||
for (si < stages.len) {
|
||||
mkdirall(outsideworks[si]);
|
||||
// An ordinary product may import the shared test library. Its artifact
|
||||
// therefore participates in normal content invalidation, while invoking
|
||||
// skip without an active test retains the preexisting abort boundary.
|
||||
let outsidebuildav: []str = [driver(stages[si]), "build", "-w",
|
||||
outsideworks[si], "-o", outsidebins[si], outside];
|
||||
runcommand(root, strings.concat("empty-skip-outside-build-", tags[si]),
|
||||
outsidebuildav, (120i64 * (time.second: i64)): time.duration, &out);
|
||||
expectexit(&out, 0);
|
||||
assert(out.stdout.len == 0 && out.stderr.len == 0
|
||||
&& os.exists(outsidebins[si]));
|
||||
let outsiderunav: []str = [outsidebins[si]];
|
||||
runcommand(root, strings.concat("empty-skip-outside-run-", tags[si]),
|
||||
outsiderunav, (30i64 * (time.second: i64)): time.duration, &out);
|
||||
expectexit(&out, 1);
|
||||
assert(out.stdout.len == 0
|
||||
&& same(out.stderr, "test.skip called outside a test"));
|
||||
outsideout[si] = strings.dup(out.stdout);
|
||||
outsideerr[si] = strings.dup(out.stderr);
|
||||
assert(!directoryhasnew(outsideworks[si])
|
||||
&& !directoryhasfragment(outsideworks[si], ".wwtxn.")
|
||||
&& !directoryhasfragment(outsideworks[si], ".install"));
|
||||
|
||||
let ordinaryav: []str = [driver(stages[si]), "test", "-w",
|
||||
works[si], "-I", root, mixed, only];
|
||||
runcommand(root, strings.concat("empty-skip-ordinary-cold-", tags[si]),
|
||||
ordinaryav, (120i64 * (time.second: i64)): time.duration, &out);
|
||||
expectexit(&out, 0);
|
||||
assert(out.stderr.len == 0);
|
||||
assert(occurrences(out.stdout,
|
||||
"skipmix.aa_empty_production ... SKIP: \n") == 1);
|
||||
assert(occurrences(out.stdout,
|
||||
"skipmix.aa_empty_same ... SKIP: \n") == 1);
|
||||
assert(occurrences(out.stdout,
|
||||
"skipmix.ad_empty_external ... SKIP: \n") == 1);
|
||||
assert(occurrences(out.stdout,
|
||||
"skiponly.aa_empty_test_only ... SKIP: \n") == 1);
|
||||
assert(occurrences(out.stdout,
|
||||
"skipmix.ac_named_same ... SKIP: named reason\n") == 1);
|
||||
assert(occurrences(out.stdout,
|
||||
"skipmix.ab_same_peer ... ok\n") == 1);
|
||||
assert(occurrences(out.stdout,
|
||||
"skipmix.ae_external_peer ... ok\n") == 1);
|
||||
assert(occurrences(out.stdout,
|
||||
"skiponly.ab_test_only_peer ... ok\n") == 1);
|
||||
assert(pos(out.stdout, "skipmix.aa_empty_same ... SKIP: \n")
|
||||
< pos(out.stdout, "skipmix.ab_same_peer ... ok\n"));
|
||||
assert(occurrences(out.stdout,
|
||||
"2 passed, 0 failed, 4 skipped, 0 harness errors\n") == 1);
|
||||
assert(occurrences(out.stdout,
|
||||
"6 discovered, 6 selected, 6 started, 6 completed\n") == 1);
|
||||
assert(occurrences(out.stdout,
|
||||
"1 passed, 0 failed, 1 skipped, 0 harness errors\n") == 1);
|
||||
assert(occurrences(out.stdout,
|
||||
"2 discovered, 2 selected, 2 started, 2 completed\n") == 1);
|
||||
assert(occurrences(out.stdout, strings.concat("ok ", mixed,
|
||||
" [skipmix]\n")) == 1);
|
||||
assert(occurrences(out.stdout, strings.concat("ok ", only,
|
||||
" [skiponly]\n")) == 1);
|
||||
assert(!has(out.stdout, "HARNESS"));
|
||||
ordinaryout[si] = strings.dup(out.stdout);
|
||||
ordinaryerr[si] = strings.dup(out.stderr);
|
||||
|
||||
// Exact repetition against persisted work preserves the externally
|
||||
// visible result and leaves no transactional residue.
|
||||
runcommand(root, strings.concat("empty-skip-ordinary-warm-", tags[si]),
|
||||
ordinaryav, (120i64 * (time.second: i64)): time.duration, &out);
|
||||
expectexit(&out, 0);
|
||||
assert(same(out.stdout, ordinaryout[si]));
|
||||
assert(same(out.stderr, ordinaryerr[si]));
|
||||
assert(!directoryhasnew(works[si])
|
||||
&& !directoryhasfragment(works[si], ".wwtxn.")
|
||||
&& !directoryhasfragment(works[si], ".install"));
|
||||
|
||||
let emptyav: []str = [driver(stages[si]), "test", "-w",
|
||||
works[si], "-I", root, "-run", "aa_empty_same", mixed];
|
||||
runcommand(root, strings.concat("empty-skip-filter-empty-", tags[si]),
|
||||
emptyav, (120i64 * (time.second: i64)): time.duration, &out);
|
||||
expectexit(&out, 0);
|
||||
assert(out.stderr.len == 0
|
||||
&& occurrences(out.stdout,
|
||||
"skipmix.aa_empty_same ... SKIP: \n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"0 passed, 0 failed, 1 skipped, 0 harness errors\n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"6 discovered, 1 selected, 1 started, 1 completed\n") == 1
|
||||
&& !has(out.stdout, "ab_same_peer ...")
|
||||
&& !has(out.stdout, "HARNESS"));
|
||||
emptyout[si] = strings.dup(out.stdout);
|
||||
emptyerr[si] = strings.dup(out.stderr);
|
||||
|
||||
let passav: []str = [driver(stages[si]), "test", "-w",
|
||||
works[si], "-I", root, "-run", "ab_same_peer", mixed];
|
||||
runcommand(root, strings.concat("empty-skip-filter-pass-", tags[si]),
|
||||
passav, (120i64 * (time.second: i64)): time.duration, &out);
|
||||
expectexit(&out, 0);
|
||||
assert(out.stderr.len == 0
|
||||
&& occurrences(out.stdout, "skipmix.ab_same_peer ... ok\n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"1 passed, 0 failed, 0 skipped, 0 harness errors\n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"6 discovered, 1 selected, 1 started, 1 completed\n") == 1
|
||||
&& !has(out.stdout, "SKIP:") && !has(out.stdout, "HARNESS"));
|
||||
passout[si] = strings.dup(out.stdout);
|
||||
passerr[si] = strings.dup(out.stderr);
|
||||
|
||||
let listav: []str = [driver(stages[si]), "test", "-w", works[si],
|
||||
"-I", root, "-list", mixed, only];
|
||||
runcommand(root, strings.concat("empty-skip-list-", tags[si]), listav,
|
||||
(120i64 * (time.second: i64)): time.duration, &out);
|
||||
expectexit(&out, 0);
|
||||
assert(out.stderr.len == 0);
|
||||
assert(occurrences(out.stdout, "skipmix.aa_empty_same\n") == 1
|
||||
&& occurrences(out.stdout, "skipmix.ad_empty_external\n") == 1
|
||||
&& occurrences(out.stdout, "skiponly.aa_empty_test_only\n") == 1
|
||||
&& !has(out.stdout, " ... ") && !has(out.stdout, "SKIP:")
|
||||
&& !has(out.stdout, " discovered, ") && !has(out.stdout, "HARNESS"));
|
||||
listout[si] = strings.dup(out.stdout);
|
||||
listerr[si] = strings.dup(out.stderr);
|
||||
|
||||
// The raw single-file compatibility route wraps the same runtime and
|
||||
// therefore accepts the same zero-byte skip payload before continuing.
|
||||
let rawav: []str = [driver(stages[si]), "test", raw];
|
||||
runcommand(root, strings.concat("empty-skip-raw-", tags[si]), rawav,
|
||||
(120i64 * (time.second: i64)): time.duration, &out);
|
||||
expectexit(&out, 0);
|
||||
assert(out.stderr.len == 0
|
||||
&& occurrences(out.stdout,
|
||||
"aa_empty_raw ... SKIP: \n") == 1
|
||||
&& occurrences(out.stdout, "ab_raw_peer ... ok\n") == 1
|
||||
&& pos(out.stdout, "aa_empty_raw ... SKIP: \n")
|
||||
< pos(out.stdout, "ab_raw_peer ... ok\n")
|
||||
&& occurrences(out.stdout,
|
||||
"1 passed, 0 failed, 1 skipped, 0 harness errors\n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"2 discovered, 2 selected, 2 started, 2 completed\n") == 1
|
||||
&& !has(out.stdout, "HARNESS"));
|
||||
rawout[si] = strings.dup(out.stdout);
|
||||
rawerr[si] = strings.dup(out.stderr);
|
||||
|
||||
// The empty-payload correction does not broaden the framed protocol:
|
||||
// an oversized reason remains an isolated harness error.
|
||||
let oversizedav: []str = [driver(stages[si]), "test", "-w",
|
||||
oversizedworks[si], oversized];
|
||||
runcommand(root, strings.concat("empty-skip-oversized-", tags[si]),
|
||||
oversizedav, (120i64 * (time.second: i64)): time.duration, &out);
|
||||
expectexit(&out, 1);
|
||||
assert(out.stderr.len == 0
|
||||
&& occurrences(out.stdout, strings.concat(
|
||||
"skipoversized.too_large ... ",
|
||||
"HARNESS (malformed or contradictory result)\n")) == 1
|
||||
&& occurrences(out.stdout,
|
||||
"0 passed, 0 failed, 0 skipped, 1 harness errors\n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"1 discovered, 1 selected, 1 started, 1 completed\n") == 1
|
||||
&& !has(out.stdout, "SKIP:"));
|
||||
oversizedout[si] = strings.dup(out.stdout);
|
||||
oversizederr[si] = strings.dup(out.stderr);
|
||||
assert(!directoryhasnew(oversizedworks[si])
|
||||
&& !directoryhasfragment(oversizedworks[si], ".wwtxn.")
|
||||
&& !directoryhasfragment(oversizedworks[si], ".install"));
|
||||
|
||||
// Running retention consumes the same result before guarded publication.
|
||||
let runningav: []str = [driver(stages[si]), "test", "-w",
|
||||
works[si], "-o", runningretained[si], "-I", root, mixed];
|
||||
runcommand(root, strings.concat("empty-skip-running-retained-", tags[si]),
|
||||
runningav, (120i64 * (time.second: i64)): time.duration, &out);
|
||||
expectexit(&out, 0);
|
||||
assert(out.stderr.len == 0
|
||||
&& occurrences(out.stdout,
|
||||
"skipmix.aa_empty_production ... SKIP: \n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"skipmix.aa_empty_same ... SKIP: \n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"skipmix.ad_empty_external ... SKIP: \n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"skipmix.ac_named_same ... SKIP: named reason\n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"2 passed, 0 failed, 4 skipped, 0 harness errors\n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"6 discovered, 6 selected, 6 started, 6 completed\n") == 1
|
||||
&& !has(out.stdout, "HARNESS")
|
||||
&& os.exists(runningretained[si])
|
||||
&& !os.exists(strings.concat(runningretained[si], ".new"))
|
||||
&& !os.exists(strings.concat(runningretained[si], ".sepwork")));
|
||||
runningout[si] = strings.dup(out.stdout);
|
||||
runningerr[si] = strings.dup(out.stderr);
|
||||
|
||||
let compileav: []str = [driver(stages[si]), "test", "-c", "-w",
|
||||
works[si], "-o", retained[si], "-I", root, mixed];
|
||||
runcommand(root, strings.concat("empty-skip-compile-", tags[si]),
|
||||
compileav, (120i64 * (time.second: i64)): time.duration, &out);
|
||||
expectexit(&out, 0);
|
||||
assert(out.stdout.len == 0 && out.stderr.len == 0
|
||||
&& os.exists(retained[si])
|
||||
&& !os.exists(strings.concat(retained[si], ".new"))
|
||||
&& !os.exists(strings.concat(retained[si], ".sepwork")));
|
||||
compileout[si] = strings.dup(out.stdout);
|
||||
compileerr[si] = strings.dup(out.stderr);
|
||||
|
||||
let directav: []str = [retained[si], "-package=skipmix"];
|
||||
runcommand(root, strings.concat("empty-skip-direct-", tags[si]),
|
||||
directav, (30i64 * (time.second: i64)): time.duration, &out);
|
||||
expectexit(&out, 0);
|
||||
assert(out.stderr.len == 0
|
||||
&& occurrences(out.stdout,
|
||||
"skipmix.aa_empty_production ... SKIP: \n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"skipmix.aa_empty_same ... SKIP: \n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"skipmix.ad_empty_external ... SKIP: \n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"skipmix.ac_named_same ... SKIP: named reason\n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"2 passed, 0 failed, 4 skipped, 0 harness errors\n") == 1
|
||||
&& occurrences(out.stdout,
|
||||
"6 discovered, 6 selected, 6 started, 6 completed\n") == 1
|
||||
&& !has(out.stdout, "HARNESS"));
|
||||
directout[si] = strings.dup(out.stdout);
|
||||
directerr[si] = strings.dup(out.stderr);
|
||||
assert(!directoryhasnew(works[si])
|
||||
&& !directoryhasfragment(works[si], ".wwtxn.")
|
||||
&& !directoryhasfragment(works[si], ".install"));
|
||||
si += 1;
|
||||
};
|
||||
|
||||
assert(same(ordinaryout[0], ordinaryout[1])
|
||||
&& same(ordinaryerr[0], ordinaryerr[1]));
|
||||
assert(same(emptyout[0], emptyout[1])
|
||||
&& same(emptyerr[0], emptyerr[1]));
|
||||
assert(same(passout[0], passout[1])
|
||||
&& same(passerr[0], passerr[1]));
|
||||
assert(same(listout[0], listout[1])
|
||||
&& same(listerr[0], listerr[1]));
|
||||
assert(same(rawout[0], rawout[1])
|
||||
&& same(rawerr[0], rawerr[1]));
|
||||
assert(same(oversizedout[0], oversizedout[1])
|
||||
&& same(oversizederr[0], oversizederr[1]));
|
||||
assert(same(runningout[0], runningout[1])
|
||||
&& same(runningerr[0], runningerr[1]));
|
||||
assert(same(readfile(runningretained[0]), readfile(runningretained[1])));
|
||||
assert(same(compileout[0], compileout[1])
|
||||
&& same(compileerr[0], compileerr[1]));
|
||||
assert(same(readfile(retained[0]), readfile(retained[1])));
|
||||
assert(same(directout[0], directout[1])
|
||||
&& same(directerr[0], directerr[1]));
|
||||
assert(same(outsideout[0], outsideout[1])
|
||||
&& same(outsideerr[0], outsideerr[1]));
|
||||
assert(same(readfile(outsidebins[0]), readfile(outsidebins[1])));
|
||||
assert(!directoryhasnew(root)
|
||||
&& !directoryhasfragment(root, ".wwtxn.")
|
||||
&& !directoryhasfragment(root, ".install"));
|
||||
clean(root);
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user