Hojun-Cho/ww
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test: accept empty skip reasons

Browse Source
This commit is contained in:
Hojun-Cho
2026-08-22 12:59:08 +09:00
parent 78c9eee82d
commit 91fce80b37
5 changed files with 610 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

225
docs/build-system.md
View File

@@ -9320,6 +9320,231 @@ fixed point and round-tripped all five WWstage tools byte-identically; the
platform gate produced a byte-identical dynamic-link result. No production
code changed after the successful ordered sequence began.
### 11.47 Implemented empty-reason test skip classification
Pinned Go's applicable semantic rule is that a test may skip without supplying
a message, remains a successful skipped test, and does not prevent the next
selected test from running. Mapping that rule to WW's already representable
`test.skip("")` call is **behavior derived from the pinned implementation**.
WW realizes that rule by writing a structurally valid `TST_SKIPPED` control
frame with a zero-length payload and accepting it when the child otherwise exits
normally with status zero. Presentation remains
`qualified.name ... SKIP: ` followed immediately by newline. Skipped
accounting increases once; failure and harness-error accounting do not.
Nonempty reasons keep their existing behavior. A 4,094-byte reason remains an
invalid control result, and skip outside an active test still aborts. Those
WW-specific frame, presentation, bound, accounting, and outside-active details
are **directly measured WW behavior** in the post-change proof below; they are
not attributed to Go's implementation.
#### Pinned evidence and fact classification
The sole semantic authority is official Go 1.26.5 at
`c19862e5f8415b4f24b189d065ed739517c548ba`:
- [`(*T).Skip` and `(*T).SkipNow` in `testing/testing.go`, lines
1231–1259](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/testing.go#L1231-L1259)
show `Skip` accepting a variadic argument list and logging
`fmt.Sprintln(args...)` before calling zero-argument `SkipNow`; `SkipNow`
marks and stops that test and explicitly permits the next test to continue;
- [the package documentation in `testing/testing.go`, lines
273–293](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/testing.go#L273-L293)
includes a zero-argument `t.Skip()`; and
- official assertions in
[`testing/sub_test.go`, lines
182–190](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/sub_test.go#L182-L190),
[237–240](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/sub_test.go#L237-L240),
and
[341–346](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/testing/sub_test.go#L341-L346)
mark message-less `SkipNow` and zero-argument `t.Skip()` cases successful.
Those methods, documentation, and official assertions are **behavior directly
implemented or asserted by pinned Go**: a message-less call is admitted and
successful, is classified skipped, and permits the next selected test to run.
That a suite containing this skip and otherwise passing selected tests has an
overall successful result is **behavior derived from the pinned
implementation**. The rule honestly applies within WW's local, dotted-import,
manifest-free model:
WW's public `test.skip` already takes a `str`, and that type already represents
the applicable empty-reason case without adding Go syntax, modules, manifests,
package-path conventions, regex filters, or a new build language.
#### Fresh four-axis audit and direct pre-fix measurements
The fresh audit classified multiple named source files, explicit
`*_test.ww` build operands, shared top-level test-process state and abort
boundaries, bare effective import bindings, and lexical import shadowing as
applicable differences that remain unselected. Leading-underscore explicit
source behavior retains a bounded applicability question, mixed declarations
are aligned, and grouped, quoted, dot, and generalized imports are inapplicable
to WW's import model. The exact explicitly named `*_test.go` build case lacks an
official pinned testdata assertion even though pinned source derives its
behavior, so it was not evidence-complete for this session. The empty-reason
skip was selected because its complete observable meaning has one bounded
shared-runtime owner and does not partially implement or redefine those broader
differences.
Before this slice, the following observations were **directly measured WW
behavior**. The fixed-path probe
`/tmp/ww-empty-skip-pre.Wkh1CW/pkg` defined production `marker()`, then test
`first`, which called `test.skip("")`, and passing test `second`. Both Cstage
and WWstage exited 1, wrote empty stderr, and emitted byte-identical 255-byte
stdout with SHA-256
`ed5015be2946b902cd42819d36ad102af34c229732b477441dc4b285ed612be3`:
```text
emptyskip.first ... HARNESS (malformed or contradictory result)
emptyskip.second ... ok
1 passed, 0 failed, 0 skipped, 1 harness errors
2 discovered, 2 selected, 2 started, 2 completed
FAIL /tmp/ww-empty-skip-pre.Wkh1CW/pkg [emptyskip] (test exit 1)
FAIL
```
Filtering to `first` failed with one harness error in both stages; filtering to
`second` succeeded; list mode succeeded and printed both names without
executing either body. `test -c -o` succeeded with empty streams and produced
stage-byte-identical 112,861-byte executables with SHA-256
`2d8f3a1adbd2f158ac605b4cecf9966a2b700ff0cc7f31aaa431bad3bc89e844`.
Direct execution of those binaries failed identically, with stdout SHA-256
`b83ef1368512dd3d2ee46c5e6d1c08369b40eb4b849eae70303aebaef80ee650`
and empty stderr. These are pre-fix measurements only.
For that fixed fixture, the directly measured pre-fix four-axis result was:
- **Go-like build:** both stages selected and built the production/test inputs
sufficiently to publish byte-identical retained test executables; this probe
did not independently trace graph/action topology;
- **Go-like test:** empty-reason classification, result presentation,
accounting, package status, and direct retained execution were different,
while filtering, list non-execution, and later-test execution were aligned;
- **Go-like package:** the production source and same-package test source formed
one runnable package result; this probe made no external/test-only-role claim;
and
- **Go-like import:** `import test` resolved and linked in both stages; this
probe made no separate qualifier, unused-import, or graph-provenance claim.
The broader unchanged role, identity, graph, persistence, and lifecycle items
below are implementation boundaries, not additional facts attributed to this
pre-fix fixture.
#### Ownership and complete four-axis contract
The true semantic owner is the shared in-binary test control-frame producer and
interpreter in `lib/test/run.ww`. Both stages link that one WW runtime; there is
no duplicate C/WW implementation. The producer rejects only a reason larger
than the control-frame payload limit and writes the empty `TST_SKIPPED` payload
normally. After the existing header and exact-length checks, the interpreter
accepts skipped code with payload length zero as well as positive length, while
retaining the requirement for normal status-zero child termination.
- **Go-like build:** ordinary source loading, graph/action construction and
scheduling, compiler, assembler, archiver, linker, naming, and output
publication semantics do not change. The source-content change invalidates
and relinks affected products through existing dependency rules. Ordinary
products that do not import `test` stay outside the changed source; an
ordinary product that explicitly imports `test` may rebuild with changed
artifact bytes, while its outside-active runtime abort remains unchanged.
- **Go-like test:** every shared-runtime descriptor accepts the empty reason as
one successful skip. This covers raw single-file, same-package,
external-test, honest test-only, filtered, coordinator-run retained, and
later direct retained-binary execution, including an active-test call reached
through production package code. Production source still contributes no test
descriptor. The result has no diagnostic or harness error, uses the existing
blank-after-colon skip line, increments only skipped accounting, and permits
later selected tests to continue.
- **Go-like package:** source roles, descriptor order, initialization, declared
names, and canonical package and variant identities do not change. A runtime
reason is result data, never identity.
- **Go-like import:** `import test`, exact dotted dependency resolution, graph
edges, initialization, qualifier binding, and unused-import behavior do not
change. No reason byte becomes import, graph, action, symbol, `.wwi`,
artifact, publication, or persistence identity.
#### Lifecycle, parity, proof, and formats
Filename and source eligibility, selected-file order, and test-role
classification are unchanged. An empty reason has no loader representation.
Graph nodes, edges, actions, ordering, product scheduling, generated
descriptors, compiler/assembler/archiver/linker operation, and private/public
output naming remain unchanged. Existing source-content invalidation rebuilds
affected runtime/test actions; there is no test-result cache.
At runtime, only the valid zero-length skip classification changes. Passes,
nonempty skips, assertion failures, signals, expected abort, premature clean
exit, ordinary abort, timeout ownership, process groups, descendant cleanup,
and the existing per-test process boundary retain their current owners and
behavior. An empty skip emits no diagnostic. Oversized reasons remain harness
errors, skip outside an active test still aborts, and all build, producer,
runtime, publication, and cleanup diagnostic channels and precedence remain
unchanged.
Affected private and retained test executables change because their shared
runtime changes; comparable Cstage and WWstage executables must remain
byte-identical. `-c`, running `-o`, guarded installation, destination modes and
names, exact null discard, private execution, and publication order are
unchanged. A successful empty skip reaches the existing success-publication
path. Producer, other runtime, and publication failures retain their existing
failure classification, cold no-partial-publication guarantee, and warm
preservation of committed generations and prior public bytes.
Persistence keys and schemas do not change. Existing content invalidation,
warm action reuse, relink, commit, rollback, parallel product isolation,
capture ownership, cancellation, interruption escalation, owned-child cleanup,
and transaction/scratch cleanup remain unchanged. No active `.new`,
`.install`, `.wwtxn.*`, adjacent `.sepwork`, capture, result, scratch, or
tool-stage transaction residue may remain outside an explicitly retained or
recipe-owned boundary.
After the change, both stages directly ran the fixed-path two-test probe with
status 0 and empty stderr. They emitted the same empty-reason skip line, ran the
later passing test, reported one pass, one skip, zero harness errors, complete
2/2 accounting, and the ordinary package `ok` result. Empty-only and pass-only
filters each succeeded, and list mode still printed both names without running
them. Both `test -c -o` invocations had empty streams and produced
byte-identical 112,861-byte mode-0755 executables with SHA-256
`257d05a99e920875e9d131ce18e7f11592179802fc6f864fc136ebf618d4a88c`.
Both retained executables then ran directly with status 0, identical empty
stderr, and identical skip/pass/accounting output.
The WW-native `empty_skip_reason_is_a_successful_skip` observer passed after
rebuilding `out/bin/test_package`. Its dynamic sources directly prove both-stage
status and stdout/stderr parity for raw single-file, same-package, external,
and honest test-only descriptors; an active-test call through production code;
later-test continuation; a nonempty-skip control; empty-only and pass-only
filters; list nonexecution; cold and warm work; compile-only and running
retention; byte-identical retained and ordinary `import test` executables;
direct retained execution; unchanged outside-active abort; the unchanged
4,094-byte oversized-reason harness error; and transaction/residue cleanup.
The complete focused `make -j1 JOBS=1 test-package` owner then passed all 53
observers with zero failures, skips, or harness errors. All ordered full gates
then passed serially after the final executable production and proof changes:
1. `make -j4 JOBS=4 test`
2. `make -j4 JOBS=4 test-commit`
3. `make -j4 JOBS=4 test-byteid`
4. `make -j1 JOBS=1 test-bootstrap`
5. `make -j1 JOBS=1 test-platform`
6. `make -j1 JOBS=1 test-all`
Every command exited zero. The byte-identity gates compared 161 language files
and 1,421 data fixtures with zero pinned-divergent fixtures; bootstrap retained
the `ww2 == ww3 == ww4` fixed point and five-tool WWstage round-trip byte
identity; and the platform dynamic-link artifact remained Cstage/WWstage
byte-identical at 8,464 bytes.
A final read-only review produced two wording-only corrections: the pinned
`Skip`/`SkipNow` description was split precisely, and the warm observer comment
was limited to persisted-work execution rather than claiming unmeasured action
reuse. With production and executable proof logic unchanged, the exact final
observer source then passed `make -j1 JOBS=1 test-package` again: all 53 tests
passed with zero failures, skips, or harness errors.
No format bump. Build workdir format remains `18`, test workdir format remains
`19`, semantic storage format remains `3`, and no test-result cache is
introduced.
## 12. Candidate architectures and hard-gate decision
Five candidates were developed as coherent systems, not as feature bins.

16
docs/spec.md
View File

@@ -930,6 +930,22 @@ and accounting in its product-local capture; a retained binary invoked directly
keeps the warning on standard error, its accounting on standard output, and has
no coordinator package result.
Calling `test.skip(reason)` from an active test records a successful skip and
stops that test body. The empty string is a valid reason: `test.skip("")` is
not a malformed harness result. Its ordinary result line is exactly the
qualified test name, ` ... SKIP: `, and a newline, with no reason bytes after
the space. It increments skipped accounting once, increments neither failed nor
harness-error accounting, and permits later selected tests to run. The same
classification applies to raw single-file, same-package, external-package, and
honest test-only descriptors; an active-test call reached through production
package code; filtered execution; a coordinator-run retained binary; and later
direct execution of a retained binary. Production files do not acquire a test
descriptor. A nonempty reason keeps the same presentation with its bytes after
`SKIP: `. An oversized reason remains an invalid control result, and
`test.skip` outside an active test still aborts. This result rule does not
change the current per-test process boundary, list mode, filtering language,
package initialization, or package/import/action identity.
After those ordered results, an ordinary explicit `ww test` request that records
an attributable setup, build, or execution failure emits exactly one standalone
`FAIL\n` on standard output. The line is command-owned: it follows even a later

15
docs/test-system-v2.md
View File

@@ -235,6 +235,21 @@ work never caches a test result. A directly invoked retained binary exposes the
warning on stderr and accounting on stdout because no coordinator combines its
descriptors or emits a package result.
The shared language runtime accepts `test.skip("")` as a successful skip. Its
zero-length reason is a structurally valid skipped control result, so a normal
zero child exit produces the existing `qualified.name ... SKIP: ` line followed
immediately by newline, increments skipped accounting, and contributes neither
a failure nor a harness error. Later selected tests continue. This one runtime
classification covers raw single-file, same-package, external-test, and honest
test-only descriptors, an active-test call reached through production package
code, filtered execution, coordinator-run retained binaries, and later direct
retained-binary execution. Production files still define no test-body
descriptor. Nonempty skips are unchanged; an oversized reason is still an
invalid harness result, and skip outside an active test still aborts. List mode
still executes no body. This rule does not merge per-test children or change
package/import initialization, product scheduling, timeout and process-group
ownership, publication, persistence, or cleanup.
Directory test binaries are always linked under the coordinator's temporary
product root. The retained-output option has the exact registered name `o` and
accepts `-o VALUE`, `--o VALUE`, `-o=VALUE`, and `--o=VALUE`. An equals form