ww test: retain directory test binaries like Go

This commit is contained in:
2026-08-20 20:13:08 +09:00
parent f28843f2f5
commit db5782eff9
10 changed files with 1014 additions and 213 deletions

View File

@@ -5474,11 +5474,13 @@ as same/external rather than blindly stripping every suffix. The private driver
descriptor is an ordered directory record:
```
--ww-package-test KIND FAMILY PRODUCTION INTERNAL EXTERNAL DIR OUTPUT STATUS
--ww-package-test KIND FAMILY PRODUCTION INTERNAL EXTERNAL DIR OUTPUT PUBLICATION STATUS
```
Missing action selectors are `-`. One descriptor owns at most one output and
one status. Canonically duplicate products and pairwise output/status/staging
Missing action selectors and absent publication are `-`. `OUTPUT` is the
request-private runnable, while optional `PUBLICATION` is its caller-visible
retained copy. One descriptor owns at most one output, publication, and status.
Canonically duplicate products and pairwise output/publication/status/staging
collisions reject before producer execution. Declared names and output stems do
not identify products or actions.
@@ -6158,6 +6160,154 @@ outer stderr to be empty, direct/raw stderr to remain separate, artifacts and
binaries to remain byte-identical, and every temporary or staged path to be
cleaned.
### 11.27 Implemented Go-like directory test-binary retention
Directory-package `ww test` now separates the request-private executable that
the coordinator may run from the optional caller-visible executable it retains.
`-c` means retain without running; `-o` means retain at the requested location
and still run unless `-c` is also present. Output naming, directory fan-out,
duplicate-name preflight, exact null-device discard, executable mode, and
no-test behavior follow the applicable Go 1.26.5 contract.
#### Pinned Go evidence and direct pre-fix measurements
The authority is official Go 1.26.5 at commit
`c19862e5f8415b4f24b189d065ed739517c548ba`:
- `CmdTest.Long` directly states that `-c` writes `pkg.test` in the current
directory and does not run it, while `-o` saves a copy and still runs unless
`-c` is present; a trailing slash or existing directory receives
`pkg.test`
([`cmd/go/internal/test/test.go`, lines 150168](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L150-L168)).
- `testNeedBinary` makes nonempty `-o` an independent retention request
([`test.go`, lines 631646](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L631-L646)).
`runTest` recognizes an existing directory or trailing separator, rejects a
multi-package non-directory output, and preflights every selected package
for duplicate test-binary names before builder execution, except when the
output is the null device
([`test.go`, lines 771804](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L771-L804)).
- `builderTest` takes the ordinary production-only branch when no test files
exist, creating no test link or retained binary
([`test.go`, lines 11331169](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L1133-L1169)).
A real test first links into its action object directory; `-c` or binary
retention adds an install action, only `-c` selects the no-op print action,
and the non-`-c` run action depends on the original build action rather than
the installed copy
([`test.go`, lines 12001313](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L1200-L1313)).
- `testBinaryName` explicitly uses the final import-path element rather than
the declared package name; its command-line-files exception uses the source
package name
([`test.go`, lines 22872300](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L2287-L2300),
[`cmd/go/internal/load/pkg.go`, lines 17271769](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1727-L1769)).
- `BuildInstallFunc` creates parents and installs a linked executable with mode
`0777` filtered by the process umask
([`cmd/go/internal/work/exec.go`, lines 19042000](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1904-L2000),
[`cmd/go/internal/work/shell.go`, lines 119220 and 283301](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/shell.go#L119-L220)).
On the pinned Unix target, only exact `/dev/null` is the null spelling
([`cmd/go/internal/base/path.go`, lines 8192](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/base/path.go#L81-L92)).
- Official `test_compile_multi_pkg.txt` requires missing nested output
directory creation, default current-directory output, rejection of a
non-directory multi-output and duplicate names, `/dev/null` acceptance, and
`-o DIR` retention while tests still run
([lines 338](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/testdata/script/test_compile_multi_pkg.txt#L3-L38)).
The separation between saved and executed paths is a conclusion derived from
the pinned action dependencies: the run consumes the temporary link action,
not the install action. Duplicate names are likewise a materialization
collision, not package identity. Go's sources do not assert that a set of
retained binaries is installed as one rollback transaction; WW keeps its
existing stronger request-wide transaction while matching the observable
accepted, rejected, and preserved outputs. No installed host Go behavior was
used as authority.
Before this slice, direct native measurements of both Cstage and WWstage showed
that single-package `-c -o FILE` retained and did not run, but `-o FILE`
without `-c` exited 2 with `-o needs -c for a package target`; multi-package
`-c -o FILE` exited 2 with the older unconditional fan-out rejection. Default
multi-package `-c` scattered `<declared-package>.test` binaries into their
source directories. Those measurements used the public driver route and
observed exits, diagnostics, files, executable behavior, and stage-equal bytes;
they were not conclusions drawn from WW source.
#### Coordinator policy and identity boundaries
`internal/wwpackage.packagecommand` is the sole owner of public output policy.
It resolves the invocation directory, computes each visible
`<import-leaf>.test` name, recognizes output-directory and `/dev/null` forms,
rejects non-directory fan-out and duplicate names, omits publication for
no-test products, and schedules execution according to `-c`. A contextual
dotted request uses its exact final component; a local path request uses its
directory leaf as the manifest-free presentation equivalent. Neither becomes
declared-name or physical-directory identity.
Every actual test product still links to `package.test` below its private plan
root. The private descriptor carries that `OUTPUT` plus an optional absolute
`PUBLICATION`. The Cstage and WWstage drivers implement only this symmetric
mechanism; they do not independently decide names or CLI policy. The
coordinator always executes `OUTPUT`, so `-o` cannot alter executable argv,
cwd, environment, null stdin, combined output, filters, action topology, or
test outcome.
Visible basename, publication path, private runnable path, declared family,
physical source directory, production/internal/external/recompiled/support/main
variants, symbols, `.wwi`, archives, action identity, and persistence keys
remain distinct. A duplicate basename rejects only the requested
materialization. It never merges, renames, folds, or rekeys either canonical
package. Compiler inputs, exported interfaces, generated main, archive order,
and linked bytes are otherwise unchanged.
#### Publication, execution, failure, and cleanup
Without explicit `-o`, `-c` retains each binary in the invocation directory.
An existing directory or a path ending in `/` receives one visible name per
selected package; missing parents are created with `0777` subject to umask. A
non-directory destination accepts exactly one selected package. Exact
`/dev/null` suppresses retained copies, permits duplicate visible names, and
does not suppress execution unless `-c` is also present. A no-test package
performs ordinary production validation, reports `[no tests]`, and creates no
binary or otherwise-unused output directory. Successful compile-only products
are silent, matching Go's no-op print action.
After linking, the driver copies the private runnable bytes to a distinct
`.new` inode opened with executable mode `0777` subject to umask. Temporary
runnable, retained copy, statuses, changed persistent actions, tool records,
and stamp then enter the existing one-request transaction. All producers and
linkers complete before installation. Any load, compile, assemble, archive,
link, stage, or install failure preserves old retained binaries and persistent
bytes, discards all stages, removes cold scratch, and rolls back only output
prefixes created by that request. Occupied or dangling `.new` paths reject
before tools and are never overwritten.
Execution begins only after publication commits. Assertion failure, signal,
timeout, or child-setup failure therefore leaves an explicitly retained binary
while retaining the established stdout/stderr result routing and sibling
isolation. Parallel products stage independent runnable/copy pairs; the shared
transaction prevents partial sibling publication and canonical result emission
order remains unchanged. Direct invocation of a retained binary continues to
inherit caller cwd, environment, and separate standard descriptors.
`-o` can accompany `-w`: unchanged semantic actions are reused, changed source
invalidates the applicable test actions, the always-run link refreshes the
private runnable and retained copy, and the test still runs. This is build
reuse, not a result cache. `-c` retains its established incompatibility with
`-w`; this slice does not invent persistent compile-only ownership.
No persisted byte schema changed. Build workdir format remains `18`, test
workdir format remains `19`, and semantic storage remains `3`.
The focused native owners are `compile_artifact_naming` and
`test_binary_publication_transaction` in `test/package/package_test.ww`. Their
Cstage/WWstage matrix covers single/default/directory/nested/multi/null output;
declared-name versus import-leaf naming; executable mode and direct execution;
temporary argv versus retained path; no-test omission; duplicate and
non-directory rejection; occupied stages; serial and parallel sibling
publication; injected late-link rollback over old files and newly created
parents; runtime-failure retention; persistent cold/warm/invalidation behavior;
diagnostic equality; retained binary byte identity; and absence of `.new`
residue. Existing package tests continue to own all action/test variants,
graph identity, output ordering, cwd/environment/stdin, timeout, and broader
transaction behavior.
## 12. Candidate architectures and hard-gate decision
Five candidates were developed as coherent systems, not as feature bins.

View File

@@ -645,6 +645,27 @@ mixed directories are accepted from their selected test files. A directory
with no selected test file validates ordinary production but creates no test
support, generated main, link, binary, result, or process.
Every test-bearing directory product links one request-private runnable.
`-c` retains an executable copy and suppresses its execution; `-o` retains a
copy at the named destination and still executes unless `-c` is present. With
no explicit output, `-c` writes `<import-leaf>.test` in the invocation
directory. An output ending in `/` or naming an existing directory receives
that basename and may have missing parent directories created. One
non-directory output may name only one package. Multiple packages whose
visible import leaves would produce the same test-binary name reject before
tools or output creation; exact `/dev/null` is the discard exception. Declared
package names, test variants, source filenames, physical directories, output
paths, and retained binary names remain presentation or loader metadata and do
not become canonical package or action identity.
The retained file is an executable, byte-identical copy of the private
runnable. It joins package artifacts and statuses in the request-wide atomic
publication transaction. Build, link, stage, or install failure preserves old
destinations and removes temporary stages and invocation-created output
prefixes. Test execution starts only after that transaction commits, so a
runtime failure leaves an explicitly retained binary. A no-test product
publishes no binary and does not create a directory solely for one.
When `ww test` executes a directory-owned product, the child process working
directory is that product's canonical absolute physical package source
directory. Its per-run environment has exactly one effective uppercase `PWD`,

View File

@@ -216,18 +216,28 @@ test binaries concurrently under `os.exec` start/poll supervision (no threads);
emission stays strictly in group order, so the byte stream is identical at
every `-j` level, and `-j 1` — the default — matches the former sequential
run loop exactly. Measured on the 31-package `lib/...` walk:
7.0s sequential, 2.4s at `-j 4`. With `-c`, it publishes each exact
`<package>.test` directory binary; the first output owns the one
shared cold sepwork containing the command-global action universe. Those become
caller-owned artifacts. `-c -o <name>`
names that artifact instead of the fixed stem for exactly one directory,
including a combined internal/external directory:
the coordinator rejects a multi-package fan-out ("cannot use -o with
multiple packages", Go's `go test -o` rule), and `-o` without `-c` is
rejected at the driver ("needs -c for a package target") because a plain
run leaves no caller-owned artifact for `-o` to name. Without `-c`, it removes the
temporary binary and scratch with its workspace. The language runtime
owns individual `@test` functions.
7.0s sequential, 2.4s at `-j 4`.
Directory test binaries are always linked under the coordinator's temporary
product root. `-c` independently requests a caller-visible executable copy and
suppresses execution. `-o` independently requests a copy and still runs the
temporary binary unless `-c` is also present. Without `-o`, `-c` publishes
`<import-leaf>.test` in the invocation directory; an output ending in `/` or
naming an existing directory receives one such name per selected package and
missing parents are created. A single non-directory output is legal for one
package only. Multi-package non-directory output and duplicate visible binary
names reject before tools; exact `/dev/null` discards every copy and permits
duplicate names. A package with no selected test source validates production
but publishes nothing and does not create an otherwise unneeded output
directory. Successful `-c` output is silent apart from no-test reporting.
The retained executable is byte-identical to the temporary runnable and has
executable mode `0777` filtered by the caller's umask, but it is never the path
executed by the coordinator. Publication participates in the driver's one
request-wide transaction: producer, linker, staging, or installation failure
preserves every prior destination and removes stages and newly created output
prefixes. A runtime failure occurs after commit and therefore leaves the saved
binary. The language runtime owns individual `@test` functions.
Every actually executed directory product gives its single generated binary
the product's canonical absolute physical source directory as child cwd. A
@@ -409,16 +419,19 @@ bounded-memory Cstage/WWstage allocation-failure parity across the complete
combined package-test graph, under independently discovered ceilings supplied
by the repository-built `sep-limitexec` helper.
`ww build`, an explicit single-file `ww test -o <stem>`, and each successful
directory-package `ww test -c` build publish `<stem>.sepwork` as a caller-owned
artifact directory. The driver acquires it with one fresh `mkdir` and refuses
an existing path; it never clears a collision. A caller keeps only the exact
artifacts it observes and removes that exact tree on every later success or
failure. `ww run` and no-output single-file `ww test` use driver-owned scratch
instead; both driver stages place that scratch and their temporary executable
beneath one freshly acquired directory, remove both after every build result,
and make cleanup failure fail the command. Make recipes build driver-produced
tools in invocation-owned directories and apply the same exact cleanup rule.
`ww build` and an explicit single-file `ww test -o <stem>` publish
`<stem>.sepwork` as a caller-owned artifact directory. The driver acquires it
with one fresh `mkdir` and refuses an existing path; it never clears a
collision. A caller keeps only the exact artifacts it observes and removes
that exact tree on every later success or failure. Directory-package test
plans instead keep their cold semantic-action scratch and runnable binary
inside the coordinator's temporary root; only the optional retained executable
escapes through the transaction above. `ww run` and no-output single-file
`ww test` use driver-owned scratch instead; both driver stages place that
scratch and their temporary executable beneath one freshly acquired directory,
remove both after every build result, and make cleanup failure fail the
command. Make recipes build driver-produced tools in invocation-owned
directories and apply the same exact cleanup rule.
`ww build -w DIR` and single-file `ww test -w DIR` replace that scratch with a
caller-owned persistent package-artifact workdir: for these direct routes the
@@ -561,8 +574,8 @@ timeout policy in this architecture.
## Open driver work
None; the package-level `-o` contract (the last carried bullet) landed as
`-c -o <name>` for exactly one package.
None; directory-package `-c` and `-o` now have the applicable Go 1.26.5
retention, naming, fan-out, execution, and publication behavior.
## Validation policy