ww test: retain directory test binaries like Go
This commit is contained in:
@@ -5474,11 +5474,13 @@ as same/external rather than blindly stripping every suffix. The private driver
|
||||
descriptor is an ordered directory record:
|
||||
|
||||
```
|
||||
--ww-package-test KIND FAMILY PRODUCTION INTERNAL EXTERNAL DIR OUTPUT STATUS
|
||||
--ww-package-test KIND FAMILY PRODUCTION INTERNAL EXTERNAL DIR OUTPUT PUBLICATION STATUS
|
||||
```
|
||||
|
||||
Missing action selectors are `-`. One descriptor owns at most one output and
|
||||
one status. Canonically duplicate products and pairwise output/status/staging
|
||||
Missing action selectors and absent publication are `-`. `OUTPUT` is the
|
||||
request-private runnable, while optional `PUBLICATION` is its caller-visible
|
||||
retained copy. One descriptor owns at most one output, publication, and status.
|
||||
Canonically duplicate products and pairwise output/publication/status/staging
|
||||
collisions reject before producer execution. Declared names and output stems do
|
||||
not identify products or actions.
|
||||
|
||||
@@ -6158,6 +6160,154 @@ outer stderr to be empty, direct/raw stderr to remain separate, artifacts and
|
||||
binaries to remain byte-identical, and every temporary or staged path to be
|
||||
cleaned.
|
||||
|
||||
### 11.27 Implemented Go-like directory test-binary retention
|
||||
|
||||
Directory-package `ww test` now separates the request-private executable that
|
||||
the coordinator may run from the optional caller-visible executable it retains.
|
||||
`-c` means retain without running; `-o` means retain at the requested location
|
||||
and still run unless `-c` is also present. Output naming, directory fan-out,
|
||||
duplicate-name preflight, exact null-device discard, executable mode, and
|
||||
no-test behavior follow the applicable Go 1.26.5 contract.
|
||||
|
||||
#### Pinned Go evidence and direct pre-fix measurements
|
||||
|
||||
The authority is official Go 1.26.5 at commit
|
||||
`c19862e5f8415b4f24b189d065ed739517c548ba`:
|
||||
|
||||
- `CmdTest.Long` directly states that `-c` writes `pkg.test` in the current
|
||||
directory and does not run it, while `-o` saves a copy and still runs unless
|
||||
`-c` is present; a trailing slash or existing directory receives
|
||||
`pkg.test`
|
||||
([`cmd/go/internal/test/test.go`, lines 150–168](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L150-L168)).
|
||||
- `testNeedBinary` makes nonempty `-o` an independent retention request
|
||||
([`test.go`, lines 631–646](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L631-L646)).
|
||||
`runTest` recognizes an existing directory or trailing separator, rejects a
|
||||
multi-package non-directory output, and preflights every selected package
|
||||
for duplicate test-binary names before builder execution, except when the
|
||||
output is the null device
|
||||
([`test.go`, lines 771–804](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L771-L804)).
|
||||
- `builderTest` takes the ordinary production-only branch when no test files
|
||||
exist, creating no test link or retained binary
|
||||
([`test.go`, lines 1133–1169](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L1133-L1169)).
|
||||
A real test first links into its action object directory; `-c` or binary
|
||||
retention adds an install action, only `-c` selects the no-op print action,
|
||||
and the non-`-c` run action depends on the original build action rather than
|
||||
the installed copy
|
||||
([`test.go`, lines 1200–1313](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L1200-L1313)).
|
||||
- `testBinaryName` explicitly uses the final import-path element rather than
|
||||
the declared package name; its command-line-files exception uses the source
|
||||
package name
|
||||
([`test.go`, lines 2287–2300](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L2287-L2300),
|
||||
[`cmd/go/internal/load/pkg.go`, lines 1727–1769](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1727-L1769)).
|
||||
- `BuildInstallFunc` creates parents and installs a linked executable with mode
|
||||
`0777` filtered by the process umask
|
||||
([`cmd/go/internal/work/exec.go`, lines 1904–2000](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1904-L2000),
|
||||
[`cmd/go/internal/work/shell.go`, lines 119–220 and 283–301](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/shell.go#L119-L220)).
|
||||
On the pinned Unix target, only exact `/dev/null` is the null spelling
|
||||
([`cmd/go/internal/base/path.go`, lines 81–92](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/base/path.go#L81-L92)).
|
||||
- Official `test_compile_multi_pkg.txt` requires missing nested output
|
||||
directory creation, default current-directory output, rejection of a
|
||||
non-directory multi-output and duplicate names, `/dev/null` acceptance, and
|
||||
`-o DIR` retention while tests still run
|
||||
([lines 3–38](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/testdata/script/test_compile_multi_pkg.txt#L3-L38)).
|
||||
|
||||
The separation between saved and executed paths is a conclusion derived from
|
||||
the pinned action dependencies: the run consumes the temporary link action,
|
||||
not the install action. Duplicate names are likewise a materialization
|
||||
collision, not package identity. Go's sources do not assert that a set of
|
||||
retained binaries is installed as one rollback transaction; WW keeps its
|
||||
existing stronger request-wide transaction while matching the observable
|
||||
accepted, rejected, and preserved outputs. No installed host Go behavior was
|
||||
used as authority.
|
||||
|
||||
Before this slice, direct native measurements of both Cstage and WWstage showed
|
||||
that single-package `-c -o FILE` retained and did not run, but `-o FILE`
|
||||
without `-c` exited 2 with `-o needs -c for a package target`; multi-package
|
||||
`-c -o FILE` exited 2 with the older unconditional fan-out rejection. Default
|
||||
multi-package `-c` scattered `<declared-package>.test` binaries into their
|
||||
source directories. Those measurements used the public driver route and
|
||||
observed exits, diagnostics, files, executable behavior, and stage-equal bytes;
|
||||
they were not conclusions drawn from WW source.
|
||||
|
||||
#### Coordinator policy and identity boundaries
|
||||
|
||||
`internal/wwpackage.packagecommand` is the sole owner of public output policy.
|
||||
It resolves the invocation directory, computes each visible
|
||||
`<import-leaf>.test` name, recognizes output-directory and `/dev/null` forms,
|
||||
rejects non-directory fan-out and duplicate names, omits publication for
|
||||
no-test products, and schedules execution according to `-c`. A contextual
|
||||
dotted request uses its exact final component; a local path request uses its
|
||||
directory leaf as the manifest-free presentation equivalent. Neither becomes
|
||||
declared-name or physical-directory identity.
|
||||
|
||||
Every actual test product still links to `package.test` below its private plan
|
||||
root. The private descriptor carries that `OUTPUT` plus an optional absolute
|
||||
`PUBLICATION`. The Cstage and WWstage drivers implement only this symmetric
|
||||
mechanism; they do not independently decide names or CLI policy. The
|
||||
coordinator always executes `OUTPUT`, so `-o` cannot alter executable argv,
|
||||
cwd, environment, null stdin, combined output, filters, action topology, or
|
||||
test outcome.
|
||||
|
||||
Visible basename, publication path, private runnable path, declared family,
|
||||
physical source directory, production/internal/external/recompiled/support/main
|
||||
variants, symbols, `.wwi`, archives, action identity, and persistence keys
|
||||
remain distinct. A duplicate basename rejects only the requested
|
||||
materialization. It never merges, renames, folds, or rekeys either canonical
|
||||
package. Compiler inputs, exported interfaces, generated main, archive order,
|
||||
and linked bytes are otherwise unchanged.
|
||||
|
||||
#### Publication, execution, failure, and cleanup
|
||||
|
||||
Without explicit `-o`, `-c` retains each binary in the invocation directory.
|
||||
An existing directory or a path ending in `/` receives one visible name per
|
||||
selected package; missing parents are created with `0777` subject to umask. A
|
||||
non-directory destination accepts exactly one selected package. Exact
|
||||
`/dev/null` suppresses retained copies, permits duplicate visible names, and
|
||||
does not suppress execution unless `-c` is also present. A no-test package
|
||||
performs ordinary production validation, reports `[no tests]`, and creates no
|
||||
binary or otherwise-unused output directory. Successful compile-only products
|
||||
are silent, matching Go's no-op print action.
|
||||
|
||||
After linking, the driver copies the private runnable bytes to a distinct
|
||||
`.new` inode opened with executable mode `0777` subject to umask. Temporary
|
||||
runnable, retained copy, statuses, changed persistent actions, tool records,
|
||||
and stamp then enter the existing one-request transaction. All producers and
|
||||
linkers complete before installation. Any load, compile, assemble, archive,
|
||||
link, stage, or install failure preserves old retained binaries and persistent
|
||||
bytes, discards all stages, removes cold scratch, and rolls back only output
|
||||
prefixes created by that request. Occupied or dangling `.new` paths reject
|
||||
before tools and are never overwritten.
|
||||
|
||||
Execution begins only after publication commits. Assertion failure, signal,
|
||||
timeout, or child-setup failure therefore leaves an explicitly retained binary
|
||||
while retaining the established stdout/stderr result routing and sibling
|
||||
isolation. Parallel products stage independent runnable/copy pairs; the shared
|
||||
transaction prevents partial sibling publication and canonical result emission
|
||||
order remains unchanged. Direct invocation of a retained binary continues to
|
||||
inherit caller cwd, environment, and separate standard descriptors.
|
||||
|
||||
`-o` can accompany `-w`: unchanged semantic actions are reused, changed source
|
||||
invalidates the applicable test actions, the always-run link refreshes the
|
||||
private runnable and retained copy, and the test still runs. This is build
|
||||
reuse, not a result cache. `-c` retains its established incompatibility with
|
||||
`-w`; this slice does not invent persistent compile-only ownership.
|
||||
|
||||
No persisted byte schema changed. Build workdir format remains `18`, test
|
||||
workdir format remains `19`, and semantic storage remains `3`.
|
||||
|
||||
The focused native owners are `compile_artifact_naming` and
|
||||
`test_binary_publication_transaction` in `test/package/package_test.ww`. Their
|
||||
Cstage/WWstage matrix covers single/default/directory/nested/multi/null output;
|
||||
declared-name versus import-leaf naming; executable mode and direct execution;
|
||||
temporary argv versus retained path; no-test omission; duplicate and
|
||||
non-directory rejection; occupied stages; serial and parallel sibling
|
||||
publication; injected late-link rollback over old files and newly created
|
||||
parents; runtime-failure retention; persistent cold/warm/invalidation behavior;
|
||||
diagnostic equality; retained binary byte identity; and absence of `.new`
|
||||
residue. Existing package tests continue to own all action/test variants,
|
||||
graph identity, output ordering, cwd/environment/stdin, timeout, and broader
|
||||
transaction behavior.
|
||||
|
||||
## 12. Candidate architectures and hard-gate decision
|
||||
|
||||
Five candidates were developed as coherent systems, not as feature bins.
|
||||
|
||||
21
docs/spec.md
21
docs/spec.md
@@ -645,6 +645,27 @@ mixed directories are accepted from their selected test files. A directory
|
||||
with no selected test file validates ordinary production but creates no test
|
||||
support, generated main, link, binary, result, or process.
|
||||
|
||||
Every test-bearing directory product links one request-private runnable.
|
||||
`-c` retains an executable copy and suppresses its execution; `-o` retains a
|
||||
copy at the named destination and still executes unless `-c` is present. With
|
||||
no explicit output, `-c` writes `<import-leaf>.test` in the invocation
|
||||
directory. An output ending in `/` or naming an existing directory receives
|
||||
that basename and may have missing parent directories created. One
|
||||
non-directory output may name only one package. Multiple packages whose
|
||||
visible import leaves would produce the same test-binary name reject before
|
||||
tools or output creation; exact `/dev/null` is the discard exception. Declared
|
||||
package names, test variants, source filenames, physical directories, output
|
||||
paths, and retained binary names remain presentation or loader metadata and do
|
||||
not become canonical package or action identity.
|
||||
|
||||
The retained file is an executable, byte-identical copy of the private
|
||||
runnable. It joins package artifacts and statuses in the request-wide atomic
|
||||
publication transaction. Build, link, stage, or install failure preserves old
|
||||
destinations and removes temporary stages and invocation-created output
|
||||
prefixes. Test execution starts only after that transaction commits, so a
|
||||
runtime failure leaves an explicitly retained binary. A no-test product
|
||||
publishes no binary and does not create a directory solely for one.
|
||||
|
||||
When `ww test` executes a directory-owned product, the child process working
|
||||
directory is that product's canonical absolute physical package source
|
||||
directory. Its per-run environment has exactly one effective uppercase `PWD`,
|
||||
|
||||
@@ -216,18 +216,28 @@ test binaries concurrently under `os.exec` start/poll supervision (no threads);
|
||||
emission stays strictly in group order, so the byte stream is identical at
|
||||
every `-j` level, and `-j 1` — the default — matches the former sequential
|
||||
run loop exactly. Measured on the 31-package `lib/...` walk:
|
||||
7.0s sequential, 2.4s at `-j 4`. With `-c`, it publishes each exact
|
||||
`<package>.test` directory binary; the first output owns the one
|
||||
shared cold sepwork containing the command-global action universe. Those become
|
||||
caller-owned artifacts. `-c -o <name>`
|
||||
names that artifact instead of the fixed stem for exactly one directory,
|
||||
including a combined internal/external directory:
|
||||
the coordinator rejects a multi-package fan-out ("cannot use -o with
|
||||
multiple packages", Go's `go test -o` rule), and `-o` without `-c` is
|
||||
rejected at the driver ("needs -c for a package target") because a plain
|
||||
run leaves no caller-owned artifact for `-o` to name. Without `-c`, it removes the
|
||||
temporary binary and scratch with its workspace. The language runtime
|
||||
owns individual `@test` functions.
|
||||
7.0s sequential, 2.4s at `-j 4`.
|
||||
|
||||
Directory test binaries are always linked under the coordinator's temporary
|
||||
product root. `-c` independently requests a caller-visible executable copy and
|
||||
suppresses execution. `-o` independently requests a copy and still runs the
|
||||
temporary binary unless `-c` is also present. Without `-o`, `-c` publishes
|
||||
`<import-leaf>.test` in the invocation directory; an output ending in `/` or
|
||||
naming an existing directory receives one such name per selected package and
|
||||
missing parents are created. A single non-directory output is legal for one
|
||||
package only. Multi-package non-directory output and duplicate visible binary
|
||||
names reject before tools; exact `/dev/null` discards every copy and permits
|
||||
duplicate names. A package with no selected test source validates production
|
||||
but publishes nothing and does not create an otherwise unneeded output
|
||||
directory. Successful `-c` output is silent apart from no-test reporting.
|
||||
|
||||
The retained executable is byte-identical to the temporary runnable and has
|
||||
executable mode `0777` filtered by the caller's umask, but it is never the path
|
||||
executed by the coordinator. Publication participates in the driver's one
|
||||
request-wide transaction: producer, linker, staging, or installation failure
|
||||
preserves every prior destination and removes stages and newly created output
|
||||
prefixes. A runtime failure occurs after commit and therefore leaves the saved
|
||||
binary. The language runtime owns individual `@test` functions.
|
||||
|
||||
Every actually executed directory product gives its single generated binary
|
||||
the product's canonical absolute physical source directory as child cwd. A
|
||||
@@ -409,16 +419,19 @@ bounded-memory Cstage/WWstage allocation-failure parity across the complete
|
||||
combined package-test graph, under independently discovered ceilings supplied
|
||||
by the repository-built `sep-limitexec` helper.
|
||||
|
||||
`ww build`, an explicit single-file `ww test -o <stem>`, and each successful
|
||||
directory-package `ww test -c` build publish `<stem>.sepwork` as a caller-owned
|
||||
artifact directory. The driver acquires it with one fresh `mkdir` and refuses
|
||||
an existing path; it never clears a collision. A caller keeps only the exact
|
||||
artifacts it observes and removes that exact tree on every later success or
|
||||
failure. `ww run` and no-output single-file `ww test` use driver-owned scratch
|
||||
instead; both driver stages place that scratch and their temporary executable
|
||||
beneath one freshly acquired directory, remove both after every build result,
|
||||
and make cleanup failure fail the command. Make recipes build driver-produced
|
||||
tools in invocation-owned directories and apply the same exact cleanup rule.
|
||||
`ww build` and an explicit single-file `ww test -o <stem>` publish
|
||||
`<stem>.sepwork` as a caller-owned artifact directory. The driver acquires it
|
||||
with one fresh `mkdir` and refuses an existing path; it never clears a
|
||||
collision. A caller keeps only the exact artifacts it observes and removes
|
||||
that exact tree on every later success or failure. Directory-package test
|
||||
plans instead keep their cold semantic-action scratch and runnable binary
|
||||
inside the coordinator's temporary root; only the optional retained executable
|
||||
escapes through the transaction above. `ww run` and no-output single-file
|
||||
`ww test` use driver-owned scratch instead; both driver stages place that
|
||||
scratch and their temporary executable beneath one freshly acquired directory,
|
||||
remove both after every build result, and make cleanup failure fail the
|
||||
command. Make recipes build driver-produced tools in invocation-owned
|
||||
directories and apply the same exact cleanup rule.
|
||||
|
||||
`ww build -w DIR` and single-file `ww test -w DIR` replace that scratch with a
|
||||
caller-owned persistent package-artifact workdir: for these direct routes the
|
||||
@@ -561,8 +574,8 @@ timeout policy in this architecture.
|
||||
|
||||
## Open driver work
|
||||
|
||||
None; the package-level `-o` contract (the last carried bullet) landed as
|
||||
`-c -o <name>` for exactly one package.
|
||||
None; directory-package `-c` and `-o` now have the applicable Go 1.26.5
|
||||
retention, naming, fan-out, execution, and publication behavior.
|
||||
|
||||
## Validation policy
|
||||
|
||||
|
||||
Reference in New Issue
Block a user