ww: protect public install outputs

This commit is contained in:
2026-08-21 04:11:01 +09:00
parent ce91f6662c
commit 1b71250ad5
7 changed files with 1064 additions and 90 deletions

View File

@@ -6214,11 +6214,10 @@ The authority is official Go 1.26.5 at commit
The separation between saved and executed paths is a conclusion derived from
the pinned action dependencies: the run consumes the temporary link action,
not the install action. Duplicate names are likewise a materialization
collision, not package identity. Go's sources do not assert that a set of
retained binaries is installed as one rollback transaction; WW keeps its
existing stronger request-wide transaction while matching the observable
accepted, rejected, and preserved outputs. No installed host Go behavior was
used as authority.
collision, not package identity. Section 11.31 completes the later install
dependency: compile-only products retain the request transaction, while a
running retained product installs independently only after its successful run.
No installed host Go behavior was used as authority.
Before this slice, direct native measurements of both Cstage and WWstage showed
that single-package `-c -o FILE` retained and did not run, but `-o FILE`
@@ -6268,23 +6267,24 @@ performs ordinary production validation, reports `[no tests]`, and creates no
binary or otherwise-unused output directory. Successful compile-only products
are silent, matching Go's no-op print action.
After linking, the driver copies the private runnable bytes to a distinct
`.new` inode opened with executable mode `0777` subject to umask. Temporary
runnable, retained copy, statuses, changed persistent actions, tool records,
and stamp then enter the existing one-request transaction. All producers and
linkers complete before installation. Any load, compile, assemble, archive,
link, stage, or install failure preserves old retained binaries and persistent
bytes, discards all stages, removes cold scratch, and rolls back only output
prefixes created by that request. Occupied or dangling `.new` paths reject
before tools and are never overwritten.
For `-c`, the driver copies the private runnable bytes to a distinct `.new`
inode opened with executable mode `0777` subject to umask. Temporary runnable,
retained copy, statuses, changed persistent actions, tool records, and stamp
then enter the existing one-request transaction. All producers and linkers
complete before installation. Any load, compile, assemble, archive, link,
stage, or install failure preserves old retained binaries and persistent bytes,
discards all stages, removes cold scratch, and rolls back only output prefixes
created by that request. Occupied or dangling `.new` paths reject before tools
and are never overwritten.
Execution begins only after publication commits. Assertion failure, signal,
timeout, or child-setup failure therefore leaves an explicitly retained binary
while retaining the established stdout/stderr result routing and sibling
isolation. Parallel products stage independent runnable/copy pairs; the shared
transaction prevents partial sibling publication and canonical result emission
order remains unchanged. Direct invocation of a retained binary continues to
inherit caller cwd, environment, and separate standard descriptors.
For running `-o`, the build transaction commits only the private runnable,
status, and semantic actions. The coordinator executes that runnable and, on a
successful result, invokes the selected driver stage's public install action.
Assertion failure, signal, timeout, interruption, or child-setup failure skips
that action and preserves any prior binary. Successful parallel products
install independently after their runs; canonical result emission order stays
unchanged. Direct invocation of a retained binary continues to inherit caller
cwd, environment, and separate standard descriptors.
`-c` and `-o` can accompany `-w`: the workdir owns only semantic actions while
the invocation/output path owns only the retained copy. Unchanged actions are
@@ -6302,9 +6302,9 @@ declared-name versus import-leaf naming; executable mode and direct execution;
temporary argv versus retained path; no-test omission; duplicate and
non-directory rejection; occupied stages; serial and parallel sibling
publication; injected late-link rollback over old files and newly created
parents; runtime-failure retention; persistent cold/warm/invalidation behavior;
diagnostic equality; retained binary byte identity; and absence of `.new`
residue. Existing package tests continue to own all action/test variants,
parents; runtime-failure preservation; persistent cold/warm/invalidation
behavior; diagnostic equality; retained binary byte identity; and absence of
`.new` residue. Existing package tests continue to own all action/test variants,
graph identity, output ordering, cwd/environment/stdin, timeout, and broader
transaction behavior.
@@ -6760,6 +6760,150 @@ output umasks, occupied stages, generalized multi-product transactions, test
runtime failure and timeout, null discard, and broader package/import graph
matrices.
### 11.31 Implemented Go-like public-output overwrite safety
Every caller-visible build and retained-test install now protects an existing
destination at the same late boundary as Go 1.26.5. After applicable producers
finish, ordinary `stat` rejects a directory and rejects a nonempty regular file
whose leading bytes do not identify a toolchain output. Absent paths, empty
regular reservations, recognized outputs, and non-directory non-regular paths
remain replaceable. Exact `/dev/null` and assembly-only `-S` have no install
action and never enter this rule.
#### Pinned Go evidence and classification
The sole authority is official Go 1.26.5 at commit
`c19862e5f8415b4f24b189d065ed739517c548ba`:
- `Shell.moveOrCopyFile` and `Shell.CopyFile` call `checkDstOverwrite` before
replacing the destination
([`cmd/go/internal/work/shell.go`, lines 119235](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/shell.go#L119-L235)).
`checkDstOverwrite` uses `os.Stat`, rejects a directory, and—unless forced—
rejects a nonempty regular file for which `isObject` is false
([lines 248261](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/shell.go#L248-L261)).
- `BuildInstallFunc` creates the destination parent and reaches
`moveOrCopyFile(..., false)` only after its build producer
([`cmd/go/internal/work/exec.go`, lines 19042000](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1904-L2000)).
`objectMagic` and `isObject` read the first 64 bytes and recognize archive,
ELF, Mach-O, PE, Plan 9, WASM, and XCOFF prefixes without consulting a file
extension
([lines 21182150](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L2118-L2150)).
- `runBuild` loads and checks every selected package before constructing the
output/install action
([`cmd/go/internal/work/build.go`, lines 459558](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L459-L558)).
- `builderTest` makes `-c` depend directly on the install action. For a running
retained test, the run consumes the private build action and the install
action additionally depends on that run
([`cmd/go/internal/test/test.go`, lines 12571364](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L1257-L1364)).
`Builder.Do` invokes an actor only when dependency failure has not propagated
(unless the action explicitly ignores failure), so a failed test run skips
`BuildInstallFunc`
([`cmd/go/internal/work/exec.go`, lines 72207](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L72-L207)).
- Official `build_output_overwrite.txt` requires refusal to replace a
nonempty source file and preservation of its contents
([lines 120](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/testdata/script/build_output_overwrite.txt#L1-L20)).
Official `test_compile_tempfile.txt` requires an existing empty reservation
to be accepted and replaced
([lines 111](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/testdata/script/test_compile_tempfile.txt#L1-L11)).
Official `build_output.txt` separately pins executable command and archive
products
([lines 4757 and 6476](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/testdata/script/build_output.txt#L47-L76)).
The destination predicate, complete magic table, load-before-install ordering,
producer-before-check ordering, and run-before-install dependency are directly
implemented by the pinned source. Non-overwrite and empty-file acceptance are
directly asserted by official testdata. Applying ELF and archive recognition
to WW's byte-identical output forms is derived from that implementation. Go
has no WW interface sidecar; recognizing only the compiler-owned
`//ww:module ` prefix is the derived local application that permits ordinary
repeat publication without letting arbitrary sidecar text be overwritten. No
installed host Go behavior was used as authority.
#### Fresh four-axis audit and pre-fix measurements
The bounded audit selected this one gap on the build axis and the shared
retained-test install axis. The package control selected the same canonical
command root twice and measured one deduplicated command action plus one
dependency action, with byte-identical Cstage/WWstage units, interfaces, and
archives. The import control placed a used alias in one source file and an
unused alias for the same dependency in a sibling; both stages emitted the
same file-local unused-import diagnostic and committed no work. Those package
and import candidates were aligned and were not changed.
Fresh public Cstage and WWstage probes directly measured the same pre-fix
behavior. Explicit command, raw command, output-directory child, library
archive/interface, compile-only test, and running retained-test destinations
containing arbitrary nonempty text were replaced successfully. A nonempty
directory at a build or test child destination was renamed to a PID-bearing
transaction backup, replaced by the executable, and left stranded because
backup cleanup could not unlink the directory. Empty reservations were already
accepted. Missing-import rejection already preceded destination handling. All
measured successful executables, archives, interfaces, diagnostics, runtime
results, and semantic artifacts were stage-identical. Those are directly
measured WW facts, not source inferences.
#### Ownership, action order, rollback, and identity
The Cstage `sep_txn_commit` and WWstage `septxncommit` publishers own the byte
predicate. Each transaction entry now explicitly distinguishes a public
install from internal status, tool-identity, stamp, and persistent-action
state. Only command/archive output, retained-copy, and published `.wwi` entries
are checked. Producers still finish before transaction commit; a rejected
destination discards all staged outputs and preserves every prior public and
persistent byte. A library archive and interface remain one rollback group, so
arbitrary text in either destination changes neither.
`internal/wwpackage.packagecommand` continues to own directory-test naming and
scheduling. A running retained descriptor withholds its public destination
from the build child. After a successful private run, the coordinator invokes
a private action in the selected driver, which stages an executable copy and
re-enters the same guarded publisher. A failed, signalled, timed-out,
interrupted, or unstartable run never invokes that action. Successful products
in a multi-package running request install independently; compile-only products
retain the established request transaction. The driver-owned raw single-file
route applies the same private build, run, and guarded-install sequence.
Package-build descriptors use `build-public` only for caller-visible command or
archive products. Private package-build placeholders, test runnables, `ww run`,
workdir-owned test binaries, null-discard products, and assembly-only products
remain internal entries. Destination path, file kind, magic, declared name,
requested alias, import leaf, physical directory, and publication order do not
enter package/import identity, graph edges, action keys, symbols, artifacts,
`.wwi` contents, or persistence keys.
The guard follows symlinks for classification, matching `os.Stat`; the existing
transaction still replaces the destination directory entry itself. It permits
FIFO and other non-directory non-regular destinations because the pinned guard
does. Diagnostics are exactly
`ww: build output "PATH" already exists and is a directory` and
`ww: build output "PATH" already exists and is not an object file` in both
stages. No guard is preflighted during loading: package/import errors still win,
and compiler, assembler, archive, or linker failure prevents the install action
from being reached.
Build runtime is inapplicable because `ww build` starts no program. Test
runtime is applicable and owns the post-run dependency above. Producer failure,
linker interruption, output-parent rollback, concurrency, occupied stages,
prior-state preservation, and residue cleanup remain governed by the existing
request/private-action transactions; the new check adds no process-global
state. Public artifact bytes and modes are unchanged on accepted installs.
There is no persisted-byte contract change: build workdir format remains `18`,
test workdir format remains `19`, and semantic storage format remains `3`.
The WW-native `public_output_overwrite_safety` observer covers both stages:
direct, default, raw, package-output-directory, library, compile-only test, and
running-test routes; late linker activity and load precedence; absent/empty,
ELF, archive, interface, arbitrary regular, directory, symlink, and FIFO
destinations; cold, warm, and invalidated persistent rollback; run-before-check
and failed-run no-install behavior; exact null and assembly-only exclusions;
runtime results; modes; diagnostic identity; public and semantic artifact-byte
identity; and `.new`, install-stage, and transaction-backup cleanup.
`test_binary_publication_transaction` pins the changed failed-run behavior and
the existing linker failure, output-parent rollback, multi-product, persistent,
and retained-binary contracts. Existing request-transaction, timeout,
interruption, and concurrent-driver owners continue to cover those unchanged
dimensions.
## 12. Candidate architectures and hard-gate decision
Five candidates were developed as coherent systems, not as feature bins.

View File

@@ -335,6 +335,22 @@ ImportPath = ident { "." ident } .
output. Output paths and directory metadata never become package, import,
graph, action,
symbol, artifact, `.wwi`, or persistence identity.
Every caller-visible build installation checks its destination after all
applicable compile, assemble, archive, and link producers finish. Ordinary
`stat` follows symlinks. An existing directory rejects as
`ww: build output "PATH" already exists and is a directory`; an existing
nonempty regular file rejects as `... is not an object file` unless its
leading bytes identify a Go 1.26.5 object/output form. The recognized table
is archive, ELF, Mach-O, PE, Plan 9, WASM, and XCOFF magic; WW additionally
recognizes its compiler-owned `//ww:module ` interface prefix. An absent
path, an empty regular reservation, or a non-directory non-regular path may
be replaced. A published non-main archive and its `.wwi` sidecar are checked
as one WW request transaction, so arbitrary caller text in either
destination preserves both old outputs and the committed persistent
generation. This safety check is output disposition only: it does not enter
package/import loading, graph or action identity, artifact bytes, or
invalidation.
Assembly-only `-S` retains the directory form's command-action selection and
no-main rejection, but it reaches no install action: destination length,
duplicate publication names, implicit destination collision, and output
@@ -714,12 +730,17 @@ paths, and retained binary names remain presentation or loader metadata and do
not become canonical package or action identity.
The retained file is an executable, byte-identical copy of the private
runnable. It joins package artifacts and statuses in the request-wide atomic
publication transaction. Build, link, stage, or install failure preserves old
destinations and removes temporary stages and invocation-created output
prefixes. Test execution starts only after that transaction commits, so a
runtime failure leaves an explicitly retained binary. A no-test product
publishes no binary and does not create a directory solely for one.
runnable. A compile-only retained binary joins package artifacts and statuses
in the request-wide atomic publication transaction. Build, link, stage, or
install failure preserves old destinations and removes temporary stages and
invocation-created output prefixes. For a running `-o` request, the private
binary executes first. Only a successful run enters the guarded install
action; a failed, signalled, timed-out, interrupted, or unstartable run
publishes no new copy and preserves any prior destination. The post-run guard
uses the same directory/nonempty-regular/object-magic rule as `ww build`.
Successful products in a multi-package running request install independently;
their visible result order remains package order. A no-test product publishes
no binary and does not create a directory solely for one.
`-w` may persist the unchanged semantic actions for either `-c` or running
retention without changing publication identity or introducing a test-result
cache.

View File

@@ -258,11 +258,25 @@ install-only destination validation or output-directory creation.
The retained executable is byte-identical to the temporary runnable and has
executable mode `0777` filtered by the caller's umask, but it is never the path
executed by the coordinator. Publication participates in the driver's one
request-wide transaction: producer, linker, staging, or installation failure
preserves every prior destination and removes stages and newly created output
prefixes. A runtime failure occurs after commit and therefore leaves the saved
binary. The language runtime owns individual `@test` functions.
executed by the coordinator. Compile-only publication participates in the
driver's one request-wide transaction: producer, linker, staging, or
installation failure preserves every prior destination and removes stages and
newly created output prefixes. A running retained request instead withholds
the public path from that build transaction. The coordinator runs the private
binary, then invokes the selected stage driver's guarded install action only
after a successful process result. Failure, signal, timeout, interruption, or
child-start failure therefore preserves any prior retained binary and creates
no new one. Successful products install independently after their runs. The
language runtime owns individual `@test` functions.
Every build or retained-test public install follows Go 1.26.5's late
destination safety rule. After applicable producers (and, for running tests,
after the successful run), ordinary `stat` rejects a directory and rejects a
nonempty regular non-object file. Empty reservations, recognized prior
outputs, and non-directory non-regular paths remain replaceable. Recognition
uses Go's archive/ELF/Mach-O/PE/Plan 9/WASM/XCOFF magic plus WW's narrow
`//ww:module ` interface prefix. Package/import rejection keeps its earlier
diagnostic precedence; exact `/dev/null` and `-S` never enter the guard.
Every actually executed directory product gives its single generated binary
the product's canonical absolute physical source directory as child cwd. A
@@ -468,8 +482,9 @@ with one fresh `mkdir` and refuses an existing path; it never clears a
collision. A caller keeps only the exact artifacts it observes and removes
that exact tree on every later success or failure. Directory-package test
plans instead keep their cold semantic-action scratch and runnable binary
inside the coordinator's temporary root; only the optional retained executable
escapes through the transaction above. `ww run` and no-output single-file
inside the coordinator's temporary root; a compile-only retained executable
escapes through the build transaction, while a running retained executable
uses the post-run install action above. `ww run` and no-output single-file
`ww test` use driver-owned scratch instead; both driver stages place that
scratch and their temporary executable beneath one freshly acquired directory,
remove both after every build result, and make cleanup failure fail the