ww: protect public install outputs
This commit is contained in:
@@ -6214,11 +6214,10 @@ The authority is official Go 1.26.5 at commit
|
||||
The separation between saved and executed paths is a conclusion derived from
|
||||
the pinned action dependencies: the run consumes the temporary link action,
|
||||
not the install action. Duplicate names are likewise a materialization
|
||||
collision, not package identity. Go's sources do not assert that a set of
|
||||
retained binaries is installed as one rollback transaction; WW keeps its
|
||||
existing stronger request-wide transaction while matching the observable
|
||||
accepted, rejected, and preserved outputs. No installed host Go behavior was
|
||||
used as authority.
|
||||
collision, not package identity. Section 11.31 completes the later install
|
||||
dependency: compile-only products retain the request transaction, while a
|
||||
running retained product installs independently only after its successful run.
|
||||
No installed host Go behavior was used as authority.
|
||||
|
||||
Before this slice, direct native measurements of both Cstage and WWstage showed
|
||||
that single-package `-c -o FILE` retained and did not run, but `-o FILE`
|
||||
@@ -6268,23 +6267,24 @@ performs ordinary production validation, reports `[no tests]`, and creates no
|
||||
binary or otherwise-unused output directory. Successful compile-only products
|
||||
are silent, matching Go's no-op print action.
|
||||
|
||||
After linking, the driver copies the private runnable bytes to a distinct
|
||||
`.new` inode opened with executable mode `0777` subject to umask. Temporary
|
||||
runnable, retained copy, statuses, changed persistent actions, tool records,
|
||||
and stamp then enter the existing one-request transaction. All producers and
|
||||
linkers complete before installation. Any load, compile, assemble, archive,
|
||||
link, stage, or install failure preserves old retained binaries and persistent
|
||||
bytes, discards all stages, removes cold scratch, and rolls back only output
|
||||
prefixes created by that request. Occupied or dangling `.new` paths reject
|
||||
before tools and are never overwritten.
|
||||
For `-c`, the driver copies the private runnable bytes to a distinct `.new`
|
||||
inode opened with executable mode `0777` subject to umask. Temporary runnable,
|
||||
retained copy, statuses, changed persistent actions, tool records, and stamp
|
||||
then enter the existing one-request transaction. All producers and linkers
|
||||
complete before installation. Any load, compile, assemble, archive, link,
|
||||
stage, or install failure preserves old retained binaries and persistent bytes,
|
||||
discards all stages, removes cold scratch, and rolls back only output prefixes
|
||||
created by that request. Occupied or dangling `.new` paths reject before tools
|
||||
and are never overwritten.
|
||||
|
||||
Execution begins only after publication commits. Assertion failure, signal,
|
||||
timeout, or child-setup failure therefore leaves an explicitly retained binary
|
||||
while retaining the established stdout/stderr result routing and sibling
|
||||
isolation. Parallel products stage independent runnable/copy pairs; the shared
|
||||
transaction prevents partial sibling publication and canonical result emission
|
||||
order remains unchanged. Direct invocation of a retained binary continues to
|
||||
inherit caller cwd, environment, and separate standard descriptors.
|
||||
For running `-o`, the build transaction commits only the private runnable,
|
||||
status, and semantic actions. The coordinator executes that runnable and, on a
|
||||
successful result, invokes the selected driver stage's public install action.
|
||||
Assertion failure, signal, timeout, interruption, or child-setup failure skips
|
||||
that action and preserves any prior binary. Successful parallel products
|
||||
install independently after their runs; canonical result emission order stays
|
||||
unchanged. Direct invocation of a retained binary continues to inherit caller
|
||||
cwd, environment, and separate standard descriptors.
|
||||
|
||||
`-c` and `-o` can accompany `-w`: the workdir owns only semantic actions while
|
||||
the invocation/output path owns only the retained copy. Unchanged actions are
|
||||
@@ -6302,9 +6302,9 @@ declared-name versus import-leaf naming; executable mode and direct execution;
|
||||
temporary argv versus retained path; no-test omission; duplicate and
|
||||
non-directory rejection; occupied stages; serial and parallel sibling
|
||||
publication; injected late-link rollback over old files and newly created
|
||||
parents; runtime-failure retention; persistent cold/warm/invalidation behavior;
|
||||
diagnostic equality; retained binary byte identity; and absence of `.new`
|
||||
residue. Existing package tests continue to own all action/test variants,
|
||||
parents; runtime-failure preservation; persistent cold/warm/invalidation
|
||||
behavior; diagnostic equality; retained binary byte identity; and absence of
|
||||
`.new` residue. Existing package tests continue to own all action/test variants,
|
||||
graph identity, output ordering, cwd/environment/stdin, timeout, and broader
|
||||
transaction behavior.
|
||||
|
||||
@@ -6760,6 +6760,150 @@ output umasks, occupied stages, generalized multi-product transactions, test
|
||||
runtime failure and timeout, null discard, and broader package/import graph
|
||||
matrices.
|
||||
|
||||
### 11.31 Implemented Go-like public-output overwrite safety
|
||||
|
||||
Every caller-visible build and retained-test install now protects an existing
|
||||
destination at the same late boundary as Go 1.26.5. After applicable producers
|
||||
finish, ordinary `stat` rejects a directory and rejects a nonempty regular file
|
||||
whose leading bytes do not identify a toolchain output. Absent paths, empty
|
||||
regular reservations, recognized outputs, and non-directory non-regular paths
|
||||
remain replaceable. Exact `/dev/null` and assembly-only `-S` have no install
|
||||
action and never enter this rule.
|
||||
|
||||
#### Pinned Go evidence and classification
|
||||
|
||||
The sole authority is official Go 1.26.5 at commit
|
||||
`c19862e5f8415b4f24b189d065ed739517c548ba`:
|
||||
|
||||
- `Shell.moveOrCopyFile` and `Shell.CopyFile` call `checkDstOverwrite` before
|
||||
replacing the destination
|
||||
([`cmd/go/internal/work/shell.go`, lines 119–235](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/shell.go#L119-L235)).
|
||||
`checkDstOverwrite` uses `os.Stat`, rejects a directory, and—unless forced—
|
||||
rejects a nonempty regular file for which `isObject` is false
|
||||
([lines 248–261](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/shell.go#L248-L261)).
|
||||
- `BuildInstallFunc` creates the destination parent and reaches
|
||||
`moveOrCopyFile(..., false)` only after its build producer
|
||||
([`cmd/go/internal/work/exec.go`, lines 1904–2000](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1904-L2000)).
|
||||
`objectMagic` and `isObject` read the first 64 bytes and recognize archive,
|
||||
ELF, Mach-O, PE, Plan 9, WASM, and XCOFF prefixes without consulting a file
|
||||
extension
|
||||
([lines 2118–2150](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L2118-L2150)).
|
||||
- `runBuild` loads and checks every selected package before constructing the
|
||||
output/install action
|
||||
([`cmd/go/internal/work/build.go`, lines 459–558](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L459-L558)).
|
||||
- `builderTest` makes `-c` depend directly on the install action. For a running
|
||||
retained test, the run consumes the private build action and the install
|
||||
action additionally depends on that run
|
||||
([`cmd/go/internal/test/test.go`, lines 1257–1364](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L1257-L1364)).
|
||||
`Builder.Do` invokes an actor only when dependency failure has not propagated
|
||||
(unless the action explicitly ignores failure), so a failed test run skips
|
||||
`BuildInstallFunc`
|
||||
([`cmd/go/internal/work/exec.go`, lines 72–207](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L72-L207)).
|
||||
- Official `build_output_overwrite.txt` requires refusal to replace a
|
||||
nonempty source file and preservation of its contents
|
||||
([lines 1–20](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/testdata/script/build_output_overwrite.txt#L1-L20)).
|
||||
Official `test_compile_tempfile.txt` requires an existing empty reservation
|
||||
to be accepted and replaced
|
||||
([lines 1–11](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/testdata/script/test_compile_tempfile.txt#L1-L11)).
|
||||
Official `build_output.txt` separately pins executable command and archive
|
||||
products
|
||||
([lines 47–57 and 64–76](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/testdata/script/build_output.txt#L47-L76)).
|
||||
|
||||
The destination predicate, complete magic table, load-before-install ordering,
|
||||
producer-before-check ordering, and run-before-install dependency are directly
|
||||
implemented by the pinned source. Non-overwrite and empty-file acceptance are
|
||||
directly asserted by official testdata. Applying ELF and archive recognition
|
||||
to WW's byte-identical output forms is derived from that implementation. Go
|
||||
has no WW interface sidecar; recognizing only the compiler-owned
|
||||
`//ww:module ` prefix is the derived local application that permits ordinary
|
||||
repeat publication without letting arbitrary sidecar text be overwritten. No
|
||||
installed host Go behavior was used as authority.
|
||||
|
||||
#### Fresh four-axis audit and pre-fix measurements
|
||||
|
||||
The bounded audit selected this one gap on the build axis and the shared
|
||||
retained-test install axis. The package control selected the same canonical
|
||||
command root twice and measured one deduplicated command action plus one
|
||||
dependency action, with byte-identical Cstage/WWstage units, interfaces, and
|
||||
archives. The import control placed a used alias in one source file and an
|
||||
unused alias for the same dependency in a sibling; both stages emitted the
|
||||
same file-local unused-import diagnostic and committed no work. Those package
|
||||
and import candidates were aligned and were not changed.
|
||||
|
||||
Fresh public Cstage and WWstage probes directly measured the same pre-fix
|
||||
behavior. Explicit command, raw command, output-directory child, library
|
||||
archive/interface, compile-only test, and running retained-test destinations
|
||||
containing arbitrary nonempty text were replaced successfully. A nonempty
|
||||
directory at a build or test child destination was renamed to a PID-bearing
|
||||
transaction backup, replaced by the executable, and left stranded because
|
||||
backup cleanup could not unlink the directory. Empty reservations were already
|
||||
accepted. Missing-import rejection already preceded destination handling. All
|
||||
measured successful executables, archives, interfaces, diagnostics, runtime
|
||||
results, and semantic artifacts were stage-identical. Those are directly
|
||||
measured WW facts, not source inferences.
|
||||
|
||||
#### Ownership, action order, rollback, and identity
|
||||
|
||||
The Cstage `sep_txn_commit` and WWstage `septxncommit` publishers own the byte
|
||||
predicate. Each transaction entry now explicitly distinguishes a public
|
||||
install from internal status, tool-identity, stamp, and persistent-action
|
||||
state. Only command/archive output, retained-copy, and published `.wwi` entries
|
||||
are checked. Producers still finish before transaction commit; a rejected
|
||||
destination discards all staged outputs and preserves every prior public and
|
||||
persistent byte. A library archive and interface remain one rollback group, so
|
||||
arbitrary text in either destination changes neither.
|
||||
|
||||
`internal/wwpackage.packagecommand` continues to own directory-test naming and
|
||||
scheduling. A running retained descriptor withholds its public destination
|
||||
from the build child. After a successful private run, the coordinator invokes
|
||||
a private action in the selected driver, which stages an executable copy and
|
||||
re-enters the same guarded publisher. A failed, signalled, timed-out,
|
||||
interrupted, or unstartable run never invokes that action. Successful products
|
||||
in a multi-package running request install independently; compile-only products
|
||||
retain the established request transaction. The driver-owned raw single-file
|
||||
route applies the same private build, run, and guarded-install sequence.
|
||||
|
||||
Package-build descriptors use `build-public` only for caller-visible command or
|
||||
archive products. Private package-build placeholders, test runnables, `ww run`,
|
||||
workdir-owned test binaries, null-discard products, and assembly-only products
|
||||
remain internal entries. Destination path, file kind, magic, declared name,
|
||||
requested alias, import leaf, physical directory, and publication order do not
|
||||
enter package/import identity, graph edges, action keys, symbols, artifacts,
|
||||
`.wwi` contents, or persistence keys.
|
||||
|
||||
The guard follows symlinks for classification, matching `os.Stat`; the existing
|
||||
transaction still replaces the destination directory entry itself. It permits
|
||||
FIFO and other non-directory non-regular destinations because the pinned guard
|
||||
does. Diagnostics are exactly
|
||||
`ww: build output "PATH" already exists and is a directory` and
|
||||
`ww: build output "PATH" already exists and is not an object file` in both
|
||||
stages. No guard is preflighted during loading: package/import errors still win,
|
||||
and compiler, assembler, archive, or linker failure prevents the install action
|
||||
from being reached.
|
||||
|
||||
Build runtime is inapplicable because `ww build` starts no program. Test
|
||||
runtime is applicable and owns the post-run dependency above. Producer failure,
|
||||
linker interruption, output-parent rollback, concurrency, occupied stages,
|
||||
prior-state preservation, and residue cleanup remain governed by the existing
|
||||
request/private-action transactions; the new check adds no process-global
|
||||
state. Public artifact bytes and modes are unchanged on accepted installs.
|
||||
There is no persisted-byte contract change: build workdir format remains `18`,
|
||||
test workdir format remains `19`, and semantic storage format remains `3`.
|
||||
|
||||
The WW-native `public_output_overwrite_safety` observer covers both stages:
|
||||
direct, default, raw, package-output-directory, library, compile-only test, and
|
||||
running-test routes; late linker activity and load precedence; absent/empty,
|
||||
ELF, archive, interface, arbitrary regular, directory, symlink, and FIFO
|
||||
destinations; cold, warm, and invalidated persistent rollback; run-before-check
|
||||
and failed-run no-install behavior; exact null and assembly-only exclusions;
|
||||
runtime results; modes; diagnostic identity; public and semantic artifact-byte
|
||||
identity; and `.new`, install-stage, and transaction-backup cleanup.
|
||||
`test_binary_publication_transaction` pins the changed failed-run behavior and
|
||||
the existing linker failure, output-parent rollback, multi-product, persistent,
|
||||
and retained-binary contracts. Existing request-transaction, timeout,
|
||||
interruption, and concurrent-driver owners continue to cover those unchanged
|
||||
dimensions.
|
||||
|
||||
## 12. Candidate architectures and hard-gate decision
|
||||
|
||||
Five candidates were developed as coherent systems, not as feature bins.
|
||||
|
||||
33
docs/spec.md
33
docs/spec.md
@@ -335,6 +335,22 @@ ImportPath = ident { "." ident } .
|
||||
output. Output paths and directory metadata never become package, import,
|
||||
graph, action,
|
||||
symbol, artifact, `.wwi`, or persistence identity.
|
||||
|
||||
Every caller-visible build installation checks its destination after all
|
||||
applicable compile, assemble, archive, and link producers finish. Ordinary
|
||||
`stat` follows symlinks. An existing directory rejects as
|
||||
`ww: build output "PATH" already exists and is a directory`; an existing
|
||||
nonempty regular file rejects as `... is not an object file` unless its
|
||||
leading bytes identify a Go 1.26.5 object/output form. The recognized table
|
||||
is archive, ELF, Mach-O, PE, Plan 9, WASM, and XCOFF magic; WW additionally
|
||||
recognizes its compiler-owned `//ww:module ` interface prefix. An absent
|
||||
path, an empty regular reservation, or a non-directory non-regular path may
|
||||
be replaced. A published non-main archive and its `.wwi` sidecar are checked
|
||||
as one WW request transaction, so arbitrary caller text in either
|
||||
destination preserves both old outputs and the committed persistent
|
||||
generation. This safety check is output disposition only: it does not enter
|
||||
package/import loading, graph or action identity, artifact bytes, or
|
||||
invalidation.
|
||||
Assembly-only `-S` retains the directory form's command-action selection and
|
||||
no-main rejection, but it reaches no install action: destination length,
|
||||
duplicate publication names, implicit destination collision, and output
|
||||
@@ -714,12 +730,17 @@ paths, and retained binary names remain presentation or loader metadata and do
|
||||
not become canonical package or action identity.
|
||||
|
||||
The retained file is an executable, byte-identical copy of the private
|
||||
runnable. It joins package artifacts and statuses in the request-wide atomic
|
||||
publication transaction. Build, link, stage, or install failure preserves old
|
||||
destinations and removes temporary stages and invocation-created output
|
||||
prefixes. Test execution starts only after that transaction commits, so a
|
||||
runtime failure leaves an explicitly retained binary. A no-test product
|
||||
publishes no binary and does not create a directory solely for one.
|
||||
runnable. A compile-only retained binary joins package artifacts and statuses
|
||||
in the request-wide atomic publication transaction. Build, link, stage, or
|
||||
install failure preserves old destinations and removes temporary stages and
|
||||
invocation-created output prefixes. For a running `-o` request, the private
|
||||
binary executes first. Only a successful run enters the guarded install
|
||||
action; a failed, signalled, timed-out, interrupted, or unstartable run
|
||||
publishes no new copy and preserves any prior destination. The post-run guard
|
||||
uses the same directory/nonempty-regular/object-magic rule as `ww build`.
|
||||
Successful products in a multi-package running request install independently;
|
||||
their visible result order remains package order. A no-test product publishes
|
||||
no binary and does not create a directory solely for one.
|
||||
`-w` may persist the unchanged semantic actions for either `-c` or running
|
||||
retention without changing publication identity or introducing a test-result
|
||||
cache.
|
||||
|
||||
@@ -258,11 +258,25 @@ install-only destination validation or output-directory creation.
|
||||
|
||||
The retained executable is byte-identical to the temporary runnable and has
|
||||
executable mode `0777` filtered by the caller's umask, but it is never the path
|
||||
executed by the coordinator. Publication participates in the driver's one
|
||||
request-wide transaction: producer, linker, staging, or installation failure
|
||||
preserves every prior destination and removes stages and newly created output
|
||||
prefixes. A runtime failure occurs after commit and therefore leaves the saved
|
||||
binary. The language runtime owns individual `@test` functions.
|
||||
executed by the coordinator. Compile-only publication participates in the
|
||||
driver's one request-wide transaction: producer, linker, staging, or
|
||||
installation failure preserves every prior destination and removes stages and
|
||||
newly created output prefixes. A running retained request instead withholds
|
||||
the public path from that build transaction. The coordinator runs the private
|
||||
binary, then invokes the selected stage driver's guarded install action only
|
||||
after a successful process result. Failure, signal, timeout, interruption, or
|
||||
child-start failure therefore preserves any prior retained binary and creates
|
||||
no new one. Successful products install independently after their runs. The
|
||||
language runtime owns individual `@test` functions.
|
||||
|
||||
Every build or retained-test public install follows Go 1.26.5's late
|
||||
destination safety rule. After applicable producers (and, for running tests,
|
||||
after the successful run), ordinary `stat` rejects a directory and rejects a
|
||||
nonempty regular non-object file. Empty reservations, recognized prior
|
||||
outputs, and non-directory non-regular paths remain replaceable. Recognition
|
||||
uses Go's archive/ELF/Mach-O/PE/Plan 9/WASM/XCOFF magic plus WW's narrow
|
||||
`//ww:module ` interface prefix. Package/import rejection keeps its earlier
|
||||
diagnostic precedence; exact `/dev/null` and `-S` never enter the guard.
|
||||
|
||||
Every actually executed directory product gives its single generated binary
|
||||
the product's canonical absolute physical source directory as child cwd. A
|
||||
@@ -468,8 +482,9 @@ with one fresh `mkdir` and refuses an existing path; it never clears a
|
||||
collision. A caller keeps only the exact artifacts it observes and removes
|
||||
that exact tree on every later success or failure. Directory-package test
|
||||
plans instead keep their cold semantic-action scratch and runnable binary
|
||||
inside the coordinator's temporary root; only the optional retained executable
|
||||
escapes through the transaction above. `ww run` and no-output single-file
|
||||
inside the coordinator's temporary root; a compile-only retained executable
|
||||
escapes through the build transaction, while a running retained executable
|
||||
uses the post-run install action above. `ww run` and no-output single-file
|
||||
`ww test` use driver-owned scratch instead; both driver stages place that
|
||||
scratch and their temporary executable beneath one freshly acquired directory,
|
||||
remove both after every build result, and make cleanup failure fail the
|
||||
|
||||
Reference in New Issue
Block a user