diff --git a/docs/build-system.md b/docs/build-system.md index d82e1166..a0851602 100644 --- a/docs/build-system.md +++ b/docs/build-system.md @@ -2852,38 +2852,55 @@ The test coordinator still discovers requested directories, enumerates the test package names, selects variants, executes independent binaries, and emits captured results in byte-sorted package order. It no longer concatenates a generated production/test root, resolves imports, or starts one package graph -per binary. Instead it sends one ordered build request per directory to the -Cstage or WWstage command. Its semantic selections are only the directory and -the selected variant/package identities; it also carries output destinations, -coordinator-private completion paths, import search roots, and the optional -shared work-directory policy. The command owns source selection, -package loading, compiler inputs, archive construction, and linking for both -non-importable roots: +per binary or directory. Instead it sends one ordered build request containing +every selected directory/variant root to the Cstage or WWstage command. Its +semantic selections are only the directory and selected variant/package +identities; it also carries output destinations, coordinator-private completion +paths, import search roots, and the optional command-scoped work-directory +policy. The command owns source selection, package loading, compiler inputs, +archive construction, and linking for every non-importable root: - `same-test` selects the byte-sorted production files followed by the byte-sorted matching `package p` test files. They form one compiler unit, so tests can use private production declarations. - `external-test` selects only matching `package p_test` files. Its `import p` - is a direct edge to a distinct production node for the same canonical - directory. That node selects every production file, emits `p.wwi` and `p.a`, - and exposes no private declaration to the external root. + is a direct edge to the canonical production action for that directory. + That action compiles with module qualifier `p`, selects every production + file, emits compiler export data and an archive, and exposes no private + declaration to the external root. If the action is first reached through + the external product, its collision-proof artifact key is derived from the + owning root, such as `__ww-test-001-external-production`; a normal import of + the same `p` and canonical directory reuses that action rather than creating + a second compilation. -The command loads those roots into one command-scoped package universe. The -roots retain distinct artifact keys (`__ww-test-same` and -`__ww-test-external`) that cannot collide with a legal import identity, -while imports of the same canonical production directory intern to one -production node. A deterministic dependency-first traversal of the union -therefore invokes the compiler and archiver once for every reachable canonical -production package, even when both test products need it. Each root is still -compiled once with its own selected sources and linked separately. The shared -plan is deliberately package-test-specific: it is not a generalized scheduler, -action schema, cache, or protocol. +The command loads all roots into one command-scoped package universe. Each root +retains an injective artifact key derived from its deterministic request ordinal +and variant, such as `__ww-test-000-same` or +`__ww-test-003-external`. Hyphens make that namespace illegal as a WW import +identity. Imports of the same canonical production directory intern to one +production node across every selected test directory. A deterministic +dependency-first traversal of the complete union therefore invokes the +compiler and archiver once for every reachable canonical production package, +even when many directory products need it. Each root is still compiled once +with its own selected sources and linked separately. The shared plan is +deliberately package-test-specific: it is not a generalized scheduler, action +schema, cache, or protocol. + +Each selected directory retains the ordinary entry-directory-first resolution +context from the local package slice: its directory, explicit `-I` roots in +command order, then the toolchain source root. Same and external variants of +one directory share that context; unrelated directory roots never acquire +lookup precedence from their request order. When multiple contexts reach one +canonical production package, the loader verifies that every directory, +folded-file, and inline import binding is identical before reusing its compile +action. A different binding is a deterministic package-resolution failure for +the roots that reach it, rather than a first-root-wins build. A production action failure blocks exactly the roots that reach it. A root-local compile or link failure does not suppress a successfully built sibling product: the command records completion per product, and the coordinator can run and report successful siblings while attributing each -missing product as a build failure. Directory builds and completed test +missing product as a build failure. The single union build and completed test products share the coordinator's existing `-j` process bound; captured output is still emitted only in byte-sorted directory/package order. @@ -2895,9 +2912,11 @@ test link still receives the root object and the complete reverse-topological archive closure. The compiler-generated `-T` dispatcher owns the implicit direct test-runtime support edge and remains embedded in each independently compiled test root; it is the narrow test-main variant, not a -coordinator-generated graph package. The shared plan compiles the common -runtime production package once. The command resolves that edge from the -selected toolchain source tree, not the user search path. Normally its graph +coordinator-generated graph package. The command-scoped plan compiles the +common runtime production package once for the complete test request. The +command resolves that edge from the selected toolchain source tree, not the +user search path; the support package's own imports are also loaded in that +toolchain context. Normally its graph qualifier is `test`, so an explicit source `import test` coalesces with the same canonical package. When a real user package occupies that identity, the command presents the runtime edge to the compiler under the reserved `__wwtest` @@ -2905,29 +2924,61 @@ qualifier. This keeps a production package named `test` available to external tests while preserving raw `w6c -T` compatibility, whose default unresolved qualifier remains `test`. +The reserved support action and an ordinary source-imported toolchain `test` +action may coexist in the command universe because their compiler qualifiers +are `__wwtest` and `test`. An external-first colocated production action uses +an owning-root artifact key so it remains collision-proof if the command also +loads a different physical package bearing the same compiler qualifier. +Same-qualifier, same-canonical normal and external-production actions always +coalesce; different logical identities for one ordinary physical directory +remain an error. These are the +only narrow action-role exceptions: each product closure is checked to contain +at most one importable action for a compiler module qualifier, so unrelated +roles can never introduce duplicate linked package symbols. + The selected test roots and a production variant reached by their imports or test-runtime closure are the only sanctioned graph nodes that may share a physical directory. This also lets a toolchain package's own tests coexist with the production variant required by the test runtime. A same-package root already defines those production symbols, so the production archive is omitted from that product's final link while the production node's dependency archives -remain in its closure. The external product includes the production archive. +remain in its closure. A reserved compiler-support action at that same physical +directory is still retained. The external product includes the production +archive. Variant-only archives are never linked into the other product. All ordinary logical and physical package-identity collision checks remain unchanged. +The Cstage linker, like the WWstage linker, passes the root object, every +reachable archive, the runtime, and explicit `-L`/`-l` values through a +structured argument vector; no fixed flattened command buffer can truncate a +large closure. WWstage emits joined `-Ldir` and `-lname` arguments accepted by +its native linker, while Cstage preserves the equivalent split forms. Generated +artifact paths are bounds-checked before any unit is opened, so distinct root +keys cannot alias by truncation. Recursive discovery groups by physical directory before sorting filenames, and -persistent coordinator work directories use one injective escaped directory -plan key. Every `*_test.ww` package variant is built even when a file only +one stable escaped request key names the persistent command work directory. +Every `*_test.ww` package variant is built even when a file only declares helpers and contains no `@test`, so its package clause and imports are still checked by the shared loader. Every built variant is run, and a successful compiler-owned dispatcher with empty output is reported as `[no tests]`. +Persistent workdirs keep a global compiler/assembler/stamp identity and +per-action committed units. When that global identity is stale, the command +first removes every old `.unit.ww` voucher while leaving artifacts recoverable. +It can then commit the new identity even if one root fails: successful actions +have freshly committed units, whereas failed or no-longer-requested actions +cannot be reused. A retry therefore recompiles the failed action without +discarding unchanged canonical packages built for independent products. + This ownership split follows Go 1.26.5's separation of production, same-package-test, external-test, and generated test-main inputs in [`cmd/go/internal/load/test.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go), while retaining WW's compiler-owned dispatcher. Go's loader returns the same -cached package pointer for repeated imports, and its command-scoped builder -deduplicates compile actions by package identity; WW borrows that one-universe, -multi-root ownership boundary without borrowing Go's cache machinery. Direct +cached package pointer for repeated imports, and `PackageList` walks arbitrary +multiple roots with pointer-based deduplication in +[`cmd/go/internal/load/pkg.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go). +Its command-scoped builder deduplicates compile actions by package identity. WW +borrows that one-universe, multi-root ownership boundary without borrowing Go's +cache machinery. Direct compile dependencies and the separately expanded link closure follow the boundary in [`cmd/go/internal/work/action.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go),