Hojun-Cho/ww
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: pin canonical package model evidence

Browse Source
This commit is contained in:
Hojun-Cho
2026-08-13 00:54:16 +09:00
parent 27a501fc31
commit 22ee8f5ae6
1 changed files with 165 additions and 97 deletions
Download Patch File Download Diff File Expand all files Collapse all files

262
docs/build-system.md
View File

@@ -2878,39 +2878,59 @@ policy. The command owns source selection, package loading, compiler inputs,
archive construction, generated-main construction, and linking for every
package variant:
- The ordinary production action selects the directory's byte-sorted
non-test files and is reused wherever that canonical package is imported.
- `same-test` is a distinct internal production-plus-test action. It selects
the byte-sorted production files followed by the byte-sorted matching
`package p` test files, so tests can use private production declarations.
- `external-test` selects only matching `package p_test` files. Its `import p`
is a direct edge to the canonical production action for that directory.
That action compiles with module qualifier `p`, selects every production
file, emits compiler export data and an archive, and exposes no private
declaration to the external root. If the action is first reached through
the external product, its collision-proof artifact key is derived from the
owning root, such as `__ww-test-001-external-production`; a normal import of
the same `p` and canonical directory reuses that action rather than creating
a second compilation.
- Each selected internal or external variant gets a distinct generated-main
package action. Its generated owner-only unit declares `package main` and
imports exactly the selected variant and test-support package. It consumes
exactly those direct `.wwi` artifacts, emits its own `.wwi/.o/.a`, and alone
receives compiler `-T --entry`.
- The ordinary production action selects the directory's byte-sorted non-test
files. A no-test requested product uses this action directly, and every
ordinary, internal-test, or external-test import of that package reuses it.
- The internal production-plus-test variant selects the byte-sorted production
files followed by byte-sorted matching `package p` test files. It is distinct
from production so its private production declarations and test declarations
remain isolated to that test product.
- The external variant selects only matching `package p_test` files. Its
`import p` is a direct edge to the same ordinary production action used by
ordinary products and transitive imports; there is no external-production
role or artifact.
- Generated main is a separate package action whose owner-only generated unit
declares `package main` and imports exactly the selected variant and test
support. It consumes those direct `.wwi` files, emits its own `.wwi/.o/.a`,
and alone receives compiler `-T --entry`.
The command loads all variants into one command-scoped package universe.
Directory variants retain injective artifact keys such as
`__ww-test-000-same` or `__ww-test-003-external`; their generated mains use
matching `__ww-test-NNN-main` keys and distinct internal package identities.
Hyphens keep artifact keys illegal as WW import identities. Repeated requests
for the same canonical directory and variant reuse one compile action, and
imports of the same canonical production directory intern to one production
action across every selected test product. A deterministic dependency-first
traversal of the complete union therefore invokes the compiler and archiver
once per package variant, even when many products share it. Generated-main
actions remain product-specific and are linked separately. The shared plan is
deliberately package-test-specific: it is not a generalized scheduler, action
schema, cache, or protocol.
The authoritative directory-action identity is the triple **(canonical
filesystem directory, canonical ordinary import path, semantic variant)**.
The semantic variants are production, internal production-plus-test, and
external `_test`; generated main and the reserved test-support alias are
explicit non-directory action classes. Source package name validates the leaf
of the bound import path but is not itself a second identity. Requested-root
state, discovery role, product ordinal, output path, persistent artifact key,
and discovery order never enter the triple.
A literal directory root may enter the interner before its full import spelling
is known. It is provisionally interned by canonical directory and variant,
binds immediately if a source import reaches it, and otherwise binds after its
sources establish the validated manifest-free package name. All binding is
finished before generated-main construction or compilation. One bound import
path mapping to two directories and one directory acquiring two incompatible
ordinary import paths are both command-global deterministic errors before any
compiler, assembler, archiver, or linker ambiguity. The same check spans
variants: an external action with ordinary base `p` cannot hide a different
directory's production `p` behind its derived compiler path `p_test`.
Artifact publication follows the semantic action instead of product order:
production uses `p`, internal uses `p-internal-test`, external uses
`p_test-external-test`, and their generated mains append `-main`. Generated
package identities are likewise variant-derived, for example
`__wwtestmain.p.internal.main` and
`__wwtestmain.p_test.external.main`. Equivalent products therefore reuse an
already-interned directory action and generated-main action and publish through
the same persistent-workdir slots regardless of discovery or product order.
The narrow raw single-file compatibility path alone retains `__root`.
A deterministic dependency-first traversal of the complete command union
invokes the compiler, assembler, and in-driver deterministic archiver once per
interned action. This is compile-time interning, not linker-argument
deduplication. Each product is then linked separately from its generated-main
or executable root archive and the complete reachable archive closure. The
shared plan remains package-test-specific; it is not a generalized scheduler,
action schema, cache, or protocol.
Each selected directory retains the ordinary entry-directory-first resolution
context from the local package slice: its directory, explicit `-I` roots in
@@ -2955,17 +2975,14 @@ tests. Explicit raw single-file `ww test FILE` fixtures retain the narrow fused
canonical directory package; that path still consumes support as a direct
export and emits a root `.wwi/.a`.
The reserved support action and an ordinary source-imported toolchain `test`
action may coexist in the command universe because their compiler qualifiers
are `__wwtest` and `test`. An external-first colocated production action uses
an owning-root artifact key so it remains collision-proof if the command also
loads a different physical package bearing the same compiler qualifier.
Same-qualifier, same-canonical normal and external-production actions always
coalesce; different logical identities for one ordinary physical directory
remain an error. These are the
only narrow action-role exceptions: each product closure is checked to contain
at most one importable action for a compiler module qualifier, so unrelated
roles can never introduce duplicate linked package symbols.
The reserved support action and an ordinary source-imported package `test` may
coexist only because the former is explicitly rebound to the compiler-only
qualifier `__wwtest`. This is the sole directory-alias exception and cannot be
created by a source import. All ordinary production actions, including an
external test's colocated production dependency, use canonical directory
identity and the global bidirectional import-path checks above. There is no
role-based tolerance for duplicate ordinary import identities and no late
product-closure ambiguity to resolve.
The selected internal/external variants and a production action reached by
their imports or test-runtime closure are the sanctioned graph nodes that may
@@ -3001,21 +3018,19 @@ have freshly committed units, whereas failed or no-longer-requested actions
cannot be reused. A retry therefore recompiles the failed action without
discarding unchanged canonical packages built for independent products.
This ownership split follows Go 1.26.5's separation of production,
same-package-test, external-test, and generated test-main inputs in
[`cmd/go/internal/load/test.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go),
while retaining WW's compiler-owned dispatcher. Go's loader returns the same
cached package pointer for repeated imports, and `PackageList` walks arbitrary
multiple roots with pointer-based deduplication in
[`cmd/go/internal/load/pkg.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go).
Its command-scoped builder deduplicates compile actions by package identity. WW
borrows that one-universe, multi-root ownership boundary without borrowing Go's
cache machinery. Direct
compile dependencies and the separately expanded link closure follow the
boundary in
[`cmd/go/internal/work/action.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go),
with compiler and linker import configurations emitted separately in
[`cmd/go/internal/work/exec.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go).
Warm reuse compares the staged owner-only unit, committed artifacts, and the
actual bytes of each direct dependency export. A changed shared dependency is
compiled once; its direct importers are reconsidered once; and propagation
stops as soon as a regenerated importer export is byte-identical. Stable
semantic artifact keys make that behavior independent of which ordinary or
test product first discovered the action.
This ownership split follows the pinned Go 1.26.5 implementation cited in
section 11.12: its loader reuses canonical cached package objects, its work
builder interns actions by mode and package, and its test loader reuses ordinary
production while constructing only the required internal, external, and
generated-main variants. WW borrows that command-global ownership boundary
without adding Go's build cache, import configuration format, or module system.
### 11.8 Implemented exact package-tool invocation slice
@@ -3226,49 +3241,102 @@ vectors across two clean
Cstage builds and two clean WWstage builds. The existing directory-package
variant regression checks separate production, internal, external, and
generated-main actions, exact generated-main direct variant/support exports,
the external-production action, and archive-only link closures. Both stages
compile and run those actions with owner-only units and byte-identical
artifacts.
canonical production reuse across ordinary and test products, and archive-only
link closures. It also reverses equivalent product descriptors, proves a
selected ordinary root imported by another selected product compiles once,
and changes a shared direct export in persistent workdirs to prove propagation
through direct importers stops at the first byte-identical regenerated export.
Both stages compile and run those actions with owner-only units and
byte-identical artifacts.
The pinned official Go 1.26.5 tag (commit
`c19862e5f8415b4f24b189d065ed739517c548ba`) supplies the design boundary:
- Directory identity starts with `ImportDir`'s named directory
([`go/build/build.go`, lines 521–524](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#521)); directory reads are name-sorted
([lines 108–111](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#108)), lookup selects directory candidates
([lines 725–809](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#725)), and the selected directory is enumerated
([lines 859–900](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#859)). The sorted iteration filters files and appends each source to exactly one ordinary/internal/external-test bucket
([lines 895–1036](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#895)).
- The loader records direct imports
([`load/pkg.go`, lines 220–225](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#220)), guarantees repeated cache loads return the same package pointer
([lines 633–636](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#633)), keys reuse by canonical import path
([lines 757–768](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#757)), and pointer-deduplicates dependency-first package lists
([lines 2776–2795](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#2776)).
- Build actions are interned by mode and package identity
([`work/action.go`, lines 437–447](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#437)); compilation depends only on direct imports
([lines 628–659](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#628)), and even package `main` receives that normal interned archive-producing compile action
([lines 641–647](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#641)). A link action places that main compile action at its first dependency and expands transitive dependencies
([lines 918–957](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#918),
[lines 1034–1068](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#1034)). The executor derives compiler import inputs from those direct actions
([`work/exec.go`, lines 864–884](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#864)), passes them separately from source files
([lines 928–930](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#928)), packs remaining objects into the package archive and records it as the built result
([lines 1017–1033](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#1017)), then supplies the root archive and all expanded package inputs to linking
([lines 1592–1624](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#1592),
[lines 1635–1647](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#1635)).
- Tests preserve four package roles
([`load/test.go`, lines 85–102](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#85)): internal production-plus-test
([lines 175–225](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#175)), external test
([lines 228–265](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#228)), and generated main
([lines 272–293](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#272)). Its support imports are loaded for that generated package
([lines 307–332](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#307)), with the selected internal/external variants attached as distinct direct imports at
[lines 351–373](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#351).
- Unified export writing re-links, re-exports, and prunes facts
([`noder/unified.go`, lines 152–165](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/unified.go#152)), finalizes self-contained data
([lines 463–470](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/unified.go#463)), selects export roots, and sorts declaration and body indices before serialization
([lines 495–570](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/unified.go#495)). Compiler import lookup opens the separately mapped artifact
([`noder/import.go`, lines 61–101](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/import.go#61)) and decodes its exports independently of source parsing
([lines 170–225](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/import.go#170)); `ReadPackage` consumes that package decoder
([`importer/ureader.go`, lines 28–62](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/importer/ureader.go#28)).
- `go/build` represents one selected directory package with its import path,
package name, ordinary files, internal-test files, external-test files, and
their imports ([`build.go`, lines 436–493](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L436-L493)).
Its directory reader is required to return name-sorted entries
([lines 108–111](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L108-L111)),
import lookup selects one directory in search order
([lines 725–767](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L725-L767)),
and the selected directory alone is scanned
([lines 859–913](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L859-L913)).
The sorted scan assigns each accepted source to that package's ordinary,
internal-test, or external-test list
([lines 948–1036](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L948-L1036)).
- `cmd/go/internal/load` expands source imports before recording their
canonical paths
([`pkg.go`, lines 658–669](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L658-L669)).
It resolves canonical path and directory before consulting the package-data
cache ([lines 833–842](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L833-L842),
[lines 863–911](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L863-L911)),
and the command-global package cache returns the existing package pointer for
a later root or import of the resolved identity
([lines 633–647](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L633-L647),
[lines 757–775](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L757-L775)).
The package's parsed import list becomes its direct package dependencies,
rather than a transitive flattening
([lines 433–440](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L433-L440),
[lines 2024–2047](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L2024-L2047)).
- `cmd/go/internal/work` keys its action cache by operation mode plus package
pointer ([`action.go`, lines 202–206](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L202-L206))
and returns the already-interned action for that key
([lines 437–447](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L437-L447)).
A compile action depends on only `p.Internal.Imports`
([lines 628–658](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L628-L658));
an executable link asks for that same cached root compile action
([lines 919–957](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L919-L957))
and separately expands the complete transitive link closure
([lines 1034–1068](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L1034-L1068)).
- The work executor derives compiler package mappings from those direct build
dependencies
([`exec.go`, lines 864–884](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L864-L884)),
compiles the package's own source list to `_pkg_.a`
([lines 928–935](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L928-L935)),
packs and publishes that package archive
([lines 1017–1033](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1017-L1033)),
and links the compiled main archive with mappings for every dependency
already expanded onto the link action
([lines 1592–1624](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1592-L1624),
[lines 1635–1647](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1635-L1647)).
- Go's test loader explicitly models production, internal
production-plus-test, external `_test`, and generated main, and states that
`ptest == p` when production can be reused
([`test.go`, lines 85–102](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L85-L102)).
Test imports use the ordinary load cache and compare canonical `ImportPath`
([lines 118–161](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L118-L161));
the internal copy is created only when needed
([lines 175–226](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L175-L226)),
while external and generated-main packages remain distinct
([lines 228–293](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L228-L293)).
Generated main receives its direct support and selected-variant imports
([lines 307–358](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L307-L358)),
and copy-on-write rewriting preserves the original package pointers/actions
for unaffected importers
([lines 421–472](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L421-L472)).
- Unified export production begins from the local package, re-exports required
dependency data, and prunes unnecessary detail
([`unified.go`, lines 147–168](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/unified.go#L147-L168)).
It type-checks the package's parsed sources and writes deterministically
ordered public/private roots
([lines 314–362](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/unified.go#L314-L362)),
then finalizes self-contained export data with sorted relocated declaration
and body indexes and a fingerprint
([lines 463–570](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/unified.go#L463-L570)).
- Compiler import handling canonicalizes each source import and rejects self
import ([`import.go`, lines 125–167](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/import.go#L125-L167)),
then independently opens and decodes each direct package archive/export
([lines 170–225](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/import.go#L170-L225)).
The complete unified section and linker fingerprint are read from that
selected package file
([lines 229–296](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/import.go#L229-L296)).
`ReadPackage` reconstructs a package from its public export root
([`ureader.go`, lines 28–62](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/importer/ureader.go#L28-L62)),
interns embedded package descriptors by canonical path and restores their
import lists
([lines 152–196](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/importer/ureader.go#L152-L196)),
and reconstructs declarations from relocated export records
([lines 391–468](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/importer/ureader.go#L391-L468)).
WW adopts those practical ownership and action semantics while retaining its
small direct CLI representation and existing self-contained `.wwi` encoding.