Hojun-Cho/ww
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: pin canonical package model evidence

Browse Source
This commit is contained in:
Hojun-Cho
2026-08-13 00:54:16 +09:00
parent 27a501fc31
commit 22ee8f5ae6
1 changed files with 165 additions and 97 deletions
Download Patch File Download Diff File Expand all files Collapse all files

262
docs/build-system.md
View File

@@ -2878,39 +2878,59 @@ policy. The command owns source selection, package loading, compiler inputs,
archive construction, generated-main construction, and linking for every archive construction, generated-main construction, and linking for every
package variant: package variant:
- The ordinary production action selects the directory's byte-sorted - The ordinary production action selects the directory's byte-sorted non-test
non-test files and is reused wherever that canonical package is imported. files. A no-test requested product uses this action directly, and every
- `same-test` is a distinct internal production-plus-test action. It selects ordinary, internal-test, or external-test import of that package reuses it.
the byte-sorted production files followed by the byte-sorted matching - The internal production-plus-test variant selects the byte-sorted production
`package p` test files, so tests can use private production declarations. files followed by byte-sorted matching `package p` test files. It is distinct
- `external-test` selects only matching `package p_test` files. Its `import p` from production so its private production declarations and test declarations
is a direct edge to the canonical production action for that directory. remain isolated to that test product.
That action compiles with module qualifier `p`, selects every production - The external variant selects only matching `package p_test` files. Its
file, emits compiler export data and an archive, and exposes no private `import p` is a direct edge to the same ordinary production action used by
declaration to the external root. If the action is first reached through ordinary products and transitive imports; there is no external-production
the external product, its collision-proof artifact key is derived from the role or artifact.
owning root, such as `__ww-test-001-external-production`; a normal import of - Generated main is a separate package action whose owner-only generated unit
the same `p` and canonical directory reuses that action rather than creating declares `package main` and imports exactly the selected variant and test
a second compilation. support. It consumes those direct `.wwi` files, emits its own `.wwi/.o/.a`,
- Each selected internal or external variant gets a distinct generated-main and alone receives compiler `-T --entry`.
package action. Its generated owner-only unit declares `package main` and
imports exactly the selected variant and test-support package. It consumes
exactly those direct `.wwi` artifacts, emits its own `.wwi/.o/.a`, and alone
receives compiler `-T --entry`.
The command loads all variants into one command-scoped package universe. The authoritative directory-action identity is the triple **(canonical
Directory variants retain injective artifact keys such as filesystem directory, canonical ordinary import path, semantic variant)**.
`__ww-test-000-same` or `__ww-test-003-external`; their generated mains use The semantic variants are production, internal production-plus-test, and
matching `__ww-test-NNN-main` keys and distinct internal package identities. external `_test`; generated main and the reserved test-support alias are
Hyphens keep artifact keys illegal as WW import identities. Repeated requests explicit non-directory action classes. Source package name validates the leaf
for the same canonical directory and variant reuse one compile action, and of the bound import path but is not itself a second identity. Requested-root
imports of the same canonical production directory intern to one production state, discovery role, product ordinal, output path, persistent artifact key,
action across every selected test product. A deterministic dependency-first and discovery order never enter the triple.
traversal of the complete union therefore invokes the compiler and archiver
once per package variant, even when many products share it. Generated-main A literal directory root may enter the interner before its full import spelling
actions remain product-specific and are linked separately. The shared plan is is known. It is provisionally interned by canonical directory and variant,
deliberately package-test-specific: it is not a generalized scheduler, action binds immediately if a source import reaches it, and otherwise binds after its
schema, cache, or protocol. sources establish the validated manifest-free package name. All binding is
finished before generated-main construction or compilation. One bound import
path mapping to two directories and one directory acquiring two incompatible
ordinary import paths are both command-global deterministic errors before any
compiler, assembler, archiver, or linker ambiguity. The same check spans
variants: an external action with ordinary base `p` cannot hide a different
directory's production `p` behind its derived compiler path `p_test`.
Artifact publication follows the semantic action instead of product order:
production uses `p`, internal uses `p-internal-test`, external uses
`p_test-external-test`, and their generated mains append `-main`. Generated
package identities are likewise variant-derived, for example
`__wwtestmain.p.internal.main` and
`__wwtestmain.p_test.external.main`. Equivalent products therefore reuse an
already-interned directory action and generated-main action and publish through
the same persistent-workdir slots regardless of discovery or product order.
The narrow raw single-file compatibility path alone retains `__root`.
A deterministic dependency-first traversal of the complete command union
invokes the compiler, assembler, and in-driver deterministic archiver once per
interned action. This is compile-time interning, not linker-argument
deduplication. Each product is then linked separately from its generated-main
or executable root archive and the complete reachable archive closure. The
shared plan remains package-test-specific; it is not a generalized scheduler,
action schema, cache, or protocol.
Each selected directory retains the ordinary entry-directory-first resolution Each selected directory retains the ordinary entry-directory-first resolution
context from the local package slice: its directory, explicit `-I` roots in context from the local package slice: its directory, explicit `-I` roots in
@@ -2955,17 +2975,14 @@ tests. Explicit raw single-file `ww test FILE` fixtures retain the narrow fused
canonical directory package; that path still consumes support as a direct canonical directory package; that path still consumes support as a direct
export and emits a root `.wwi/.a`. export and emits a root `.wwi/.a`.
The reserved support action and an ordinary source-imported toolchain `test` The reserved support action and an ordinary source-imported package `test` may
action may coexist in the command universe because their compiler qualifiers coexist only because the former is explicitly rebound to the compiler-only
are `__wwtest` and `test`. An external-first colocated production action uses qualifier `__wwtest`. This is the sole directory-alias exception and cannot be
an owning-root artifact key so it remains collision-proof if the command also created by a source import. All ordinary production actions, including an
loads a different physical package bearing the same compiler qualifier. external test's colocated production dependency, use canonical directory
Same-qualifier, same-canonical normal and external-production actions always identity and the global bidirectional import-path checks above. There is no
coalesce; different logical identities for one ordinary physical directory role-based tolerance for duplicate ordinary import identities and no late
remain an error. These are the product-closure ambiguity to resolve.
only narrow action-role exceptions: each product closure is checked to contain
at most one importable action for a compiler module qualifier, so unrelated
roles can never introduce duplicate linked package symbols.
The selected internal/external variants and a production action reached by The selected internal/external variants and a production action reached by
their imports or test-runtime closure are the sanctioned graph nodes that may their imports or test-runtime closure are the sanctioned graph nodes that may
@@ -3001,21 +3018,19 @@ have freshly committed units, whereas failed or no-longer-requested actions
cannot be reused. A retry therefore recompiles the failed action without cannot be reused. A retry therefore recompiles the failed action without
discarding unchanged canonical packages built for independent products. discarding unchanged canonical packages built for independent products.
This ownership split follows Go 1.26.5's separation of production, Warm reuse compares the staged owner-only unit, committed artifacts, and the
same-package-test, external-test, and generated test-main inputs in actual bytes of each direct dependency export. A changed shared dependency is
[`cmd/go/internal/load/test.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go), compiled once; its direct importers are reconsidered once; and propagation
while retaining WW's compiler-owned dispatcher. Go's loader returns the same stops as soon as a regenerated importer export is byte-identical. Stable
cached package pointer for repeated imports, and `PackageList` walks arbitrary semantic artifact keys make that behavior independent of which ordinary or
multiple roots with pointer-based deduplication in test product first discovered the action.
[`cmd/go/internal/load/pkg.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go).
Its command-scoped builder deduplicates compile actions by package identity. WW This ownership split follows the pinned Go 1.26.5 implementation cited in
borrows that one-universe, multi-root ownership boundary without borrowing Go's section 11.12: its loader reuses canonical cached package objects, its work
cache machinery. Direct builder interns actions by mode and package, and its test loader reuses ordinary
compile dependencies and the separately expanded link closure follow the production while constructing only the required internal, external, and
boundary in generated-main variants. WW borrows that command-global ownership boundary
[`cmd/go/internal/work/action.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go), without adding Go's build cache, import configuration format, or module system.
with compiler and linker import configurations emitted separately in
[`cmd/go/internal/work/exec.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go).
### 11.8 Implemented exact package-tool invocation slice ### 11.8 Implemented exact package-tool invocation slice
@@ -3226,49 +3241,102 @@ vectors across two clean
Cstage builds and two clean WWstage builds. The existing directory-package Cstage builds and two clean WWstage builds. The existing directory-package
variant regression checks separate production, internal, external, and variant regression checks separate production, internal, external, and
generated-main actions, exact generated-main direct variant/support exports, generated-main actions, exact generated-main direct variant/support exports,
the external-production action, and archive-only link closures. Both stages canonical production reuse across ordinary and test products, and archive-only
compile and run those actions with owner-only units and byte-identical link closures. It also reverses equivalent product descriptors, proves a
artifacts. selected ordinary root imported by another selected product compiles once,
and changes a shared direct export in persistent workdirs to prove propagation
through direct importers stops at the first byte-identical regenerated export.
Both stages compile and run those actions with owner-only units and
byte-identical artifacts.
The pinned official Go 1.26.5 tag (commit The pinned official Go 1.26.5 tag (commit
`c19862e5f8415b4f24b189d065ed739517c548ba`) supplies the design boundary: `c19862e5f8415b4f24b189d065ed739517c548ba`) supplies the design boundary:
- Directory identity starts with `ImportDir`'s named directory - `go/build` represents one selected directory package with its import path,
([`go/build/build.go`, lines 521–524](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#521)); directory reads are name-sorted package name, ordinary files, internal-test files, external-test files, and
([lines 108–111](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#108)), lookup selects directory candidates their imports ([`build.go`, lines 436–493](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L436-L493)).
([lines 725–809](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#725)), and the selected directory is enumerated Its directory reader is required to return name-sorted entries
([lines 859–900](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#859)). The sorted iteration filters files and appends each source to exactly one ordinary/internal/external-test bucket ([lines 108–111](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L108-L111)),
([lines 895–1036](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#895)). import lookup selects one directory in search order
- The loader records direct imports ([lines 725–767](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L725-L767)),
([`load/pkg.go`, lines 220–225](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#220)), guarantees repeated cache loads return the same package pointer and the selected directory alone is scanned
([lines 633–636](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#633)), keys reuse by canonical import path ([lines 859–913](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L859-L913)).
([lines 757–768](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#757)), and pointer-deduplicates dependency-first package lists The sorted scan assigns each accepted source to that package's ordinary,
([lines 2776–2795](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#2776)). internal-test, or external-test list
- Build actions are interned by mode and package identity ([lines 948–1036](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L948-L1036)).
([`work/action.go`, lines 437–447](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#437)); compilation depends only on direct imports - `cmd/go/internal/load` expands source imports before recording their
([lines 628–659](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#628)), and even package `main` receives that normal interned archive-producing compile action canonical paths
([lines 641–647](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#641)). A link action places that main compile action at its first dependency and expands transitive dependencies ([`pkg.go`, lines 658–669](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L658-L669)).
([lines 918–957](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#918), It resolves canonical path and directory before consulting the package-data
[lines 1034–1068](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#1034)). The executor derives compiler import inputs from those direct actions cache ([lines 833–842](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L833-L842),
([`work/exec.go`, lines 864–884](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#864)), passes them separately from source files [lines 863–911](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L863-L911)),
([lines 928–930](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#928)), packs remaining objects into the package archive and records it as the built result and the command-global package cache returns the existing package pointer for
([lines 1017–1033](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#1017)), then supplies the root archive and all expanded package inputs to linking a later root or import of the resolved identity
([lines 1592–1624](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#1592), ([lines 633–647](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L633-L647),
[lines 1635–1647](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#1635)). [lines 757–775](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L757-L775)).
- Tests preserve four package roles The package's parsed import list becomes its direct package dependencies,
([`load/test.go`, lines 85–102](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#85)): internal production-plus-test rather than a transitive flattening
([lines 175–225](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#175)), external test ([lines 433–440](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L433-L440),
([lines 228–265](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#228)), and generated main [lines 2024–2047](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L2024-L2047)).
([lines 272–293](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#272)). Its support imports are loaded for that generated package - `cmd/go/internal/work` keys its action cache by operation mode plus package
([lines 307–332](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#307)), with the selected internal/external variants attached as distinct direct imports at pointer ([`action.go`, lines 202–206](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L202-L206))
[lines 351–373](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#351). and returns the already-interned action for that key
- Unified export writing re-links, re-exports, and prunes facts ([lines 437–447](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L437-L447)).
([`noder/unified.go`, lines 152–165](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/unified.go#152)), finalizes self-contained data A compile action depends on only `p.Internal.Imports`
([lines 463–470](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/unified.go#463)), selects export roots, and sorts declaration and body indices before serialization ([lines 628–658](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L628-L658));
([lines 495–570](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/unified.go#495)). Compiler import lookup opens the separately mapped artifact an executable link asks for that same cached root compile action
([`noder/import.go`, lines 61–101](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/import.go#61)) and decodes its exports independently of source parsing ([lines 919–957](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L919-L957))
([lines 170–225](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/import.go#170)); `ReadPackage` consumes that package decoder and separately expands the complete transitive link closure
([`importer/ureader.go`, lines 28–62](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/importer/ureader.go#28)). ([lines 1034–1068](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L1034-L1068)).
- The work executor derives compiler package mappings from those direct build
dependencies
([`exec.go`, lines 864–884](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L864-L884)),
compiles the package's own source list to `_pkg_.a`
([lines 928–935](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L928-L935)),
packs and publishes that package archive
([lines 1017–1033](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1017-L1033)),
and links the compiled main archive with mappings for every dependency
already expanded onto the link action
([lines 1592–1624](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1592-L1624),
[lines 1635–1647](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1635-L1647)).
- Go's test loader explicitly models production, internal
production-plus-test, external `_test`, and generated main, and states that
`ptest == p` when production can be reused
([`test.go`, lines 85–102](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L85-L102)).
Test imports use the ordinary load cache and compare canonical `ImportPath`
([lines 118–161](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L118-L161));
the internal copy is created only when needed
([lines 175–226](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L175-L226)),
while external and generated-main packages remain distinct
([lines 228–293](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L228-L293)).
Generated main receives its direct support and selected-variant imports
([lines 307–358](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L307-L358)),
and copy-on-write rewriting preserves the original package pointers/actions
for unaffected importers
([lines 421–472](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L421-L472)).
- Unified export production begins from the local package, re-exports required
dependency data, and prunes unnecessary detail
([`unified.go`, lines 147–168](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/unified.go#L147-L168)).
It type-checks the package's parsed sources and writes deterministically
ordered public/private roots
([lines 314–362](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/unified.go#L314-L362)),
then finalizes self-contained export data with sorted relocated declaration
and body indexes and a fingerprint
([lines 463–570](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/unified.go#L463-L570)).
- Compiler import handling canonicalizes each source import and rejects self
import ([`import.go`, lines 125–167](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/import.go#L125-L167)),
then independently opens and decodes each direct package archive/export
([lines 170–225](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/import.go#L170-L225)).
The complete unified section and linker fingerprint are read from that
selected package file
([lines 229–296](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/import.go#L229-L296)).
`ReadPackage` reconstructs a package from its public export root
([`ureader.go`, lines 28–62](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/importer/ureader.go#L28-L62)),
interns embedded package descriptors by canonical path and restores their
import lists
([lines 152–196](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/importer/ureader.go#L152-L196)),
and reconstructs declarations from relocated export records
([lines 391–468](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/importer/ureader.go#L391-L468)).
WW adopts those practical ownership and action semantics while retaining its WW adopts those practical ownership and action semantics while retaining its
small direct CLI representation and existing self-contained `.wwi` encoding. small direct CLI representation and existing self-contained `.wwi` encoding.