From a88078faf89ae226f89ace1f6354643dd173a8a6 Mon Sep 17 00:00:00 2001 From: Hojun-Cho Date: Wed, 12 Aug 2026 18:42:26 +0900 Subject: [PATCH] docs: record direct package export boundary --- docs/build-system.md | 168 +++++++++++++++++++++++++++-------------- docs/test-system-v2.md | 12 +-- 2 files changed, 117 insertions(+), 63 deletions(-) diff --git a/docs/build-system.md b/docs/build-system.md index c6076556..ffe291df 100644 --- a/docs/build-system.md +++ b/docs/build-system.md @@ -2820,18 +2820,20 @@ retaining the deeper declarations' original package identity. Checked fixed array dimensions are emitted as numeric type facts, so a public layout never requires exposing the private constant spelling that produced its length. -A package compilation unit contains one `.wwi` for each byte-sorted **direct** -import and no separately injected transitive interface. Origin-tagged facts -inside those direct artifacts are compiler data, not source imports: a source -qualifier is visible only when its owning package directly imports it, and -private members, transitive-only qualifiers, bare values, and bare types remain -compiler errors. In `-c` package mode the compiler coalesces repeated exported -type/constant facts with the same origin, kind, and name, preserving one nominal -type identity across diamonds; raw non-package `w6c` retains its existing -duplicate behavior. The source-like `.wwi` syntax remains a transitional export -encoding pending the binary `.wwe` format described above, but the direct-input -ownership boundary is now live in production Cstage and WWstage compilers and -drivers. +A package compilation unit contains only that package's own byte-sorted sources +and deterministic `//ww:module-reset` separators. Each **direct** import is a +separate `--import ` compiler input, sorted by +canonical path and deduplicated by the loader; no transitive `.wwi` is passed. +Origin-tagged facts inside those direct artifacts are compiler data, not source +imports: a source qualifier is visible only when its owning package directly +imports it, and private members, transitive-only qualifiers, bare values, and +bare types remain compiler errors. In `-c` package mode the compiler parses each +export independently, then coalesces repeated exported type/constant facts with +the same origin, kind, and name, preserving one nominal type identity across +diamonds; raw non-package `w6c` retains its existing one-source behavior. The +source-like `.wwi` syntax remains a transitional export encoding pending the +binary `.wwe` format described above, but the separate direct-input ownership +boundary is live in production Cstage and WWstage compilers and drivers. An ordinary root is linked with the full reachable object closure into the requested executable (legacy WW programs may use a package name other than @@ -2914,8 +2916,9 @@ is still emitted only in byte-sorted directory/package order. Every non-root dependency is always a production variant, so dependency `*_test.ww` files never enter the graph. Imports that occur only in selected -test files add edges only to that test root. The compiler unit for each package -contains only its byte-sorted direct dependency `.wwi` artifacts; the final +test files add edges only to that test root. Each compiler unit contains only +the variant's owned source set, while its invocation receives only the +byte-sorted direct dependency `.wwi` artifacts as separate inputs. The final test link still receives the root object and the complete reverse-topological archive closure. The compiler-generated `-T` dispatcher owns the implicit direct test-runtime support edge and remains embedded in each independently @@ -3013,14 +3016,14 @@ assembler remains attributed to its owning package in both stages. The Cstage linker also sets executable mode with `chmod(2)` on the exact output path rather than invoking an ambient command. -This changes only process invocation and publication. Source imports still own -the graph, each directory is still one production package, compiler actions -still consume direct dependency export data through `.unit.ww`, and links still -receive the complete per-root `.a` closure. Repository-native coverage wraps -all three real stage tools at executable paths containing spaces, records every -argument boundary, inspects `.unit.ww`, `.wwi`, `.a`, and root object placement, -runs the published test binary, compares Cstage/WWstage artifacts and traces, -and injects a compiler failure to compare package attribution. +Source imports still own the graph, each directory is still one production +package, compiler actions receive direct dependency exports as individual +arguments beside an owner-only `.unit.ww`, and links receive the complete +per-root `.a` closure. Repository-native coverage wraps all three real stage +tools at executable paths containing spaces, records every argument boundary, +inspects `.unit.ww`, `.wwi`, `.a`, and root object placement, runs the published +binary, compares repeated Cstage/WWstage artifacts and traces, and removes one +direct export at compiler entry to compare package-attributed diagnostics. Go 1.26.5 keeps the same responsibility boundary: its work executor passes the selected compiler or linker tool and a constructed argument slice to the @@ -3089,12 +3092,13 @@ stamp. A warm invocation byte-compares all applicable live executables before considering any committed unit reusable. A missing or changed driver copy invalidates every `.unit.ww` voucher before compilation; old artifacts may remain recoverable, but none can be reused without a freshly committed unit. -The workdir format revisions are 5 for ordinary builds and 6 for tests. +The workdir format revisions are 6 for ordinary builds and 7 for tests. This closes a real hidden-input boundary. The driver, rather than `w6c`, owns -canonical directory interning, source-derived graph construction, direct-export -unit composition, deterministic dependency ordering, single-member package -archive serialization, and the artifact commit sequence. Unit equality alone +canonical directory interning, source-derived graph construction, owner-only +unit composition, direct-export argument construction, deterministic dependency +ordering, single-member package archive serialization, and the artifact commit +sequence. Unit equality alone cannot identify changes to those algorithms, and a manually maintained format number can be forgotten. Exact driver bytes conservatively cover them during the transitional plain-file reuse scheme. This may rebuild after an unrelated @@ -3110,8 +3114,8 @@ Repository-native coverage runs a three-directory import graph through copied, independently mutable Cstage and WWstage drivers. Exact compiler, assembler, and linker wrappers prove a cold dependency-first build, an unchanged warm compile/assemble skip with a deliberate relink, and full package invalidation -after only the invoking driver's bytes change. The test inspects direct -`.unit.ww` inputs, `.wwi`, `.a`, identity files, exact tool arguments, +after only the invoking driver's bytes change. The test inspects owner-only +`.unit.ww` inputs, separate direct `.wwi` arguments, `.a`, identity files, transitive link order, diagnostics before tool execution, published binary bytes, runtime exit, and stage equivalence. @@ -3133,9 +3137,9 @@ import, including imports selected only by a package-test variant and the compiler-generated test-support edge. The loader makes one ordered pass for `//`; it never probes `/.ww`. Unit composition consequently writes only the owning package's byte-sorted source -files after its direct dependency interfaces and never recursively folds an -imported source body. Missing imports retain the importing source position and -the same stable diagnostic in both stages. +files and never copies a dependency interface or imported source body. Missing +imports retain the importing source position and the same stable diagnostic in +both stages. Root selection remains a separate compatibility boundary. A literal `.ww` CLI target, or a bare CLI target found as `/.ww` after the global @@ -3162,35 +3166,83 @@ artifacts, link and run a transitive archive closure, and repeat the resolution through a real directory-package test. With only the file root present, both stages reject the import with byte-identical package-attributed stderr. The existing exact-tool observer remains the non-duplicated proof that each -canonical production action compiles once, compiler units contain only direct -`.wwi` inputs, and linker argument vectors contain the complete reachable `.a` -closure and no `.wwi` path. +canonical production action compiles once, compiler units contain only owned +sources, compile argument vectors contain exactly direct `.wwi` inputs, and +linker vectors contain the complete reachable `.a` closure and no `.wwi` path. -This follows Go 1.26.5's concrete directory ownership. `ImportDir` is defined -as importing the package in a named directory; import lookup accepts directory -candidates, then reads and sorts that directory's immediate entries -([`go/build/build.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#521), -[`go/build/build.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#725), -[`go/build/build.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#859)). -Repeated loads reuse the canonical package object, and `PackageList` performs a -pointer-deduplicated postorder walk -([`cmd/go/internal/load/pkg.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#633), -[`cmd/go/internal/load/pkg.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#2776)). +### 11.12 Implemented direct compiler-export input slice -Go's build action is interned by mode and package identity and receives only -the package's direct imports, while linking explicitly expands all transitive -link dependencies -([`cmd/go/internal/work/action.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#437), -[`cmd/go/internal/work/action.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#646), -[`cmd/go/internal/work/action.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#1034)). -Its compiler export writer finalizes self-contained public data in sorted index -order, which is why direct compiler artifacts suffice -([`cmd/compile/internal/noder/unified.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/unified.go#463)). -Finally, Go constructs distinct ordinary, internal-test, external-test, and -generated-main package variants through the same import loader -([`cmd/go/internal/load/test.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#85)). -WW adopts these package-ownership boundaries without adding modules, manifests, -network lookup, a generalized action graph, or a cache protocol. +Cstage and WWstage now share one small package-compiler convention: +`--import ` may repeat before the one owning +source unit. It is valid only with `-c`; paths must be nonempty, strictly sorted, +and unique. Both compilers read and parse every export independently under the +supplied canonical identity before parsing the owner unit, then pass the merged +semantic declaration list through the existing checker, deterministic export +writer, and primary-only code generator. Missing export bytes therefore fail at +the compiler boundary as `w6c: import : cannot read `, followed by +the driver's stable owning-package attribution. No import configuration file, +manifest, schema, package database, or network lookup is involved. + +Both drivers construct those arguments directly from the package node's sorted, +deduplicated outgoing edges. They never walk grandchildren for compilation. +Every `.unit.ww` contains only the node's byte-sorted source files and reset +separators, while executable linking independently walks the full reachable +package closure and passes archives, never interfaces. The same path handles an +ordinary package, the production-plus-internal-test variant, the external test +package and its reused production package, compiler-generated test main, and +the reserved test-support package. Persistent workdirs compare a newly emitted +export with its committed predecessor before allowing a direct importer to +reuse owner-identical artifacts, retaining correctness without a new cache +schema or identity record. + +The exact-argv regression uses the real diamond +`base -> {left,right} -> root`. It proves one compile per node; no input for +`base`; only `base.wwi` for each middle node; only sorted `left.wwi` and +`right.wwi` for `root`; exact owner-only unit bytes; the complete three-archive +link closure; no link-time `.wwi`; exit status 42; and byte-identical units, +exports, archives, executables, and tool argument vectors across two clean +Cstage builds and two clean WWstage builds. The existing directory-package +variant regression checks exact direct inputs for internal and external +generated roots and the external-production action; both stages also compile +and run ordinary production and support actions with owner-only units and +byte-identical artifacts. + +The pinned Go 1.26.5 implementation supplies the design boundary: + +- Directory identity starts with `ImportDir`'s named directory + ([`go/build/build.go`, lines 521–524](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#521)); directory reads are name-sorted + ([lines 108–111](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#108)), lookup selects directory candidates + ([lines 725–809](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#725)), and the selected directory is enumerated + ([lines 859–900](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#859)). +- The loader records direct imports + ([`load/pkg.go`, lines 220–225](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#220)), guarantees repeated cache loads return the same package pointer + ([lines 633–636](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#633)), keys reuse by canonical import path + ([lines 757–768](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#757)), and pointer-deduplicates dependency-first package lists + ([lines 2776–2795](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#2776)). +- Build actions are interned by mode and package identity + ([`work/action.go`, lines 437–447](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#437)); compilation depends only on direct imports + ([lines 628–659](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#628)), whereas link actions expand transitive dependencies + ([lines 918–957](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#918), + [lines 1034–1068](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#1034)). The executor derives compiler import inputs from those direct actions + ([`work/exec.go`, lines 864–884](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#864)), passes them separately from source files + ([lines 928–930](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#928)), and separately emits the expanded linker closure + ([lines 1592–1647](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#1592)). +- Tests preserve four package roles + ([`load/test.go`, lines 85–102](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#85)): internal production-plus-test + ([lines 175–225](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#175)), external test + ([lines 228–265](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#228)), and generated main + ([lines 272–293](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#272)), with their distinct imports attached at + [lines 351–373](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#351). +- Unified export writing re-links, re-exports, and prunes facts + ([`noder/unified.go`, lines 152–165](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/unified.go#152)), finalizes self-contained data + ([lines 463–470](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/unified.go#463)), and sorts declarations and bodies before serialization + ([lines 514–570](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/unified.go#514)). Compiler import lookup opens the separately mapped artifact + ([`noder/import.go`, lines 61–101](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/import.go#61)) and decodes its exports independently of source parsing + ([lines 170–225](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/import.go#170)); `ReadPackage` consumes that package decoder + ([`importer/ureader.go`, lines 28–62](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/importer/ureader.go#28)). + +WW adopts those practical ownership and action semantics while retaining its +small direct CLI representation and existing self-contained `.wwi` encoding. ## 12. Candidate architectures and hard-gate decision diff --git a/docs/test-system-v2.md b/docs/test-system-v2.md index 491d6efc..4f56b9fa 100644 --- a/docs/test-system-v2.md +++ b/docs/test-system-v2.md @@ -238,11 +238,13 @@ caller-owned persistent package-artifact workdir: the directory must already exist, is never cleaned by the driver, and holds one committed unit, `.wwi`, `.s`, `.o`, and dep `.a` per package plus byte copies of the invoking driver, compiler, and assembler and a small mode stamp. A package is reused only when -its freshly composed unit byte-equals the committed unit and every applicable -recorded executable byte-equals the live executable — content identity only, -no mtimes, no hashes, every decision reproducible with `cmp` against plain -files. The driver identity covers the graph, unit-composition, archive, and -commit algorithms that neither unit bytes nor compiler identity can name. +its freshly composed owner unit byte-equals the committed unit, no recompiled +direct dependency emitted changed export bytes, and every applicable recorded +executable byte-equals the live executable — content identity only, no mtimes, +no hashes, every decision reproducible with `cmp` against plain files. The +driver identity covers graph construction, owner-unit composition, +direct-export argument construction, archive creation, and commit algorithms +that neither unit bytes nor compiler identity can name. Recompiled artifacts land at staged `.new` names and commit by rename with the unit renamed last, so an interrupted build forces a recompile rather than a false reuse; the link always reruns. One workdir serves one invocation at a