Hojun-Cho/ww
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ww: canonicalize directory package identities

Browse Source
This commit is contained in:
Hojun-Cho
2026-08-13 02:48:12 +09:00
parent 853090174b
commit 0aff801def
4 changed files with 781 additions and 163 deletions
Download Patch File Download Diff File Expand all files Collapse all files

156
docs/build-system.md
View File

@@ -2837,24 +2837,28 @@ binary `.wwe` format described above, but the separate direct-input ownership
boundary is live in production Cstage and WWstage compilers and drivers.
An ordinary executable directory root is one normal package action. Its
declared package identity tags its owner-only unit; it receives only direct
exports, emits `.wwi`, `.o`, and a deterministic `.a`, and is compiled exactly
once. The narrow compiler `--entry` flag controls only bare `main` codegen and is
independent of export production. The linker receives that root archive first,
then the complete reachable package-archive closure and runtime archive; it
never receives `.wwi`. The linkers seed `main` before archive selection, so the
existing WWAR member protocol needs no special root object or format change.
finalized canonical import identity tags its owner-only unit; it receives only
direct exports, emits `.wwi`, `.o`, and a deterministic `.a`, and is compiled
exactly once. The declared package name only validates the last component of
that identity. The narrow compiler `--entry` flag controls only bare `main`
codegen and is independent of export production. The linker receives that root
archive first, then the complete reachable package-archive closure and runtime
archive; it never receives `.wwi`. The linkers seed `main` before archive
selection, so the existing WWAR member protocol needs no special root object or
format change.
`ww build -p -o lib.a DIR` explicitly requests a non-main package product: it
emits a deterministic archive at `lib.a` and its compiler interface at
`lib.a.wwi`, without invoking the linker. A logical target retains its full
identity (`ww build -p -I ROOT -o bar.a foo.bar` emits `foo.bar.*` symbols),
while a literal directory uses its declared leaf package. Package output
requires a directory and `-p` cannot be combined with assembly-only `-S`. Two
cold builds with identical inputs are required to produce byte-identical
requested products. Compiler intrinsics keep their package-mode runtime ABI
independent of transitive source interfaces (for example, `alloc` lowers to the
runtime allocator without requiring an `rt.wwi` compiler input).
while a literal directory is reverse-resolved through the active source roots
or receives the deterministic local identity described below. Its declared
leaf can never invent or truncate that identity. Package output requires a
directory and `-p` cannot be combined with assembly-only `-S`. Two cold builds
with identical inputs are required to produce byte-identical requested
products. Compiler intrinsics keep their package-mode runtime ABI independent
of transitive source interfaces (for example, `alloc` lowers to the runtime
allocator without requiring an `rt.wwi` compiler input).
### 11.7 Implemented directory package-test slice
@@ -2904,22 +2908,63 @@ state, discovery role, product ordinal, output path, persistent artifact key,
and discovery order never enter the triple.
A literal directory root may enter the interner before its full import spelling
is known. It is provisionally interned by canonical directory and variant,
binds immediately if a source import reaches it, and otherwise binds after its
sources establish the validated manifest-free package name. All binding is
finished before generated-main construction or compilation. One bound import
path mapping to two directories and one directory acquiring two incompatible
ordinary import paths are both command-global deterministic errors before any
compiler, assembler, archiver, or linker ambiguity. The same check spans
variants: an external action with ordinary base `p` cannot hide a different
directory's production `p` behind its derived compiler path `p_test`.
is known. It is provisionally interned by canonical directory and variant, and
a later source import of that directory binds and reuses the provisional action.
After all source discovery, but before generated-main construction or any tool
invocation, each still-unbound directory is finalized by this exact algorithm:
1. For every import-resolution context that reached the directory, walk that
context's roots in its normal forward precedence: the selected package's
directory, explicit `-I` roots in command order, then `WW_SRCLIB` or the
selected toolchain source root. Recursive package-test requests privately
insert their symlink-resolved discovery root before user `-I` roots, so
descendants retain their complete relative identity.
2. Canonicalize each candidate root and require the package directory to be a
strict descendant. Every relative path component must be a non-keyword WW
identifier. Convert separators to dots, then resolve that relative spelling
again through the complete ordered context. Accept it only if ordinary
forward lookup selects the same canonical directory. Thus an earlier shadow
invalidates a name inferred from a later or nested root.
3. Bind the first precedence-valid candidate from each reaching context through
the command-global bidirectional interner. An identity supplied by successful
logical package lookup, such as `-p encoding.utf8`, is already bound and is
preserved exactly. That forward-selected identity is authoritative: reverse
derivation applies only to still-unbound literal roots, so a nested active
root cannot rename an explicitly resolved package.
4. If no active root can represent the directory, bind the reserved,
non-source-importable identity
`__wwlocal.p<escaped-canonical-absolute-directory>.<declared-leaf>`. The
escape is injective and reversible over path bytes: ASCII letters and digits
are copied, `_` becomes `_u`, `/` becomes `_s`, and every other byte becomes
`_xHH` with lowercase hexadecimal. Source imports of `__wwlocal` or any of
its children are rejected, so this command-local identity creates no alias.
The selected full identity's final component is then validated against the
ordinary declared package name (or against the ordinary leaf obtained by
removing `_test` for the external variant). There is no fallback from an empty
import path to a declaration name. Relative, absolute, and symlink spellings
converge through the canonical directory; two unrelated local directories with
the same declaration therefore remain distinct. One bound import path mapping
to two directories and one directory acquiring two incompatible ordinary
import paths are command-global deterministic errors before any compiler,
assembler, archiver, or linker ambiguity. The same check spans variants: an
external action cannot hide a different directory's production package behind
its derived `_test` compiler path.
The derivation and diagnostics are implemented symmetrically in
`cmd/ww/main.c` and `selfhost/cmd/ww/main.ww`. The package coordinator in
`internal/wwpackage/package.ww` supplies the canonical recursive discovery root,
preserves an explicitly resolved logical request identity, and keys a persistent
request workdir only by the canonical discovery directory. `w6c` and `wcc`
continue to consume and validate the finalized dotted identity; neither tool
performs directory lookup or introduces a package registry.
Artifact publication follows the semantic action instead of product order:
production uses `p`, internal uses `p-internal-test`, external uses
`p_test-external-test`, and their generated mains append `-main`. Generated
package identities are likewise variant-derived, for example
`__wwtestmain.p.internal.main` and
`__wwtestmain.p_test.external.main`. Equivalent products therefore reuse an
production uses the full finalized ordinary identity, internal appends
`-internal-test`, external appends `_test-external-test`, and their generated
mains append `-main`. Generated package identities are likewise derived from
the full variant identity, for example `__wwtestmain.a.foo.internal.main` and
`__wwtestmain.b.foo_test.external.main`. Equivalent products therefore reuse an
already-interned directory action and generated-main action and publish through
the same persistent-workdir slots regardless of discovery or product order.
The narrow raw single-file compatibility path alone retains `__root`.
@@ -3004,7 +3049,10 @@ arguments accepted by its native linker, while Cstage preserves the equivalent
split forms. Generated artifact paths are bounds-checked before any unit is
opened, so distinct root keys cannot alias by truncation.
Recursive discovery groups by physical directory before sorting filenames, and
one stable escaped request key names the persistent command work directory.
one stable escape of the canonical discovery directory names the persistent
command work directory. It contains neither a declared package leaf nor a
product ordinal, so equivalent path spellings and reordered products select the
same request state.
Every `*_test.ww` package variant is built even when a file only
declares helpers and contains no `@test`, so its package clause and imports are
still checked by the shared loader. Every built variant is run, and a successful
@@ -3128,7 +3176,9 @@ stamp. A warm invocation byte-compares all applicable live executables before
considering any committed unit reusable. A missing or changed driver copy
invalidates every `.unit.ww` voucher before compilation; old artifacts may
remain recoverable, but none can be reused without a freshly committed unit.
The workdir format revisions are 6 for ordinary builds and 7 for tests.
The workdir format revisions are 8 for ordinary builds and 9 for tests. The
identity-finalization change bumps both formats so no leaf-keyed unit voucher
can be reused as a full-path package action.
This closes a real hidden-input boundary. The driver, rather than `w6c`, owns
canonical directory interning, source-derived graph construction, owner-only
@@ -3231,6 +3281,19 @@ export with its committed predecessor before allowing a direct importer to
reuse owner-identical artifacts, retaining correctness without a new cache
schema or identity record.
The canonical-root regressions additionally prove the following in both
stages: a literal root under one import root publishes its complete dotted
identity; logical, literal absolute, equivalent, relative, and symlink routes
emit byte-identical `.unit.ww`, `.wwi`, and `.a`; root-only and combined
root/import requests emit those same bytes; dependency-first and root-first
discovery each compile the shared production action once; recursive `a/foo`
and `b/foo` directories declaring the same `package foo` publish distinct
`a.foo` and `b.foo` variants; and two outside-root `package foo` directories
coexist in one command under distinct reversible local identities. Equivalent
recursive spellings reuse the same persistent request directory without new
compilation, while source imports of the reserved local namespace reject before
tool invocation.
The exact-argv regression uses the real diamond
`base -> {left,right} -> root`. It proves one compile per node; no input for
`base`; only `base.wwi` for each middle node; only sorted `left.wwi` and
@@ -3242,12 +3305,11 @@ Cstage builds and two clean WWstage builds. The existing directory-package
variant regression checks separate production, internal, external, and
generated-main actions, exact generated-main direct variant/support exports,
canonical production reuse across ordinary and test products, and archive-only
link closures. It also reverses equivalent product descriptors, proves a
selected ordinary root imported by another selected product compiles once,
and changes a shared direct export in persistent workdirs to prove propagation
through direct importers stops at the first byte-identical regenerated export.
Both stages compile and run those actions with owner-only units and
byte-identical artifacts.
link closures. It also reverses equivalent product descriptors and compares
exact compiler and linker trace bytes, then changes a shared direct export in
persistent workdirs to prove propagation through direct importers stops at the
first byte-identical regenerated export. Both stages compile and run those
actions with owner-only units and byte-identical artifacts.
The pinned official Go 1.26.5 tag (commit
`c19862e5f8415b4f24b189d065ed739517c548ba`) supplies the design boundary:
@@ -3255,25 +3317,39 @@ The pinned official Go 1.26.5 tag (commit
- `go/build` represents one selected directory package with its import path,
package name, ordinary files, internal-test files, external-test files, and
their imports ([`build.go`, lines 436–493](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L436-L493)).
Those fields remain separate in Go; WW's final-component/name equality is its
existing language validation layered on the canonical identity, not a claim
that Go conflates `Name` with `ImportPath`.
Its directory reader is required to return name-sorted entries
([lines 108–111](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L108-L111)),
import lookup selects one directory in search order
and local directory loading reverse-derives a complete import path by checking
`GOROOT/src` first and then `GOPATH` roots in order. A candidate under a later
root is rejected when the same relative path resolves through an earlier root
to another directory; an outside-root directory remains without an ordinary
import path
([lines 612–665](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L612-L665)).
Forward import lookup selects one directory in search order
([lines 725–767](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L725-L767)),
and the selected directory alone is scanned
([lines 859–913](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L859-L913)).
The sorted scan assigns each accepted source to that package's ordinary,
internal-test, or external-test list
([lines 948–1036](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L948-L1036)).
- `cmd/go/internal/load` expands source imports before recording their
canonical paths
- `cmd/go/internal/load` derives an outside-root local directory's deterministic
pseudo-import path from its slash-form absolute directory and establishes the
package-data cache/promise boundary around that resolved key
([`pkg.go`, lines 633–647](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L633-L647)).
WW uses the same reserved full-directory principle but a reversible byte
escape, strengthening it so two canonical directory spellings cannot collapse
merely through character sanitization. The Go loader expands source imports
before recording their canonical paths
([`pkg.go`, lines 658–669](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L658-L669)).
It resolves canonical path and directory before consulting the package-data
cache ([lines 833–842](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L833-L842),
[lines 863–911](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L863-L911)),
and the command-global package cache returns the existing package pointer for
a later root or import of the resolved identity
([lines 633–647](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L633-L647),
[lines 757–775](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L757-L775)).
([lines 757–775](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L757-L775)).
The package's parsed import list becomes its direct package dependencies,
rather than a transitive flattening
([lines 433–440](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L433-L440),