5331 lines
322 KiB
Markdown
5331 lines
322 KiB
Markdown
# WW package, dependency, build, and bootstrap architecture
|
||
|
||
Status: **binding architecture decision**
|
||
|
||
Decision date: 2026-08-10
|
||
|
||
Implementation status: specified, not yet implemented
|
||
|
||
This document selects the production architecture that replaces WW's current
|
||
package driver, source-like interface protocol, work-directory reuse scheme,
|
||
Make orchestration, test coordinator, and bootstrap chain. It is a greenfield
|
||
decision. Migration effort is recorded only to plan implementation; it did not
|
||
influence the selection.
|
||
|
||
The words **MUST**, **MUST NOT**, **SHOULD**, and **MAY** are normative.
|
||
|
||
## 1. Executive decision
|
||
|
||
WW will have one integrated command backed by one typed, content-addressed
|
||
action graph. Language imports describe the language-package subgraph. A small,
|
||
declarative `ww.mod` file describes only facts that source imports cannot:
|
||
distribution requirements, products, generated inputs, native providers, and
|
||
unusual link steps. Both descriptions lower to the same graph, scheduler,
|
||
sandbox, cache, and explanation machinery. There is no general build language
|
||
and no arbitrary build script. A one-directory, zero-dependency executable
|
||
needs no manifest. Distributed projects use an exact lock file; fetching and
|
||
updating are explicit commands, while build, run, test, documentation, and
|
||
installation never access the network or rewrite project metadata. Packages
|
||
produce target-specific binary export data and one object, importers consume
|
||
only direct export data, and final products link a declared ordered closure.
|
||
Every cached action names its compiler, target, profile, tools, sysroot, runtime,
|
||
native inputs, environment, and content. The official toolchain owns these
|
||
protocols and ships a pinned tool closure, but WW does not permanently own an
|
||
assembler or linker.
|
||
|
||
The decisive insight is that **the package graph and the build graph are not the
|
||
same graph**. Imports are a complete and desirable description of WW-language
|
||
dependencies. They cannot honestly describe a C header tree, a host generator,
|
||
an assembler, a linker script, a CRT, or a sysroot. Making imports pretend to do
|
||
so hides native inputs; making every project use a programmable build framework
|
||
destroys the simple ordinary path. Two small declarative front ends lowering to
|
||
one action engine are simpler as a system than either lie.
|
||
|
||
The architecture is named **WW Action Build** in this document. That is a label,
|
||
not another user-facing product: the command remains `ww`.
|
||
|
||
### 1.1 The Pike lens and attribution
|
||
|
||
Pike explicitly documented and defended these Go design choices and principles:
|
||
|
||
- language-defined imports make dependencies explicit, clear, and mechanically
|
||
computable;
|
||
- unused imports and import cycles are errors; rejecting cycles improves package
|
||
boundaries and independent maintenance;
|
||
- compilation speed and short edit/build cycles are primary design properties;
|
||
- a direct dependency's compiled artifact can carry the deeper public type facts
|
||
needed by its clients, so an importer opens only direct dependency artifacts;
|
||
and
|
||
- orthogonal, predictable concepts, fewer ways to express a construct, and a
|
||
simple user experience are worth substantial implementation work.
|
||
|
||
Those points are stated in Pike's 2009 Go talk, the 2012 SPLASH article, and his
|
||
2015 simplicity talk
|
||
([2009 talk](https://go.dev/talks/2009/go_talk-20091030.pdf),
|
||
[2012 article](https://go.dev/talks/2012/splash.article),
|
||
[Simplicity is Complicated](https://go.dev/talks/2015/simplicity-is-complicated.slide)).
|
||
The collective Plan 9 papers add system-wide placement of complexity, focused
|
||
interfaces, and transparent text or explicitly encoded binary data. `Go in Go`
|
||
documents one contingent case in which owning more of the toolchain simplified
|
||
Go; it does not establish permanent toolchain ownership as a general Pike
|
||
principle ([Go in Go](https://go.dev/talks/2015/gogo.slide)).
|
||
|
||
Plan 9's `mk` constructs
|
||
the dependency graph before execution, rejects cycles and ambiguous recipes,
|
||
and schedules independent work in parallel. Plan 9 used a target-specific
|
||
compiler/assembler/loader family and portably encoded target object conventions
|
||
([mk](https://9p.io/sys/doc/mk.html),
|
||
[mkfiles](https://9p.io/sys/doc/mkfiles.html),
|
||
[compilers](https://9p.io/sys/doc/comp.html)).
|
||
|
||
Modern module-path/version semantics, Minimal Version Selection, `go.sum`, the modern Go build cache,
|
||
automatic toolchain selection, and current supply-chain policy are later Go-team
|
||
designs, not principles uniquely attributable to Pike
|
||
([module reference](https://go.dev/ref/mod),
|
||
[`go` command](https://go.dev/cmd/go/),
|
||
[toolchain selection](https://go.dev/doc/toolchain),
|
||
[toolchain rebuilding](https://go.dev/blog/rebuild),
|
||
[supply-chain policy](https://go.dev/blog/supply-chain)). This decision borrows
|
||
some invariants from those systems but does not attribute them to Pike.
|
||
|
||
The following are this document's inferences from the documented principles:
|
||
|
||
- strict directory packages and direct binary export data are the smallest way
|
||
to keep dependencies computable and compilation fast;
|
||
- a declarative native/action layer is necessary for an unmanaged language,
|
||
because omitting it moves complexity into ambient shell state;
|
||
- one shared action engine is simpler than independent language and outer-build
|
||
caches;
|
||
- given WW's complete-graph, frozen-build, and explainable-key hard gates,
|
||
arbitrary graph-producing programs are rejected; they would require executing
|
||
dependency host code before the graph is inspectable and add another permanent
|
||
user programming model; and
|
||
- WW should specify a toolchain closure but should not maintain an assembler and
|
||
linker forever when pinned external tools make the whole system smaller.
|
||
|
||
Modern native requirements force deliberate departures from historical Plan 9
|
||
and early Go: cryptographic source identities, lock files, explicit build/host/
|
||
target separation, sysroot and SDK identity, hostile dependency acquisition,
|
||
cross-platform sandboxes, and cache-miss explanations were not their complete
|
||
problem statement. WW adopts their architectural restraint, not their ambient
|
||
host assumptions.
|
||
|
||
### 1.2 What follows from being unmanaged and native
|
||
|
||
Being unmanaged/native genuinely requires the build model to know:
|
||
|
||
- the target data layout and C ABI;
|
||
- foreign symbol spelling and visibility;
|
||
- object format, relocation model, CPU features, and assembly dialect;
|
||
- ordered objects, archives, shared libraries, linker scripts, and archive-group
|
||
semantics;
|
||
- the libc, CRT, dynamic loader, SDK, runtime, and sysroot closure;
|
||
- freestanding entry and runtime policy;
|
||
- build-machine tools that generate host- or target-machine inputs; and
|
||
- ABI compatibility among compiler, runtime, native providers, and final link.
|
||
|
||
It does **not** follow that WW needs multiple dependency versions, semantic
|
||
version ranges, a network resolver in every build, programmable build scripts,
|
||
feature unification, a global namespace, or its own linker. In particular, the
|
||
absence of a garbage collector says nothing about version resolution.
|
||
|
||
### 1.3 Binding answers to the critical questions
|
||
|
||
| Question | Binding answer |
|
||
|---|---|
|
||
| Package identity | One canonical import path: the owning module identity for its root package, otherwise that identity plus `/` and the normalized package-relative path. The declared package name is a source qualifier, not identity. |
|
||
| Identity versus location/origin/version/content | All are separate. A resolver record maps identity and selected version to an origin and source-tree digest; a workspace maps identity to a local location. |
|
||
| Directory membership | Exactly one package per directory. Immediate selected source files belong to it. Nested directories never do. |
|
||
| Single-file packages | Deleted. A one-file directory package remains configuration-free. |
|
||
| Language dependency graph | The compiler-parsed imports alone define it. Manifest native/action edges extend the build graph, never the language graph. |
|
||
| Import interfaces | Direct dependencies only. Each direct `.wwe` contains the deep public type information needed to understand its own API. |
|
||
| `.wwi` | Deleted and replaced by deterministic, versioned binary `.wwe` export data. Canonical source prototypes are not an interchange format. |
|
||
| Package invalidation | The package action key changes when its selected own sources/generated inputs, direct export digests, compiler/toolchain, target/profile, declared environment, or protocol changes. A private transitive change does not invalidate it. |
|
||
| Cache key | Domain-separated SHA-256 over the canonical action record defined in section 6.6. |
|
||
| Cache scope | A per-user global local content store plus a project-local graph-history index. Remote import/export is explicit, never part of ordinary build. |
|
||
| Corruption/upgrades | Every object is rehashed on read; corrupt entries are quarantined. Tools and protocol versions are content inputs, so upgrades change keys. |
|
||
| Multiple versions | One selected version of a module identity. Incompatible major releases use distinct module identities ending `/vN`, so those identities may coexist. |
|
||
| Build scripts | Arbitrary scripts are forbidden. A finite declarative action may run a pinned build-machine tool in a denied-by-default sandbox. |
|
||
| Action authority | Exact readable inputs, writable outputs, argv, environment, execution platform, and tool closure. No network, shell, ambient `PATH`, clock, randomness, or undeclared filesystem access. |
|
||
| Acquisition | `add`, `update`, `lock`, `fetch`, `toolchain fetch`, cache transfer, and explicitly authorized remote observation may use the network. Artifact build/analysis and ordinary tests may not; running the finished user program is outside acquisition. |
|
||
| Manifest for a trivial program | No. A standalone directory containing a `main` package is sufficient. A manifest is required for distribution dependencies, multiple products, native providers, or generated inputs. |
|
||
| Configuration placement | Imports in source; identity, requirements, products, native declarations, and actions in `ww.mod`; exact selected closure in `ww.lock`; local paths in `ww.work`; ephemeral target/profile/output choices on the command line. |
|
||
| Local overrides | `ww.work` maps a module identity to a local source tree and records its observed digest. Imports do not change. |
|
||
| Target-specific files | A fixed filename-suffix selection rule; no source-level build expressions and no user-programmable selector. |
|
||
| OS distribution | A distributor may vendor the locked source closure or supply an exact `ww.work`/native-provider map. Substituted files and tools get new digests; frozen mode never silently consults the host. |
|
||
| Reproduction input | For supported official targets: project source, complete locked dependency-source bytes (vendor/CAS export), `ww.lock`, named immutable toolchain bundle, target/profile, and every declared external seed/signing input. Hashes without bytes are insufficient. Impure profiles forfeit the promise. |
|
||
| Assembler/linker ownership | No permanent ownership. The toolchain descriptor pins complete implementations. The current WW tools may bridge migration only. |
|
||
| Stage zero | One release-generated, checked-in portable C99 compiler snapshot plus a tiny declarative bootstrap plan and digest file. |
|
||
|
||
### 1.4 Corrective protocol boundary (2026-08-11)
|
||
|
||
The first Phase 0 experiment over-scoped the protocol freeze. It turned package
|
||
resolution, manifest parsing, compiler projections, action lowering, provider
|
||
recursion, graph traversal, scheduling, cache policy, failure precedence, and
|
||
bootstrap assertions into a declarative expression language. Its checker then
|
||
implemented those operations again. That experiment is preserved as recoverable
|
||
migration evidence, but it is not the production architecture.
|
||
|
||
The correction follows the separation visible in the pinned Go source. Go reads
|
||
imports from source with an imports-only parse and resolves them in ordinary
|
||
loader code ([`go/build/read.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/read.go#272),
|
||
[`cmd/go/internal/load/pkg.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#1775)).
|
||
The compiler writes a narrow export representation in compiler code
|
||
([`cmd/compile/internal/noder/writer.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/writer.go)),
|
||
while `cmd/go` builds and schedules an in-memory action graph with ordinary Go
|
||
functions ([`work/action.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#85),
|
||
[`work/exec.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#73)).
|
||
Action IDs and cache storage/validation are executable hashing and storage
|
||
operations, not schema programs
|
||
([`work.buildActionID`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#260),
|
||
[`internal/cache`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/cache/cache.go#95)).
|
||
Go's `cmd/dist` performs concrete staged builds and checks that the final targets
|
||
are not stale. Separate compiler reproducibility tests compare repeated outputs
|
||
byte-for-byte, while the release process independently rebuilds and compares
|
||
archives bit-for-bit
|
||
([`cmd/dist/build.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/dist/build.go#1404),
|
||
[`reproduciblebuilds_test.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/test/reproduciblebuilds_test.go),
|
||
[`rebuild` account](https://go.dev/blog/rebuild)).
|
||
|
||
WW adopts that division, not Go's module/network/toolchain policy. Normal typed
|
||
C/WW code MUST own source loading, parsing, resolution, compiler behavior,
|
||
lowering, orchestration, storage, and bootstrap execution. Declarative schemas
|
||
MUST describe wire representation only. Tests MUST verify executable behavior;
|
||
a schema or proof-shaped record MUST NOT stand in for running it. A generator is
|
||
permitted only for repetitive codec data and MUST be small, generic,
|
||
deterministic, and byte-for-byte reproducible.
|
||
|
||
Phase 0 therefore freezes only WWAR framing and primitive canonical encoding;
|
||
record/enum/union tags, field order, encoded defaults and record kinds; exact
|
||
domain-separated digest and action-key byte formulas; compact positive and
|
||
malformed-wire vectors; a small reference codec; deterministic data-only codec
|
||
generation; and its repository gate. It does not freeze algorithms for deriving
|
||
the represented records. Every declared record tag remains encoded, including
|
||
an optional field's empty `encoded_default`; absence is not default insertion.
|
||
|
||
The owning implementation phases are binding:
|
||
|
||
| Behavior removed from the Phase 0 experiment | Owning phase |
|
||
|---|---|
|
||
| source imports, package graph/cycles, `.wwe`/`.wwlm`, compiler export and public type/ABI projections | Phase 1 |
|
||
| action construction, graph traversal, scheduling, CAS/cache, environment/sandbox and build failure behavior | Phase 2 |
|
||
| manifest/lock/work/vendor text parsing, module/source resolution, fetching, source-store policy and source-tree construction | Phase 3 |
|
||
| native/provider recursion, lowering, link-plan construction, tool adapters and platform policy | Phase 4 |
|
||
| actual staged bootstrap, fixed-point rebuild and byte comparison | Phase 6 |
|
||
|
||
WW-specific guarantees remain stronger and explicit: frozen artifact builds are
|
||
deterministic and offline, selections are locked, artifacts are content-addressed,
|
||
cached objects are rehashed on read, and bootstrap is established by rebuilding and
|
||
comparing actual bytes. At cutover there is one user-facing build path, as
|
||
already required by the migration plan.
|
||
|
||
## 2. Normative vocabulary
|
||
|
||
| Term | Exact meaning |
|
||
|---|---|
|
||
| **package** | The WW declarations selected from one directory, compiled together under one declared package name and one package identity. |
|
||
| **module** | A distributable, versioned source tree rooted by one `ww.mod`, declaring one globally stable module identity and containing zero or more packages. |
|
||
| **project** | The module or standalone package selected by the user's current command, including its declared products. |
|
||
| **workspace** | A local, non-published set of module-identity-to-directory overlays described by `ww.work`. It changes location, never identity. |
|
||
| **dependency** | A typed directed edge: package import, generated-input edge, tool edge, native-provider edge, runtime edge, ordered link edge, source-input edge, or bootstrap-record edge. A source-input edge is content-rooted and has no producer action; it is valid in template/final action inputs but never in `GraphEdgeV1`. A bootstrap-record edge selects the producer action-record or action-result record for `bootstrap.compare`. The edge kind is never implicit. |
|
||
| **target/platform descriptor** | A canonical architecture/platform/ABI/object/CPU/runtime description. An action labels descriptors by role: execution `B`, product `H`, and optional compiler-output `T`. A target triple is only a short lookup name. |
|
||
| **artifact** | An immutable byte string or canonical directory tree produced by an action and named by a content digest. Materialized files are copies or links, not the artifact's identity. |
|
||
| **toolchain** | An immutable descriptor and content closure containing the compiler, action protocol, export/ABI versions, target descriptors, resource files, runtime implementations, and pinned assembler/linker/archive tools. |
|
||
| **sysroot** | A content-identified target filesystem tree containing the exact headers, libraries, CRT objects, loader metadata, linker scripts, and SDK files exposed to target actions. |
|
||
| **source identity** | `sha256` of the canonical source-tree encoding in section 4.5. It is independent of download URL and checkout path. |
|
||
| **version** | An immutable SemVer release label associated with one module identity and one source identity. It is selection metadata, not package identity. |
|
||
| **product** | A named requested result: executable, static library, shared library, object bundle, test binary, generated tree, documentation tree, or toolchain component. |
|
||
| **action** | A pure, finite build step with a typed canonical record, declared input artifacts, one execution platform, and declared output paths. |
|
||
| **build platform (B)** | The platform on which the build actions execute. |
|
||
| **host platform (H)** | The platform on which the requested product will execute. |
|
||
| **target platform (T)** | For a compiler-like product, the platform for which that product emits code. It is absent for an ordinary executable or library. |
|
||
|
||
Module and package identities are slash-separated ASCII paths. They are
|
||
NFC-normalized, case-sensitive, contain no empty, `.` or `..` segment, and do
|
||
not depend on filesystem case folding. A non-root package identity is written
|
||
`module-id/package/path`; the root package identity is `module-id`. The selected
|
||
module catalog records which module owns each package identity. If two selected
|
||
modules would supply the same package identity, resolution fails rather than
|
||
choosing a longer prefix.
|
||
|
||
A no-manifest invocation gives its sole root package the reserved internal
|
||
identity `@standalone/root`; any external test package gets the reserved identity
|
||
`@test/<first-128-bits-of-SHA256(production-package-identity)>`. These namespaces cannot be
|
||
declared by a module or imported from ordinary source. The default standalone
|
||
executable materializes as `main`, independent of directory basename. A
|
||
standalone package cannot contain/import another local package or be published
|
||
until `ww init` gives it a stable module/package identity.
|
||
|
||
## 3. Source and package rules
|
||
|
||
### 3.1 One directory, one package
|
||
|
||
A package directory contains its immediate regular source files only. The build
|
||
does not follow source symlinks. Nested directories are separate packages. Every
|
||
selected production source MUST begin with the same canonical `package name;`
|
||
clause. The declared name MUST be a valid WW identifier. It need not repeat the
|
||
directory leaf because identity and source qualifier are separate concepts.
|
||
|
||
The following current forms are errors after the migration:
|
||
|
||
- importing a single `.ww` file as if it were a package;
|
||
- placing multiple package blocks in one compilation unit;
|
||
- satisfying an unresolved import from an inline package block;
|
||
- selecting a literal source file as the build root; and
|
||
- finding packages through ordered `-I` search roots.
|
||
|
||
`ww build ./cmd/tool` selects a directory. A directory with one source file is
|
||
still the smallest package and needs no extra metadata.
|
||
|
||
### 3.2 File membership and target selection
|
||
|
||
Production candidates are immediate regular files ending `.ww`, excluding
|
||
`*_test.ww` and generated outputs. Names are byte-sorted after UTF-8 validity and
|
||
normalization checks. A source symlink, duplicate normalized name, case-fold
|
||
collision, or non-regular candidate is a loud error on every host.
|
||
|
||
Target variants use this only convention:
|
||
|
||
```text
|
||
stem[+os][+arch][+environment].ww
|
||
```
|
||
|
||
Recognized tags come from the selected target descriptor, not from the host.
|
||
Files are grouped by `stem`. The matching member with the greatest number of
|
||
tags wins; the untagged member is the fallback. Two equally specific matches are
|
||
an error. Examples are `poll.ww`, `poll+linux.ww`, and
|
||
`poll+linux+amd64+gnu.ww`. This is replacement selection, not additive feature
|
||
selection; additive code uses a distinct stem. Unknown tags are errors. There
|
||
are no boolean selectors, glob expressions, or manifest-defined tag meanings.
|
||
|
||
Test variants put the same tags before the reserved suffix, for example
|
||
`poll+linux_test.ww`; their grammar is `stem[+tags]_test.ww` and the identical
|
||
most-specific rule applies within the test set. A production stem ending
|
||
`_test` is reserved and rejected, preventing a tagged test from being mistaken
|
||
for production source.
|
||
|
||
The selected file-name list is itself an action-key input. Therefore adding or
|
||
removing a more-specific file invalidates the package even when the old files'
|
||
bytes do not change.
|
||
|
||
CPU features and optimization mode always enter the compile key but do not add
|
||
another WW source-selection language. WW-level specialization uses compiler
|
||
intrinsics/runtime dispatch or a distinct package; CPU/float-ABI/PIC-sensitive C
|
||
or assembly uses the finite native `when` constraints in section 8.4. This keeps
|
||
ordinary source membership conventional while still making exceptional native
|
||
selection exact and inspectable.
|
||
|
||
### 3.3 Imports, names, and resolution
|
||
|
||
The canonical forms are:
|
||
|
||
```ww
|
||
import "example.org/codec/hex";
|
||
import wire "example.org/protocol/hex";
|
||
```
|
||
|
||
The quoted string is the package identity. A module package may abbreviate its
|
||
own module prefix with a relative import written `import "./sub/path";`;
|
||
resolution replaces `./` with the importing module identity and normalizes the
|
||
remainder without permitting `..`. Bare dotted imports and filesystem imports
|
||
are deleted. Standalone packages cannot use relative imports.
|
||
|
||
The default source qualifier is the imported package's declared name. An alias
|
||
changes only that qualifier. Two imports producing the same qualifier are an
|
||
error unless one is explicitly aliased. Importing the same identity twice,
|
||
resolving one identity to two sources, or resolving two selected module records
|
||
to the same module identity is a loud collision error.
|
||
|
||
Resolution is exact:
|
||
|
||
1. Build the locked/workspace package catalog by joining every selected module
|
||
identity with its package directories.
|
||
2. Require exactly one catalog owner for the requested package identity; zero is
|
||
unresolved and two is an identity collision, even when one module prefix is
|
||
longer.
|
||
3. Verify that the catalog directory exists in that module source tree and has
|
||
the expected package clause.
|
||
4. Never search another root and never choose by filesystem accident.
|
||
|
||
Imports are parsed by the compiler front end, not a line scanner. Their union
|
||
forms package edges, but name visibility remains file-scoped. Every imported
|
||
qualifier must be used. Package cycles, including self-imports, are reported
|
||
before compilation with one stable identity path through the cycle.
|
||
|
||
If an import's catalog owner is outside the importing module, that owner MUST be
|
||
a direct `require` in the importing module's own `ww.mod`; availability through
|
||
another dependency is not enough. This keeps distribution dependencies as
|
||
explicit as package imports and prevents accidental reliance on a transitive
|
||
selection. `ww add` creates the requirement; source is never rewritten.
|
||
|
||
The sole exception is the toolchain's intrinsic standard module, whose identity
|
||
and source-tree digest are part of the selected toolchain descriptor. It is
|
||
available without a manifest requirement, including to a standalone package;
|
||
it is not searched from an installation directory or upgraded independently.
|
||
|
||
### 3.4 Visibility and internal packages
|
||
|
||
Existing exported-versus-private declaration rules survive. An `internal`
|
||
directory segment adds one resolution rule: a package under
|
||
`M/P/internal/Q` may be imported only by the package `M/P` or a package having
|
||
`M/P/` as a segment prefix. For `M/internal/Q`, the allowed root is module `M`
|
||
and its descendants. This is checked against the catalog's owning module and
|
||
package identities, not checkout paths. There are no friend lists or manifest
|
||
visibility overrides.
|
||
|
||
### 3.5 Tests, examples, documentation, and generated WW
|
||
|
||
Only `*_test.ww` files are test sources. `package p;` tests compile with package
|
||
`p`; `package p_test;` tests compile as a separate external package importing
|
||
the production package normally. Test sources of dependencies are never in a
|
||
consumer graph. `ww test .` tests one package; `ww test ./...` discovers package
|
||
directories under the selected module, excluding hidden, underscore-prefixed,
|
||
vendor, cache, and output directories. Discovery is deterministic and does not
|
||
follow directory symlinks. Test compilation is cached; each selected test binary
|
||
is executed on every command and independent binaries may run in parallel.
|
||
|
||
Each test invocation gets only its declared `test-data` mounted read-only under
|
||
`/data`, its literal `test-env`, a private writable temporary directory, and the
|
||
selected runner/runtime closure. Project/home/host files and network are denied.
|
||
Because execution is an observation rather than an artifact action, real clock,
|
||
process IDs, scheduling, and OS randomness may be exposed and are recorded as
|
||
runner capabilities; their output is never cached or part of byte reproduction.
|
||
|
||
The same-package test action uses the production package identity with the
|
||
non-importable action variant `same-test`; it compiles production and test
|
||
sources together so private names remain visible. An external test uses the
|
||
reserved `@test/<128-bit-production-identity-digest>` package identity and has a
|
||
normal direct import of the production package. Neither identity can collide
|
||
with or be imported by published source.
|
||
|
||
An example is an ordinary package or named product under `examples/`; it has no
|
||
special dependency semantics. Documentation is derived from source comments and
|
||
`.wwe` declarations, not by compiling examples during an ordinary build.
|
||
|
||
Generated WW source MUST be declared as an output of an action and as a generated
|
||
input of exactly one package. It uses the suffix `.wwgen`, does not appear in the
|
||
source tree, may not contain `import` or `package` clauses, and therefore cannot
|
||
discover new graph edges after graph construction. It may refer to built-ins and
|
||
declarations already in its owner package. A generator needing another package
|
||
must have a checked-in owner file that imports it. This restriction keeps the
|
||
complete package graph inspectable before executing generators.
|
||
|
||
A generated fragment may contain a foreign declaration only when the owning
|
||
`package` clause already names its provider slot and the declaration explicitly
|
||
names that slot. It cannot add a native requirement. After generation, the
|
||
compiler verifies imports/package clauses are absent and the observed foreign
|
||
slots exactly match the predeclared set; mismatch is a generator/protocol error.
|
||
|
||
## 4. Dependency distribution model
|
||
|
||
### 4.1 When metadata is required
|
||
|
||
A standalone one-directory executable with no non-toolchain dependency builds
|
||
without metadata. `ww init MODULE` creates a module when the program needs a
|
||
stable import identity, distribution dependencies, multiple packages/products,
|
||
generated inputs, or native declarations.
|
||
|
||
A module tree contains exactly one `ww.mod` at its root; nested manifests are an
|
||
error. A workspace composes separate module roots instead of nesting ownership.
|
||
|
||
`ww.mod` is declarative UTF-8 data. It is not WW code: it has no expressions,
|
||
variables, imports, include files, macros, loops, user functions, or host
|
||
conditionals. Strings use JSON escaping; lists preserve source order and record
|
||
key order is non-semantic. Unknown fields are errors unless a later manifest schema is explicitly
|
||
selected.
|
||
|
||
A minimal module is:
|
||
|
||
```text
|
||
ww-manifest 1
|
||
module = "example.org/hello"
|
||
language = "1"
|
||
toolchain = { id = "ww.org/toolchain", minimum = "v1.4.0" }
|
||
|
||
require "example.org/codec" {
|
||
minimum = "v1.2.3"
|
||
source-index = "https://example.org/codec/.well-known/ww-source"
|
||
}
|
||
```
|
||
|
||
The complete set of top-level clause kinds in schema 1 is `require`, `product`,
|
||
`package`, `action`, and `native`. Global scalar keys are only `module`,
|
||
`language`, and `toolchain`. `require` has required `minimum` and optional
|
||
credential-free `source-index`; the latter maps origin without changing identity.
|
||
The schema tables in section 6.9 close all remaining fields. There is
|
||
deliberately no general `[settings]` escape hatch.
|
||
|
||
The root `main` package is the default executable product, named after the
|
||
module's last segment, with explicit normalized linkage `dynamic` and runtime
|
||
`hosted`. Libraries need no product declaration to be imported.
|
||
Additional or non-default outputs are explicit:
|
||
|
||
```text
|
||
product "inspect" {
|
||
kind = "exe"
|
||
root = "cmd/inspect"
|
||
linkage = "dynamic"
|
||
}
|
||
```
|
||
|
||
### 4.2 Selection rule
|
||
|
||
Versions are `vMAJOR.MINOR.PATCH` SemVer labels with the usual prerelease order.
|
||
A requirement is one minimum version, never a range. Selection chooses the
|
||
greatest minimum requested for each module identity over the complete transitive
|
||
closure and repeats until stable. This is intentionally the small, monotonic
|
||
part of Minimal Version Selection, not every behavior of the Go module command.
|
||
The selected result is written exactly to `ww.lock` by `add`, `update`, or
|
||
`lock`; build never resolves a newer version.
|
||
|
||
Exactly one version of a module identity is selected. A backward-incompatible
|
||
major version `N >= 2` MUST declare a module identity ending `/vN`, and imports
|
||
name that identity. Consequently incompatible releases may coexist as distinct
|
||
identities without an aliasing version resolver. Two versions of the same
|
||
identity cannot coexist.
|
||
|
||
There are no feature sets, optional-dependency activation, target-dependent
|
||
version constraints, upper bounds, wildcard versions, or dependency-wide
|
||
configuration unification. Target variation belongs in source selection and
|
||
native-provider declarations after one source closure is selected.
|
||
|
||
The selected source manifests also contribute one minimum for the same
|
||
toolchain identity. The root lock chooses one exact installed/catalog version at
|
||
least as high as every minimum and records its descriptor/bundle digests. A
|
||
different toolchain identity or unsupported language/export/runtime protocol is
|
||
an error; SemVer alone never overrides protocol compatibility. Dependency
|
||
manifests do not pin the consumer to their development compiler, while an
|
||
application remains exactly reproducible from its lock.
|
||
|
||
### 4.3 Lock file
|
||
|
||
`ww.lock` is generated, canonical, and committed for applications and toolchains.
|
||
Published libraries SHOULD commit it for their own tests, but consumers resolve
|
||
from `ww.mod` requirements. A lock record is:
|
||
|
||
```text
|
||
ww-lock 1
|
||
root-manifest = "sha256:9c..."
|
||
toolchain "ww.org/toolchain" {
|
||
version = "v1.4.0"
|
||
descriptor = "sha256:31..."
|
||
bundle = "sha256:80..."
|
||
origin = "https://dist.wwlang.org/toolchain/v1.4.0/"
|
||
}
|
||
module "example.org/codec" {
|
||
version = "v1.2.3"
|
||
origin = "https://example.org/codec/.ww/v1.2.3.tar.zst"
|
||
archive = "sha256:4a..."
|
||
tree = "sha256:f7..."
|
||
manifest = "sha256:55..."
|
||
signature = "ed25519:key-id:base64..."
|
||
provenance = "https://example.org/codec/.ww/v1.2.3.intoto.jsonl"
|
||
}
|
||
```
|
||
|
||
Records are sorted by identity. Required semantic fields are version, immutable
|
||
origin, archive digest, canonical source-tree digest, and manifest digest.
|
||
Signature and provenance are optional records whose verification policy is
|
||
configured by the user or distributor; hashes are never optional. A signature,
|
||
when present, covers schema, module identity, version, tree digest, and manifest
|
||
digest. Lock files never contain local overlay paths or credentials.
|
||
|
||
### 4.4 Acquisition and network policy
|
||
|
||
`ww add M@V` discovers `M` by the HTTPS convention
|
||
`https://M/.well-known/ww-source`, unless `--from=URL` or a user-configured
|
||
longest-prefix source map supplies an index. The index returns immutable archive
|
||
locations and signed digest records. Redirects and the final URL are recorded.
|
||
The downloaded module manifest MUST declare exactly `M`; the signed index/lock
|
||
record, not source text, binds `V` to its tree digest. Private indexes use the
|
||
same protocol and obtain credentials from the fetch command's credential
|
||
helper; credentials never enter build actions or lock files.
|
||
|
||
`(module identity, version)` is immutable: observing two signed tree digests for
|
||
the same pair is an equivocation error recorded in the source store, never an
|
||
automatic replacement. Yank metadata may prevent new selection but cannot alter
|
||
or invalidate an already locked digest.
|
||
|
||
Only these operations may initiate network requests:
|
||
|
||
- `ww add`, `ww update`, and `ww lock` while selecting metadata;
|
||
- `ww fetch --locked` while materializing the already locked source/toolchain
|
||
closure;
|
||
- `ww toolchain fetch` for an explicitly named toolchain; and
|
||
- explicit `ww cache pull` and `ww cache push`; and
|
||
- explicitly authorized `ww observe` remote execution, which is not a build,
|
||
test-build, or cached artifact action.
|
||
|
||
The build/analysis phases of `ww build`, `run`, `test`, `doc`, `install`,
|
||
`graph`, and `explain` deny network access even when an input is absent. They report the missing source digest and
|
||
the exact `ww fetch --locked` command. They never modify `ww.mod`, `ww.lock`, or
|
||
`ww.work`. `--frozen` additionally requires those files to be present,
|
||
canonical, mutually consistent, and unchanged by selection. `--offline` is an
|
||
explicit assertion of the already mandatory no-network build policy.
|
||
|
||
### 4.5 Source identity and storage
|
||
|
||
A module source tree contains only directories and regular files; symlinks,
|
||
devices, sockets, FIFOs, absolute paths, `..`, duplicate normalized paths, and
|
||
case-fold collisions are rejected. Its identity is:
|
||
|
||
```text
|
||
SHA256("ww-source-tree-v1\0" ||
|
||
for each byte-sorted relative path:
|
||
LP(path) || type || executable-bit || LP(SHA256(file-bytes)))
|
||
```
|
||
|
||
`LP(x)` is an unsigned 64-bit big-endian byte length followed by `x`. Directory
|
||
entries are included with type `dir`; regular files with type `file`. Ownership,
|
||
timestamps, archive compression, checkout path, and non-executable permission
|
||
bits are excluded. Archives are checked both against their blob digest and the
|
||
unpacked tree digest. The source store is immutable and keyed by tree digest.
|
||
|
||
### 4.6 Workspaces, vendoring, and distributors
|
||
|
||
`ww.work` is local, declarative, and normally uncommitted:
|
||
|
||
```text
|
||
ww-work 1
|
||
use "example.org/codec" {
|
||
path = "../codec"
|
||
expect = "sha256:f7..."
|
||
}
|
||
```
|
||
|
||
An overlay replaces only the location for the named module identity. The module
|
||
at that path must declare the same identity. Its current canonical tree digest
|
||
is an action input; `expect` makes accidental drift loud but may be updated by
|
||
`ww work sync`. No import or lock identity changes.
|
||
|
||
`ww vendor` materializes every locked module under `vendor/sha256/<tree>` and
|
||
writes a canonical identity-to-tree `vendor/index.wwv`. Frozen builds may select
|
||
that source store with `--vendor`; the vendored bytes must match the lock. There
|
||
is no flattened import tree and no rewritten import statement.
|
||
|
||
An operating-system distributor has three honest options: ship this vendor
|
||
store, prefill WW's immutable source store, or provide an exact workspace/source
|
||
map to distro-owned trees. Native system libraries are substituted only through
|
||
the explicit provider mechanism in section 8.8. A mutable `/usr` lookup is an
|
||
impure system profile, is local-cache-only, and is rejected by frozen builds.
|
||
|
||
### 4.7 Closed metadata grammar
|
||
|
||
`ww.mod`, `ww.lock`, `ww.work`, vendor indexes, and toolchain/native descriptors
|
||
share this lexical grammar; each schema separately closes its allowed headers,
|
||
clauses, fields, value types, and cardinalities:
|
||
|
||
```text
|
||
document = header newline { statement } EOF
|
||
header = schema-name SP unsigned
|
||
statement = assignment | clause
|
||
assignment = key ws "=" ws value ws newline
|
||
clause = key ws string ws "{" newline
|
||
{ assignment } "}" ws newline
|
||
value = string | unsigned | boolean | list | record
|
||
list = "[" ws [ value { ws "," ws value } [ ws "," ] ] ws "]"
|
||
record = "{" ws [ pair { ws "," ws pair } [ ws "," ] ] ws "}"
|
||
pair = key ws "=" ws value
|
||
key = ALPHA { ALPHA | DIGIT | "_" | "-" }
|
||
string = JSON-string-with-valid-UTF-8
|
||
unsigned = "0" | ("1"…"9" { DIGIT })
|
||
boolean = "true" | "false"
|
||
ws = { SP | TAB | newline | comment }
|
||
comment = "#" { any-character-except-newline }
|
||
```
|
||
|
||
`schema-name` is exactly `ww-manifest`, `ww-lock`, `ww-work`, `ww-vendor`,
|
||
`ww-toolchain`, `ww-native-map`, `ww-native-sidecar`, `ww-install`, or
|
||
`ww-bootstrap`. A clause
|
||
body contains assignments only, so nesting cannot grow into a language.
|
||
Duplicate keys, duplicate singleton clauses, invalid UTF-8, unknown fields,
|
||
integer overflow, and a comment marker inside an unclosed string are errors.
|
||
|
||
Whitespace, comments, assignment order, record-key order, and clause order where
|
||
the schema declares identity keys are non-semantic. List order is semantic.
|
||
Parsing produces a typed record whose canonical semantic encoding is
|
||
`WWAR(record)`, not the original text. Its semantic digest is the applicable
|
||
kind/schema-bound `record_id` from section 6.6. `ww fmt` writes two-space
|
||
canonical text; generated lock/vendor files MUST already equal that rendering
|
||
in frozen mode.
|
||
|
||
## 5. Build model and graph construction
|
||
|
||
### 5.1 One graph, constructed before execution
|
||
|
||
For every command, `ww` constructs a typed graph in these deterministic phases:
|
||
|
||
1. Select the project, manifest, lock, workspace, toolchain descriptor, target,
|
||
and profile. Verify their schemas, canonical identities, and content digests.
|
||
Missing locked inputs are errors; this phase never fetches.
|
||
2. Enumerate selected checked-in package files by section 3.2. Ask the compiler
|
||
front end to parse package clauses, imports, checked-in foreign declarations,
|
||
and test metadata. Generated artifacts are known future input slots but do not
|
||
yet exist. The build driver never scans source lines itself.
|
||
3. Resolve every import by identity, reject collisions/internal violations, and
|
||
compute the complete acyclic package graph.
|
||
4. Add statically declared generated-input, host-tool, native-provider,
|
||
toolchain, runtime, archive, ordered link, and bootstrap-record comparison
|
||
edges. Match target clauses and reject zero or multiple providers. All
|
||
generated output names and consumers are known here.
|
||
5. Lower nodes to the complete **action-template DAG** and report it. A template
|
||
names every edge/input slot, tool, policy, and output, but its final key remains
|
||
unresolved until every predecessor output or selected record content digest
|
||
is known. Only after this point may a cache be read or a tool execute.
|
||
6. As verified cache results or completed predecessors resolve input artifacts,
|
||
finalize ready action records/keys, query the cache, schedule misses, publish
|
||
successful artifacts atomically, then materialize requested products.
|
||
|
||
No executed action may add a node, input, output, import, library, flag, or
|
||
follow-up command. Native C/assembly declarations name complete header/source
|
||
trees rather than learning dependencies from an ambient compiler depfile. This
|
||
may conservatively rebuild for an unused header change, but the graph remains
|
||
complete before execution and the key is correct.
|
||
|
||
`ww graph --actions --format=json` emits the graph after phase 5. Its canonical
|
||
JSON contains node kind, execution platform, typed input/output slots, incoming
|
||
edge kinds, target/profile/toolchain digests, and the exact ordered link plan.
|
||
A resolved node also has its key/cache status; otherwise it has `key: "pending"`,
|
||
`cache: "unknown"`, and a byte-sorted `waiting-on` list. It contains logical
|
||
paths only. A collision, cycle, missing provider,
|
||
undeclared target, or unresolved tool fails graph construction even if a stale
|
||
cache entry might otherwise satisfy the product.
|
||
|
||
### 5.2 Built-in action kinds
|
||
|
||
Schema 1 has this closed set of semantic action kinds:
|
||
|
||
- `ww.package`: compile one package to export data, object code, and link
|
||
metadata;
|
||
- `ww.init`: synthesize one deterministic retained package-initialization
|
||
dispatcher from precomputed package link metadata;
|
||
- `native.compile`: compile one declared C or assembly source unit;
|
||
- `archive`: construct a static library from an ordered object list;
|
||
- `link`: construct an executable or shared library from an ordered link plan;
|
||
- `generate`: execute one declared build-platform tool;
|
||
- `doc`: render a documentation tree from sources and export data;
|
||
- `bootstrap.compare`: compare canonical stage outputs and manifests.
|
||
|
||
Adding an action kind changes the action schema. There is no generic rule engine,
|
||
phony target, implicit suffix rule, command-string target, or shell recipe. An
|
||
archive is made only for an explicit static-library product or native provider;
|
||
WW packages are not automatically wrapped in one-member archives.
|
||
|
||
`run` and `test.run` are non-cacheable invocation nodes, not semantic artifact
|
||
actions. `test.run` consumes the declared data/environment/runner and uses the
|
||
test sandbox above on every request. `ww run`, after its network-denied build,
|
||
launches the user's program with the user's runtime authority/environment unless
|
||
`--sandbox` is explicitly requested; that execution still cannot affect a build
|
||
cache entry. Both store exit status/logs only as observations. Materialization/install is a third category:
|
||
a request-local side effect consuming an immutable artifact/install manifest.
|
||
Neither category can satisfy or poison an artifact-action cache entry.
|
||
|
||
The semantic graph is independent of process boundaries. An implementation MAY
|
||
run compiler workers in-process or in a bounded pool, but each `ww.package`
|
||
action still has an independent canonical record and outputs. There is no
|
||
required background daemon and no daemon state may affect an output.
|
||
|
||
### 5.3 The finite escape hatch
|
||
|
||
Unusual generation and packaging use a declarative `action`, not a build script:
|
||
|
||
```text
|
||
action "protocol-bindings" {
|
||
tool = "product:tools/schema-gen"
|
||
platform = "build"
|
||
inputs = {
|
||
schema = { file = "protocol/schema.idl" }
|
||
}
|
||
outputs = {
|
||
ww = { file = "generated/protocol.wwgen" }
|
||
}
|
||
argv = ["--input", "/in/schema", "--ww", "/out/ww"]
|
||
env = { LANG = "C", TZ = "UTC" }
|
||
}
|
||
|
||
package "protocol" {
|
||
generated = ["action:protocol-bindings:ww"]
|
||
}
|
||
```
|
||
|
||
The tool is either a named executable in the immutable toolchain or a named WW
|
||
product built for `H = B`. `/in/NAME` inputs are read-only mounts and
|
||
`/out/NAME` outputs are initially absent, exclusive writable mounts. The working
|
||
directory is the empty logical `/work`. Output type is exactly `file` or `tree`;
|
||
undeclared files fail the action. `argv` is passed directly, never through a
|
||
shell. Input and output names are identifiers and each path appears through its
|
||
fixed mount, so there is no template language.
|
||
|
||
The sandbox exposes only the declared tool closure at fixed logical `/tool`
|
||
paths, `/in`, `/out`, the literal
|
||
environment map, deterministic locale/time-zone data, and bounded CPU/memory
|
||
resources. Network, process inspection, host devices, user/home directories,
|
||
ambient `PATH`, ambient environment, wall clock, writable source, and filesystem
|
||
paths outside the mounts are denied. Randomness is absent unless a declared
|
||
seed artifact is mounted. A tool may spawn only executable inputs declared in
|
||
its tool closure. These isolation rules apply equally to built-in compiler, C,
|
||
assembler, archiver, linker, documentation, test, and bootstrap executions.
|
||
Strong enforcement is required for frozen/shared-cache builds;
|
||
an unsupported host must fail rather than silently weaken isolation.
|
||
|
||
This action can perform arbitrary computation over finite declared inputs, so it
|
||
is sufficient for code generators, image/file-system builders, binding tools,
|
||
and signing-input preparation. It cannot inspect the project and invent more
|
||
work. Dependency modules may declare actions only for outputs consumed by their
|
||
own packages/products; they cannot register hooks that run merely because the
|
||
module is present.
|
||
|
||
Every artifact-producing action declares `reproducibility = "required"` in a
|
||
frozen or official build. Isolation removes undeclared external inputs, but it
|
||
cannot prove that arbitrary tool internals avoid PIDs, uninitialized memory,
|
||
ASLR-derived values, or race-dependent output. Such variance is a tool/action
|
||
defect. Official releases and first shared-cache publication of a custom
|
||
generator repeat it from clean sandboxes and compare outputs. A node explicitly
|
||
classified `observation` is never result-cached/shared and may not feed an
|
||
artifact action; tests and hardware execution use that class. Impure development
|
||
actions are local-only and outside the byte promise.
|
||
|
||
### 5.4 Build, host, and target
|
||
|
||
WW uses the conventional three-platform meaning rigorously:
|
||
|
||
```text
|
||
B: execution platform on which build actions run
|
||
H: platform ABI of the produced artifact; executable products are intended to run here
|
||
T: output platform of a compiler-like artifact that itself is built for H
|
||
```
|
||
|
||
For an ordinary program `T` is absent. The familiar
|
||
`ww build --target=aarch64-unknown-linux-gnu` spelling sets `H`; it means “build
|
||
the program that runs on this target.” A compiler product may additionally set
|
||
`--host=H --target=T`. Any generator used while producing it still executes on
|
||
`B`; if the generator is itself WW source, its product is compiled with `H = B`.
|
||
|
||
Every action record carries all applicable descriptors, even when two are equal.
|
||
No rule may infer `H` or `T` from the kernel running `ww`. Cross compilation is
|
||
therefore the same graph with a different explicit host/target descriptor, not
|
||
a mode that edits environment variables.
|
||
|
||
In record/JSON field names these are `execution-platform`, `product-platform`,
|
||
and optional `compiler-output-platform`. Source suffix selection, ordinary
|
||
native-provider `when`, sysroot, CRT, runtime, and linker selection always match
|
||
the product platform H. A build-tool dependency instead has its own H equal to
|
||
the parent action's B. CLI `--target` is only the familiar spelling for selecting
|
||
the ordinary product platform; it does not rename the GNU roles internally.
|
||
|
||
### 5.5 Scheduling and failure
|
||
|
||
After graph construction, ready actions run in a deterministic priority order
|
||
with a user-selected concurrency bound. Priority affects latency only; output
|
||
bytes and link order come from records, never completion order. Independent
|
||
actions may finish after another branch fails, but no dependent action starts.
|
||
On the first observed failure WW stops launching work and cancels its owned
|
||
in-flight actions. Which failure triggers cancellation is observational and may
|
||
vary with concurrency; every concurrently observed failure is sorted by stable
|
||
logical node in the report. `--keep-going` instead continues branches whose
|
||
dependency closure remains healthy and reports all failures in that stable
|
||
order. Interrupts cancel only processes owned by this invocation and leave no
|
||
published partial result.
|
||
|
||
Tool stdout and stderr are captured as artifacts and streamed with node labels.
|
||
Diagnostics use module-relative logical paths. `--verbose` may display physical
|
||
mount paths separately, marked non-semantic. A successful action is published
|
||
only after all declared outputs exist, have valid type/mode, are canonicalized
|
||
where required, and have been hashed. A failed action is never entered in the
|
||
action cache.
|
||
|
||
### 5.6 Atomic publication and materialization
|
||
|
||
CAS files and action-result records are written to same-filesystem unique
|
||
temporary names, flushed, rehashed, then atomically renamed to their digest
|
||
locations. For crash durability, WW flushes the parent directory after rename;
|
||
it publishes and flushes every output before the result mapping. A concurrent
|
||
publisher of the same digest verifies equality and
|
||
discards its temporary file. A directory artifact is a canonical tree object
|
||
whose leaves are CAS blobs. An action-result mapping is published last, so no
|
||
reader can observe a result before its outputs.
|
||
|
||
Materialization is a request-local side effect outside the action-template DAG.
|
||
By default requested
|
||
products appear in `out/<target>/<profile>/`; `--out` and `ww install --prefix`
|
||
change only where immutable artifacts are copied or copy-on-write reflinked.
|
||
Hardlinks/symlinks are permitted only when the backing store is enforced
|
||
immutable against the user and mode changes cannot affect its inode. Executable bits are
|
||
set by the artifact record, never by a later ambient `chmod`. Replacement uses
|
||
temporary siblings and atomic rename. `ww clean` removes materialized/project
|
||
state only; `ww cache gc` is the separate explicit global-cache operation.
|
||
|
||
### 5.7 Reproducibility contract
|
||
|
||
For official supported targets, WW promises byte-identical artifacts when every
|
||
artifact action satisfies `reproducibility = "required"` and these are identical:
|
||
|
||
- canonical project source and `ww.lock`;
|
||
- immutable toolchain descriptor and complete bundle;
|
||
- target descriptor and build profile; and
|
||
- all declared action inputs, including generated seeds and signing material.
|
||
|
||
The engine guarantees input isolation, logical paths, and canonical publication;
|
||
repeat-build certification checks arbitrary tool determinism. The resulting promise is independent of absolute checkout, source-store, cache, output,
|
||
and temporary paths; wall time, locale, process order, username, UID, and host
|
||
environment are absent. Logical paths are module/package paths. Debug information
|
||
uses those logical paths and fixed prefix maps. Archive metadata is canonical;
|
||
timestamps and ownership are zeroed; deterministic linker build IDs derive from
|
||
the link key. Official toolchains reject tools that cannot meet this contract.
|
||
|
||
A project source archive plus the complete locked dependency source bytes (a
|
||
vendor/source-CAS export), its lock, the named immutable toolchain bundle, and
|
||
every declared external seed/signing input is therefore a complete offline
|
||
reproduction input. A lock's hashes alone cannot recreate absent bytes. Runtime behavior that depends on a shared library
|
||
outside the pinned runtime/sysroot closure is not covered, and frozen official
|
||
profiles prohibit such a dependency. An explicitly selected impure system
|
||
profile receives no byte-identity promise, cannot publish to a shared cache, and
|
||
prints every ambient input it accepted. `ww verify reproducible` runs isolated
|
||
uncached builds under two physical roots and compares every result artifact and
|
||
action manifest, not just the final executable.
|
||
|
||
## 6. Action records and cache protocol
|
||
|
||
### 6.1 Canonical encoding
|
||
|
||
Action records use **WWAR 1**, this deterministic byte encoding:
|
||
|
||
```text
|
||
WWAR(record) = 0x57 0x57 0x41 0x52 | u16be(1) | value(record)
|
||
value(v) = type:u8 | u64be(payload-length) | payload
|
||
|
||
type 0x01 bytes: payload is the bytes
|
||
type 0x02 string: payload is valid NFC UTF-8 with no NUL
|
||
type 0x03 uint: payload is minimal unsigned big-endian; zero is one 0x00
|
||
type 0x04 bool: payload is exactly 0x00 or 0x01
|
||
type 0x05 list: u32be(count) | each (u64be(value-length) | value)
|
||
type 0x06 map: u32be(count) | each (u64be(key-length) | key-UTF-8 |
|
||
u64be(value-length) | value)
|
||
type 0x07 record: u32be(count) | each (u32be(field-tag) |
|
||
u64be(value-length) | value)
|
||
```
|
||
|
||
Record fields are strictly increasing by numeric tag. Map entries are strictly
|
||
increasing by raw UTF-8 key bytes. Duplicate/out-of-order keys or tags, leading
|
||
zeroes in a uint, invalid booleans/UTF-8/NFC, mismatched counts/lengths, unknown
|
||
schema tags, and trailing bytes are errors. Lists preserve declared order.
|
||
Schema defaults are always encoded, so no semantic field is inferred from
|
||
absence. Floats, signed integers, null, and indefinite lengths do not exist.
|
||
|
||
One byte string/string is at most `2^31-1` bytes, a container has at most
|
||
`2^24-1` members, and nesting depth is at most 64. A content/container-relative
|
||
logical path string uses `/`, is relative, has no NUL/backslash, empty/`.`/`..`
|
||
segment, and passes the schema's ASCII-identity or NFC-source-path rule. A field
|
||
that explicitly permits `.` as its complete root sentinel is the sole exception.
|
||
These are protocol limits, not host `size_t` limits. Human-readable JSON is a
|
||
lossless rendering, not the hashed representation. Phase-0 golden vectors
|
||
include empty/nested records, ordered lists, sorted maps, every rejection, and
|
||
their complete bytes/digests.
|
||
The normative empty-record vector is
|
||
`57574152000107000000000000000400000000`, SHA-256
|
||
`138c6acb7f01e91df73cb1d9c3356d18f19d7b8eb8b0a15426bef32e515d0de0`.
|
||
|
||
Each schema assigns every path field a path class. Artifact, source, generated
|
||
output, install-destination, bundle-relative, sysroot-relative, and
|
||
vendor-relative paths use the relative rule above. Sandbox-execution paths are
|
||
path-independent absolute paths only in the closed virtual namespaces `/work`,
|
||
`/in`, `/out`, `/tool`, and `/data`; schema-1 action working directory is exactly
|
||
`/work`, and exact sandbox path spellings in argv are encoded. Platform-validated
|
||
target-runtime paths are a separate type and may be absolute in H's namespace.
|
||
Workspace locations and observation physical paths are separately typed and
|
||
never enter an artifact action record or key as host locations. Absolute host
|
||
paths, filesystem device/inode numbers, mtimes, cache locations, and command
|
||
process IDs are invalid in action templates and final action records. A physical
|
||
input enters only through a logical name, content digest, type, and semantic
|
||
mode.
|
||
|
||
### 6.2 Required action-record fields
|
||
|
||
Every record contains, in this order:
|
||
|
||
1. WWAR schema and action kind/version;
|
||
2. language edition, compiler protocol, export protocol, object ABI, runtime ABI,
|
||
manifest schema, and lock schema;
|
||
3. B, H, and optional T descriptor digests plus the expanded target fields;
|
||
4. toolchain identity, selected descriptor-slice/closure digest,
|
||
compiler/backend digest, and every executable/shared/resource digest actually
|
||
used by the action; the distribution bundle root/version authenticates
|
||
acquisition but unused targets/tools do not invalidate this action;
|
||
5. profile fields: optimization, debug, assertions, overflow, panic, sanitizers,
|
||
LTO, relocation/code model, symbol stripping, and reproducibility policy;
|
||
6. logical package/product/action identity and sandbox-virtual working directory
|
||
(schema 1 exactly `/work`);
|
||
7. exact argument vector and a sorted literal environment map;
|
||
8. byte-sorted named inputs, each with edge kind, logical path, semantic artifact
|
||
kind, semantic mode, content digest, and—where applicable—origin package identity;
|
||
9. direct export-data inputs byte-sorted by package identity for `ww.package`
|
||
actions;
|
||
10. selected source-membership list and target-selection explanation;
|
||
11. typed native declarations: headers, objects, archives, shared libraries,
|
||
sysroot, SDK, libc, CRT, dynamic loader, linker scripts and their include
|
||
closures, assembler/linker/archive tools, and ABI-provider slots;
|
||
12. the exact ordered link plan, retaining archive groups, whole-archive markers,
|
||
as-needed state, and repeated libraries;
|
||
13. named output paths, types, modes, and canonicalization policies; and
|
||
14. sandbox policy/version, resource bounds, and reproducibility classification.
|
||
|
||
Fields irrelevant to an action are encoded as empty values, not inferred. Native
|
||
flags exist only as typed fields whose meaning is part of a tool adapter. A raw
|
||
flag can be used only in a custom toolchain declaration and then its exact bytes
|
||
are part of the record; the official profile has no hidden default flags.
|
||
|
||
### 6.3 Environment and tool discovery
|
||
|
||
No inherited environment variable is semantic. Built-in actions receive the
|
||
fixed environment specified by their toolchain adapter. A declarative action
|
||
receives only its `env` record. `PATH`, compiler-driver defaults, `pkg-config`,
|
||
shell initialization, host include/library directories, and current directory
|
||
are never consulted to discover an input.
|
||
|
||
User configuration may choose a cache location, concurrency, output directory,
|
||
credential helper, source mirror, or display preference; these do not enter an
|
||
action because they cannot alter output bytes. Choosing a toolchain, target,
|
||
profile, workspace overlay, native provider, environment value, raw option, or
|
||
impure system mapping can alter bytes and therefore always enters the record.
|
||
|
||
### 6.4 Package action inputs and invalidation
|
||
|
||
A `ww.package` action consumes:
|
||
|
||
- the exact selected production or test source files and generated fragments;
|
||
- their ordered membership metadata;
|
||
- only the `.wwe` artifacts of direct imported packages;
|
||
- the compiler/backend and toolchain resources;
|
||
- B/H/T, target descriptor, profile, language/compiler/export/object/runtime
|
||
protocols, and manifest/lock schemas;
|
||
- package-specific predeclared native-provider slots, distinct from selected
|
||
concrete provider declarations; and
|
||
- its literal built-in environment and sandbox policy.
|
||
|
||
It emits `.wwe`, one target object, and canonical link metadata. A private change
|
||
in a dependency changes that dependency's object and the final link key, but not
|
||
the importer's action key. A public change changes the dependency's `.wwe` and
|
||
therefore its direct importers. If a rebuilt importer emits byte-identical
|
||
`.wwe`, the invalidation stops there. Link-only input changes only invalidate
|
||
link/archive actions; materialization merely recopies a newly selected immutable
|
||
artifact when its requested result digest changes.
|
||
|
||
A package's own source bytes remain part of its action key even if a compiler
|
||
could prove a change dead. A target/profile/tool/runtime ABI change creates a
|
||
different key. There is no timestamp freshness shortcut and no “artifact exists”
|
||
predicate.
|
||
|
||
### 6.5 Link action inputs
|
||
|
||
The link action consumes every reachable package/native object digest, explicit
|
||
archive/shared-library digest, CRT object, runtime object, dynamic-loader choice,
|
||
linker script closure, sysroot descriptor, target/profile, exact linker tool and
|
||
resources, and the ordered plan. Objects are ordered by stable package identity;
|
||
an explicit static product's `members` list controls only that archive's member
|
||
order. Native archives retain declared
|
||
order; repeated archives remain repeated; group and whole-archive boundaries are
|
||
semantic. `-L`/`-l` token collections are not an internal representation.
|
||
|
||
The link result is cacheable. A warm identical build need not invoke the linker.
|
||
Changing output/materialization path alone does not change the link key. Changing
|
||
a private package implementation normally preserves importer objects but changes
|
||
the final link key through that package's object digest.
|
||
|
||
### 6.6 Complete cache-key formula
|
||
|
||
Let `R` be the complete WWAR record from sections 6.2–6.5. The action key is:
|
||
|
||
```text
|
||
K = SHA256("WW-ACTION-KEY\0" || uint64be(len(WWAR(R))) || WWAR(R))
|
||
```
|
||
|
||
Input entries contain the SHA-256 digest of their canonical artifact bytes/tree,
|
||
not merely the producer's action key. Thus semantically identical outputs stop
|
||
rebuild propagation even when their producing source/action key changed. Ordered
|
||
fields remain ordered; only fields specified as maps are sorted. The domain
|
||
separator and schemas prevent a digest from one protocol being reinterpreted in
|
||
another.
|
||
|
||
A successful result record contains `K`, result-schema version, output
|
||
name/type/mode/digest tuples only. Stdout, stderr, exit/diagnostic presentation,
|
||
timing, worker identity, resource use, and physical paths go in a separate
|
||
invocation-observation record. Thus one action key has exactly one semantic
|
||
successful result even if its logs differ. Observation records may be
|
||
content-addressed, but never participate in action mapping, cache hits, or
|
||
reproducibility comparison.
|
||
|
||
CAS identities are type-separated. Blob bytes use their own domain; every
|
||
structured object is additionally bound to its top-level record kind and schema:
|
||
|
||
```text
|
||
blob_id = SHA256("WW-BLOB\0" || u64be(length) || bytes)
|
||
|
||
record_id(kind, schema, record) =
|
||
SHA256("WW-RECORD\0" || u32be(kind) || u32be(schema) ||
|
||
u64be(length(WWAR(record))) || WWAR(record))
|
||
```
|
||
|
||
Schema 1 reserves these top-level `kind` numbers: 1 tree, 2 manifest, 3 lock,
|
||
4 workspace, 5 vendor index, 6 native map, 7 platform descriptor, 8 toolchain,
|
||
9 profile, 10 action template, 11 action record, 12 action result,
|
||
13 observation, 14 graph snapshot, 15 native sidecar, 16 native ABI contract,
|
||
17 link plan, 18 export, 19 package link metadata, 20 install manifest,
|
||
21 bootstrap plan, and 22 selected-tool closure. An unknown kind is never
|
||
decoded as another record. `TypedDigestV1` is a record with tag 1 domain
|
||
(`blob` or `record`), tag 2 algorithm (schema 1 only `sha256`), tag 3 record kind
|
||
(`0` for a blob), tag 4 record schema (`0` for a blob), and tag 5 the exact
|
||
32 digest bytes. A field whose type is “typed digest” always means this record;
|
||
a bare hexadecimal string is only a text rendering.
|
||
|
||
A tree record is schema 1 plus a list sorted by entry-name UTF-8 bytes. An entry
|
||
is `(name, kind=file|tree, executable:boolean, typed-child-digest)`. `name` is
|
||
one normalized path segment. Empty directories are explicit tree children;
|
||
symlinks, hardlink identity, devices, xattrs, uid/gid, mtimes, and non-executable
|
||
permission bits do not exist. Duplicate normalized or case-fold-colliding names
|
||
are errors. Validation recursively checks every typed child to a blob; verifying
|
||
only a root digest is insufficient.
|
||
|
||
### 6.7 Storage, sharing, and corruption
|
||
|
||
The default cache is per-user, global across that user's checkouts, local, and
|
||
private to the account:
|
||
|
||
```text
|
||
<cache>/v1/cas/sha256/aa/bb...
|
||
<cache>/v1/actions/sha256/aa/bb...
|
||
<cache>/v1/quarantine/
|
||
```
|
||
|
||
The first path stores blob objects and kind/schema-bound structured records; the
|
||
second maps an action key to `(action-record digest, result-object digest)`. A small ignored
|
||
project index `.ww/state-v1` references the previous successful graph snapshot,
|
||
whose logical nodes point to action-record/key/result digests. It roots that
|
||
history until replacement so explanation can compare records and follow causes;
|
||
it is disposable and never proves freshness. After explicit GC removes history,
|
||
`explain` reports `history-unavailable` rather than inventing “stale.” A
|
||
system-wide cache service requires authenticated isolated writers and the same
|
||
signed-mapping policy as a remote cache.
|
||
|
||
On every cache read, WW verifies the requested object's digest and canonical
|
||
type, decodes the mapped action record, recomputes `K` from it, and requires
|
||
`recomputed K = lookup K = ActionResultV1.tag2` plus
|
||
`ActionResultV1.tag3 = the mapping's typed action-record digest`. It then verifies
|
||
all output objects before use or materialization. A mismatch moves only that
|
||
explicit entry to quarantine, removes its action mapping, reports corruption,
|
||
and rebuilds.
|
||
`ww cache verify` walks the store; `ww cache gc` traces retained action results
|
||
and materializations. A tool upgrade changes its content/descriptor fields and
|
||
cannot reuse the old key.
|
||
|
||
If two executions of one `reproducibility=required` action key produce different
|
||
semantic result digests, WW publishes neither as an authoritative replacement,
|
||
records both observations/artifact sets in quarantine, and fails with a
|
||
nondeterminism diagnostic. Impure actions have no reusable action mapping.
|
||
|
||
Shared caches are opt-in explicit transports. `ww cache pull` imports only
|
||
content-addressed objects and action mappings in an Ed25519 signed envelope over
|
||
`"WW-CACHE-MAP\0"`, cache namespace, action key, action-record digest, result
|
||
digest/schema, and reproducibility/policy classification. The envelope carries
|
||
a signing-key ID; configured trust policy handles rotation/revocation. Hashes
|
||
prove bytes; the trusted cache signing key authorizes the asserted key-to-result
|
||
mapping. All hashes are reverified. An unsigned/untrusted mapping is treated as a miss
|
||
even if its referenced blobs exist. `ww cache push` refuses impure,
|
||
non-reproducible, secret-bearing, or policy-incompatible actions. Literal secret
|
||
environment values are forbidden; a required secret is a classified file input,
|
||
redacted from JSON/explain, and makes the action non-shareable. Ordinary build
|
||
does not contact a shared cache.
|
||
|
||
### 6.8 Explainability
|
||
|
||
For every node, WW retains its last local record and current record. `ww explain
|
||
NODE` reports one of `hit`, `not-built`, `missing-result`, `corrupt-result`,
|
||
`policy-rejected`, or `key-changed`. For `key-changed` it prints the first and,
|
||
with `--all`, every differing typed field, for example:
|
||
|
||
```text
|
||
codec/hex: key changed
|
||
input direct-export example.org/base: 71… -> a4…
|
||
caused by base: exported type Header layout changed
|
||
link hello: key changed
|
||
package-object example.org/codec/hex: 19… -> 27…
|
||
```
|
||
|
||
`ww explain --path NODE` follows the shortest changed-input path to a source,
|
||
tool, target, native provider, or policy root. `--format=json` exposes both WWAR
|
||
renderings and field paths. Export-data differences use the normative `ExportV1`
|
||
semantic field/type diff; if old content was explicitly GC'd the command reports
|
||
`history-unavailable`. Cache misses are never explained merely as “stale.”
|
||
|
||
### 6.9 Version-1 semantic record schemas
|
||
|
||
The following tables freeze schema-1 semantic fields and WWAR numeric tags.
|
||
`1` means exactly one, `0/1` optional, `*` a list, and `map` unique string keys.
|
||
Every absent optional value encodes the stated empty/default. Identity-keyed
|
||
lists are byte-sorted by identity; lists marked `ordered` preserve source/link
|
||
order. Nested records use the field tags in their named table. Enums reject
|
||
unknown values rather than passing them to a tool.
|
||
|
||
#### Project and distribution records
|
||
|
||
| Record/tag | Field | Type/cardinality | Rule/default |
|
||
|---|---|---|---|
|
||
| `ManifestV1/1` | schema | uint/1 | `1` |
|
||
| `/2` | module | string/1 | canonical module identity |
|
||
| `/3` | language | string/1 | language edition |
|
||
| `/4` | toolchain | `ToolchainRef`/1 | compatible ID and minimum |
|
||
| `/5` | requires | `Require`/* | sorted by module |
|
||
| `/6` | products | `Product`/* | sorted by name |
|
||
| `/7` | packages | `PackageConfig`/* | sorted by relative path |
|
||
| `/8` | actions | `GenerateDecl`/* | sorted by name |
|
||
| `/9` | natives | `NativeProvider`/* | sorted by name |
|
||
| `ToolchainRef/1` | id | string/1 | toolchain identity |
|
||
| `/2` | minimum | string/1 | minimum compatible SemVer |
|
||
| `Require/1` | module | string/1 | module identity |
|
||
| `/2` | minimum | string/1 | SemVer minimum |
|
||
| `/3` | source-index | string/0/1 | empty means HTTPS convention |
|
||
| `Product/1` | name | string/1 | unique identifier |
|
||
| `/2` | kind | enum/1 | `exe`, `static`, `shared`, `object`, `generated` |
|
||
| `/3` | root | string/0/1 | package-relative path; required except generated |
|
||
| `/4` | linkage | enum/1 | `dynamic`, `pie`, `static`, `static-pie`, `shared`, `none`; kind-valid |
|
||
| `/5` | runtime | string/1 | `hosted` default, `minimal`, `none`, or slot |
|
||
| `/6` | entry | string/0/1 | empty selects typed toolchain default |
|
||
| `/7` | native | string/* | required slots, sorted |
|
||
| `/8` | linker-script | `ArtifactRef`/0/1 | empty |
|
||
| `/9` | providers | `ProviderSelection`/* | sorted by slot |
|
||
| `/10` | members | string/* ordered | static/archive members; root only by default |
|
||
| `/11` | action | string/0/1 | required only for generated product |
|
||
| `/12` | install-name | string/0/1 | platform-validated, empty |
|
||
| `ProviderSelection/1` | slot | string/1 | ABI slot |
|
||
| `/2` | use | string/1 | `module#native-clause` |
|
||
| `PackageConfig/1` | path | string/1 | normalized relative path; `.` root |
|
||
| `/2` | generated | string/* | sorted `action:NAME:OUTPUT` refs |
|
||
| `/3` | native | string/* | sorted provider slots |
|
||
| `/4` | test-data | `InputDecl`/* | sorted names, read-only under `/data` |
|
||
| `/5` | test-env | map | literal non-secret test environment |
|
||
|
||
`ArtifactRefV1` is permitted in declarative configuration records (including
|
||
toolchain/native records) and action templates, but never in a final action
|
||
record. Its tags are: 1 `ArtifactNamespace` (`source`, `generated`, `package`,
|
||
`toolchain`, `sysroot`, `provider-output`, `cas`, or `graph`); 2 owner identity
|
||
(empty only for a root source);
|
||
3 normalized logical name/path; 4 semantic artifact kind (`file`, `tree`,
|
||
`object`, `archive`, `shared`, `import-library`, `script`, `crt`, `loader`,
|
||
`native-sidecar`, `native-abi`, `export`, `package-link`, `action-record`, or
|
||
`action-result`);
|
||
5 optional expected `TypedDigestV1`; and 6 mode (`data` or `executable`). A local
|
||
source may omit tag 5 because analysis hashes it. A `cas`, external prebuilt,
|
||
toolchain, or sysroot reference must include it. A generated/package/provider
|
||
output gets its digest only from the declared predecessor output. Absolute host
|
||
paths are invalid.
|
||
|
||
`InputSlotRefV1`, `TemplateInputV1`, `ResolvedInputV1`, built-in/template/final
|
||
action-output records, and `ResultOutputV1` use that same closed semantic
|
||
artifact-kind enum; `GenerateDecl.OutputDecl` remains restricted to `file` or
|
||
`tree`. In schema 1, `file`, `object`, `archive`, `shared`, `import-library`,
|
||
`script`, `crt`, and `loader` require a blob digest. `tree` requires record kind
|
||
1, `native-sidecar` kind 15, `native-abi` kind 16, `export` kind 18, and
|
||
`package-link` kind 19, each at record schema 1. Schema-1 artifact-kind values 14
|
||
`action-record` and 15 `action-result` require record kinds 11 and 12,
|
||
respectively, at record schema 1. They are input-only and valid only for
|
||
`bootstrap.compare`; they are invalid in built-in, template, or final action
|
||
outputs and in `ResultOutputV1`. Any other digest domain, record kind, or schema
|
||
is invalid kind substitution.
|
||
|
||
The `graph` namespace has one exact form. Its consumer is `bootstrap.compare`,
|
||
the `TemplateInputV1` edge kind is `bootstrap-record`, and
|
||
`ArtifactRefV1.tag2` is the producer logical node. Tag 3 is the Identifier
|
||
selector `action_record` for artifact kind `action-record` or `action_result`
|
||
for artifact kind `action-result`; tag 5 is absent. `ArtifactRefV1.tag6` and
|
||
`TemplateInputV1.tag5` are `data`, `TemplateInputV1.tag4` repeats the
|
||
corresponding artifact kind, and `TemplateInputV1.tag6` is empty. No other
|
||
consumer, edge kind, selector, kind, expected digest, or mode is valid for this
|
||
namespace. These inputs and edges are bijective: each `graph` template input has
|
||
exactly one `bootstrap-record` `GraphEdgeV1`, and each such edge has exactly one
|
||
`graph` template input. The edge's consumer node is the enclosing template node,
|
||
its consumer input slot equals `TemplateInputV1.tag1`, its producer node equals
|
||
`ArtifactRefV1.tag2`, and its selector equals `ArtifactRefV1.tag3`.
|
||
|
||
`InputSlotRefV1` has tag 1 slot name and tag 2 expected semantic artifact type.
|
||
A final action record contains no `ArtifactRefV1`: every artifact-bearing field is recursively lowered
|
||
to an `InputSlotRefV1`. `TemplateInputV1` tags are 1 unique slot name, 2 edge
|
||
kind, 3 `ArtifactRefV1`, 4 expected semantic type, 5 semantic mode, and 6
|
||
optional origin package identity. `ResolvedInputV1` tags are 1 the same slot
|
||
name, 2 edge kind, 3 normalized logical name/path, 4 semantic artifact type, 5
|
||
semantic mode, 6 the resolved `TypedDigestV1`, and 7 optional originating
|
||
package identity. It contains no producer node, producer action key, action-
|
||
template digest, physical output path, or unresolved filesystem lookup. Thus all
|
||
content that a native plan, link plan, source-selection record, or tool closure can read is
|
||
also present exactly once in action-record tag 10 under a named slot.
|
||
|
||
`GenerateDecl` fields are fixed as follows: tag 1 name; 2 tool artifact/product
|
||
reference; 3 platform enum (schema 1 only `build`); 4 `TargetConstraint` or empty;
|
||
5 input map of `InputDecl`; 6 output map of `OutputDecl`; 7 ordered string argv;
|
||
8 literal string environment map; 9 `ResourcePolicy`; 10 reproducibility enum
|
||
`required` or `impure`. `InputDecl` is tag 1 kind (`file`, `tree`, `artifact`,
|
||
`tool`), 2 logical reference, 3 optional expected typed digest, 4 semantic mode.
|
||
`OutputDecl` is tag 1 kind (`file`, `tree`), 2 logical output path, 3 executable
|
||
boolean. `ResourcePolicy` is tags 1 max CPU count, 2 memory bytes, 3 output bytes,
|
||
4 process count; zero selects the toolchain's recorded bound, never “unlimited.”
|
||
|
||
| Record/tag | Field | Type/cardinality | Rule/default |
|
||
|---|---|---|---|
|
||
| `LockV1/1` | schema | uint/1 | `1` |
|
||
| `/2` | root-manifest | typed digest/1 | semantic manifest record |
|
||
| `/3` | toolchain | `ToolchainLock`/1 | exact closure |
|
||
| `/4` | modules | `ModuleLock`/* | sorted identity |
|
||
| `/5` | native-map | `LockedObject`/0/1 | empty |
|
||
| `ToolchainLock/1…5` | id, version, descriptor, bundle, origin | strings/digests | all required |
|
||
| `ModuleLock/1` | module | string/1 | identity |
|
||
| `/2` | version | string/1 | selected SemVer |
|
||
| `/3` | origin | string/1 | exact final archive URL |
|
||
| `/4` | archive | blob digest/1 | required |
|
||
| `/5` | tree | tree digest/1 | required |
|
||
| `/6` | manifest | record digest/1 | required |
|
||
| `/7` | signature | bytes/0/1 | empty |
|
||
| `/8` | provenance | string/0/1 | empty |
|
||
| `LockedObject/1…3` | origin, digest, signature | string/digest/bytes | origin+digest required |
|
||
|
||
`WorkV1` is tag 1 schema, tag 2 sorted `Use` records, tag 3 sorted local provider
|
||
overrides. `Use` tags are module, path, expected source-tree digest. A provider
|
||
override has slot, provider ID, product-platform constraint, contract digest,
|
||
artifact-tree digest, and provenance in tags 1–6. `VendorV1` is tag 1 schema,
|
||
tag 2 lock-record digest, tag 3 sorted entries `(module, version, source-tree
|
||
digest, vendor-relative path)` in tags 1–4. `NativeMapV1` is tag 1 schema, tag 2
|
||
exact product-platform descriptor digest, tag 3 sorted provider overrides, and
|
||
tag 4 signer/provenance record.
|
||
|
||
#### Profiles, templates, actions, results, and trees
|
||
|
||
A profile is toolchain data, not an open project map:
|
||
|
||
| Tag | `ProfileV1` field | Values |
|
||
|---|---|---|
|
||
| 1 | name | identity |
|
||
| 2 | optimization | `0`, `1`, `2`, `3`, `size` |
|
||
| 3 | debug | `none`, `line`, `full` |
|
||
| 4 | assertions | boolean |
|
||
| 5 | overflow | `trap`, `wrap` |
|
||
| 6 | panic | `abort`, `runtime` |
|
||
| 7 | sanitizers | sorted toolchain capability IDs |
|
||
| 8 | LTO | `none`, `thin`, `full` |
|
||
| 9 | relocation | effective `static`, `pic`, `pie` |
|
||
| 10 | code-model | exact target capability ID |
|
||
| 11 | TLS default | exact target capability ID |
|
||
| 12 | strip | `none`, `debug`, `all` |
|
||
| 13 | reproducibility | `required`, `impure` |
|
||
|
||
| Tag | `ActionTemplateV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | schema | `1` |
|
||
| 2 | kind/version | exact built-in kind or generate version |
|
||
| 3 | protocol record | language/compiler/export/object/runtime/manifest/lock; compiler protocol is distinct from compiler/backend byte identity |
|
||
| 4 | platform roles | B, H, optional T descriptor refs |
|
||
| 5 | selected tool closure | identity plus semantic closure digest/resources |
|
||
| 6 | profile | complete `ProfileV1` |
|
||
| 7 | logical identity/cwd | normalized identity; cwd exactly `/work`, path-independent |
|
||
| 8 | argv | ordered strings |
|
||
| 9 | environment | sorted literal map, no secrets |
|
||
| 10 | input slots | sorted `(name, edge kind, producer/output or source ref, type, mode)` |
|
||
| 11 | source selection | ordered selected membership plus reasons |
|
||
| 12 | native/link template | closed predeclared provider slots, concrete providers, and ordered link records |
|
||
| 13 | outputs | sorted name/path/type/mode/canonicalization |
|
||
| 14 | sandbox/reproducibility | exact policy/version/bounds/class |
|
||
|
||
`ActionRecordV1` has the same top-level tags, but tag 10 contains sorted
|
||
`ResolvedInputV1` records and every `ArtifactRefV1` elsewhere in the template is
|
||
replaced by the corresponding `InputSlotRefV1`. The tag-12 native/link value is
|
||
therefore a resolved plan; it cannot contain an unresolved artifact reference,
|
||
producer, physical path, or readable artifact locator/digest outside tag 10.
|
||
ABI/layout/contract digests embedded in a referenced sidecar or contract are
|
||
semantic verification values, not authority to read another object. No producer
|
||
action key substitutes for a content digest.
|
||
|
||
The tag-12 native/link record separately encodes the sorted predeclared provider
|
||
slots, concrete selected-provider declarations, and the optional ordered link
|
||
policy or resolved plan. A `ww.package` action contains exactly its
|
||
`PackageConfigV1` native-slot list and empty concrete-provider and link-policy/
|
||
plan values; finalization copies that slot list unchanged. Concrete provider
|
||
selection does not enter a package action merely because the provider satisfies
|
||
one of those slots.
|
||
|
||
Finalization interns every source, direct export, package/native object, archive,
|
||
shared library, header/sysroot tree, generated output, tool/resource, CRT,
|
||
loader, script, and init dispatcher into exactly one named template input.
|
||
For `bootstrap.compare`, it also interns every selected raw action record and
|
||
action result as a separate named input. Finalization erases the `graph`
|
||
namespace, producer node, and unresolved selector form. The corresponding
|
||
tag-10 `ResolvedInputV1` retains edge kind `bootstrap-record`, logical selector
|
||
`action_record` or `action_result`, matching artifact kind, `data` mode, and the
|
||
resolved typed digest; that digest is the sole authority to read the raw record.
|
||
Predecessor output and selected-record digests resolve those slots lazily. The
|
||
producer logical node, producer output path, producer key, and template digest
|
||
are graph/provenance facts only and do not enter the consumer's `ActionRecordV1`
|
||
or `K`. Two producers that deliver the same typed bytes to the same semantic
|
||
slot therefore produce the same downstream record and key.
|
||
|
||
`ActionResultV1` tags are: 1 schema, 2 the 32-byte action key, 3 typed
|
||
action-record digest, and 4 sorted `ResultOutputV1` records. `ResultOutputV1`
|
||
tags are 1 unique output name, 2 semantic artifact type, 3 mode (`data` or
|
||
`executable`), and 4 `TypedDigestV1`. `ObservationV1` separately uses tags 1 schema,
|
||
2 logical invocation, 3 optional action key, 4 exit status/signal, 5 stdout blob,
|
||
6 stderr blob, 7 timing/resources, and 8 physical runner metadata; it is never
|
||
an action result.
|
||
|
||
`TreeV1` tags are 1 schema and 2 ordered entries. `TreeEntryV1` tags are 1 name,
|
||
2 kind (`file`, `tree`), 3 executable boolean (false for tree), and 4 typed child
|
||
digest. `GraphSnapshotV1` tags are 1 schema, 2 logical root, 3 sorted
|
||
`GraphNodeV1` records, and 4 sorted `GraphEdgeV1` records. `GraphNodeV1` tags are
|
||
1 logical node ID, 2 typed action-template digest, 3 optional 32-byte action key,
|
||
4 optional typed action-record digest, and 5 optional typed action-result digest.
|
||
`GraphEdgeV1` tags are 1 consumer node ID, 2 consumer input slot, 3 producer node
|
||
ID, 4 producer output name, and 5 edge kind; edges sort by that five-field tuple.
|
||
Tag 4 is an ordinary producer output name except that a `bootstrap-record` edge
|
||
uses selector `action_record` or `action_result`. That branch resolves the
|
||
actual producer `GraphNodeV1.tag4` or tag 5, respectively; it never selects an
|
||
`ActionOutputV1` or `ResultOutputV1`. If the selected producer tag is absent,
|
||
the input remains unresolved and blocks finalization. The `source-input` kind is
|
||
invalid in `GraphEdgeV1`. Non-action source inputs live only in the consumer
|
||
template rather than invented graph nodes. The project index contains only its
|
||
typed graph-snapshot digest.
|
||
|
||
#### Target, toolchain, native, interface, and handoff records
|
||
|
||
| Tag | `PlatformDescriptorV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | schema | `1` |
|
||
| 2 | name | canonical lookup name |
|
||
| 3 | arch/vendor/os/environment/object format | five exact enums |
|
||
| 4 | endian/address spaces/pointers | complete integer-width map |
|
||
| 5 | integer/data-layout | widths, alignments, aggregate algorithm |
|
||
| 6 | C ABI/data model | exact IDs and calling-convention table |
|
||
| 7 | float/variadic/name decoration | exact ABI records |
|
||
| 8 | CPU baseline/features/atomics | baseline plus required/forbidden sets |
|
||
| 9 | minimum OS/SDK | typed version record |
|
||
| 10 | TLS/unwind capabilities | sets plus defaults |
|
||
| 11 | relocation/code capabilities | supported sets plus defaults |
|
||
| 12 | executable/shared/page rules | typed object-format rules |
|
||
| 13 | hosted policy | hosted/freestanding plus capability set |
|
||
| 14 | object/runtime ABI protocols | exact IDs |
|
||
|
||
`TargetConstraintV1` tags 1–17 are, respectively: optional exact descriptor
|
||
digest; arch; vendor; OS; environment; object format; hosted; C ABI; data model;
|
||
float ABI; CPU baseline; required feature set; forbidden feature set; minimum
|
||
SDK; relocation; code model; PIC requirement. Empty scalar/set means no
|
||
constraint. Matching is exactly section 8.4; no expression field exists.
|
||
|
||
`ToolchainV1` tags are: 1 schema; 2 ID; 3 version; 4 the same complete protocol
|
||
record used by action tag 3; 5 sorted `Tool` records; 6 sorted platform
|
||
descriptors; 7 sorted profiles; 8 sorted link policies; 9 runtime/provider
|
||
records; 10 bundle tree digest/signature provenance.
|
||
A `Tool` is `(name, bundle-relative path, executable blob digest, ordered dynamic
|
||
tool dependencies, resource-tree digests, adapter record)` tags 1–6. A link
|
||
policy is `LinkPolicyV1`: tag 1 product-platform descriptor; 2 product kind; 3
|
||
linkage; 4 profile constraint; 5 runtime selector (`hosted`, `minimal`, `none`,
|
||
or an exact provider slot); 6 one ordered link-policy token template; and 7
|
||
output ABI/install policy. Policies sort by the five-field selection key and a
|
||
zero/multiple match is an error. CRTs and scripts are `ArtifactRefV1` tokens,
|
||
not basenames. A dynamic loader is one restricted `dynamic-loader` token holding
|
||
both its artifact and runtime interpreter path; PE/COFF platform-image policy is
|
||
an ordered provider token rather than a fabricated loader artifact.
|
||
`SelectedToolClosureV1` deterministically
|
||
projects only the relevant tools/resources/platform/profile/policy into tags
|
||
1–7; that projection—not unrelated bundle members—is action-key input.
|
||
|
||
#### Native artifact and ABI subrecords
|
||
|
||
Compact manifest paths are lowered to `ArtifactRefV1` before WWAR encoding and
|
||
then to action input-slot references before execution. The native records are:
|
||
|
||
| Tag | `IncludeTreeRefV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | tree | `ArtifactRefV1` of kind `tree` |
|
||
| 2 | class | `quote`, `user`, `system`, or `framework` |
|
||
| 3 | subdirectory | normalized tree-relative path; `.` default |
|
||
|
||
The provider's include list is ordered because header search order is semantic.
|
||
The same tree may occur more than once with another class or subdirectory.
|
||
|
||
| Tag | `NativeSourceV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | name | unique provider-local identity |
|
||
| 2 | source | `ArtifactRefV1` of kind `file` |
|
||
| 3 | language | exact toolchain capability ID, such as `c11` or `gnu-assembly` |
|
||
| 4 | preprocessing | `none` or `c-preprocessor` |
|
||
| 5 | include-indices | ordered indexes into the provider include list; empty means all |
|
||
| 6 | defines | sorted literal macro map; duplicates with provider defines error |
|
||
|
||
Target, profile, relocation/PIC/code/TLS policy, tool, and dialect adapter come
|
||
from the enclosing `native.compile` record. Raw source flags do not exist.
|
||
|
||
| Tag | `PrebuiltObjectV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | name | unique provider-local identity |
|
||
| 2 | object | `ArtifactRefV1` of kind `object` |
|
||
| 3 | sidecar | `ArtifactRefV1` of kind `native-sidecar` |
|
||
| 4 | contract | `ArtifactRefV1` of kind `native-abi` |
|
||
|
||
| Tag | `ArchiveV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | name | unique provider-local identity |
|
||
| 2 | archive | `ArtifactRefV1` of kind `archive` |
|
||
| 3 | sidecar | archive-level `ArtifactRefV1` of kind `native-sidecar` |
|
||
| 4 | members | ordered `ArchiveMemberV1` list in physical order |
|
||
| 5 | contract | `ArtifactRefV1` of kind `native-abi` |
|
||
|
||
`ArchiveMemberV1` tags are 1 member name, 2 member blob `TypedDigestV1`, and 3
|
||
object-sidecar `ArtifactRefV1`. Duplicate names are legal only at distinct
|
||
positions; member order is never sorted. The referenced archive sidecar records
|
||
the ordered member-sidecar record digests as well as the physical member facts.
|
||
|
||
| Tag | `SharedImportLibraryV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | name | unique provider-local identity |
|
||
| 2 | kind | `elf-shared`, `macho-dylib`, or `pe-import` |
|
||
| 3 | link-artifact | shared object/dylib/import-library `ArtifactRefV1` |
|
||
| 4 | link-sidecar | `ArtifactRefV1` of kind `native-sidecar` for tag 3 |
|
||
| 5 | runtime-identity | exact SONAME, install-name, or DLL name |
|
||
| 6 | runtime-artifact | exact deployable shared object/dylib/DLL `ArtifactRefV1` |
|
||
| 7 | runtime-sidecar | `ArtifactRefV1` of kind `native-sidecar` for tag 6 |
|
||
| 8 | contract | `ArtifactRefV1` of kind `native-abi` |
|
||
| 9 | runtime-requires | sorted `NativeRuntimeRequirementV1` list |
|
||
|
||
ELF and Mach-O tags 3 and 6 may resolve to the same bytes. For PE, tag 3 is
|
||
the import library and tag 6 its matching DLL. A platform image still supplies
|
||
tag 6 as a content-identified artifact within that image.
|
||
|
||
`NativeProviderV1` tags are therefore: 1 name; 2 provided slot; 3
|
||
`TargetConstraintV1`; 4 ordered `IncludeTreeRefV1`; 5 ordered `NativeSourceV1`;
|
||
6 sorted provider define map; 7 ordered `PrebuiltObjectV1`; 8 ordered
|
||
`ArchiveV1`; 9 ordered `SharedImportLibraryV1`; 10 sorted required slots; 11
|
||
ordered link-token templates; and 12 an `ArtifactRefV1` of kind `native-abi`.
|
||
Every ABI contract and sidecar is an independently encoded, content-addressed
|
||
record. In a resolved action, those records and every artifact field above are
|
||
`InputSlotRefV1` values; source/action/provider output digests live only in
|
||
action tag 10. A sidecar's internal artifact digest must equal the corresponding
|
||
object/archive/shared input-slot digest, and its contract digest must equal the
|
||
kind-16 `TypedDigestV1` of the referenced native-ABI input record.
|
||
Section/layout/provenance digests inside a sidecar are verification facts, not
|
||
locators from which the action may read undeclared content.
|
||
|
||
| Tag | `NativeABIContractV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | schema | `1` |
|
||
| 2 | slot | exact ABI-provider slot |
|
||
| 3 | platform | `NativeABIPlatformV1` |
|
||
| 4 | symbols | sorted `NativeSymbolContractV1` list |
|
||
| 5 | types | sorted `NativeTypeContractV1` list |
|
||
| 6 | runtime-requires | sorted `NativeRuntimeRequirementV1` list |
|
||
| 7 | features | `NativeFeatureContractV1` |
|
||
| 8 | minimum-platform | optional `PlatformVersionV1` |
|
||
| 9 | code | `NativeCodeContractV1` |
|
||
|
||
Its typed record identity is `record_id(16, 1, contract)` as defined in section
|
||
6.6. Every schema-1 digest identifying a complete `NativeABIContractV1` is the
|
||
corresponding record-domain, kind-16, schema-1 `TypedDigestV1`. WW computes it; a
|
||
supplied digest is never accepted in place of the record. Subordinate layout,
|
||
calling-convention, type-contract, and header-contract digests remain their
|
||
separately specified semantic values.
|
||
|
||
| Tag | `NativeABIPlatformV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | object-format | exact format capability ID |
|
||
| 2 | object-class | exact class/word-size ID |
|
||
| 3 | endian | `little` or `big` |
|
||
| 4 | machine-ABI | exact architecture object ABI ID |
|
||
| 5 | C-ABI | exact C ABI ID |
|
||
| 6 | data-model | exact data-model ID |
|
||
| 7 | data-layout | canonical layout digest |
|
||
| 8 | calling-conventions | canonical convention-table digest |
|
||
| 9 | float-ABI | exact ID |
|
||
| 10 | variadic-ABI | exact ID |
|
||
| 11 | symbol-ABI | exact decoration/versioning ABI ID |
|
||
| 12 | object-ABI | exact object protocol ID |
|
||
| 13 | runtime-ABI | required WW runtime ABI ID or empty |
|
||
|
||
`NativeSymbolContractV1` tags are: 1 exact external name; 2 exact version or
|
||
empty; 3 kind (`function`, `data`, `tls`, `ifunc`); 4 role (`define`, `require`);
|
||
5 binding (`strong`, `weak`); 6 visibility (`default`, `protected`, `hidden`);
|
||
7 calling-convention ID or empty; 8 canonical function/object type-contract
|
||
digest; and 9 optional byte size. Symbols sort by `(name,version,kind,role)`;
|
||
duplicate keys error.
|
||
|
||
`NativeTypeContractV1` tags are: 1 stable binding/header-qualified identity; 2
|
||
kind (`opaque`, `scalar`, `enum`, `struct`, `union`, `function`); 3 exposure
|
||
(`opaque`, `layout`); 4 canonical target-specific layout/signature digest; 5
|
||
optional size; 6 optional alignment; and 7 optional canonical header/macro
|
||
contract digest. Types sort by identity and duplicates error.
|
||
|
||
`NativeRuntimeRequirementV1` tags are 1 provider slot, 2 required ABI-contract
|
||
digest, 3 phase (`link`, `load`, `both`), and 4 optional SONAME/install-name/DLL
|
||
identity. They sort by `(slot,phase,runtime-identity)`; conflicting requirements
|
||
for one slot error. `NativeFeatureContractV1` tags are 1 CPU baseline or empty,
|
||
2 sorted required feature IDs, 3 sorted forbidden feature IDs, and 4 sorted
|
||
atomic-capability IDs; required and forbidden sets must be disjoint.
|
||
|
||
`PlatformVersionV1` tags are 1 version-family ID and unsigned 2 major, 3 minor,
|
||
4 patch, 5 revision. Versions compare lexicographically over tags 2–5 only after
|
||
tag 1 equality. `NativeCodeContractV1` tags are 1 PIC (`any`, `required`,
|
||
`forbidden`); 2 sorted TLS-model IDs; 3 unwind ABI ID or `none`; 4 sorted
|
||
personality/runtime symbols; and 5 sorted required/forbidden relocation records,
|
||
each record being tag 1 capability ID and tag 2 requirement (`required` or
|
||
`forbidden`).
|
||
|
||
`NativeSidecarV1` is evidence, not a second contract. Its tags are: 1 schema; 2
|
||
artifact `TypedDigestV1`; 3 evidenced `NativeABIPlatformV1`; 4 sorted evidenced
|
||
`SectionFactV1`; 5 sorted evidenced `SymbolFactV1`; 6 sorted evidenced
|
||
`RelocationFactV1`; 7 evidenced sorted architecture attribute/notes map; 8
|
||
evidenced `NativeMachineFactsV1`; 9 native ABI-contract digest; 10 sorted
|
||
`NativeRuntimeRequirementV1`; and 11 `ProvenanceV1`. An evidenced value is
|
||
`EvidenceV1`: tag 1 enum (`inspected` or `declared`) and tag 2 the value whose
|
||
type is fixed by the containing field. `NativeMachineFactsV1` tags are 1
|
||
`NativeFeatureContractV1` and 2 `NativeCodeContractV1`.
|
||
|
||
`SectionFactV1` tags are 1 name, 2 format type, 3 flag set, 4 size, 5 alignment,
|
||
6 optional content digest. `SymbolFactV1` tags are the nine
|
||
`NativeSymbolContractV1` fields plus tag 10 section and tag 11 value/offset.
|
||
`RelocationFactV1` tags are 1 section, 2 offset, 3 exact relocation ID, 4 symbol,
|
||
5 signed addend encoded as `(negative:boolean,magnitude:uint)`, and 6 target
|
||
section. `ProvenanceV1` tags are 1 producer/tool typed digest, 2 source/build
|
||
record typed digest, 3 attestation bytes, and 4 signer/policy ID. A frozen opaque
|
||
input requires accepted attestation for every `declared` value.
|
||
|
||
`LinkTokenTemplateV1` and final `LinkTokenV1` share tags: 1 kind; 2 artifact; 3
|
||
provider slot; 4 string value; 5 enabled boolean; 6 validated path; 7 runtime
|
||
path; and 8 ordered nested tokens. Tag 2 is `ArtifactRefV1` in a template and
|
||
`InputSlotRefV1` in a final action. Valid nonempty combinations are exactly:
|
||
object/archive/shared/linker-script/version-script/export-map/def-file use tag 2;
|
||
the template-only provider kind uses tag 3; group/whole use tag 8; as-needed uses tags 5 and 8;
|
||
runtime-search uses tags 4 and 6; install-name, entry, and retain use tag 4; and
|
||
dynamic-loader uses tags 2 and 7. Every other field encodes its empty/default
|
||
value. A final `LinkTokenV1` forbids `provider`; provider selection and its
|
||
ordered fragment are recursively expanded at that exact position before the
|
||
plan is final. Section 8.6 constrains valid nesting. A toolchain `LinkPolicyV1`
|
||
template additionally permits `splice` (tag 4 is exactly `product-objects`,
|
||
`init-dispatch`, `native-providers`, or `product-controls`) and
|
||
`script-slot` (tag 2 is an optional policy-default script). Those two kinds are
|
||
forbidden in project/native link declarations and in a final plan.
|
||
|
||
The selected policy template is flattened in list order. Each splice kind occurs
|
||
exactly once and expands to its already computed ordered product list;
|
||
the single required `script-slot` becomes one `linker-script` at the same list
|
||
position using the product's explicit script when present, otherwise the policy
|
||
default, and disappears only when both are empty. A product script therefore
|
||
replaces, never combines with, the default. Ordinary policy tokens—including CRT
|
||
objects, compiler runtime, dynamic loader, and system-provider slots—stay exactly
|
||
where declared. System providers are ordered template tokens, not a sorted set.
|
||
Each expands to its selected provider's concrete token fragment; dependencies
|
||
expand recursively, and a provider cycle is an analysis error. Every expanded
|
||
artifact, sidecar, and ABI contract is a tag-10 input. A provider such as a PE
|
||
platform image that adds no linker token instead contributes its resolved
|
||
contract slot to the plan's non-link policy field. After expansion no provider
|
||
or splice partition remains.
|
||
|
||
`LinkPlanV1` tags are 1 schema; 2 the exact product-platform/kind/linkage/
|
||
profile/runtime-policy selection record; 3 the single fully expanded ordered
|
||
`LinkTokenV1` stream; 4 selected linker/tool/resource slot refs; 5 selected
|
||
non-link platform/runtime/provider ABI-contract and sidecar slot refs; and 6
|
||
output ABI/install policy. Every file-bearing token and verification record
|
||
references action tag 10. Only
|
||
this resolved flattened plan enters the link action key.
|
||
|
||
`ExportV1` (the WWAR body after `.wwe` magic) tags are: 1 schema; 2 reader
|
||
capabilities; 3 language/type protocol; 4 product-platform/C/object/runtime ABI;
|
||
5 package identity/name; 6 sorted exported-surface origin/type contract table; 7
|
||
canonical type graph; 8 sorted exported declarations/constants/foreign symbols;
|
||
9 public initialization/ABI facts; 10 public-type digest; 11 public-ABI digest. Section 7.2
|
||
defines excluded non-semantic fields. `PackageLinkV1` (`.wwlm`) tags are 1 schema,
|
||
2 package identity, 3 platform/object/runtime ABI, 4 package-object digest, 5
|
||
defined/required foreign symbols, 6 predeclared provider slots/contracts, 7 init
|
||
symbol/dependency facts, 8 install/link requirements. Tag 6 contains exactly one
|
||
compiler-derived `ProviderContractV1` for each package action tag-12 predeclared
|
||
slot, and its sorted slot projection must equal that predeclared list. It cannot
|
||
add a provider slot or graph edge.
|
||
|
||
`InstallManifestV1` tags are 1 schema, 2 product identity/key, 3 sorted entries
|
||
`(artifact digest, mode, relative destination)`, 4 runtime-resolution policy, 5
|
||
sorted shared/runtime closure, 6 target/toolchain/ABI provenance. Absolute prefix
|
||
is deliberately absent. `BootstrapPlanV1` tags are 1 schema, 2 bootstrap-host
|
||
contract, 3 ordered source refs/digests, 4 portable-C compiler/output settings,
|
||
5 recorded host-C command/tool closure, 6 stage-1 outputs, 7 production toolchain
|
||
closure, 8 identical logical stage-2/3/4 action roots, 9 semantic fixed-point
|
||
output roles, and 10 raw-record fixed-point roles. No clause or executable step
|
||
exists beyond that closed plan.
|
||
|
||
These sections fix the architectural fields, but they do not make Phase 0 an
|
||
executable specification of every reference or key transformation. Phase 0
|
||
transcribes only their wire-visible record fields, tags, enum values, union
|
||
discriminants, encoded defaults, field order, record kinds, wrapper framing, and
|
||
digest preimage formulas into the checked-in compact schemas. Cross-field
|
||
validity, construction, resolution, projection, lowering, and failure behavior
|
||
belong to the executable phase that implements them. The Phase 0 generator and
|
||
golden vectors determine bytes, not future build-engine semantics. No phase may
|
||
silently add a wire field, renumber an assignment, change an encoded default, or
|
||
alter a frozen digest formula.
|
||
|
||
## 7. Interface and artifact protocol
|
||
|
||
### 7.1 Package outputs
|
||
|
||
Every `ww.package` action, including a root package, emits the same three named
|
||
artifacts:
|
||
|
||
```text
|
||
export.wwe deterministic binary export data
|
||
package.o one target object
|
||
link.wwlm deterministic package link metadata
|
||
```
|
||
|
||
These are names inside an immutable action result, not globally meaningful
|
||
filenames. The cache is keyed by digests and logical package identity, so there
|
||
is no `__root` special case and no dotted import path used as an artifact
|
||
basename. A root object and dependency object obey identical protocols.
|
||
|
||
`link.wwlm` declares the package object's target, object ABI, defined/required
|
||
foreign symbols, required native-provider slots, initialization ordering, and
|
||
runtime ABI. It attests facts/edges already present in the action-template DAG;
|
||
it may not introduce a provider, dependency, or action after compilation, and a
|
||
mismatch is a compiler/build-protocol error. It does not contain raw linker flags. Ordinary package objects are
|
||
fed directly to the product link. An archive exists only when an explicit
|
||
static-library product asks the `archive` action to combine its ordered declared
|
||
`members` (default: root package only). Transitive package/native dependencies
|
||
remain typed link requirements in the library's install manifest and are not
|
||
silently copied into multiple archives. A deliberately self-contained archive
|
||
must list every member explicitly and pass duplicate-symbol/provider checks.
|
||
|
||
Language initialization never depends on linker input order. `ww.init` consumes
|
||
all reachable `.wwlm` artifacts, topologically orders initialization by package
|
||
imports with byte-sorted ties, diagnoses duplicate/cyclic init facts, and emits
|
||
one dispatcher object. The link plan explicitly roots that dispatcher and every
|
||
referenced init symbol against section garbage collection.
|
||
|
||
### 7.2 WW Export Data 1 (`.wwe`)
|
||
|
||
`.wwe` is a cache/build protocol, not source text and not a long-term binary
|
||
distribution promise. It starts with the eight-byte magic `WWEX\0\0\0\1` and a
|
||
WWAR-encoded body. The magic is fixed framing that is reconstructed and verified
|
||
around the body; it does not create a second blob identity. The typed `.wwe`
|
||
identity is `record_id(18, 1, ExportV1)` over that body. The body contains:
|
||
|
||
1. export schema, language edition, type-system protocol, and required reader
|
||
capabilities;
|
||
2. target descriptor, C ABI, object ABI, and runtime ABI digests;
|
||
3. full package identity and declared package name;
|
||
4. a sorted table of originating package/type identities and declaration-level
|
||
public ABI digests actually referenced by the exported surface;
|
||
5. a canonical type graph sufficient for type checking, layout, calling
|
||
convention, and code generation of every exported declaration;
|
||
6. exported constants, variables, functions, methods, types, and explicit
|
||
foreign symbols; and
|
||
7. the public initialization/ABI facts needed by an importer.
|
||
|
||
Declarations are sorted by `(kind, exported name, stable overload discriminator)`;
|
||
type-graph nodes are assigned by deterministic structural traversal. Integer and
|
||
floating constants use canonical target-independent bit encodings until a
|
||
target conversion is part of their type. Function parameter **names**, source
|
||
locations, comments, unused imports, private function bodies, and declaration
|
||
order are not semantic and are omitted. Documentation/source mapping is a
|
||
separate optional artifact and cannot invalidate an importer.
|
||
|
||
Unmanaged layout sometimes depends on facts that are private at the source
|
||
level. An exported representation therefore records size, alignment, field/base
|
||
offsets, calling convention, niche/tag rules, and relevant private padding or
|
||
opaque-field descriptors without exposing private names. `@repr("c")` types
|
||
also record the exact C data model and layout algorithm version. Opaque types
|
||
record only the operations and layout promises permitted to clients.
|
||
|
||
### 7.3 Direct imports with deep public closure
|
||
|
||
An importer opens one `.wwe` for each direct import and no transitive interface
|
||
file. If a direct dependency's API mentions a type originating in a deeper
|
||
package, its `.wwe` embeds a canonical deep descriptor for the portion of that
|
||
type required to understand and lay out the direct API. The descriptor retains
|
||
the originating package/type identity and digest of that exact referenced
|
||
contract—not the originating package's entire public ABI. It does not pretend
|
||
the type belongs to the middle package.
|
||
|
||
This rule gives both correctness and bounded reads:
|
||
|
||
```text
|
||
source/package imports -> direct .wwe inputs
|
||
direct .wwe -> complete meaning of that direct API
|
||
link graph -> all reachable package objects
|
||
```
|
||
|
||
A public change in a leaf rebuilds direct reverse dependencies. Propagation
|
||
continues only while each rebuilt package's `.wwe` bytes change. A private leaf
|
||
change never enters an importer key. This replaces source-like transitive
|
||
interface prepending and its quadratic composed units.
|
||
|
||
This bounds interface **opens and reparsing**, not necessarily total descriptor
|
||
bytes: heavily re-exported type graphs can duplicate deep descriptors. Phase 1
|
||
measures total `.wwe` size and structural duplication on the real library graph.
|
||
Only if that is material may a later export-schema revision intern immutable
|
||
per-declaration descriptors; direct-import semantics do not change.
|
||
|
||
### 7.4 Public and ABI digests
|
||
|
||
The `.wwe` content digest identifies its complete target-specific bytes. It also
|
||
contains two domain-separated hashes:
|
||
|
||
- **public type digest** over names, types, constants, visibility, and language
|
||
semantics; and
|
||
- **public ABI digest** over target layouts, calling conventions, exported
|
||
symbol contracts, runtime ABI, and representation facts.
|
||
|
||
Compile actions normally depend on the whole `.wwe` content digest. Tools such as
|
||
documentation may depend only on the public type digest when their action kind
|
||
explicitly permits it. Link compatibility checks use the ABI digest. Digest
|
||
subsetting is protocol-defined; callers cannot choose arbitrary ignored fields.
|
||
|
||
### 7.5 Symbols and identity
|
||
|
||
Internal WW symbols are mangled from a protocol-versioned hash of the full
|
||
package identity plus declaration identity, never from a leaf name or artifact
|
||
filename. Resolver rules ensure only one source/version supplies that package
|
||
identity. An explicitly foreign symbol is exactly the source-declared spelling
|
||
and participates in duplicate-provider checks.
|
||
|
||
Package version and distribution origin are absent from mangling because they
|
||
are not identity. An incompatible major version has a different `/vN` module
|
||
identity and therefore different WW symbols. Native C symbols do not gain this
|
||
protection; their provider slots and link collision rules must reject
|
||
incompatible co-selection.
|
||
|
||
### 7.6 Compatibility and deterministic serialization
|
||
|
||
A consumer accepts only the exact export/type/object/runtime protocol combination
|
||
declared compatible by its immutable toolchain descriptor. A new optional record
|
||
still requires a new export schema and reader capability. Unknown records are
|
||
not silently dropped. Target descriptor and ABI mismatches are errors before a
|
||
compiler or linker runs.
|
||
|
||
Because `.wwe` is rebuilt from locked source, WW does not need an indefinitely
|
||
stable compiler-internal export format. A toolchain upgrade changes action keys
|
||
and may rebuild the graph. Public native-library ABI stability is a separate,
|
||
explicit provider contract. Release/bootstrap byte comparisons include `.wwe`,
|
||
objects, link metadata, action records, and executables.
|
||
|
||
## 8. Native integration and cross compilation
|
||
|
||
### 8.1 Complete target descriptors
|
||
|
||
A target triple is a user-facing alias. Before graph construction it expands to
|
||
an immutable target descriptor containing at least:
|
||
|
||
- architecture, vendor, operating system, environment, and object format;
|
||
- endianness, pointer widths/address spaces, integer/long widths, alignment and
|
||
aggregate-layout rules;
|
||
- C ABI/data model, calling conventions, name decoration, variadic convention,
|
||
TLS ABI capabilities/default, floating ABI, and unwind model;
|
||
- baseline CPU, required/forbidden CPU features, minimum OS/SDK version, and
|
||
atomic capability;
|
||
- supported/default relocation and code models, executable/shared-library rules,
|
||
and page constraints;
|
||
- hosted versus freestanding policy; and
|
||
- compatible object ABI and runtime ABI protocol identifiers.
|
||
|
||
The triple `x86_64-unknown-linux-gnu` is insufficient by itself to identify CPU
|
||
features, sysroot, glibc, loader, or linker. Those are separate descriptor/input
|
||
digests. `--cpu` and `--feature` produce a new canonical expanded descriptor;
|
||
the host CPU is never probed to select target features unless the user explicitly
|
||
requests the impure alias `native`, which is rejected by frozen/shared builds.
|
||
|
||
The descriptor supplies ABI invariants plus supported/default policy values. A
|
||
profile/product selects the effective relocation, code, PIC/PIE, and TLS policy
|
||
from those permitted sets; the normalized effective values appear once in the
|
||
action record. A conflicting or unsupported selection is rejected, never
|
||
resolved by precedence between duplicate fields.
|
||
|
||
This follows the native facts exposed by LLVM data layouts and Clang's cross
|
||
compilation/toolchain documentation, while making their often-driver-selected
|
||
inputs explicit
|
||
([LLVM data layout](https://llvm.org/docs/LangRef.html#data-layout),
|
||
[Clang cross compilation](https://clang.llvm.org/docs/CrossCompilation.html),
|
||
[Clang toolchain](https://clang.llvm.org/docs/Toolchain.html)).
|
||
|
||
### 8.2 Toolchain closure (`toolchain.wwt`)
|
||
|
||
An immutable toolchain bundle has a canonical `toolchain.wwt` descriptor with:
|
||
|
||
```text
|
||
ww-toolchain 1
|
||
id = "ww.org/toolchain"
|
||
version = "v1.4.0"
|
||
protocols = { language = "1", compiler = "1", export = "1", object = "1",
|
||
runtime = "1", manifest = 1, lock = 1 }
|
||
|
||
tool "wwc" { path = "bin/wwc", digest = "sha256:..." }
|
||
tool "cc" { path = "bin/clang", digest = "sha256:..." }
|
||
tool "as" { path = "bin/llvm-mc", digest = "sha256:..." }
|
||
tool "ld" { path = "bin/ld.lld", digest = "sha256:..." }
|
||
tool "archive" { path = "bin/llvm-ar", digest = "sha256:..." }
|
||
|
||
target "aarch64-unknown-linux-gnu" {
|
||
descriptor = "targets/aarch64-linux-gnu.wwt"
|
||
sysroot = { tree = "sha256:...", path = "sysroots/aarch64-linux-gnu" }
|
||
runtime = "runtime:aarch64-linux-gnu@1"
|
||
libc = "c:glibc@2.39"
|
||
link-policies = [
|
||
{ kind = "exe", linkage = "dynamic", profile = ["debug", "release"],
|
||
runtime = "hosted",
|
||
tokens = [
|
||
{ kind = "object", artifact = "sysroot:lib/crt1.o" },
|
||
{ kind = "object", artifact = "sysroot:lib/crti.o" },
|
||
{ kind = "splice", value = "product-objects" },
|
||
{ kind = "splice", value = "init-dispatch" },
|
||
{ kind = "splice", value = "native-providers" },
|
||
{ kind = "splice", value = "product-controls" },
|
||
{ kind = "provider", slot = "c:compiler-rt@1" },
|
||
{ kind = "provider", slot = "c:glibc@2.39" },
|
||
{ kind = "dynamic-loader",
|
||
artifact = "sysroot:lib/ld-linux-aarch64.so.1",
|
||
runtime-path = "/lib/ld-linux-aarch64.so.1" },
|
||
{ kind = "script-slot", artifact = "toolchain:lib/ldscripts/elf.lds" },
|
||
{ kind = "object", artifact = "sysroot:lib/crtn.o" }
|
||
] }
|
||
]
|
||
}
|
||
```
|
||
|
||
Every path is bundle-relative and every executable, shared tool dependency,
|
||
resource directory, built-in header tree, target descriptor, runtime, sysroot,
|
||
CRT, default script, and adapter is covered by the bundle's canonical tree
|
||
digest. The engine invokes exact paths and passes explicit target/sysroot/resource
|
||
arguments. A tool's compiled-in search outside the sandbox cannot resolve.
|
||
The compact `sysroot:`/`toolchain:` references in the example normalize to full
|
||
`ArtifactRefV1` records containing the individual typed digest obtained from
|
||
that authenticated tree; the shorthand itself never enters WWAR.
|
||
|
||
The whole bundle digest authenticates acquisition. An action key uses the
|
||
transitive selected descriptor slice/tool/resource/sysroot/runtime closure only;
|
||
adding an unused target or unrelated tool to a republished bundle does not cause
|
||
global recompilation. Changing any selected byte/protocol still changes the key.
|
||
|
||
Official bundles may use LLVM, GNU binutils, or another implementation per
|
||
target; the architecture does not expose that choice as project semantics.
|
||
Tool adapters translate WW's typed record to exact argv and declare all injected
|
||
inputs. A custom bundle must do the same and pass conformance/reproducibility
|
||
tests before frozen mode accepts it.
|
||
|
||
WW permanently owns the language compiler, export/object/runtime ABI protocols,
|
||
target descriptor schema, action engine, and official bundle definitions. It
|
||
does **not** permanently own the assembler, linker, archiver, C compiler, or SDK.
|
||
The current `w6a`/`w6l` may serve as migration inputs, then are removed once a
|
||
pinned external closure passes parity. This is the smaller long-term system.
|
||
|
||
### 8.3 C ABI and foreign declarations
|
||
|
||
Foreign declarations are explicit source contracts:
|
||
|
||
```ww
|
||
@abi("c") @symbol("write")
|
||
@provider("c:libc")
|
||
fn c_write(fd s32, data *u8, count usize) ssize;
|
||
|
||
@repr("c")
|
||
type Header struct { tag u32; length u16; };
|
||
```
|
||
|
||
The compiler checks that every type has a defined representation for the
|
||
selected C ABI and records symbol, calling convention, variadic status, layout,
|
||
and explicitly named provider slot in `.wwe`/`.wwlm`. `@provider` is mandatory
|
||
for each foreign declaration (a group annotation may supply it lexically), and
|
||
the owning package metadata must require that slot; WW never infers it from a
|
||
symbol spelling or link position. A foreign declaration with no provider, two
|
||
definitions of a strong symbol, incompatible calling conventions, or mismatched
|
||
layout digest is a pre-link diagnostic where possible and a mandatory link
|
||
failure otherwise.
|
||
|
||
C headers are not searched or parsed implicitly. Bindings are either checked-in
|
||
WW source produced by an explicit `ww bindgen c` command, or a declared
|
||
`generate` action whose inputs include the exact header trees, target descriptor,
|
||
preprocessor, macro map, include roots, and binding tool. The generated result
|
||
records the declared whole input-tree digests, observed include trace as
|
||
non-semantic audit metadata, and C ABI digest. The trace cannot add an input; a
|
||
future finer-grained scan would require a new built-in action still keyed by the
|
||
complete allowed include-tree digest. No build invokes ambient
|
||
`pkg-config`; `ww native snapshot-pkg-config` is an explicit, impure acquisition
|
||
command that converts one selected host configuration into a reviewable native
|
||
provider record and content snapshot.
|
||
|
||
### 8.4 Native declarations
|
||
|
||
Schema 1 uses two connected declarations. A package lists the ABI slots it
|
||
requires:
|
||
|
||
```text
|
||
package "compress/zlib" {
|
||
native = ["c:zlib@1"]
|
||
generated = ["action:zlib-bindings:ww"]
|
||
}
|
||
```
|
||
|
||
A provider declares exact target artifacts and dependencies:
|
||
|
||
```text
|
||
native "zlib-linux-aarch64" {
|
||
provides = "c:zlib@1"
|
||
when = { os = "linux", arch = "aarch64", environment = "gnu",
|
||
c-abi = "aapcs64", float-abi = "hard",
|
||
requires-features = ["neon"],
|
||
minimum-sdk = "linux:5.10.0.0" }
|
||
include-trees = [
|
||
{ tree = "native/zlib/include", class = "user", subdirectory = "." }
|
||
]
|
||
sources = [
|
||
{ name = "adler32", source = "native/zlib/adler32.c",
|
||
language = "c11", preprocessing = "c-preprocessor",
|
||
include-indices = [0], defines = {} }
|
||
]
|
||
defines = { ZLIB_CONST = "1" }
|
||
objects = []
|
||
archives = []
|
||
shared = []
|
||
requires = ["c:libc"]
|
||
link = [
|
||
{ kind = "object",
|
||
artifact = { namespace = "provider-output",
|
||
owner = "zlib-linux-aarch64",
|
||
name = "adler32", kind = "object", mode = "data" } }
|
||
]
|
||
abi = { file = "native/zlib.wwabi" }
|
||
}
|
||
```
|
||
|
||
Allowed provider fields are exactly `provides`, `when`, `include-trees`,
|
||
`sources`, `defines`, `objects`, `archives`, `shared`, `requires`, `link`, and
|
||
`abi`. `when` is a finite conjunction. It may contain exact
|
||
`descriptor-digest`, `arch`, `vendor`, `os`, `environment`, `object-format`,
|
||
`hosted`, `c-abi`, `data-model`, `float-abi`, `cpu-baseline`, `relocation`,
|
||
`code-model`, and `pic`; `requires-features`/`forbids-features` use subset/
|
||
disjoint-set matching; `minimum-sdk` matches only a product platform whose
|
||
declared minimum is at least that value. It has no general expression. All
|
||
matching candidates are retained: identical provider/artifact digests coalesce,
|
||
while multiple different matches require the product's explicit provider map
|
||
rather than a specificity guess.
|
||
|
||
The textual `abi = { file = PATH }` form parses `PATH` as one complete canonical
|
||
`NativeABIContractV1` subdocument, encodes it as top-level record kind 16, and
|
||
lowers the provider field to an `ArtifactRefV1` containing that typed record
|
||
digest; an inline complete record is equivalent. The local source blob and
|
||
parsed contract are both hashed. The file form is not a digest assertion or a
|
||
build-time include, and partial contracts are invalid.
|
||
|
||
`sources` entries support only toolchain-declared C language editions and
|
||
assembly dialects. Each source has an exact file digest and sees only the listed
|
||
include trees/defines and toolchain headers. `objects`, `archives`, and `shared`
|
||
name content-identified prebuilt artifacts plus their target/object/ABI records.
|
||
A local `path` is content-hashed during analysis; a literal expected digest is
|
||
needed only for an external/prebuilt record. A provider may mix source and prebuilt inputs, but every produced object is its
|
||
own `native.compile` action. There is no filesystem library search.
|
||
|
||
`abi` is not an opaque user assertion. It is the recomputed digest of a canonical
|
||
native ABI contract containing slot, product-platform ABI/data model, calling
|
||
conventions, symbol names/versions/kinds, referenced C layout/header-contract
|
||
digests, required runtime slots, CPU/features, minimum SDK, and PIC/TLS/unwind
|
||
requirements. Foreign bindings carry the expected contract or compatible
|
||
declaration-level subset. WW cross-checks the contract against compiled objects,
|
||
shared/import-library sidecars, and declared providers before link.
|
||
|
||
### 8.5 Assembly and object files
|
||
|
||
Assembly source declares an official external-tool dialect (`gnu`,
|
||
`llvm-integrated`, or another exact toolchain capability), preprocessing mode,
|
||
target, and CPU feature contract. The current `w6a` Plan-9-style dialect is a
|
||
migration input and is not accepted after cutover.
|
||
The selected assembler executable and resources are action inputs. A source for
|
||
one target cannot be selected for another by extension alone.
|
||
|
||
Prebuilt objects carry a sidecar native record with content digest, object
|
||
format, architecture, ABI, required CPU features, defined/undefined symbols,
|
||
PIC/TLS/unwind properties, and producer provenance. WW verifies the object
|
||
format/class/endian/machine header, sections, symbol table, relocations, notes,
|
||
and architecture attributes against every mechanically inferable sidecar fact;
|
||
the same inspection recurses into archive members and shared/import libraries.
|
||
Source-level C contract, libc/runtime compatibility, and provenance are not
|
||
fully inferable from object bytes, so frozen opaque prebuilts additionally need
|
||
a signature/attestation accepted by policy. A missing or contradictory record
|
||
is an error, not permission to ask the host linker what happens.
|
||
|
||
### 8.6 Static/shared libraries and ordered linking
|
||
|
||
Native `link` is an ordered list of typed template tokens:
|
||
|
||
```text
|
||
{ kind = "object", artifact = "object:NAME" }
|
||
{ kind = "archive", artifact = "archive:NAME" }
|
||
{ kind = "shared", artifact = "shared:NAME" }
|
||
{ kind = "provider", slot = "c:zlib@1" }
|
||
{ kind = "group", items = [...] }
|
||
{ kind = "whole", items = [...] }
|
||
{ kind = "as-needed", enabled = true, items = [...] }
|
||
{ kind = "linker-script", artifact = "file:NAME" }
|
||
{ kind = "version-script", artifact = "file:NAME" }
|
||
{ kind = "export-map", artifact = "file:NAME" }
|
||
{ kind = "def-file", artifact = "file:NAME" }
|
||
{ kind = "runtime-search", policy = "origin-relative", path = "lib" }
|
||
{ kind = "install-name", value = "@rpath/libname.so" }
|
||
{ kind = "dynamic-loader", artifact = "file:LOADER", runtime-path = "/lib/ld.so" }
|
||
{ kind = "entry", symbol = "_start" }
|
||
{ kind = "retain", symbol = "ww_init_abcd" }
|
||
```
|
||
|
||
Those are the complete schema-1 project/native template kinds. `provider` is
|
||
recursively replaced at its exact position by the chosen provider fragment;
|
||
the final kinds are every listed kind except `provider`. The toolchain-only
|
||
`splice` and `script-slot` kinds also lower away as specified in section 6.9.
|
||
Every file-bearing final token resolves
|
||
to a declared typed artifact and digest. Nested `items` contain only link tokens;
|
||
repetition is represented by repeating a list entry and is never deduplicated.
|
||
`group` contains archives or provider templates that resolve only to archives,
|
||
`whole` contains archives only, and `as-needed` contains shared inputs or
|
||
provider templates that resolve only to shared inputs; any other expansion or
|
||
nesting is invalid.
|
||
Runtime-search/install-name values are validated by the selected platform
|
||
adapter; frozen bundled policy permits only relocatable origin-relative paths.
|
||
Entry/retain tokens become the pinned linker's typed entry/undefined-root
|
||
mechanism. Raw flags exist only inside a content-identified custom toolchain
|
||
adapter.
|
||
|
||
The plan is not a set and is never alphabetically reordered. The selected
|
||
toolchain policy template determines global position: it can place start CRT
|
||
before the product-object splice and end CRT after runtime providers, rather
|
||
than relying on one universal ordering rule. Inside `product-objects`, WW
|
||
objects use stable package-identity order. Inside `native-providers`, fragments
|
||
use requester-before-provider topological order, which gives `libA` before the
|
||
`libB` it requires. `product-controls` carries the declared entry/retain/install
|
||
tokens; `init-dispatch` carries its single retained object when needed. Cyclic
|
||
static archives must be represented by one explicit `group`; an undeclared
|
||
provider cycle is an error. Repetition, whole-archive, as-needed, export maps,
|
||
and symbol-version scripts remain exact records in the flattened canonical plan.
|
||
|
||
An archive action preserves declared member order and canonicalizes header
|
||
timestamps, ownership, modes, and string tables. A shared-library input includes
|
||
its link-time artifact, SONAME/install-name, ABI digest, transitive runtime
|
||
requirements, and deployable runtime artifact digest. Merely finding the same
|
||
basename in a host directory is never equivalence.
|
||
|
||
The installation manifest also fixes runtime resolution. ELF bundled policy
|
||
copies the exact shared closure under a digest-namespaced relative `lib/` and
|
||
uses an origin-relative RUNPATH; Mach-O uses exact `@rpath`/install names; Windows
|
||
places named DLL artifacts in the declared application directory beside their
|
||
matching import libraries. A platform-image provider may instead bind an exact
|
||
loader/system tree. An OS-managed mutable shared library is an impure runtime
|
||
policy: link bytes can still be recorded, but WW does not promise that execution
|
||
will load a particular digest.
|
||
|
||
Linker scripts are declared content inputs. The adapter resolves/audits only
|
||
file-bearing directives such as `INCLUDE`, `INPUT`, `GROUP`, and `SEARCH_DIR`
|
||
for the pinned linker dialect; included files and permitted sysroot trees are in
|
||
the record. The pinned linker—not WW—interprets section placement, expressions,
|
||
symbols, memory regions, and target semantics inside the closed sandbox.
|
||
Unresolved `SEARCH_DIR`, absolute host paths, and implicit default scripts are
|
||
errors. GNU `ld` documents that scripts and archive order change link semantics;
|
||
WW therefore preserves rather than abstracts them away
|
||
([GNU ld scripts](https://sourceware.org/binutils/docs/ld/Scripts.html),
|
||
[GNU linker](https://sourceware.org/binutils/docs/ld.html)).
|
||
|
||
### 8.7 libc, CRT, SDK, loader, and freestanding products
|
||
|
||
A hosted platform entry names one exact sysroot/SDK and the available libc,
|
||
system, compiler-runtime, and WW-runtime providers; it does not name one
|
||
universal CRT sequence or loader. The selected toolchain link policy for
|
||
`(product-platform descriptor, product kind, linkage, profile, runtime
|
||
selector)` supplies the exact ordered token template containing CRTs, compiler
|
||
runtime, system-provider slots, script slot, and any platform-appropriate loader
|
||
contract. Compiler and linker
|
||
drivers run through no-defaults adapters and receive only that declared closure,
|
||
so they cannot fall back to B's `/usr`.
|
||
|
||
A product chooses a runtime policy:
|
||
|
||
```text
|
||
product "kernel" {
|
||
kind = "exe"
|
||
root = "kernel"
|
||
linkage = "static"
|
||
runtime = "none"
|
||
entry = "_start"
|
||
native = ["freestanding:boot@1"]
|
||
linker-script = "native/kernel.ld"
|
||
}
|
||
```
|
||
|
||
For an executable, `linkage` is exactly `dynamic`, `pie`, `static`, or
|
||
`static-pie`; shared-library products use `shared`. The toolchain maps
|
||
`(product kind, linkage, profile, product-platform, runtime selector)` to one
|
||
exact ordered policy template; selection never ignores `runtime`. Loader
|
||
presence follows the platform's executable rules. On an ABI with
|
||
an explicit program interpreter, `dynamic` and `pie` executables must name its
|
||
runtime path and exact provider artifact; `static` and `static-pie` must not. A
|
||
shared library has a runtime identity and dependencies but no executable program
|
||
interpreter. PE/COFF-style platforms without a separate interpreter bind the
|
||
exact platform-image/loader contract through system-provider policy rather than
|
||
inventing a pathname. Different product policies therefore cannot accidentally
|
||
share one CRT/loader sequence.
|
||
|
||
`runtime` is `hosted`, `minimal`, `none`, or a named provider. `none` supplies no
|
||
libc, CRT, loader, or WW runtime; compiler helper routines must be supplied by a
|
||
declared provider or rejected. `minimal` names an exact freestanding runtime.
|
||
Entry symbol, memory/linker script, relocation/code model, panic/stack policy,
|
||
and any boot image action are explicit. Kernel-style targets never inherit the
|
||
hosted target's defaults.
|
||
|
||
Schema 1 permits `none` only for object/static products and `static` or
|
||
`static-pie` executables. Its selected link template must contain no CRT,
|
||
dynamic-loader, libc, compiler-runtime, or WW-runtime token. `minimal` and named
|
||
providers declare their valid product/linkage set as capabilities; `hosted`
|
||
uses the platform's hosted set. A product/linkage/runtime tuple outside that set
|
||
is rejected during graph construction. The policy's single `script-slot` uses
|
||
the product's `linker-script` when present, replacing the default at the same
|
||
ordered position.
|
||
|
||
The target/toolchain declares the runtime capability required by every
|
||
compiler-emitted helper and language operation. During graph construction a
|
||
`none`/`minimal` product is rejected if selected source operations require an
|
||
unavailable allocation, panic, stack, arithmetic, TLS, unwind, or other runtime
|
||
capability; this is not deferred to an unexplained undefined linker symbol.
|
||
|
||
### 8.8 Provider conflicts and system substitution
|
||
|
||
One link namespace may select exactly one provider digest for an ABI slot such
|
||
as `c:zlib@1`, `c:libc`, or `runtime:ww@1`. Multiple requirements for the same
|
||
slot coalesce only if they resolve to the same provider and ABI digest. Different
|
||
providers, ABI major slots that export colliding unversioned symbols, and two
|
||
native modules claiming the same strong symbols are loud identity-collision
|
||
errors. WW never chooses whichever library appears first.
|
||
|
||
When more than one contract-compatible provider matches, the root product must
|
||
select one explicitly:
|
||
|
||
```text
|
||
product "hello" {
|
||
kind = "exe"
|
||
root = "."
|
||
linkage = "dynamic"
|
||
providers = [
|
||
{ slot = "c:zlib@1", use = "example.org/zlib#zlib-linux-aarch64" }
|
||
]
|
||
}
|
||
```
|
||
|
||
The provider ID is `(declaring module identity, native-clause name)`, rendered
|
||
with `#` only in metadata. Selection cannot change a dependency's required ABI
|
||
contract; the chosen provider must satisfy every declaration-level contract.
|
||
|
||
Two incompatible native versions can coexist only if they use distinct provider
|
||
slots **and** their symbols/runtime names are namespaced or versioned so the link
|
||
record proves no collision. Otherwise the build must adapt one behind a wrapper,
|
||
use dynamic isolation, or fail. Language-level multiple-version selection cannot
|
||
solve a C global-symbol collision.
|
||
|
||
A distro system provider is a normal provider record mapping exact logical
|
||
artifacts to content digests and ABI metadata. If those files live under `/usr`,
|
||
the mapping snapshots/re-hashes them before graph construction and changes the
|
||
key whenever they change. It is marked impure unless the directory tree itself
|
||
is immutable and content-identified. Raw `-L`, `-l`, `LD_LIBRARY_PATH`, compiler
|
||
defaults, and build-time `pkg-config` are not accepted substitutes.
|
||
|
||
A portable distributor substitution uses a closed `ww-native-map 1` file:
|
||
|
||
```text
|
||
ww-native-map 1
|
||
target = "sha256:product-platform-descriptor..."
|
||
provider "c:zlib@1" {
|
||
use = "distro.example/native#zlib"
|
||
contract = "sha256:..."
|
||
artifact-tree = "sha256:..."
|
||
provenance = "https://distro.example/provenance/zlib.jsonl"
|
||
}
|
||
```
|
||
|
||
`ww lock --native-map=FILE` records the map digest/origin in `ww.lock`; frozen
|
||
mode accepts only that exact signed/content-verified map. `ww.work` may contain
|
||
the same `provider` clause for local development, but it is an impure overlay and
|
||
frozen mode rejects it. This gives OS packagers an offline substitution mechanism
|
||
without changing imports or silently consulting `/usr`.
|
||
|
||
### 8.9 Cross-compilation behavior
|
||
|
||
All code-generating tools execute on B. WW/C/assembly compilation for the
|
||
requested ordinary product emits H objects using only H's target descriptor,
|
||
sysroot, headers, runtime, and native providers. A compiler-like product built
|
||
for H may later emit T code, but no T program executes during its own build.
|
||
Object headers and sidecars are checked before linking, so a host object cannot
|
||
silently enter a target product.
|
||
|
||
`ww test --target=H` always builds target test binaries. It executes them only
|
||
when `H = B` or the toolchain declares an explicit content-identified runner
|
||
(local emulator or simulator plus immutable image) as an invocation tool. Otherwise
|
||
it reports “built, not run” unless `--require-run` was requested, in which case
|
||
it fails. The runner and its platform image are action inputs; no ambient emulator
|
||
is discovered.
|
||
|
||
Remote hardware/device testing is a separate explicit
|
||
`ww observe test --runner=NAME` operation with declared endpoint/capability
|
||
authority. It may use network/devices but is a non-build observation: remote
|
||
state is reported, it never populates artifact/shared caches, and it is outside
|
||
byte-reproducibility claims. Ordinary `ww test` retains the no-network policy.
|
||
|
||
This model supports new targets without running a compiler on them: an existing
|
||
host toolchain adds a target descriptor, backend, object adapter, sysroot/runtime,
|
||
and native providers, then builds and tests through a declared runner or hardware
|
||
step. GNU's build/host/target distinction is useful vocabulary, but WW records
|
||
the complete descriptors rather than only triplets
|
||
([Autoconf triplets](https://www.gnu.org/software/autoconf/manual/autoconf-2.71/html_node/Specifying-Target-Triplets.html)).
|
||
|
||
## 9. Command-line design
|
||
|
||
### 9.1 The ordinary path
|
||
|
||
The default workflow is deliberately short:
|
||
|
||
```sh
|
||
ww init example.org/hello
|
||
ww build
|
||
ww run -- argument
|
||
ww test .
|
||
```
|
||
|
||
`ww init MODULE` creates `ww.mod`, a root `main.ww` only when the directory is
|
||
empty, and a lock selecting the currently invoked immutable toolchain. It does
|
||
not add a dependency. In an existing one-directory `main` package, `ww build`
|
||
works without `init`; the invoking toolchain and standalone source identity are
|
||
shown in verbose output.
|
||
|
||
`ww build [DIR|PRODUCT]` builds the default root product or one named product.
|
||
The default profile is the fully specified `debug` profile; `--profile=release`
|
||
selects the toolchain's immutable release profile. `ww run` first performs that
|
||
same build, then runs only a product with `H = B`; arguments after `--` are never
|
||
interpreted by the build.
|
||
|
||
There is no command that means “build and opportunistically download whatever is
|
||
missing.” If a locked source or toolchain is absent, the diagnostic names its
|
||
digest and asks for `ww fetch --locked`.
|
||
|
||
### 9.2 Tests, examples, docs, and installation
|
||
|
||
```sh
|
||
# Exactly one package.
|
||
ww test ./internal/codec
|
||
|
||
# Every package below the current module root.
|
||
ww test ./...
|
||
|
||
# Compile cross-target tests and require a declared runner.
|
||
ww test ./... --target=aarch64-unknown-linux-gnu --require-run
|
||
|
||
# Build an ordinary example product and render documentation.
|
||
ww build example:examples/packet
|
||
ww doc ./... --out=out/doc
|
||
|
||
# Materialize a named release product under a prefix.
|
||
ww install inspect --profile=release --prefix=/opt/ww
|
||
```
|
||
|
||
Test package discovery and result reporting remain deterministic under `-j N`.
|
||
Compilation is cached, but every selected test binary runs. An installation
|
||
manifest lists every copied artifact, digest, mode, runtime dependency, and
|
||
relative destination and is materialized as `<product>.wwinstall`. Installation never discovers libraries in the prefix and
|
||
never mutates the cache artifact.
|
||
|
||
### 9.3 Inspecting the graph and cache
|
||
|
||
```sh
|
||
ww graph --packages
|
||
ww graph --actions
|
||
ww graph --actions --format=json > graph.json
|
||
ww explain example.org/hello/internal/codec
|
||
ww explain --path --all product:hello
|
||
ww cache verify
|
||
ww verify reproducible --profile=release
|
||
```
|
||
|
||
`graph` performs analysis but no build action. A node whose producer output does
|
||
not yet exist has a known action shape and incoming edges but a `pending` key;
|
||
its final key is computed when predecessor content digests become available.
|
||
`explain` compares available current records with the project index and likewise
|
||
does not build. These commands never fetch.
|
||
|
||
### 9.4 Adding, fetching, updating, freezing, and vendoring
|
||
|
||
```sh
|
||
# Query the module's conventional source index, add a minimum, and lock closure.
|
||
ww add example.org/codec@v1.2.3
|
||
|
||
# The same operation with an explicit private or non-conventional origin.
|
||
ww add corp.example/codec@v1.2.3 --from=https://packages.corp/codec/
|
||
|
||
# Materialize exactly the existing lock without changing any project file.
|
||
ww fetch --locked
|
||
|
||
# Change one direct minimum and recompute the complete lock atomically.
|
||
ww update example.org/codec@v1.4.0
|
||
|
||
# Recompute after an intentional manifest edit.
|
||
ww lock
|
||
|
||
# Materialize the exact locked source closure in the project.
|
||
ww vendor --locked
|
||
|
||
# A frozen, offline build from already present locked inputs.
|
||
ww build --frozen --offline --vendor
|
||
```
|
||
|
||
`add` and `update` write `ww.mod` and `ww.lock` together through temporary files
|
||
only after the entire selection and integrity check succeeds. `lock` writes only
|
||
`ww.lock`. `fetch` and `vendor` never change selection. Without `--vendor`, WW
|
||
uses the immutable source store; with it, every vendor digest is reverified.
|
||
`add --from=URL` records that credential-free index as the requirement's
|
||
`source-index`; archive redirects/final origin remain exact lock metadata.
|
||
|
||
An ordinary non-frozen build still never updates an inconsistent lock: it fails
|
||
with the exact `ww lock` command. Frozen mode additionally rejects workspaces,
|
||
impure providers, noncanonical metadata, unlocked tools, and any source digest
|
||
not named by the lock. `--offline` is useful in automation as an assertion but
|
||
does not weaken or strengthen the normal build network denial.
|
||
|
||
### 9.5 Cross compilation and toolchain selection
|
||
|
||
```sh
|
||
# Select a product host platform (the familiar cross-build form).
|
||
ww build --target=aarch64-unknown-linux-gnu --profile=release
|
||
|
||
# Select exact CPU semantics rather than probing the build machine.
|
||
ww build --target=x86_64-unknown-linux-gnu \
|
||
--cpu=x86-64-v3 --feature=-avx512f
|
||
|
||
# Build a compiler that runs on H and emits code for T.
|
||
ww build compiler:wwc \
|
||
--host=aarch64-unknown-linux-gnu \
|
||
--target=riscv64-unknown-none-elf
|
||
|
||
# Acquire and lock an exact toolchain before building; only fetch uses network.
|
||
ww toolchain fetch ww.org/toolchain@v1.4.0
|
||
ww lock --toolchain=ww.org/toolchain@v1.4.0
|
||
ww build --toolchain=ww.org/toolchain@v1.4.0
|
||
```
|
||
|
||
The selected command toolchain must match the common ID, satisfy every minimum,
|
||
and equal the root lock's exact descriptor. A digest
|
||
selection is exact; an ID/version selection resolves only through the lock or
|
||
installed signed catalog and never during build. `ww toolchain list --targets`
|
||
shows descriptors and sysroot/runtime digests, not just triples. `ww lock
|
||
--toolchain` changes only the lock's exact toolchain selection when the chosen
|
||
version satisfies every manifest minimum; it does not rewrite those minimums.
|
||
|
||
### 9.6 Deliberately absent commands/options
|
||
|
||
There is no `ww build --fetch`, build-time package-manager hook, raw `-L`/`-l`,
|
||
global import search path, arbitrary compiler/linker environment injection,
|
||
manifest evaluator, or command that installs dependencies into a mutable global
|
||
source namespace. Expert escape hatches are explicit toolchains, native provider
|
||
records, and finite declarative actions; all remain visible in the graph/key.
|
||
|
||
## 10. Toolchain and bootstrap design
|
||
|
||
### 10.1 The durable stage-zero seed
|
||
|
||
The smallest credible recovery seed is:
|
||
|
||
```text
|
||
bootstrap/ww0.c generated portable C99 snapshot
|
||
bootstrap/bootstrap.plan canonical source/tool/output plan
|
||
bootstrap/SHA256SUMS expected seed and plan digests
|
||
```
|
||
|
||
`ww0.c` is generated at release time from the same compiler sources as the
|
||
production compiler, with a deliberately non-optimizing portable C backend and
|
||
the minimal support routines concatenated into one translation unit. It is not
|
||
a second hand-maintained compiler and is never used in ordinary production
|
||
builds. The resulting `ww0` retains that portable C-emission path: it contains
|
||
only enough compiler, export writer, C output, and fixed-plan execution to emit
|
||
stage-1 C from the enumerated source closure and have the named host C closure
|
||
compile it. It has
|
||
no resolver, network client, general manifest engine, cache, test coordinator,
|
||
or installation framework.
|
||
|
||
The release process regenerates `ww0.c` and fails if its bytes differ from the
|
||
checked-in snapshot. Keeping portable source rather than four opaque host
|
||
binaries permits recovery on a new architecture with a C99 implementation. The
|
||
cost of the simple C emitter is accepted because it bounds and exposes the seed;
|
||
it is a release backend, not another production path.
|
||
|
||
The bootstrap-host contract is narrower than “any C99”: hosted C99 with
|
||
`CHAR_BIT == 8`, exact `uint8_t/uint32_t/uint64_t`, two's-complement signed
|
||
integers, binary file I/O, at least 32-bit address space and the published source/
|
||
object size limits. The snapshot uses no host floating-point result or undefined
|
||
signed overflow. Compile-time assertions plus a tiny I/O/integer conformance
|
||
probe run before compilation. Supported implementation modes and required
|
||
headers are enumerated in `bootstrap.plan`; an implementation outside the
|
||
contract is not silently called portable.
|
||
|
||
### 10.2 Stage transitions and fixed point
|
||
|
||
Recovery uses these exact stages:
|
||
|
||
1. A named host C implementation compiles `ww0.c` to `ww0`. Its executable,
|
||
version output, command, headers, libc, assembler, and linker are recorded in
|
||
`bootstrap-host.wwar`; they are part of the trusted base, not silently blessed.
|
||
2. `ww0 bootstrap/bootstrap.plan` runs on B, emits portable C for the locked
|
||
production compiler/driver, and uses that same recorded host C closure to
|
||
produce **stage 1 with H = B**. Stage 1 must run locally; distributable cross
|
||
compilers are built only after the local fixed point.
|
||
3. Stage 1 uses the pinned production target/tool closure to perform a normal
|
||
frozen build of the same sources for `H = B`, producing **stage 2**.
|
||
4. Stage 2 repeats the identical logical build to produce **stage 3**.
|
||
5. Stage 3 repeats it to produce **stage 4**.
|
||
6. `bootstrap.compare` first requires stage-2 and stage-3 **semantic output
|
||
sets**—compiler/driver executables, export/object/runtime artifacts, and the
|
||
installed semantic bundle tree—to be byte-identical. Producer action records
|
||
and provenance are excluded because stage 2 was built by stage 1 while stage
|
||
3 was built by stage 2. Once those compiler bytes converge, it requires
|
||
stage-3 and stage-4 semantic outputs **and raw action/result records** to be
|
||
byte-identical; their producer compiler digests are then equal. Each raw
|
||
record is a separately named `action-record` or `action-result` input in
|
||
action tag 10; comparison metadata or a digest outside tag 10 is not read
|
||
authority.
|
||
|
||
Stage 1 need not equal stage 2: portable-C and production backends may generate
|
||
different code. Stage 2 MUST equal stage 3 semantically, and stage 3 MUST equal
|
||
stage 4 completely. A canonical stage semantic manifest lists only role/output
|
||
artifact digests, never its producer. A mismatch reports the first differing
|
||
output or action field and is a release failure. Two absolute roots and two
|
||
concurrency levels are used for the official fixed-point job.
|
||
|
||
This fixed point proves self-consistency and path/order reproducibility; it does
|
||
not by itself defeat Ken Thompson's trusting-trust attack. Official releases
|
||
also perform **diverse seed compilation** with two independently sourced host C
|
||
toolchains where available, publish both bootstrap-host records, regenerate the
|
||
C snapshot from the converged WW source/compiler, and compare converged semantic
|
||
outputs before signing. This is additional evidence, not a claim of formal
|
||
diverse-double-compilation proof.
|
||
|
||
### 10.3 Trusted-computing-base accounting
|
||
|
||
The bootstrap report lists, by content digest:
|
||
|
||
- `ww0.c`, `bootstrap.plan`, and their tiny C support layer;
|
||
- host C compiler/preprocessor, headers, libc, assembler, linker, loader, and OS
|
||
kernel used to create/run `ww0`;
|
||
- bootstrap assembler/linker/archive tools and their shared/resource closure;
|
||
- source/lock/toolchain descriptor trees; and
|
||
- the SHA-256 and signature verification implementations/keys.
|
||
|
||
That is the reproducible software/input closure. The operational trusted
|
||
computing base additionally includes CPU, firmware, memory/storage behavior,
|
||
kernel, and execution environment; the report names the available hardware/
|
||
firmware attestations but does not pretend a source hash removes physical trust.
|
||
|
||
Nothing is called “trusted” merely because it was found on `PATH`. A recovery
|
||
build with unpinned system tools may establish a new local fixed point but cannot
|
||
claim byte identity with an official release. An official reconstruction uses a
|
||
published content-identified host/bootstrap closure and compares its advertised
|
||
digests.
|
||
|
||
### 10.4 Toolchain distribution and selection
|
||
|
||
Each release publishes source plus locked-source-closure archive, `ww0.c`, lock
|
||
file, platform toolchain bundles, `toolchain.wwt`, stage-2/3/4 semantic and
|
||
action/result manifests, fixed-point report,
|
||
digests, signatures, and provenance. Binary bundles are conveniences verified
|
||
against the descriptor, not irreplaceable seeds. A user either selects the
|
||
project-locked toolchain or an exact command-line digest; WW never downloads a
|
||
new compiler while building.
|
||
|
||
Runtime ABI, export protocol, object ABI, and action-schema compatibility are
|
||
declared independently. A compiler refuses a runtime whose ABI it does not
|
||
support. Toolchain upgrades can retain source compatibility while intentionally
|
||
invalidating all relevant action keys.
|
||
|
||
### 10.5 New-architecture and no-compiler recovery
|
||
|
||
For a new H satisfying the bootstrap-host contract, compile `ww0.c`; `ww0` emits
|
||
the modified production compiler as portable C and the H C compiler builds the
|
||
locally runnable stage 1. Add/pin H's production object/assembler/linker closure,
|
||
then reach the stage-2/3/4 fixed point. Alternatively an existing B toolchain may
|
||
cross-build that H compiler, but it executes only through an explicit runner.
|
||
For a new T only, no target-local compiler is required: add the backend/target
|
||
descriptor, object tests, sysroot or freestanding runtime, and linker provider
|
||
to an existing B/H toolchain.
|
||
|
||
When no suitable WW compiler exists, the checked-in C seed is sufficient. When
|
||
no C compiler exists either, a platform must provide one previously built `ww0`
|
||
binary plus its exact source/digest, or bootstrap a C implementation; WW does not
|
||
claim a smaller physical trusted base than the machine can execute.
|
||
|
||
## 11. Current-system assessment
|
||
|
||
This section records the verified baseline at commit
|
||
`ca2cadeb4173e8190cd4c8bcc25e7da25bcdb0bc`. Line references describe that
|
||
commit and are implementation evidence, not requirements for the replacement.
|
||
|
||
### 11.1 Current invariants
|
||
|
||
- POSIX Make selects ambient `CC`, `AR`, optional `ccache`, and flags; builds the
|
||
Cstage tools; then uses the Cstage driver to build WWstage tools into separate
|
||
work directories. Make repeats manually enumerated transitive source
|
||
prerequisites for each self-hosted target
|
||
([Makefile](../Makefile), lines 7–20 and 105–279).
|
||
- `ww build` creates fixed-array package nodes keyed by dotted import spelling,
|
||
discovers imports with a hand-written scanner separate from the compiler
|
||
parser, rejects directory-package cycles, visits dependencies in
|
||
DFS postorder, and invokes packages serially. The graph is capped at 256
|
||
packages ([C driver](../cmd/ww/main.c), lines 333–370 and 670–700).
|
||
- An imported directory is a separately compiled package. An imported `.ww`
|
||
file is recursively folded into the importing unit and has no node, identity,
|
||
interface, object, or link artifact of its own. Inline package blocks can also
|
||
satisfy otherwise missing imports (C driver, lines 396–428 and 498–613).
|
||
- A directory package selects all immediate `.ww` files except `*_test.ww`, sorts
|
||
them by bytes, and has no target-specific selection rule (C driver, lines
|
||
239–301). The build path follows source symlinks while the test coordinator
|
||
rejects them.
|
||
- Resolution translates dotted imports to paths and performs a global
|
||
directory-before-file search over the entry root, `-I` roots, and inferred
|
||
source library. Thus a directory in a later root beats a file in an earlier
|
||
root (C driver, lines 152–224 and 966–1003).
|
||
- Every dependency emits source-like `.wwi`. Every importer receives the whole
|
||
transitive `.wwi` closure, tagged with out-of-band module comments and prepended
|
||
to a composed `.unit.ww`; the compiler reparses that unit (C driver, lines
|
||
702–809; [interface writer](../cmd/w6c/wwi.c), lines 1–24 and 505–570).
|
||
- The root compiles without compiler `-I`, emits no `__root.wwi`, bypasses the
|
||
dependency export-signature path, and receives special `main` handling; only
|
||
dependency nodes emit interfaces. `ww test` also injects the resolvable `test`
|
||
package as a synthetic root edge even when source has no such import, while
|
||
runtime is an implicit link edge. Imports do not fully describe even today's
|
||
complete graph (C driver, lines 1066–1083 and 1131–1146).
|
||
- `.wwi` records exported prototypes and direct import text but no compiler,
|
||
format, target, data-layout, object ABI, or runtime ABI identity. It includes
|
||
non-semantic parameter names and import spelling. The compiler's `-I` flag
|
||
both requests interface output and changes `main` symbol handling
|
||
([compiler entry](../cmd/w6c/main.c), lines 25–56 and 83–110).
|
||
- For `P` reachable directory packages including root, a normal driver build
|
||
launches `P` compiler processes, `P` assembler processes, and one linker.
|
||
It writes `P-1` dependency archives itself. The link is root object, dependency
|
||
archives in reverse topological order, runtime, then separately accumulated
|
||
`-L` and `-l` values, losing their original interleaving (C driver, lines
|
||
1095–1249 and 1368–1460).
|
||
- `-w DIR` is a caller-owned mutable reuse directory, not a cache. Freshness is
|
||
exact composed-unit bytes, copied driver/compiler/assembler bytes, a text mode
|
||
stamp, and artifact existence/nonzero size. The graph and units are rebuilt
|
||
in memory and the final executable is relinked on every invocation.
|
||
Publication through `.new` files with the unit committed last is usefully
|
||
atomic (C driver, lines 867–952, 1016–1059, and 1110–1213).
|
||
- Without `-w`, a build creates and deliberately retains
|
||
`<output-stem>.sepwork`; repeating while it exists fails. With `-w`, the caller
|
||
must create and serialize the directory. Normal driver products are always
|
||
root executables linked with the runtime; there is no library-only product
|
||
path, and `-S` merely stops after assembly.
|
||
- `ww test DIR` delegates to a separate coordinator. It groups same-package
|
||
`package p;` and external `package p_test;` tests from `*_test.ww`, excludes
|
||
dependency tests, composes generated source roots, and parallelizes independent
|
||
test binaries with deterministic reporting
|
||
([package coordinator](../internal/wwpackage/package.ww), lines 278–417,
|
||
508–648, and 725–1057).
|
||
- Explicit `ww test FILE` bypasses the directory `*_test.ww` classifier and
|
||
accepts an arbitrarily named source root. Universal directory packages delete
|
||
that distinct test mode.
|
||
- Current bootstrap is mixed C/self-hosted. Make keeps the C driver fixed, uses
|
||
self-hosted compiler stages to produce `ww2`, `ww3`, and `ww4`, and compares
|
||
`ww2 == ww3` and `ww3 == ww4`. The self-hosted driver itself is outside that
|
||
fixed-point chain. The planned four stage-zero binaries are absent; the
|
||
“no C compiler” route still uses host `ar` ([Makefile](../Makefile), lines
|
||
825–883; [bootstrap notes](../BOOTSTRAP.md)).
|
||
|
||
### 11.2 Conflated identities and accidental behavior
|
||
|
||
This table records the baseline that the implemented slices below replaced;
|
||
sections 11.6–11.18 are authoritative where they conflict with it.
|
||
|
||
| Concept that must be separate | Current conflation or accident |
|
||
|---|---|
|
||
| package identity | Before the local-package slices, dotted import spelling was simultaneously graph key, module/symbol prefix, artifact basename, and link identity. The implemented loader now separates source spelling, expanded canonical identity, physical directory, and storage locator. |
|
||
| declared name | Before section 11.18, imported directories required their declared name to equal the import-path leaf and `.wwi` retained only that leaf. The implemented compiler/export path now carries the declaration independently. |
|
||
| filesystem location | Ordered search roots silently choose/shadow a location; the same physical directory may be compiled under two import identities, while duplicate locations for one spelling produce no collision diagnostic. Paths are lexical, not content identities. |
|
||
| package versus file | Directories create separate-compilation nodes; files disappear into owners. The same `package` syntax means two compilation models. |
|
||
| artifact versus identity | `<dotted-path>.wwi/.s/.o/.a` names artifacts; root aliases to `__root`, which can collide with a real import. |
|
||
| public versus non-semantic interface | Parameter names and AST-preserved type/import spellings influence `.wwi` bytes and reverse rebuilds. Imports/declarations are otherwise canonically sorted; whitespace/comments and original declaration order generally do not. |
|
||
| compiler interface mode versus link identity | `w6c -I` both emits `.wwi` and classifies the package as a dependency for `main` mangling. |
|
||
| native dependency versus linker search | Raw `-L` and `-l` names carry no selected file, ABI, order relationship, target, or content identity. |
|
||
| cache location versus cache key | The explicit `-w` directory is both mutable namespace and freshness state; callers must serialize it. |
|
||
| source root versus command UX | Help historically describes `.` like a basename file, while implementation stats and builds it as a directory. |
|
||
|
||
Other accidental constraints include fixed 256/1024-byte name/path buffers. The
|
||
compiler parser silently truncates dotted full imports beyond 255 bytes, while
|
||
the C driver scanner can stop advancing and hang on an import identifier at that
|
||
limit; the dynamically sized WWstage scanner differs. There is no regression
|
||
test for this stage divergence. Compiler, assembler, and linker launches now use
|
||
structured argument vectors in both stages. Both drivers honor exact executable
|
||
paths in `WW_W6C`, `WW_W6A`, and `WW_W6L` and otherwise select their
|
||
stage-specific sibling tools. Both drivers distinguish the package-source root
|
||
selected by `WW_SRCLIB` from the runtime-artifact root selected by `WW_LIB`, and
|
||
apply the same empty-value and repository/install fallbacks. The two
|
||
implementations remain parallel production algorithms rather than one protocol
|
||
implementation.
|
||
|
||
Build and test disagree about source symlinks. External package tests are built
|
||
from a generated single-file root plus `-I`; an external import of a multi-file
|
||
production package can resolve and fold only its canonical same-named file rather
|
||
than the directory package. Test work-directory names flatten `/` to `_`, so
|
||
distinct lexical paths can collide. These are consequences of routing tests
|
||
around, rather than through, one package model.
|
||
|
||
`make install` copies only `ww`, `wwtest`, and `libwcc.a`, while the driver needs
|
||
sibling compiler/assembler/linker tools and `libwwrt.a`; the installed result is
|
||
not a self-contained functional toolchain outside the build tree.
|
||
|
||
### 11.3 Scaling, invalidation, and hidden inputs
|
||
|
||
The driver performs deterministic linear action interning but now grows every
|
||
package dependency vector dynamically (section 11.14). Every package compiler
|
||
reads exactly one interface per direct dependency; transitive dependencies enter
|
||
only the executable archive closure. Package compiler/assembler work within one
|
||
driver remains serial; Make gains parallelism only by launching independent
|
||
top-level driver builds.
|
||
|
||
The observed invalidation rules are:
|
||
|
||
- a private change in a directory dependency rebuilds that package and the
|
||
unconditional final link, but not importers;
|
||
- an exported change rebuilds direct importers and continues through an ancestor
|
||
only while the regenerated direct-dependency export bytes change, stopping at
|
||
the first byte-identical regenerated interface;
|
||
- a private change in a folded file import rebuilds its entire owner;
|
||
- a link-only option reruns the always-executed link but not package compiles;
|
||
- changing copied compiler or assembler bytes rebuilds every package; and
|
||
- nonzero corruption of `.s`, `.o`, `.a`, or `.wwi` may be accepted because
|
||
content is not rehashed.
|
||
|
||
Hidden or incompletely modeled inputs include `CC`, `AR`, `PATH`, `ccache`, Make
|
||
flags, compiler built-ins, assembler/linker defaults,
|
||
inferred `argv[0]` library locations, current working directory, file mode, runtime
|
||
archive, linker binary, native-library resolution, host libc/CRT/loader, SDK,
|
||
CPU, target, and environment. Make does not invalidate existing C objects when
|
||
the host compiler or C flags change. Persistent package workdirs now bind the
|
||
exact driver executable, conservatively covering its graph, unit, archive, and
|
||
commit semantics; the remaining ambient inputs are not thereby promoted into a
|
||
cache protocol. WW has no target triple, sysroot, conditional source, generator,
|
||
manifest, lock, source digest, or frozen/offline concept.
|
||
|
||
### 11.4 Measurements and disposable experiments
|
||
|
||
Measurements ran on Linux 6.12.76_1 x86-64 with eight logical CPUs, GCC 14.2.1,
|
||
binutils 2.44, and no `ccache`. They used successful clean builds and isolated
|
||
temporary source fixtures; no production migration was begun. Clean/full means
|
||
use two timed samples and representative cold means three, without CPU isolation
|
||
or OS-cache flushing, so they are observed baselines rather than universal
|
||
performance claims.
|
||
|
||
| Measurement | Result |
|
||
|---|---|
|
||
| clean `make all`, `-j1` | 24.847 s mean |
|
||
| clean `make all`, `-j8` | 15.515 s mean; only 1.60× speedup |
|
||
| warm `make all -j8` | 0.02365 s mean at Make level |
|
||
| clean tool trace | 69 package compiles/assembles in six driver workdirs; 71 `w6c`, 76 `w6a`, six `w6l`, 63 in-driver archives; package work inside each driver stayed serial |
|
||
| small / 8-package / 15-package cold driver build | 0.0103 / 0.1326 / 9.6706 s |
|
||
| same warm driver builds | 0.00814 / 0.01347 / 0.05302 s; each still linked |
|
||
| `out/` | 34,464,015 bytes, 398 files |
|
||
| six WW build workdirs | 28,609,092 bytes, 351 files; 9,954,428 bytes (34.8%) duplicate beyond the first content copy |
|
||
| 15-package root composed unit | 14 `.wwi` sections, 1,464,581 bytes; 62 interface insertions across graph, 1,928,898 composed-unit bytes versus 23,664 distinct interface bytes |
|
||
|
||
A three-package `root -> mid -> leaf` fixture confirmed that root consumes both
|
||
direct and transitive interfaces. A private leaf implementation edit rebuilt
|
||
only leaf plus link. Adding a compatible public leaf API rebuilt all three even
|
||
though `mid` emitted identical `.wwi`. A link-only `-L` change ran only the link.
|
||
|
||
Switching the compiler executable by byte content invalidated all packages.
|
||
Appending data to a nonzero cached `mid.s` was not detected; the old object was
|
||
reused and the build succeeded. This directly rejects artifact-presence caching.
|
||
|
||
Two builds from different absolute source roots produced the same 398 relative
|
||
paths and all sampled WW-generated executables/package artifacts were byte
|
||
identical. The full trees were not: 45 Cstage host objects/copied tools differed,
|
||
including GCC `DW_AT_comp_dir`. Thus current checkout-independent byte identity
|
||
holds for measured WW artifacts, not for the complete build.
|
||
|
||
Cstage and WWstage drivers building the same eight-package graph produced a
|
||
byte-identical final executable and all 38 non-tool artifacts, but took 0.138 s
|
||
and 0.356 s respectively. The audit also found that Cstage formerly returned
|
||
success after an ambient `chmod` lookup failed and left mode 0644, while WWstage
|
||
created mode 0755 directly. Cstage now calls `chmod(2)` on the exact output path
|
||
and reports failure, removing that host-tool and path-splitting asymmetry.
|
||
|
||
### 11.5 What survives and what is deleted
|
||
|
||
The replacement retains these sound concepts:
|
||
|
||
- explicit source imports, package clauses, exported/private declarations, and
|
||
loud directory-package-cycle errors, strengthened to every package after file
|
||
folding (whose cycles were merely visit-deduplicated) is deleted;
|
||
- directory package boundaries, made universal rather than optional;
|
||
- same-package and external `*_test.ww` semantics, dependency-test exclusion,
|
||
deterministic discovery/reporting, and always executing selected tests;
|
||
- deterministic sorting/serialization, atomic artifact publication, fixed-point
|
||
bootstrap checks, and byte-identity tests; and
|
||
- the ordinary `ww build`, `run`, and `test` user experience.
|
||
|
||
The replacement deletes these concepts rather than emulating them indefinitely:
|
||
|
||
- file imports, inline multi-package units, bare dotted imports, `-I` search
|
||
roots, directory-before-file precedence, and the `__root` artifact alias;
|
||
- `.wwi`, `//ww:module` wrappers, `.unit.ww`, transitive interface prepending,
|
||
source-prototype interchange, automatic per-dependency archives, and `-w`;
|
||
- raw ambient `-L`/`-l`, compiler/linker/sysroot defaults, and build-time
|
||
`pkg-config` discovery;
|
||
- separate C and self-hosted production drivers, the production Make graph, and
|
||
the separate source-composing test coordinator; and
|
||
- the current Cstage bootstrap path and permanently owned assembler/linker after
|
||
the generated C seed and pinned external tool closure replace them.
|
||
|
||
There will be no compatibility alias that silently translates an old import,
|
||
interface, workdir, or link search into the new model.
|
||
|
||
### 11.6 Implemented local package slice
|
||
|
||
The first executable package slice is intentionally smaller than the final
|
||
module design above. It is local, offline, and manifest-free. The supported
|
||
form is:
|
||
|
||
```sh
|
||
out/bin/ww build -I /work/acme -o app /work/acme/cmd/app
|
||
```
|
||
|
||
Every selected source uses the existing syntax:
|
||
|
||
```ww
|
||
package main;
|
||
import lib.math;
|
||
```
|
||
|
||
A source import is translated from dots to path separators, expanded through
|
||
the nearest eligible local `vendor` directory described in section 11.16, then
|
||
falls back to directory lookup through the entry package's directory, explicit
|
||
`-I` roots in command order, and the toolchain source-library root. A same-named
|
||
`.ww` file is neither a match nor a shadow for an import, so a later root
|
||
containing the directory wins over an earlier file decoy. There is no network,
|
||
manifest, or imported-file fallback. Explicit single-file CLI roots retain
|
||
their raw-unit compatibility path. The loader uses the compiler frontend's
|
||
imports-only parser, retains every real import occurrence with its owning
|
||
source file and position, byte-sorts and deduplicates the resulting canonical
|
||
direct edges, interns canonical directory actions, and reports self-imports and
|
||
stable cycle chains before compilation. Occurrence retention makes contextual
|
||
`internal` and `vendor` checks run at every import site; it does not duplicate
|
||
package actions or compiler inputs.
|
||
|
||
A directory package consists of its immediate `.ww` entries whose basenames do
|
||
not begin `.` or `_`: regular files and symlinks targeting regular files are
|
||
included under the entry name, while symlinks targeting directories are
|
||
ignored. The production variant excludes `*_test.ww`; each variant retains
|
||
byte-sorted filename order. Every selected production file must declare the
|
||
same package name, but that declaration is independent of the directory name
|
||
and every component of the canonical import path. A selected command directory
|
||
declares `package main` while retaining its complete canonical import identity.
|
||
A source import of a command package from a
|
||
different directory is rejected; the one same-directory exception is an
|
||
external `main_test` variant's canonical import of its production action. Two
|
||
ordinary logical identities for one physical directory are rejected rather
|
||
than compiled twice; section 11.16 records the deliberate exception for
|
||
distinct expanded vendor routes that converge through symlinks.
|
||
|
||
Packages compile serially in dependency-first postorder. The compiler emits the
|
||
existing deterministic `.wwi` interface for every directory-package action,
|
||
including an executable root. Its primary section contains that package's
|
||
byte-sorted direct imports and exported declarations. The compiler then appends
|
||
byte-sorted, origin-tagged sections for only the foreign type and constant facts
|
||
recursively reachable from the primary public signatures. Reachable owner-local
|
||
private nominal types are carried without `export`: they make the export
|
||
self-contained for type checking, but qualified source lookup still rejects
|
||
their names. Checked fixed array dimensions are emitted as numeric type facts,
|
||
so a public layout never requires exposing the private constant spelling that
|
||
produced its length.
|
||
|
||
A package compilation unit contains only that package's own byte-sorted sources,
|
||
deterministic `//ww:module-reset` separators, and sorted driver-private
|
||
resolution metadata; it never contains a dependency source body. Each
|
||
**direct** import is a separate
|
||
`--import <canonical-path> <dependency.wwi>` compiler input, sorted by canonical
|
||
path and deduplicated by the loader; a source spelling expanded through
|
||
`vendor` additionally receives the non-dependency `--import-map` described in
|
||
section 11.16. No transitive `.wwi` is passed.
|
||
Origin-tagged facts inside those direct artifacts are compiler data, not source
|
||
imports: a source qualifier is visible only in the source file that directly
|
||
imports it. The qualifier is the imported export's declared package name, not
|
||
the source spelling or import-path leaf. Private members, transitive-only
|
||
qualifiers, bare values, and bare types remain compiler errors. In `-c` package
|
||
mode the compiler parses each
|
||
export independently, then coalesces repeated exported type/constant facts with
|
||
the same origin, kind, and name, preserving one nominal type identity across
|
||
diamonds; raw non-package `w6c` retains its existing one-source behavior. The
|
||
source-like `.wwi` syntax remains a transitional export encoding pending the
|
||
binary `.wwe` format described above, but the separate direct-input ownership
|
||
boundary is live in production Cstage and WWstage compilers and drivers.
|
||
|
||
A selected production root is one normal package action. Its finalized
|
||
canonical import identity tags its owner-only unit; it receives only direct
|
||
exports, emits `.wwi`, `.o`, and a deterministic `.a`, and is compiled exactly
|
||
once. The declared package name is semantic package content but never validates,
|
||
shortens, aliases, or replaces that identity. After loading and identity
|
||
finalization, the
|
||
declaration also selects the terminal build action: `package main` is a command
|
||
and every other valid declaration is a compile-only library. A command root
|
||
receives the narrow compiler `--entry` flag, which controls bare `main` codegen,
|
||
then the linker receives that root archive first, the complete reachable
|
||
package-archive closure, and the runtime archive; it never receives `.wwi`. A
|
||
non-main root receives no `--entry` and never enters the linker. `main` validates
|
||
command kind but never replaces or truncates an identity such as `cmd.tool`.
|
||
The explicit package-less single-file compatibility path has no directory
|
||
package declaration to classify and remains a raw command unit; it does not
|
||
participate in canonical directory-package interning.
|
||
The package driver still uses the parser-only `--command-package` marker for
|
||
command test variants that must remain ordinary archive code. Neither marker
|
||
changes export identity, and imported interfaces retain independent canonical
|
||
owner and declared-name records. The linkers seed `main` before archive selection, so the existing
|
||
WWAR member protocol needs no special root object or format change.
|
||
|
||
Publication is separate from that semantic action choice. `ww build -o lib.a
|
||
DIR` or `ww build -I ROOT -o bar.a foo.bar` automatically publishes a non-main
|
||
root's deterministic archive and self-contained compiler export at
|
||
`FILE.wwi`, without invoking the linker; the latter retains `foo.bar.*` symbols.
|
||
Without `-o`, a non-main root and its dependencies are compiled in the selected
|
||
scratch or persistent work directory and no cwd product is invented. For a
|
||
command, `-o` continues to name the executable publication path. Output names,
|
||
request order, and whether publication was requested never enter package or
|
||
action identity. The historical action-selecting `-p` exception is removed and
|
||
rejected as an unknown build flag. Assembly-only `-S` still stops before object,
|
||
archive, publication, or link production. A literal directory is
|
||
reverse-resolved through the active source roots or receives the deterministic
|
||
local identity described below; its declaration can never invent or truncate
|
||
that identity. Two cold builds with identical inputs are required to produce
|
||
byte-identical requested products. Compiler intrinsics keep their package-mode
|
||
runtime ABI independent of transitive source interfaces (for example, `alloc`
|
||
lowers to the runtime allocator without requiring an `rt.wwi` compiler input).
|
||
|
||
This rule follows the pinned official Go 1.26.5 source at commit
|
||
`c19862e5f8415b4f24b189d065ed739517c548ba`. `go/build.Package` stores source
|
||
directory, declared name, and import path independently, and defines a command
|
||
solely as a package named `main`
|
||
([`go/build/build.go`, lines 436–449](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L436-L449),
|
||
[`go/build/build.go`, lines 514–519](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L514-L519)).
|
||
The Go builder's `AutoAction` links only `main` and returns the archive compile
|
||
action for every other package
|
||
([`cmd/go/internal/work/action.go`, lines 450–456](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L450-L456)).
|
||
The build command invents a default output only for one `main`, applies an
|
||
explicit `-o` to either AutoAction result, and otherwise builds each requested
|
||
package without conflating publication and semantic kind
|
||
([`cmd/go/internal/work/build.go`, lines 473–478](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L473-L478),
|
||
[`cmd/go/internal/work/build.go`, lines 508–548](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L508-L548),
|
||
[`cmd/go/internal/work/build.go`, lines 551–558](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L551-L558)).
|
||
|
||
Both WW stages represent the rule with the existing loaded declaration and
|
||
stable root action index; no package, dependency, ownership, locator, traversal,
|
||
or closure allocation is added. `ww run` adds only a command-kind requirement:
|
||
a successfully loaded, cycle-free non-main root produces the deterministic
|
||
`package PATH is not a main package` diagnostic before any compiler, assembler,
|
||
or linker invocation; package and graph failures retain precedence. Before a
|
||
non-main `-o` build invokes a producer, both stages validate the longest atomic
|
||
publication spelling, `FILE.wwi.new`, so an incomplete archive/export pair is
|
||
never caused by a late path-overflow failure. Package loading, cycle detection,
|
||
and closure validation retain diagnostic precedence over this publication-only
|
||
check, and `-S` does not validate a publication path it never consumes.
|
||
Build workdir format 14 and test workdir format 13 invalidate older unit
|
||
vouchers before reuse because source binding and vendor-directory identity now
|
||
participate in compiler argv and persistent unit semantics. Thereafter
|
||
an equivalent warm library build invokes no tools, a private dependency change
|
||
stops at its unchanged export, and an export change recompiles its direct
|
||
importer under the existing propagation rule.
|
||
|
||
### 11.7 Implemented directory package-test slice
|
||
|
||
Directory tests now enter that same local package loader and build path. The
|
||
supported manifest-free forms include `ww build DIR...`, `ww build DIR/...`,
|
||
`ww test DIR...`, and `ww test DIR/...`, with overlapping direct and recursive
|
||
roots. Test retains its existing `-run`, `-filter`, `-list`, `-timeout-ms`,
|
||
`-j`, `-c`, and `-w` forms. `ww test -c -o test.bin DIR` names the result when
|
||
the selected directory has one test variant; the coordinator rejects one output
|
||
name for a multi-variant or recursive test request. Explicit `ww test FILE`
|
||
retains its compatibility path.
|
||
|
||
The test coordinator still discovers requested directories, enumerates the
|
||
test package names, selects variants, executes independent binaries, and emits
|
||
captured results in byte-sorted package order. It no longer concatenates a
|
||
generated production/test root, resolves imports, or starts one package graph
|
||
per binary or directory. Instead it sends one ordered build request containing
|
||
every selected directory/variant root to the Cstage or WWstage command. Its
|
||
semantic selections are only the directory and selected variant/package
|
||
identities; it also carries output destinations, coordinator-private completion
|
||
paths, import search roots, and the optional command-scoped work-directory
|
||
policy. The command owns source selection, package loading, compiler inputs,
|
||
archive construction, generated-main construction, and linking for every
|
||
package variant:
|
||
|
||
- The ordinary production action selects the directory's byte-sorted non-test
|
||
files. A no-test requested product uses this action directly, and every
|
||
ordinary, internal-test, or external-test import of that package reuses it.
|
||
- The internal production-plus-test variant selects the byte-sorted production
|
||
files followed by byte-sorted matching `package p` test files. It is distinct
|
||
from production so its private production declarations and test declarations
|
||
remain isolated to that test product.
|
||
- When production sources establish `p`, the external variant selects only
|
||
matching `package p_test` files, where `p` is the production declaration
|
||
rather than an import-path leaf. A test-only directory may establish its own
|
||
one consistent test package name, matching pinned `go/build` classification.
|
||
A real source import of the production package uses its canonical import
|
||
path and is a direct edge to the same ordinary production action used by
|
||
ordinary products and transitive imports; there is no external-production
|
||
role or artifact.
|
||
- Generated main is a separate package action whose owner-only generated unit
|
||
declares `package main` and imports exactly the selected variant and test
|
||
support. It consumes those direct `.wwi` files, emits its own `.wwi/.o/.a`,
|
||
and alone receives compiler `-T --entry`. Its target edge also receives the
|
||
compiler-private `--test-target-package <canonical-path>` binding. That
|
||
private canonical-path qualifier prevents a tested command declared `main`
|
||
from colliding with generated main; it is not user alias syntax.
|
||
|
||
The authoritative directory-action identity is the triple **(canonical
|
||
filesystem directory, canonical ordinary import path, semantic variant)**.
|
||
The semantic variants are production, internal production-plus-test, and
|
||
external `_test`; generated main and the reserved test-support alias are
|
||
explicit non-directory action classes. The declared package name is stored
|
||
separately and is not part of canonical directory/path interning. Requested-root
|
||
state, discovery role, product ordinal, output path, persistent artifact key,
|
||
and discovery order never enter the triple.
|
||
|
||
A literal directory root may enter the interner before its full import spelling
|
||
is known. It is provisionally interned by canonical directory and variant, and
|
||
a later source import of that directory binds and reuses the provisional action.
|
||
After all source discovery, but before generated-main construction or any tool
|
||
invocation, each still-unbound directory is finalized by this exact algorithm:
|
||
|
||
1. For every import-resolution context that reached the directory, walk that
|
||
context's roots in its normal forward precedence: the selected package's
|
||
directory, explicit `-I` roots in command order, then `WW_SRCLIB` or the
|
||
selected toolchain source root. A request pattern never becomes an import
|
||
root and therefore cannot shorten, replace, or donate package identity.
|
||
2. Canonicalize each candidate root and require the package directory to be a
|
||
strict descendant. Every relative path component must be a non-keyword WW
|
||
identifier. Convert separators to dots, then resolve that relative spelling
|
||
again through the complete ordered context. Accept it only if ordinary
|
||
forward lookup selects the same canonical directory. Thus an earlier shadow
|
||
invalidates a name inferred from a later or nested root.
|
||
3. Bind the first precedence-valid candidate from each reaching context through
|
||
the command-global bidirectional interner. An identity supplied by successful
|
||
logical package lookup, such as `encoding.utf8`, is already bound and is
|
||
preserved exactly. That forward-selected identity is authoritative: reverse
|
||
derivation applies only to still-unbound literal roots, so a nested active
|
||
root cannot rename an explicitly resolved package.
|
||
4. If no active root can represent the directory, bind the reserved,
|
||
non-source-importable identity
|
||
`__wwlocal.p<escaped-canonical-absolute-directory>`. The
|
||
escape is injective and reversible over path bytes: ASCII letters and digits
|
||
are copied, `_` becomes `_u`, `/` becomes `_s`, and every other byte becomes
|
||
`_xHH` with lowercase hexadecimal. Source imports of `__wwlocal` or any of
|
||
its children are rejected, so this command-local identity creates no alias.
|
||
|
||
The selected full identity is never validated against the ordinary declared
|
||
package name. Production and internal variants retain the production
|
||
declaration; when production exists, an external variant is admitted only as
|
||
`<production-declared-name>_test`. A test-only directory instead establishes
|
||
one consistent test package declaration itself. The one command-kind rule is that a selected
|
||
command family declares `main`/`main_test` while keeping the finalized ordinary
|
||
identity unchanged. A source import of that command from another
|
||
directory rejects as a program before tools; a colocated external command test
|
||
may reuse the canonical production action. There is no fallback from an empty
|
||
import path to a declaration name. Relative, absolute, and symlink spellings
|
||
converge through the canonical directory; two unrelated local directories with
|
||
the same declaration therefore remain distinct. One bound import path mapping
|
||
to two directories and one directory acquiring two incompatible ordinary
|
||
import paths are command-global deterministic errors before any compiler,
|
||
assembler, archiver, or linker ambiguity. The same check spans variants: an
|
||
external action cannot hide a different directory's production package behind
|
||
its derived `_test` compiler path.
|
||
|
||
The derivation and diagnostics are implemented symmetrically in
|
||
`cmd/ww/main.c` and `selfhost/cmd/ww/main.ww`. The package coordinator in
|
||
`internal/wwpackage/package.ww` supplies canonical selected directories and
|
||
variant descriptors, preserves an explicitly resolved identity only for one
|
||
direct request, and forwards a caller's `-w` semantic-action store unchanged.
|
||
It never derives identity or persistent layout from a pattern traversal prefix
|
||
and does not add that prefix to import search. `w6c` and `wcc` consume the
|
||
finalized dotted identity as export/symbol owner while reading the declared
|
||
name independently from export data; neither tool performs directory lookup or
|
||
introduces a package registry.
|
||
|
||
Artifact publication follows the semantic action instead of product order:
|
||
production uses the full finalized ordinary identity, internal appends
|
||
`-internal-test`, external appends `_test-external-test`, and their generated
|
||
mains append `-main`. Generated package identities are likewise derived from
|
||
the full variant identity, for example `__wwtestmain.a.foo.internal.main` and
|
||
`__wwtestmain.b.foo_test.external.main`. Equivalent products therefore reuse an
|
||
already-interned directory action and generated-main action and publish through
|
||
the same persistent-workdir slots regardless of discovery or product order.
|
||
The narrow raw single-file compatibility path alone retains `__root`.
|
||
|
||
A deterministic dependency-first traversal of the complete command union
|
||
invokes the compiler, assembler, and in-driver deterministic archiver once per
|
||
interned action. This is compile-time interning, not linker-argument
|
||
deduplication. Each product is then linked separately from its generated-main
|
||
or executable root archive and the complete reachable archive closure. The
|
||
shared plan remains package-test-specific; it is not a generalized scheduler,
|
||
action schema, cache, or protocol.
|
||
|
||
Each selected directory retains the ordinary entry-directory-first resolution
|
||
context from the local package slice: its directory, explicit `-I` roots in
|
||
command order, then the toolchain source root. Same and external variants of
|
||
one directory share that context; unrelated directory roots never acquire
|
||
lookup precedence from their request order. When multiple contexts reach one
|
||
canonical production package, the loader verifies that every directory import
|
||
binding is identical before reusing its compile action. A different binding is
|
||
a deterministic package-resolution failure for the roots that reach it, rather
|
||
than a first-root-wins build.
|
||
|
||
A production action failure blocks exactly the roots that reach it. A
|
||
root-local compile or link failure does not suppress a successfully built
|
||
sibling product: the command records completion per product, and the
|
||
coordinator can run and report successful siblings while attributing each
|
||
missing product as a build failure. The single union build and completed test
|
||
products share the coordinator's existing `-j` process bound; captured output
|
||
is still emitted only in byte-sorted directory/package order.
|
||
|
||
Every source-imported dependency is a production variant, so dependency
|
||
`*_test.ww` files never enter the graph. Imports that occur only in selected
|
||
test files add edges only to that internal or external variant. Each compiler
|
||
unit contains only its action's owned source set, while its invocation receives
|
||
only the byte-sorted direct dependency `.wwi` artifacts as separate inputs.
|
||
Variant compiles receive `--test-package`, which validates and retains private
|
||
`@test` declarations as compiler-only export metadata without synthesizing an
|
||
entry point. The distinct generated-main action consumes that metadata from its
|
||
direct variant export and synthesizes the dispatcher with `-T`.
|
||
|
||
The generated-main action, rather than the tested variant, owns the implicit
|
||
direct test-runtime support edge. The command-scoped plan compiles the common
|
||
support production package once for the complete test request. The command
|
||
resolves that edge from the selected toolchain source tree, not the user search
|
||
path; the support package's own imports are also loaded in that toolchain
|
||
context. Normally its graph
|
||
qualifier is `test`, so an explicit source `import test` coalesces with the same
|
||
canonical package. When a real user package occupies that identity, the command
|
||
presents the runtime edge to the compiler under the reserved `__wwtest`
|
||
qualifier. This keeps a production package named `test` available to external
|
||
tests. Explicit raw single-file `ww test FILE` fixtures retain the narrow fused
|
||
`-T` compatibility path because an anonymous multi-package raw unit is not a
|
||
canonical directory package; that path still consumes support as a direct
|
||
export and emits a root `.wwi/.a`.
|
||
|
||
The reserved support action and an ordinary source-imported package `test` may
|
||
coexist only because the former is explicitly rebound to the compiler-only
|
||
qualifier `__wwtest`. This is the sole role-based directory alias and cannot be
|
||
created by a source import. The separate expanded-vendor-route exception in
|
||
section 11.16 is canonical source-tree identity, not a role alias. All ordinary
|
||
production actions, including an external test's colocated production
|
||
dependency, use canonical directory identity and the global bidirectional
|
||
import-path checks above. There is no role-based tolerance for duplicate
|
||
ordinary import identities and no late product-closure ambiguity to resolve.
|
||
|
||
The selected internal/external variants and a production action reached by
|
||
their imports or test-runtime closure are the sanctioned graph nodes that may
|
||
share a physical directory. This also lets a toolchain package's own tests
|
||
coexist with the production action required by the test runtime. An internal
|
||
variant already defines those production symbols, so the colocated production
|
||
archive is omitted from that product's final link while the production action's
|
||
dependency archives remain in the closure. The external product includes the
|
||
production archive. Variant-only archives are never linked into another
|
||
product. All ordinary logical and physical package-identity collision checks
|
||
remain unchanged; only distinct expanded vendor routes receive the section
|
||
11.16 symlink-convergence exception.
|
||
|
||
Both stage linkers receive the generated-main archive first, followed by the
|
||
complete reverse-topological reachable package-archive closure, runtime, and
|
||
explicit `-L`/`-l` values through a structured argument vector. No `.wwi` or
|
||
special root `.o` appears in linker argv, and no fixed flattened command buffer
|
||
can truncate a large closure. WWstage emits joined `-Ldir` and `-lname`
|
||
arguments accepted by its native linker, while Cstage preserves the equivalent
|
||
split forms. Generated artifact paths are bounds-checked before any unit is
|
||
opened, so distinct root keys cannot alias by truncation.
|
||
Recursive discovery groups by physical directory before sorting filenames, and
|
||
one stable escape of the canonical discovery directory names the persistent
|
||
command work directory. It contains neither a declared package leaf nor a
|
||
product ordinal, so equivalent path spellings and reordered products select the
|
||
same request state.
|
||
Every `*_test.ww` package variant is built even when a file only
|
||
declares helpers and contains no `@test`, so its package clause and imports are
|
||
still checked by the shared loader. Every built variant is run, and a successful
|
||
compiler-owned dispatcher with empty output is reported as `[no tests]`.
|
||
|
||
Persistent workdirs keep a global driver/compiler/assembler/stamp identity and
|
||
per-action committed units. When that global identity is stale, the command
|
||
first removes every old `.unit.ww` voucher while leaving artifacts recoverable.
|
||
It can then commit the new identity even if one root fails: successful actions
|
||
have freshly committed units, whereas failed or no-longer-requested actions
|
||
cannot be reused. A retry therefore recompiles the failed action without
|
||
discarding unchanged canonical packages built for independent products.
|
||
|
||
Warm reuse compares the staged owner-only unit, committed artifacts, and the
|
||
actual bytes of each direct dependency export. A changed shared dependency is
|
||
compiled once; its direct importers are reconsidered once; and propagation
|
||
stops as soon as a regenerated importer export is byte-identical. Stable
|
||
semantic artifact keys make that behavior independent of which ordinary or
|
||
test product first discovered the action.
|
||
|
||
This ownership split follows the pinned Go 1.26.5 implementation cited in
|
||
section 11.12: its loader reuses canonical cached package objects, its work
|
||
builder interns actions by mode and package, and its test loader reuses ordinary
|
||
production while constructing only the required internal, external, and
|
||
generated-main variants. WW borrows that command-global ownership boundary
|
||
without adding Go's build cache, import configuration format, or module system.
|
||
|
||
### 11.8 Implemented exact package-tool invocation slice
|
||
|
||
The local package builder now launches the compiler, assembler, and linker as
|
||
an executable plus an argument vector in both Cstage and WWstage. No package
|
||
source path, work-directory artifact path, output path, test-support qualifier,
|
||
or link-closure member is flattened into a shell command. Paths containing
|
||
spaces therefore retain one argument boundary from the package coordinator
|
||
through compilation, assembly, and final executable linking.
|
||
|
||
`WW_W6C`, `WW_W6A`, and `WW_W6L` each name one exact executable path. They are
|
||
not shell fragments and are not searched through `PATH`. With no override,
|
||
Cstage keeps its `w6c`/`w6a`/`w6l` siblings and WWstage keeps its
|
||
`w6c_ww`/`w6a_ww`/`w6l_ww` siblings. The coordinator preserves these variables
|
||
when it starts the one command-scoped package build, so the same contract covers
|
||
ordinary directory builds, same-package tests, external tests, recursive test
|
||
requests, and persistent-workdir tool identity. A failed overridden compiler or
|
||
assembler remains attributed to its owning package in both stages. The Cstage
|
||
linker also sets executable mode with `chmod(2)` on the exact output path rather
|
||
than invoking an ambient command.
|
||
|
||
Source imports still own the graph, each directory is still one production
|
||
package, compiler actions receive direct dependency exports as individual
|
||
arguments beside an owner-only `.unit.ww`, and links receive the complete
|
||
per-root `.a` closure. Repository-native coverage wraps all three real stage
|
||
tools at executable paths containing spaces, records every argument boundary,
|
||
inspects `.unit.ww`, `.wwi`, `.a`, and root archive placement, runs the published
|
||
binary, compares repeated Cstage/WWstage artifacts and traces, and removes one
|
||
direct export at compiler entry to compare package-attributed diagnostics.
|
||
|
||
Go 1.26.5 keeps the same responsibility boundary: its work executor passes the
|
||
selected compiler or linker tool and a constructed argument slice to the
|
||
builder, while package loading and action construction remain separate
|
||
([`cmd/go/internal/work/exec.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go)).
|
||
WW adopts that exact-tool boundary without adding a command schema, generalized
|
||
action graph, scheduler, manifest, cache protocol, or package-manager behavior.
|
||
|
||
### 11.9 Implemented package and runtime library-root parity slice
|
||
|
||
Cstage and WWstage now apply the same two-root contract. A nonempty
|
||
`WW_SRCLIB` selects the toolchain package-source search root. It is appended
|
||
after the entry directory and explicit `-I` roots for every package context,
|
||
and compiler-generated test support is resolved from that toolchain root rather
|
||
than from a user package that happens to have the same name. A nonempty
|
||
`WW_LIB` independently selects runtime artifacts; executable and test links use
|
||
`$WW_LIB/libwwrt.a`. If that archive is absent, both stages pass their sibling
|
||
`start.o` and `syscall.o` runtime objects instead. Missing and explicitly empty
|
||
values use the same deterministic fallbacks in both stages.
|
||
|
||
Bare logical build and run targets search the current directory, explicit
|
||
`-I` roots, then one selected library root. That last root is a nonempty
|
||
`WW_SRCLIB`, else a nonempty `WW_LIB`, else an existing
|
||
`<selfdir>/../../lib`, else an existing `./lib`, else
|
||
`<selfdir>/../lib`. Once a target is resolved, its import graph retains the
|
||
ordinary entry-directory-first context and its separate source-root policy:
|
||
without `WW_SRCLIB`, that policy prefers the repository and `./lib` source
|
||
trees before falling back to the runtime root. This preserves Cstage's
|
||
existing distinction between locating a requested package and selecting the
|
||
toolchain source tree used by that package's imports.
|
||
|
||
The variables select local directories only. They do not add manifests,
|
||
dependency declarations, network lookup, package-manager behavior, or a second
|
||
graph: source `import` declarations still create every language edge, resolved
|
||
directories still intern to canonical package identities, compilers still
|
||
receive only direct `.wwi` exports, and final links still receive the complete
|
||
reachable `.a` closure. Directory tests continue to build same-package and
|
||
external-package variants through the same command-scoped package universe.
|
||
|
||
Repository-native coverage uses distinct source and runtime roots whose paths
|
||
contain spaces, resolves a bare logical target found only through `WW_SRCLIB`,
|
||
places a same-named decoy under `WW_LIB`, records exact compiler, assembler, and
|
||
linker arguments, inspects `.unit.ww`, `.wwi`, `.a`, diagnostics, binaries, and
|
||
runtime results, and compares both stages. It separately exercises the runtime
|
||
object fallback and explicitly empty variables. The package-test observer copies
|
||
the toolchain source library to a selected root, marks the compiler-generated
|
||
test-support source, and proves that both the marker and the selected runtime
|
||
archive reach real same-package test builds through the coordinator.
|
||
|
||
The ownership boundary follows the pinned Go 1.26.5 implementation: source
|
||
headers supply imports in
|
||
[`go/build/read.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/read.go#272),
|
||
the loader canonicalizes and reuses packages in
|
||
[`cmd/go/internal/load/pkg.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#633),
|
||
tests remain real package variants in
|
||
[`cmd/go/internal/load/test.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#85),
|
||
and direct compile dependencies are expanded separately for linking in
|
||
[`cmd/go/internal/work/action.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#628).
|
||
|
||
### 11.10 Implemented persistent-workdir driver identity slice
|
||
|
||
The package driver is now an explicit content input to every persistent
|
||
`-w DIR` package action. Both stages copy the exact invoking executable to
|
||
`.wwtool.ww`, alongside `.wwtool.w6c`, `.wwtool.w6a`, and the mode/format
|
||
stamp. A warm invocation byte-compares all applicable live executables before
|
||
considering any committed unit reusable. A missing or changed driver copy
|
||
invalidates every `.unit.ww` voucher before compilation; old artifacts may
|
||
remain recoverable, but none can be reused without a freshly committed unit.
|
||
That slice introduced workdir format revisions 8 for ordinary builds and 9 for
|
||
tests. Later package-identity slices supersede those revisions; the current
|
||
formats are recorded in section 11.16.
|
||
|
||
This closes a real hidden-input boundary. The driver, rather than `w6c`, owns
|
||
canonical directory interning, source-derived graph construction, owner-only
|
||
unit composition, direct-export argument construction, deterministic dependency
|
||
ordering, single-member package archive serialization, and the artifact commit
|
||
sequence. Unit equality alone
|
||
cannot identify changes to those algorithms, and a manually maintained format
|
||
number can be forgotten. Exact driver bytes conservatively cover them during
|
||
the transitional plain-file reuse scheme. This may rebuild after an unrelated
|
||
driver change, but it cannot falsely reuse a package after a relevant one.
|
||
|
||
The linker remains outside the recorded package identity because persistent
|
||
reuse never skips a final link: each invocation reconstructs the complete
|
||
reachable `.a` closure, selects current runtime inputs and link flags, and runs
|
||
the selected linker. Thus a linker or runtime change affects the requested
|
||
binary immediately without forcing unrelated package compilation.
|
||
|
||
Repository-native coverage runs a three-directory import graph through copied,
|
||
independently mutable Cstage and WWstage drivers. Exact compiler, assembler,
|
||
and linker wrappers prove a cold dependency-first build, an unchanged warm
|
||
compile/assemble skip with a deliberate relink, and full package invalidation
|
||
after only the invoking driver's bytes change. The test inspects owner-only
|
||
`.unit.ww` inputs, separate direct `.wwi` arguments, `.a`, identity files,
|
||
transitive link order, diagnostics before tool execution, published binary
|
||
bytes, runtime exit, and stage equivalence.
|
||
|
||
Go 1.26.5 draws the same semantic line with a richer cache: its build action ID
|
||
binds compiler/assembler tool identities, configuration, selected source
|
||
content, and direct dependency content IDs in
|
||
[`cmd/go/internal/work/exec.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#261),
|
||
while its link action ID separately binds linker configuration and the package
|
||
closure in
|
||
[`cmd/go/internal/work/exec.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#1504).
|
||
WW adopts only the correctness boundary in its existing inspectable `cmp`-based
|
||
workdir. It does not add build IDs, hashes, a CAS, an action graph, a scheduler,
|
||
or a manifest.
|
||
|
||
### 11.11 Implemented directory-only source-import slice
|
||
|
||
Cstage and WWstage now use directory packages for every parsed source import,
|
||
including imports selected only by a package-test variant and real imports in
|
||
the test-support package. Source imports first probe the bounded vendor
|
||
candidates in section 11.16, then make one ordered fallback pass for
|
||
`<root>/<import-path>/`; they never probe `<root>/<import-path>.ww`. The
|
||
compiler-generated edge to test support remains synthetic. Unit composition
|
||
consequently writes only the owning package's byte-sorted source files and
|
||
never copies a dependency interface or imported source body. Missing imports
|
||
retain the importing source position and the same stable diagnostic in both
|
||
stages.
|
||
|
||
Root selection remains a separate compatibility boundary. A literal `.ww` CLI
|
||
target, or a bare CLI target found as `<root>/<name>.ww` after the global
|
||
directory search, can still create one raw single-file root. Its historical
|
||
inline package clauses may satisfy compiler-fixture bindings inside that raw
|
||
unit. Directory roots cannot use that exemption, and no filesystem source
|
||
import can reach it. This preserves low-level compiler fixtures without
|
||
weakening package-graph identity.
|
||
|
||
The self-hosted tools no longer depend on the removed behavior. `w6a/` and
|
||
`w6l/` are executable `package main` directories whose sorted source sets are
|
||
compiled once. The compiler backend is one `wcc/` directory package with a
|
||
narrow exported check/codegen façade; `w6c` and `wwdump` import that package
|
||
from its parent search root instead of importing its implementation files.
|
||
Legacy test fixtures were converted to directories, except for one intentional
|
||
compiler leaf-collision probe that now invokes `w6c` on an explicitly composed
|
||
raw unit. The Lisp example likewise imports a `lispcore/` directory package.
|
||
|
||
The focused native regression puts only `example/foo.ww` in an earlier import
|
||
root and a two-source `example/foo/` package in a later root. Both stages select
|
||
the directory, emit the exact sorted package-owned unit, consume its direct
|
||
dependency export, produce byte-identical deterministic `.wwi` and `.a`
|
||
artifacts, link and run a transitive archive closure, and repeat the resolution
|
||
through a real directory-package test. With only the file root present, both
|
||
stages reject the import with byte-identical package-attributed stderr. The
|
||
existing exact-tool observer remains the non-duplicated proof that each
|
||
canonical production action compiles once, compiler units contain only owned
|
||
sources, compile argument vectors contain exactly direct `.wwi` inputs, and
|
||
linker vectors contain the complete reachable `.a` closure and no `.wwi` path.
|
||
|
||
### 11.12 Implemented direct compiler-export input slice
|
||
|
||
Cstage and WWstage now share one small package-compiler convention:
|
||
`--import <canonical-import-path> <export.wwi>` may repeat before the one owning
|
||
source unit. It is valid only with `-c`; paths must be nonempty, strictly sorted,
|
||
and unique. Both compilers read and parse every export independently under the
|
||
supplied canonical identity before parsing the owner unit, then pass the merged
|
||
semantic declaration list through the existing checker, deterministic export
|
||
writer, and primary-only code generator. Missing export bytes therefore fail at
|
||
the compiler boundary as `w6c: import <path>: cannot read <file>`, followed by
|
||
the driver's stable owning-package attribution. No import configuration file,
|
||
manifest, schema, package database, or network lookup is involved.
|
||
|
||
Both drivers construct those arguments directly from the package node's sorted,
|
||
deduplicated outgoing edges. They never walk grandchildren for compilation.
|
||
Every `.unit.ww` contains the node's byte-sorted source files, reset separators,
|
||
and any sorted driver-private vendor/import-map voucher comments, but no
|
||
dependency body. When exact source spelling differs from a selected expanded
|
||
canonical identity, the drivers also pass sorted, unique
|
||
`--import-map <source-spelling> <canonical-import-path>` triples. A map must
|
||
target an ordinary direct `--import`; it adds no export input or graph edge.
|
||
The compilers rewrite only the matching primary import's canonical semantic key
|
||
before merging interfaces. They then obtain that target's declared package name
|
||
from its direct export and install it as the default qualifier in the owning
|
||
source file. Source spelling and position remain intact; generated/synthetic
|
||
imports use their explicit compiler-owned bindings. Executable linking
|
||
independently walks the full reachable package
|
||
closure and passes archives, never interfaces. The same path handles an
|
||
ordinary package, the production-plus-internal-test variant, the external test
|
||
package and its reused production package, compiler-generated test main, and
|
||
the reserved test-support package. Persistent workdirs compare a newly emitted
|
||
export with its committed predecessor before allowing a direct importer to
|
||
reuse owner-identical artifacts, retaining correctness without a new cache
|
||
schema or identity record.
|
||
|
||
The canonical-root regressions additionally prove the following in both
|
||
stages: a literal root under one import root publishes its complete dotted
|
||
identity; logical, literal absolute, equivalent, relative, and symlink routes
|
||
emit byte-identical `.unit.ww`, `.wwi`, and `.a`; root-only and combined
|
||
root/import requests emit those same bytes; dependency-first and root-first
|
||
discovery each compile the shared production action once; recursive `a/foo`
|
||
and `b/foo` directories declaring the same `package foo` publish distinct
|
||
`a.foo` and `b.foo` variants; and two outside-root `package foo` directories
|
||
coexist in one command under distinct reversible local identities. Equivalent
|
||
recursive spellings reuse the same persistent semantic-action store without new
|
||
compilation, while source imports of the reserved local namespace reject before
|
||
tool invocation. The exact-argv command root declares `package main` but keeps
|
||
its non-`main` canonical identity in the unit, export, archive, compiler argv,
|
||
and linker argv; the bootstrap self-rebuild independently proves the same rule
|
||
for the real `w6a` and `w6l` command directories. A focused command-test row
|
||
proves distinct production, internal, external, and generated-main actions,
|
||
colocated external-production reuse, parser-only command markers, stage-equal
|
||
unit/export/archive bytes, both test binaries' runtime behavior, and pre-tool
|
||
rejection when another directory tries to import the command.
|
||
|
||
The exact-argv regression uses the real diamond
|
||
`base -> {left,right} -> root`. It proves one compile per node; no input for
|
||
`base`; only `base.wwi` for each middle node; only sorted `left.wwi` and
|
||
`right.wwi` for `root`; exact owner-only unit bytes; the complete four-package
|
||
link closure including the root archive; no link-time `.wwi`; exit status 42;
|
||
and byte-identical units, exports, archives, executables, and tool argument
|
||
vectors across two clean
|
||
Cstage builds and two clean WWstage builds. The existing directory-package
|
||
variant regression checks separate production, internal, external, and
|
||
generated-main actions, exact generated-main direct variant/support exports,
|
||
canonical production reuse across ordinary and test products, and archive-only
|
||
link closures. It also reverses equivalent product descriptors and compares
|
||
exact compiler and linker trace bytes, then changes a shared direct export in
|
||
persistent workdirs to prove propagation through direct importers stops at the
|
||
first byte-identical regenerated export. Both stages compile and run those
|
||
actions with owner-only units and byte-identical artifacts.
|
||
|
||
The pinned official Go 1.26.5 tag (commit
|
||
`c19862e5f8415b4f24b189d065ed739517c548ba`) supplies the design boundary:
|
||
|
||
- `go/build` represents one selected directory package with its import path,
|
||
package name, ordinary files, internal-test files, external-test files, and
|
||
their imports ([`build.go`, lines 436–493](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L436-L493)).
|
||
Those fields remain separate in Go; WW's final-component/name equality is its
|
||
existing language validation layered on the canonical identity, not a claim
|
||
that Go conflates `Name` with `ImportPath`.
|
||
Its directory reader is required to return name-sorted entries
|
||
([lines 108–111](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L108-L111)),
|
||
and local directory loading reverse-derives a complete import path by checking
|
||
`GOROOT/src` first and then `GOPATH` roots in order. A candidate under a later
|
||
root is rejected when the same relative path resolves through an earlier root
|
||
to another directory; an outside-root directory remains without an ordinary
|
||
import path
|
||
([lines 612–665](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L612-L665)).
|
||
Forward import lookup selects one directory in search order
|
||
([lines 725–767](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L725-L767)),
|
||
and the selected directory alone is scanned
|
||
([lines 859–913](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L859-L913)).
|
||
The sorted scan assigns each accepted source to that package's ordinary,
|
||
internal-test, or external-test list
|
||
([lines 948–1036](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L948-L1036)).
|
||
- `cmd/go/internal/load` derives an outside-root local directory's deterministic
|
||
pseudo-import path from its slash-form absolute directory and establishes the
|
||
package-data cache/promise boundary around that resolved key
|
||
([`pkg.go`, lines 633–647](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L633-L647)).
|
||
WW uses the same reserved full-directory principle but a reversible byte
|
||
escape, strengthening it so two canonical directory spellings cannot collapse
|
||
merely through character sanitization. The Go loader expands source imports
|
||
before recording their canonical paths
|
||
([`pkg.go`, lines 658–669](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L658-L669)).
|
||
It resolves canonical path and directory before consulting the package-data
|
||
cache ([lines 833–842](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L833-L842),
|
||
[lines 863–911](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L863-L911)),
|
||
and the command-global package cache returns the existing package pointer for
|
||
a later root or import of the resolved identity
|
||
([lines 757–775](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L757-L775)).
|
||
Go keeps a command-line `Name == "main"` package as the selected command and
|
||
rejects an import from a different directory, while permitting the
|
||
same-directory test-loader edge
|
||
([lines 799–805](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L799-L805)).
|
||
The package's parsed import list becomes its direct package dependencies,
|
||
rather than a transitive flattening
|
||
([lines 433–440](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L433-L440),
|
||
[lines 2024–2047](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L2024-L2047)).
|
||
- `cmd/go/internal/work` keys its action cache by operation mode plus package
|
||
pointer ([`action.go`, lines 202–206](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L202-L206))
|
||
and returns the already-interned action for that key
|
||
([lines 437–447](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L437-L447)).
|
||
A selected package whose independent `Name` is `main` receives a link action
|
||
([lines 450–455](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L450-L455))
|
||
while retaining its ordinary compiled archive action.
|
||
A compile action depends on only `p.Internal.Imports`
|
||
([lines 628–658](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L628-L658));
|
||
an executable link asks for that same cached root compile action
|
||
([lines 919–957](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L919-L957))
|
||
and separately expands the complete transitive link closure
|
||
([lines 1034–1068](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L1034-L1068)).
|
||
- The work executor derives compiler package mappings from those direct build
|
||
dependencies
|
||
([`exec.go`, lines 864–884](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L864-L884)),
|
||
compiles the package's own source list to `_pkg_.a`
|
||
([lines 928–935](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L928-L935)),
|
||
packs and publishes that package archive
|
||
([lines 1017–1033](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1017-L1033)),
|
||
and links the compiled main archive with mappings for every dependency
|
||
already expanded onto the link action
|
||
([lines 1592–1624](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1592-L1624),
|
||
[lines 1635–1647](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1635-L1647)).
|
||
- Go's test loader explicitly models production, internal
|
||
production-plus-test, external `_test`, and generated main, and states that
|
||
`ptest == p` when production can be reused
|
||
([`test.go`, lines 85–102](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L85-L102)).
|
||
Test imports use the ordinary load cache and compare canonical `ImportPath`
|
||
([lines 118–161](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L118-L161));
|
||
the internal copy is created only when needed
|
||
([lines 175–226](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L175-L226)),
|
||
while external and generated-main packages remain distinct
|
||
([lines 228–293](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L228-L293)).
|
||
Generated main receives its direct support and selected-variant imports
|
||
([lines 307–358](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L307-L358)),
|
||
and copy-on-write rewriting preserves the original package pointers/actions
|
||
for unaffected importers
|
||
([lines 421–472](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L421-L472)).
|
||
- Unified export production begins from the local package, re-exports required
|
||
dependency data, and prunes unnecessary detail
|
||
([`unified.go`, lines 147–168](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/unified.go#L147-L168)).
|
||
It type-checks the package's parsed sources and writes deterministically
|
||
ordered public/private roots
|
||
([lines 314–362](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/unified.go#L314-L362)),
|
||
then finalizes self-contained export data with sorted relocated declaration
|
||
and body indexes and a fingerprint
|
||
([lines 463–570](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/unified.go#L463-L570)).
|
||
- Compiler import handling canonicalizes each source import and rejects self
|
||
import ([`import.go`, lines 125–167](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/import.go#L125-L167)),
|
||
then independently opens and decodes each direct package archive/export
|
||
([lines 170–225](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/import.go#L170-L225)).
|
||
The complete unified section and linker fingerprint are read from that
|
||
selected package file
|
||
([lines 229–296](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/noder/import.go#L229-L296)).
|
||
`ReadPackage` reconstructs a package from its public export root
|
||
([`ureader.go`, lines 28–62](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/importer/ureader.go#L28-L62)),
|
||
interns embedded package descriptors by canonical path and restores their
|
||
import lists
|
||
([lines 152–196](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/importer/ureader.go#L152-L196)),
|
||
and reconstructs declarations from relocated export records
|
||
([lines 391–468](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/importer/ureader.go#L391-L468)).
|
||
|
||
WW adopts those practical ownership and action semantics while retaining its
|
||
small direct CLI representation and existing self-contained `.wwi` encoding.
|
||
|
||
### 11.13 Implemented unbounded semantic package-identity storage slice
|
||
|
||
Canonical package identity is no longer stored in, derived from, or bounded by
|
||
one internal filesystem component. In both drivers every package action now
|
||
keeps these values separately:
|
||
|
||
- `path`: the complete compiler/import identity;
|
||
- `import_base`: the complete canonical ordinary directory identity;
|
||
- `canon`: the complete canonical directory location;
|
||
- `variant` and `role`: the semantic test/action tags; and
|
||
- `storage`: an internal scratch basename that is never passed as package
|
||
identity.
|
||
|
||
`canon` participates in command-local directory-action interning, diagnostics,
|
||
and storage-address derivation. The complete `path`, together with the
|
||
semantic `variant` and `role`, is the persisted/compiler owner carried through
|
||
source-import edges, module-reset and export ownership, compiler `--import`
|
||
arguments, generated-main construction, and symbol qualification; `storage`
|
||
participates in none of those identities. The reversible outside-root form is
|
||
`__wwlocal.p<escaped-canonical-absolute-directory>` and is
|
||
allocated to its exact length. It is neither truncated nor replaced by a
|
||
digest, and `__wwlocal` remains unavailable to source imports. Package
|
||
declarations classify package kind and contribute semantic export content; they
|
||
do not validate a path leaf, supply a missing identity, or alter a command
|
||
package's canonical path.
|
||
|
||
Short actions retain their established `.unit.ww`, `.wwi`, `.s`, `.o`, and
|
||
`.a` basenames when the basename plus `.unit.new` fits the 255-byte supported
|
||
filesystem component bound and the complete path fits the host pathname API.
|
||
An action that does not fit uses this bounded storage locator:
|
||
|
||
```text
|
||
__wwpkg.v<variant>.r<role>.h<lowercase-sha256>
|
||
```
|
||
|
||
The SHA-256 byte input is exactly:
|
||
|
||
```text
|
||
"ww-package-storage-v2:"
|
||
|| ASCII(<variant> ":" <role> ":")
|
||
|| complete semantic path
|
||
|| NUL
|
||
|| complete canonical directory
|
||
```
|
||
|
||
WW package paths and host paths cannot contain NUL, so that boundary is
|
||
unambiguous. Variant and role are present in both the digest input and the
|
||
visible locator tag. The digest is only an action-storage address: units still
|
||
begin with `//ww:module-reset <complete-path>` and may append deterministic
|
||
driver-private resolution comments, while exports begin with
|
||
`//ww:module <complete-path>`, compiler imports carry the complete path, and
|
||
qualified declarations use it in generated symbols. A selected executable
|
||
entry retains its intentional bare linker spelling. User-selected `-o`
|
||
publication paths bypass this derivation completely.
|
||
|
||
Storage assignment is command-global and finishes before any tool is invoked.
|
||
If two actions prefer the same legacy basename, every unhashed member is
|
||
readdressed through the complete-action formula instead of rejecting a valid
|
||
package graph. If two already-addressed, unequal complete actions ever produce
|
||
the same locator, both drivers issue the same full-identity storage-collision
|
||
diagnostic before compilation. A persistent workdir also validates every
|
||
existing regular `.unit.ww` voucher against the requested complete semantic
|
||
owner before stale-tool invalidation or reuse; a missing voucher is cold state,
|
||
while a malformed, non-regular, or wrong-owner voucher is a pre-tool error.
|
||
Thus ordinary preferred-name collisions are resolved, and a digest collision
|
||
cannot silently alias two live or warm package actions. That storage slice
|
||
introduced build version 10 and test version 11 so an older flat-layout voucher
|
||
was never accepted as current state. Section 11.16 records the current
|
||
superseding formats.
|
||
|
||
The persisted semantic owner is deliberately the complete canonical import
|
||
path, not the canonical host directory: host location must not enter compiler
|
||
artifacts. Variant and role are encoded in the locator itself. Canonical
|
||
directory remains part of command-local action interning and the digest input,
|
||
so distinct locations normally receive distinct slots. In the hypothetical
|
||
case that two locations with the same import path, variant, and role also
|
||
collide in SHA-256, they are still one semantic package identity: the freshly
|
||
composed owner unit must byte-equal the committed unit before reuse, so
|
||
different sources rebuild and identical sources produce the same deterministic
|
||
artifacts. A digest collision between different semantic paths fails the
|
||
existing complete-path owner check before tools. This preserves collision
|
||
checking without serializing machine-specific canonical directories into units
|
||
or exports and without a sidecar, registry, or new metadata protocol.
|
||
|
||
The package coordinator derives no persistent container from a request
|
||
directory or pattern. `-w DIR` names the driver's semantic-action store itself,
|
||
and the driver independently derives every action locator in that store from
|
||
the full semantic tuple above and validates each committed semantic owner.
|
||
Equivalent direct, logical, relative, absolute, dotted, recursive, duplicate,
|
||
and explicit-root-symlink requests that select the same canonical actions can
|
||
therefore reuse the same slots; request shape cannot split or alias persistent
|
||
state. For a delegated request only, a missing caller `-w` directory is created
|
||
after graph, identity, visibility, cycle, closure, and output preflight. An
|
||
explicit root symlink is followed and canonicalized; a source entry symlink to
|
||
a regular file is followed under the entry name, a source-shaped symlink to a
|
||
directory is ignored, and symlinked recursive children are not traversed.
|
||
|
||
The old `SEP_IMPORT_PATH_MAX` and all corresponding 255-byte WWstage import,
|
||
variant, local-identity, and generated-main checks are removed. Package names,
|
||
canonical identities, reverse-resolved dotted identities, generated-main
|
||
identities, compiler import paths, and Cstage assembler symbols now use exact
|
||
allocations. Cstage's assembler no longer copies a line, operand, `TEXT`, or
|
||
`DATA` symbol through 256-byte arrays, and the C checker no longer resolves a
|
||
qualified type through a 128-byte prefix buffer. `PATH_MAX` remains only at
|
||
actual host pathname and syscall boundaries; the 255-byte constant remains
|
||
only as the conservative internal basename component bound. The former
|
||
`SEP_MAXPKG`, `SEP_MAXPRODUCT`, and `SEP_MAXCONTEXT` action-count limits are
|
||
removed by the dynamically sized package-universe implementation in the next
|
||
section.
|
||
|
||
The existing native observers exercise the new boundary with ordinary dotted
|
||
identities over 255 bytes and punctuation-heavy reversible local identities
|
||
over 255 bytes. They inspect the complete owner in units and exports, exact
|
||
direct sorted/deduplicated `.wwi` compiler inputs, long mangled assembler
|
||
symbols, complete archive-only link closures, runtime results, and independent
|
||
Cstage/WWstage artifact bytes. Two deep outside-root directories declaring
|
||
the same leaf coexist under distinct reversible identities. A long command
|
||
package and its production, internal, external, and two generated-main actions
|
||
remain distinct. Logical and literal roots, dependency-first and root-first
|
||
discovery, equivalent and symlink spellings, reordered products, and warm
|
||
requests reuse the same production action and persistent slot. The persistent
|
||
diamond observer also changes a shared dependency export, proves rebuilding of
|
||
its direct importers, and proves propagation stops when the regenerated export
|
||
is byte-identical.
|
||
|
||
This separation follows the pinned official Go 1.26.5 tag at commit
|
||
`c19862e5f8415b4f24b189d065ed739517c548ba`, without adopting Go's cache,
|
||
build IDs, importcfg, module machinery, or scheduler:
|
||
|
||
- Source imports are expanded to canonical paths before being recorded
|
||
([`pkg.go`, lines 658–669](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L658-L669),
|
||
[`pkg.go`, lines 1150–1178](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1150-L1178)).
|
||
Local directories receive a deterministic pseudo-import identity derived
|
||
from the full directory while directory and import path stay separate
|
||
([`pkg.go`, lines 633–647](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L633-L647),
|
||
[`pkg.go`, lines 863–907](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L863-L907)).
|
||
Resolution/package caches retain and reuse those complete values
|
||
([`pkg.go`, lines 833–842](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L833-L842),
|
||
[`pkg.go`, lines 909–985](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L909-L985),
|
||
[`pkg.go`, lines 1008–1029](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1008-L1029)).
|
||
- `go/build` reverse-resolves directory ownership through ordered roots and
|
||
performs the matching forward lookup while keeping directory, import path,
|
||
declared name, and source lists separate
|
||
([`build.go`, lines 612–665](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L612-L665),
|
||
[`build.go`, lines 725–767](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L725-L767),
|
||
[`build.go`, lines 436–493](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L436-L493)).
|
||
- Go interns an action by operation and package pointer, returns an existing
|
||
action for repeated compilation requests, and attaches only direct package
|
||
dependencies
|
||
([`action.go`, lines 202–206](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L202-L206),
|
||
[`action.go`, lines 437–447](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L437-L447),
|
||
[`action.go`, lines 628–658](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L628-L658)).
|
||
- Its `Action` keeps `Package`, `Objdir`, and `Target` as independent fields,
|
||
then assigns a short `bNNN/` object directory unrelated to import identity
|
||
([`action.go`, lines 84–109](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L84-L109),
|
||
[`action.go`, lines 383–394](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L383-L394)).
|
||
Direct dependency identities map independently to archive paths, and the
|
||
owner is compiled to fixed `_pkg_.a`
|
||
([`exec.go`, lines 864–884](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L864-L884),
|
||
[`exec.go`, lines 928–935](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L928-L935),
|
||
[`exec.go`, lines 1017–1033](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1017-L1033)).
|
||
- Linking reuses the compiled root action and separately expands the complete
|
||
reachable archive closure
|
||
([`action.go`, lines 919–957](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L919-L957),
|
||
[`action.go`, lines 1034–1068](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L1034-L1068),
|
||
[`exec.go`, lines 1592–1624](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1592-L1624),
|
||
[`exec.go`, lines 1635–1647](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1635-L1647)).
|
||
User `-o` separately controls the publication target
|
||
([`build.go`, lines 508–548](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L508-L548)).
|
||
- Go's test loader models ordinary production, internal production-plus-test,
|
||
external `_test`, and generated-main packages separately, reusing ordinary
|
||
production when possible
|
||
([`test.go`, lines 85–102](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L85-L102),
|
||
[`test.go`, lines 175–226](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L175-L226),
|
||
[`test.go`, lines 228–293](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L228-L293)).
|
||
|
||
### 11.14 Implemented dynamically sized command-global package universe
|
||
|
||
Package, dependency, resolution-context, selected-product, traversal, support,
|
||
order, and closure storage no longer has an arbitrary 256-element boundary.
|
||
This is a storage correction, not a new build abstraction: source imports still
|
||
form one command-global canonical package graph; each semantic package variant
|
||
still has one action; each compiler still receives exactly its direct exports;
|
||
and each executable linker still receives its complete reachable archive
|
||
closure.
|
||
|
||
The Cstage representation is exact and deliberately small:
|
||
|
||
- `sepgraph.pkg` is a dynamically allocated `struct seppkg *` with logical
|
||
count `n` and capacity `pkgcap`;
|
||
- `sepgraph.context` is a dynamically allocated `struct sepcontext *` with
|
||
logical count `ncontext` and capacity `contextcap`;
|
||
- each `seppkg.deps` is a dynamically allocated `int *` with `ndeps` and
|
||
`depcap`;
|
||
- each `seppkg.context_state` is a lazily extended, zero-filled
|
||
`unsigned char *` with `context_cap`;
|
||
- each package owns a dynamically grown import-occurrence vector recording
|
||
kind, source spelling, source file, line, column, and stable dependency
|
||
action index; its separate dependency vector remains sorted/deduplicated;
|
||
- parsed `sepproduct` values are a dynamically allocated vector, and each
|
||
product stores its support-action index directly; and
|
||
- package-load frames and topological-DFS frames are temporary dynamic vectors,
|
||
replacing recursion proportional to graph depth.
|
||
|
||
The WWstage representation is isomorphic. `sepgraph.pkg: []seppkg` and
|
||
`sepgraph.context: []sepcontext` use allocated slice length as capacity and keep
|
||
separate `n`/`ncontext` logical counts. Every `seppkg` owns a dynamically grown
|
||
`bindings: []sepbind`, a dynamically grown `deps: []i32` with `ndeps`, and a
|
||
lazily zero-extended `contextstate: []u8`.
|
||
Products, load frames, and topological frames use typed dynamically allocated
|
||
slices. `internal/wwpackage` continues to construct one union command for all
|
||
selected directory-test groups, but now checks the complete
|
||
`12 + 6*products + 2*includes` builder argument count before allocating or
|
||
starting the driver.
|
||
|
||
All graph and product growth starts at capacity 8 and doubles until it covers
|
||
the requested element count. Cstage clamps before `INT_MAX`, checks the element
|
||
count against `SIZE_MAX / sizeof(element)`, and publishes a `realloc` result only
|
||
after success. WWstage checks against the same signed 32-bit count boundary,
|
||
allocates a replacement typed slice, copies the live prefix, and publishes it
|
||
only after success. Context-state growth copies old bytes and explicitly zeros
|
||
the new tail. The shared deterministic failures are `ww: package graph is too
|
||
large` for an unrepresentable count and `ww: out of memory` for failed storage;
|
||
compiler and linker argument-count arithmetic is checked before allocation and
|
||
before any affected tool invocation. Each driver records allocation/size
|
||
failure during graph discovery and propagates it as a command-fatal load
|
||
result, rather than treating it as one product's semantic failure and starting
|
||
tools for a sibling root. The coordinator uses fallible dynamic storage for
|
||
discovered paths, source/folder/group/plan vectors, process handles, tool
|
||
environments, and complete builder/run argument vectors; it reports an
|
||
oversized product set or allocation failure before the corresponding
|
||
`exec.start` and cleans an already-created request temporary tree. Host pathname,
|
||
filesystem-component, process-argument, and available-memory boundaries remain
|
||
real host constraints; none is used as a disguised package-count maximum.
|
||
|
||
Vector growth never changes semantic references. Dependency edges, resolution
|
||
contexts, selected-product roots and variant roots, generated-main/support
|
||
edges, load/topological frames, order entries, and closure membership are all
|
||
stable `int`/`i32` indices. Code reserves a graph slot before taking an element
|
||
pointer and never carries an element pointer across a graph reserve. Capacity,
|
||
addresses, request order, product order, output names, and workdir location
|
||
therefore cannot enter action identity, sorting, diagnostics, storage locators,
|
||
or artifact bytes. Dependency lists retain byte-sorted insertion and duplicate
|
||
elimination. The iterative loader retains mark-before-child and post-child
|
||
command-import validation; the iterative tri-color DFS retains deterministic
|
||
postorder and the complete live path for cycle diagnostics.
|
||
|
||
No fixed package, product, context, action, support-map, traversal, order, or
|
||
closure cardinality remains in either driver. The unrelated `SEP_MAXLFLAGS ==
|
||
32` limit is retained solely for the existing `-L`/`-l` command-line interface;
|
||
it neither indexes nor bounds package actions. Compiler and linker tools already
|
||
allocate their import/input tables from `argc`; their genuine remaining process
|
||
boundary is the host's executable-argument limit.
|
||
|
||
The native package observers generate rather than commit large fixture trees.
|
||
The extended `long_shared_link_closure_is_complete` builds and runs a chain of
|
||
300 ordinary directory packages under independent cold Cstage and WWstage work
|
||
roots. Its command root directly imports all 300 packages and repeats one import,
|
||
proving an action beyond index 256 compiles, the root receives exactly 300
|
||
sorted/deduplicated direct `.wwi` inputs, every ordinary action receives only its
|
||
one direct export, every `.unit.ww` contains only its two byte-sorted owner
|
||
sources, and the linker receives the root plus all 300 archives exactly once and
|
||
no `.wwi`. A second command root imports only `p000`; it reuses all 300 ordinary
|
||
actions and its exact linker line still contains the root followed by the full
|
||
`p000` through `p299` transitive archive chain and runtime archive, proving that
|
||
closure construction—not the wide root's direct imports—crosses the old boundary.
|
||
The observer compares every unit, export, assembly, object, archive, and binary
|
||
across stages. A second equivalent request against the same persistent store invokes no compiler or
|
||
assembler. Changing `p257`'s export recompiles exactly `p257`, direct importer
|
||
`p256`, and the wide root, and stops before `p255` after `p256` regenerates a
|
||
byte-identical export. Closing the chain at `p299 -> p000` produces the complete
|
||
stage-identical 300-node cycle diagnostic with empty compiler, assembler, and
|
||
linker traces.
|
||
|
||
`dynamic_package_universe_crosses_former_boundary` first selects 309 package-test
|
||
products through 257 directory spellings in one direct request per stage.
|
||
Fifty-two real directories each produce ordinary production, internal
|
||
production-plus-test, external `_test`, and two generated-main actions; the
|
||
shared support closure brings that command-global universe to 270 actions. Two
|
||
hundred five explicit symlink spellings of `p000` add distinct valid products
|
||
and resolution contexts while reusing its one canonical internal action. The
|
||
observer reverses product and variant order between stages, proves one compile
|
||
per action, checks all 309 binaries exist, runs boundary products, validates
|
||
variant-owned units, and compares every action artifact plus representative
|
||
binaries byte-for-byte.
|
||
|
||
The same observer then exercises the public `ww test <tree>/...` coordinator
|
||
path. The generated tree has 257 real directories: the 52 test-bearing
|
||
directories above plus 205 production-only directories. The coordinator forms
|
||
and passes 309 products and 258 contexts (including support) in one driver
|
||
request. Reusing the direct request's persistent semantic-action store preserves all 270
|
||
existing actions and adds exactly 410 production/generated-main actions, for 680
|
||
distinct actions. Both stages run every product, produce byte-identical ordered
|
||
coordinator output and all 680 action artifacts, and perform no compilation on a
|
||
second equivalent public request. It checks all 104 test-bearing result labels
|
||
and all 205 no-test result labels, while a builder-boundary observer records
|
||
exactly one invocation containing all 309 descriptors for each cold and warm
|
||
public request. Existing focused observers continue to prove
|
||
dependency-first/root-first canonical reuse, root-only/combined artifact
|
||
identity, exact reordered-product trace bytes, and request-shape-independent
|
||
persistent action reuse.
|
||
|
||
This representation follows the semantic separation and scalable action
|
||
construction in the pinned official Go 1.26.5 source, identified by
|
||
[`VERSION`, lines 1–2](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/VERSION#L1-L2),
|
||
at commit `c19862e5f8415b4f24b189d065ed739517c548ba`:
|
||
|
||
- Go's loader states that repeated package lookup returns the same pointer
|
||
([`pkg.go`, lines 633–636](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L633-L636)),
|
||
resolves canonical path and directory before package-data lookup
|
||
([lines 863–911](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L863-L911)),
|
||
and reuses the package cached under the resolved `ImportPath`
|
||
([lines 757–768](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L757-L768)).
|
||
- A Go builder has one command-global action cache, while each action's
|
||
dependencies are a dynamically accumulated slice
|
||
([`action.go`, lines 38–45](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L38-L45),
|
||
[lines 84–89](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L84-L89)).
|
||
The cache key is operation plus canonical package pointer and returns the
|
||
existing action
|
||
([lines 202–206](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L202-L206),
|
||
[lines 437–447](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L437-L447)).
|
||
- Go constructs an archive compile action and dynamically appends actions only
|
||
for the package's direct imports
|
||
([lines 628–659](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L628-L659)).
|
||
It separately interns a link action rooted in that cached compile action
|
||
([lines 919–958](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L919-L958))
|
||
and dynamically expands the complete transitive link closure
|
||
([lines 1034–1068](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L1034-L1068)).
|
||
Requested package actions are likewise accumulated with `append`
|
||
([`build.go`, lines 519–534](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L519-L534),
|
||
[lines 551–558](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L551-L558)).
|
||
- During execution, Go builds one package from its own source list
|
||
([`exec.go`, lines 721–790](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L721-L790),
|
||
[lines 928–935](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L928-L935)),
|
||
maps its direct action dependencies into compiler inputs
|
||
([lines 864–884](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L864-L884)),
|
||
and links the root archive with mappings for the complete link-action closure
|
||
([lines 1592–1647](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/exec.go#L1592-L1647)).
|
||
- `go/build` keeps directory, import identity, declared name, and ordinary,
|
||
internal-test, and external-test file lists separate
|
||
([`build.go`, lines 436–493](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L436-L493));
|
||
`ImportDir` explicitly processes the named directory
|
||
([lines 521–525](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L521-L525)),
|
||
reads precisely that directory
|
||
([lines 859–900](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L859-L900)),
|
||
and assigns accepted files to the separate package-owned lists
|
||
([lines 948–1039](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L948-L1039)).
|
||
- Go's test loader explicitly returns generated main, internal production-plus-
|
||
test, and external-test packages, reusing production when valid
|
||
([`test.go`, lines 85–102](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L85-L102));
|
||
constructs the internal, external, and generated-main variants separately
|
||
([lines 175–293](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L175-L293));
|
||
dynamically appends, sorts, and deduplicates generated-main imports
|
||
([lines 315–376](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L315-L376));
|
||
and uses copy-on-write test variants while preserving unaffected package
|
||
objects
|
||
([lines 421–474](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L421-L474)).
|
||
|
||
WW adopts those package/action distinctions and scalable dependency
|
||
accumulation, but not Go's build IDs, module system, importcfg, cache/CAS,
|
||
preloader, parallel action scheduler, or network behavior. Normal local WW
|
||
builds and tests remain offline, manifest-free, registry-free, database-free,
|
||
CAS-free, and network-free.
|
||
|
||
### 11.15 Implemented internal-package import visibility
|
||
|
||
A source import whose complete canonical import path contains a directory
|
||
component named exactly `internal` is now contextual. Locate the final such
|
||
component. Its parent directory is the ownership boundary, and the import is
|
||
legal only when the importing source package's canonical filesystem directory
|
||
is that boundary or a descendant at a real path-component boundary. Thus
|
||
`domain.client` may import `domain.internal.secret`, while `outsider` and
|
||
`domainx.client` may not. `internalx` has no special meaning. For
|
||
`domain.internal.outer.internal.deep`, the final `internal` wins and the owner
|
||
is `domain.internal.outer`, not `domain`.
|
||
|
||
The complete dotted import identity determines whether the rule applies and
|
||
how many components comprise the final `internal` plus its following suffix.
|
||
WW removes those components from that source edge's resolved lexical target
|
||
route, canonicalizes the resulting owner directory physically, and compares it
|
||
with the already canonical physical importer directory. Stripping precedes
|
||
physical canonicalization because a symlink at or below `internal` may point to
|
||
a target with a different name or depth; an action spelling interned by an
|
||
earlier edge is never reused for this contextual calculation. Cstage obtains
|
||
directory canonicalization through `realpath`; WWstage uses its equivalent
|
||
`chdir`/`getcwd` canonicalization. For explicit single-file compatibility
|
||
roots, Cstage uses `realpath` and WWstage's component walker uses
|
||
`lstat`/`readlink` plus the same canonical current-directory representation;
|
||
both preserve component and trailing-directory semantics across symlinks. The
|
||
equality-or-`/`-boundary comparison never uses a raw string prefix. This
|
||
division is intentional: an unrelated physical
|
||
ancestor literally named `internal` does not impose visibility on an import
|
||
whose canonical identity has no such component, while a symlink spelling at or
|
||
below an import-path `internal` cannot move the effective owner or importer.
|
||
Explicit single-file compatibility roots use their canonicalized containing
|
||
directory as the importer context. Directly selecting an internal directory as
|
||
a command root remains legal because selection is not a source import.
|
||
|
||
Visibility is an import-edge property, not part of canonical package or action
|
||
identity. Both drivers resolve the complete target, intern or reuse its one
|
||
canonical production action, and then perform the importer-context check before
|
||
accepting the source binding or dependency edge. Consequently an allowed
|
||
importer may load and compile the target normally, but that cached action cannot
|
||
authorize a later forbidden importer. Reversing requested products or visiting
|
||
the forbidden importer first produces the same result. No declared package
|
||
name, leaf, output name, product ordinal, storage locator, hash, or test variant
|
||
participates in the decision. Legal edges therefore retain the existing
|
||
owner-only byte-sorted unit, exact sorted/deduplicated direct `.wwi` compiler
|
||
inputs, deterministic archive, and complete archive-only linker closure.
|
||
|
||
Cstage represents the rule with the bounded
|
||
`sep_internal_parent_count`/`sep_internal_import_allowed` helpers in
|
||
`cmd/ww/main.c`. WWstage has the isomorphic
|
||
`sepinternalparentcount`/`sepinternalimportallowed` helpers in
|
||
`selfhost/cmd/ww/main.ww`. Neither adds a package field, fixed-size package
|
||
table, second action universe, or action-key input. A distinct
|
||
`SEP_LOAD_INTERNAL` result propagates through the iterative loader. On rejection
|
||
both stages emit the importing parser position followed by exactly:
|
||
|
||
```text
|
||
use of internal package <canonical-import-path> not allowed
|
||
```
|
||
|
||
The command returns immediately from graph loading, before directory-identity
|
||
finalization, unit composition, workdir owner validation, stale-voucher
|
||
invalidation, tool-identity recording, compiler, assembler, in-driver archive
|
||
production, linker, publication, or execution. A cold rejection therefore
|
||
commits no target/importer artifact or `.wwtool.*` state. A forbidden warm
|
||
request cannot use a previously committed target to bypass the check and leaves
|
||
the already committed target voucher and all tool records byte-unchanged.
|
||
Allocation or path canonicalization failure remains a deterministic
|
||
command-fatal pre-tool error; the visibility helper does not change stable
|
||
package indices or growth behavior.
|
||
|
||
Every selected source file reaches the same driver scan. The rule therefore
|
||
applies to ordinary production sources, same-package test sources in the
|
||
internal production-plus-test variant, external `_test` sources, and real
|
||
source imports inside the test-support package. Generated-main-to-variant,
|
||
generated-main-to-support, and coordinator product wiring remain synthetic
|
||
edges and are not retroactively treated as source imports. No coordinator
|
||
change is required: `cmd/wwtest` only dispatches, and
|
||
`internal/wwpackage/package.ww` only discovers and classifies source groups,
|
||
constructs the union request, and runs the driver; neither resolves a source
|
||
import or owns its importing position.
|
||
|
||
This follows only the pinned official Go 1.26.5 source at commit
|
||
`c19862e5f8415b4f24b189d065ed739517c548ba`:
|
||
|
||
- `loadImport` resolves and reuses the canonical package first, then explicitly
|
||
checks the rule on every import because the result depends on the importing
|
||
code, and attaches the importing position
|
||
([`pkg.go`, lines 787–791](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L787-L791)).
|
||
- The rule is the tree rooted at the parent of the target's `internal`
|
||
directory
|
||
([`pkg.go`, lines 1463–1471](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1463-L1471)).
|
||
A package named directly on the command line is not an import
|
||
([lines 1498–1502](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1498-L1502)),
|
||
and the import-path boundary is located before filesystem containment
|
||
([lines 1505–1515](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1505-L1515)).
|
||
- Go's filesystem branch cleans the importer and owner, requires a
|
||
path-component-aware prefix, and retries with both paths symlink-expanded
|
||
([`pkg.go`, lines 1534–1546](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1534-L1546)).
|
||
Its rejection text is exactly the diagnostic above
|
||
([lines 1564–1571](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1564-L1571)),
|
||
and its exact-component search deliberately selects the final `internal`
|
||
element as the most restrictive rule
|
||
([lines 1574–1590](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1574-L1590)).
|
||
- Same-package and external test imports both pass through ordinary
|
||
`loadImport` with their own source positions before variant construction
|
||
([`test.go`, lines 102–161](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L102-L161)).
|
||
Generated test-main dependencies are synthesized separately
|
||
([lines 307–330](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L307-L330)).
|
||
- Go's downstream action cache keys canonical package actions independently
|
||
and consumes the already validated direct package imports, so visibility does
|
||
not belong in action identity
|
||
([`action.go`, lines 437–455](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L437-L455),
|
||
[lines 628–657](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L628-L657)).
|
||
|
||
The focused native observer `internal_package_import_visibility` generates its
|
||
lexical package/import tree under a temporary physical ancestor also named
|
||
`internal`, while deliberate symlink destinations live outside that ancestor,
|
||
so that unrelated-host-path case is exercised rather than documented only. It
|
||
builds and runs allowed descendant, nested-final, `internalx`, and symlinked
|
||
physical-owner cases; rejects outsider, sibling-prefix, nested-final, and
|
||
symlink-escape cases at exact source positions with empty tool traces; maps an
|
||
internal target symlinked to a differently shaped physical path back to its
|
||
lexical owner's canonical directory; builds an internal package directly; and
|
||
reverses requested allowed/forbidden product-descriptor order around one reused
|
||
target. It checks ordinary, internal-test, and external-test
|
||
source imports, generated-main transitive-export isolation, owner-only source
|
||
order, direct export inputs, archive-only link closure, runtime results, warm
|
||
no-op package production, rejection-state preservation, and the primary
|
||
production chain's unit, export, assembly, object, archive, binary, and exact
|
||
normalized tool arguments across independent cold Cstage and WWstage work
|
||
roots. Test variants additionally compare their unit/export/archive bytes and
|
||
exercise byte-equivalent runtime output in both stages.
|
||
|
||
### 11.16 Implemented manifest-free local vendor-directory imports
|
||
|
||
The local package loader now expands only imports parsed from source. For a
|
||
source package whose resolved lexical route is `<root>/domain/app`, an import
|
||
of `lib.math` probes these directory packages in order:
|
||
|
||
```text
|
||
<root>/domain/app/vendor/lib/math
|
||
<root>/domain/vendor/lib/math
|
||
<root>/vendor/lib/math
|
||
<ordinary ordered-root lookup for lib/math>
|
||
```
|
||
|
||
The walk stops at that edge's applicable active source root. It never walks an
|
||
arbitrary filesystem ancestor and never acquires a boundary from another
|
||
requested product. Each resolution context carries the current package's
|
||
lexical route and source-root boundary. An explicitly identified root derives
|
||
the boundary by removing and round-trip validating exactly its dotted identity
|
||
components. An unbound literal root chooses the first precedence-valid strict
|
||
ancestor in its own ordered search roots, or the selected directory itself.
|
||
An ordinary child records the exact root that selected it; a vendored child
|
||
inherits the parent's boundary. These values are contextual resolution state,
|
||
not package/action identity.
|
||
|
||
Before source scanning, a literal directory whose lexical route is
|
||
representable below that boundary binds the complete relative dotted identity,
|
||
including any `vendor` components. A truly rootless literal keeps its reversible
|
||
`__wwlocal` identity and cannot be coalesced with a same-physical vendored
|
||
action. Thus selecting a vendored directory directly remains legal without
|
||
letting product order donate its action identity to or from a source import.
|
||
|
||
A candidate shadows outer and ordinary candidates only when its directory
|
||
contains an observed non-directory name ending in `.ww`. This deliberately
|
||
includes `_test.ww` and the bare name `.ww`, matching Go's suffix probe; an
|
||
actual subdirectory named `x.ww` and an unreadable candidate with no observed
|
||
source do not shadow. Once a candidate is selected, normal package enumeration
|
||
reports its real errors, including a test-only or otherwise production-empty
|
||
directory, rather than falling through.
|
||
|
||
Source retains only the effective spelling, such as `lib.math`. Selection
|
||
assigns the target the complete expanded canonical identity, such as
|
||
`domain.app.vendor.lib.math`, and separately canonicalizes its physical
|
||
directory. The target's declared name independently supplies the default
|
||
qualifier in each importing source file. The action key is that expanded
|
||
identity and canonical directory,
|
||
plus the existing variant/role. Different physical vendor copies are distinct;
|
||
different expanded vendor routes remain distinct even when symlinks converge
|
||
on one physical directory; repeated resolutions of the same pair reuse one
|
||
action. Source spelling, importer context, product order, output name, declared
|
||
leaf, and allocation/discovery order do not enter that key.
|
||
|
||
The final exact non-terminal dotted component named `vendor` determines the
|
||
effective suffix and owner. A nested path uses its final `vendor`; `vendorx` is
|
||
ordinary; and a path ending exactly in `vendor` names an ordinary package.
|
||
After resolution and action interning, every source edge first performs vendor
|
||
visibility and then verifies source spelling. An allowed importer that directly
|
||
spells an expanded path receives the source-position diagnostic:
|
||
|
||
```text
|
||
<expanded-path> must be imported as <effective-suffix>
|
||
```
|
||
|
||
An outside importer receives, with visibility taking diagnostic precedence:
|
||
|
||
```text
|
||
use of vendored package not allowed
|
||
```
|
||
|
||
Directly selecting a directory below `vendor` remains legal because a command
|
||
root is not a source import.
|
||
|
||
Visibility derives the owner from the current edge's resolved lexical vendor
|
||
route and only then canonicalizes that owner physically. It never strips
|
||
components from the already-canonical target, whose symlink shape may have a
|
||
different depth. The importer is its canonical physical directory. Equality or
|
||
a real `/`-component descendant is allowed; a raw string prefix is not.
|
||
Consequently an importer reached through a symlink is judged by its physical
|
||
containment, while a vendor target symlinked to a differently shaped physical
|
||
directory keeps the owner established by the lexical vendor route. The check
|
||
runs after intern/reuse on every source edge, so an action loaded by an allowed
|
||
importer cannot authorize a later forbidden spelling or importer.
|
||
|
||
Cstage represents the resolution with `route` and `source_root` in
|
||
`sepcontext`, the bounded `sep_resolve_source_import` and vendor helpers, stable
|
||
integer action references in typed source bindings, and transient
|
||
`{package,context}` loader children. WWstage uses the isomorphic `sepcontext`,
|
||
`sepresolvesourceimport`, typed bindings, and transient child vector. The
|
||
package-global dependency set remains the sorted/deduplicated canonical action
|
||
set used for compilation and linking; contextual child traversal never creates
|
||
a second universe or contaminates action identity. All new storage grows with
|
||
checked allocation, and package/context vectors continue to expose only stable
|
||
integer references across growth.
|
||
|
||
A compile still receives one sorted, deduplicated
|
||
`--import <expanded-path> <dependency.wwi>` triple for each direct dependency.
|
||
When source spelling differs, the driver additionally supplies the sorted,
|
||
unique auxiliary mapping:
|
||
|
||
```text
|
||
--import-map <source-spelling> <expanded-path>
|
||
```
|
||
|
||
Both compilers require the map target to be an existing direct `--import`,
|
||
require source keys to be sorted and unique, and require a matching import in
|
||
the primary source input. No leaf-equality condition exists. After parsing the
|
||
primary input and before prepending imported interfaces, the compiler replaces
|
||
its semantic import key while preserving source position and spelling. It then
|
||
reads the expanded target's declared name from the direct `.wwi` and installs
|
||
that name only in the declaring source file. Thus the map adds no dependency or
|
||
export input, expanded
|
||
identity flows into self-contained `.wwi` ownership and symbols, and direct
|
||
exports still have no transitive leakage. Linking remains independent: each
|
||
executable consumes its root archive and complete reachable archive closure,
|
||
never `.wwi` or import-map inputs.
|
||
|
||
Persistent units record resolution identity in ignored, deterministic comments:
|
||
|
||
```text
|
||
//ww:vendor-dir <hex-canonical-package-directory>
|
||
//ww:import-map <source-spelling> <expanded-path> <hex-canonical-target-directory>
|
||
```
|
||
|
||
Hex encoding keeps arbitrary legal filesystem bytes inside one comment. The
|
||
metadata makes ordinary-to-vendor changes and vendor symlink retargeting
|
||
invalidate the importer even when its source bytes and both already-warm export
|
||
bytes happen to match. It contains no dependency body and does not alter source
|
||
positions. The current workdir formats are build 14 and test 13. An equivalent
|
||
warm request remains a package-production no-op; an export change propagates
|
||
only through ordinary direct-export comparison.
|
||
|
||
Production, same-package internal-test, external `_test`, and real source
|
||
imports inside the test-support package all use this source-edge resolver.
|
||
Generated-main-to-variant, generated-main-to-support, and coordinator product
|
||
edges remain synthetic and receive no map or retroactive source legality.
|
||
`cmd/wwtest` remains a dispatcher. `internal/wwpackage` still discovers and
|
||
classifies source groups and submits one command-global union; it performs no
|
||
vendor resolution. It creates only its removable command-temporary coordination
|
||
tree. The private driver creates a missing delegated `-w` directory and a
|
||
requested output directory only after all semantic graph and output preflight
|
||
succeeds; a rejected request therefore leaves neither directory behind.
|
||
|
||
Resolution, identity collision checks, contextual legality, dependency-failure
|
||
propagation, cycles, command kind, publication paths, and action closures all
|
||
finish before cold scratch acquisition, product status removal, stale-voucher
|
||
invalidation, tool identity records, producer execution, publication, or
|
||
runtime execution. A request whose every product is already invalid returns
|
||
without acquiring scratch; independent valid sibling products retain the
|
||
existing isolated-build behavior. A forbidden warm request leaves committed
|
||
vouchers, status files, tool records, and products byte-unchanged.
|
||
|
||
This is only manifest-free local source-tree behavior. It does not implement
|
||
Go modules, module vendor mode, `go.mod`, `vendor/modules.txt`, importcfg, build
|
||
IDs, a package database, a CAS, registry access, or network lookup, and it is
|
||
separate from the future locked vendor store in section 4.6.
|
||
|
||
The rule follows the pinned official Go 1.26.5 source at commit
|
||
`c19862e5f8415b4f24b189d065ed739517c548ba`:
|
||
|
||
- source-derived imports are expanded and the expanded vendor path becomes the
|
||
canonical import path
|
||
([`pkg.go`, lines 658–668](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L658-L668));
|
||
- resolution and canonical package-cache reuse occur before the contextual
|
||
internal/vendor checks, which still run with the importing position on every
|
||
source edge
|
||
([`pkg.go`, lines 722–796](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L722-L796));
|
||
- vendor lookup walks importer ancestors nearest-first to the applicable root,
|
||
requires a source-bearing directory, and records the expanded identity
|
||
([`pkg.go`, lines 1213–1263](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1213-L1263));
|
||
- the source-bearing probe accepts any non-directory `.go` suffix and ignores
|
||
directory-read failure
|
||
([`pkg.go`, lines 1418–1429](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1418-L1429));
|
||
- command roots remain legal, while an expanded source spelling must use its
|
||
effective suffix
|
||
([`pkg.go`, lines 1593–1617](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1593-L1617));
|
||
- vendor ownership uses component-aware physical containment and symlink
|
||
expansion
|
||
([`pkg.go`, lines 1620–1667](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1620-L1667));
|
||
- the final exact non-terminal `vendor` component controls the rule
|
||
([`pkg.go`, lines 1670–1688](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L1670-L1688));
|
||
- production, internal-test, and external-test imports retain distinct raw
|
||
source spellings/positions but each resolves canonically, while generated
|
||
test-main wiring is synthetic
|
||
([`test.go`, lines 85–173](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L85-L173),
|
||
[`test.go`, lines 175–266](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L175-L266),
|
||
[`test.go`, lines 272–373](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L272-L373),
|
||
[`go/build/build.go`, lines 415–493](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L415-L493)); and
|
||
- action caching consumes the already-resolved package graph, while output
|
||
publication remains a later concern
|
||
([`action.go`, lines 437–455](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L437-L455),
|
||
[`action.go`, lines 628–659](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L628-L659),
|
||
[`build.go`, lines 470–558](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L470-L558)).
|
||
|
||
The native observer `vendor_directory_import_resolution` generates every source
|
||
tree temporarily and runs both stages from independent cold work roots. It
|
||
proves nearest/outer/root/ordinary selection and the active-root boundary;
|
||
source-bearing versus empty candidates; distinct and reused actions, including
|
||
same-physical symlink targets; exact spelling/visibility diagnostics and
|
||
product-order reversal; final-component, terminal-name, component-prefix, and
|
||
symlink behavior; direct vendored roots; production/internal/external/support
|
||
source imports and synthetic-main isolation; owner-only sorted units, exact
|
||
direct exports/import maps, archive-only link closures, runtime output,
|
||
stage-equal normalized argv/artifacts/binaries, warm no-op production, and cold
|
||
and warm rejection-state preservation.
|
||
|
||
### 11.17 Implemented manifest-free recursive local package-pattern selection
|
||
|
||
Build and test now share one local request selector. A positional spelling with
|
||
no `...` is one explicit directory root. In a spelling containing `...`, each
|
||
occurrence in a valid UTF-8 spelling has Go's regular-expression wildcard
|
||
semantics; an invalid UTF-8 pattern matches nothing, and a final `/...` also
|
||
matches the directory before that suffix. WW applies the local matcher to its
|
||
existing manifest-free `DIR/...` interface as well as `./...`, `../...`, and
|
||
absolute spellings. It does not interpret the non-filesystem portion as a
|
||
module or registry path. Multiple direct roots and patterns may be mixed in one
|
||
command.
|
||
For build, `--` before the first positional ends option parsing and every
|
||
following argument is a package selector, including a spelling that begins
|
||
with `-`. Once the first positional has already ended flag parsing, a later
|
||
`--` is itself another package argument.
|
||
|
||
Pattern expansion is request processing only. For each pattern, traversal starts
|
||
at the directory prefix before the first `...` and is bounded to that physical
|
||
tree. The selector never obtains a traversal or source-root boundary from
|
||
another product. It produces canonical physical directory roots; only imports
|
||
parsed from their real source files add dependency edges. The request spelling,
|
||
wildcard prefix, match membership, output name, request order, and discovery
|
||
order never become a package import identity or action key and never enter an
|
||
import search path.
|
||
|
||
The recursive eligibility rules are:
|
||
|
||
- Directory entries are read completely and byte-sorted before processing.
|
||
Every recursively encountered directory whose basename begins `.` or `_`, or
|
||
equals `testdata`, is pruned with its subtree. An explicit literal selection
|
||
bypasses these traversal exclusions, so those directories remain legal direct
|
||
roots.
|
||
- A directory owns only its immediate `.ww` directory entries. Basenames
|
||
beginning `.` or `_` are ignored. As in `go/build`, a source symlink whose
|
||
target is a regular file is read under the symlink entry's byte-sorted name;
|
||
a source-shaped symlink to a directory is ignored. A subdirectory never
|
||
donates sources to its parent.
|
||
- A directory with at least one production or `*_test.ww` source is eligible.
|
||
A recursively encountered source-empty directory is silently skipped. A
|
||
direct source-empty root is an error. A malformed source-bearing directory is
|
||
retained as a root and fails during ordinary package-clause or driver loading;
|
||
malformed files in excluded or source-empty trees do not poison the request.
|
||
- The traversal does not prune `vendor`. Instead a wildcard cannot consume a
|
||
non-terminal exact path component named `vendor`. Thus `DIR/...` may select a
|
||
code-bearing terminal `DIR/vendor` but not `DIR/vendor/x`.
|
||
`DIR/vendor/...` explicitly selects that vendor root and its descendants until
|
||
another non-terminal `vendor` becomes a barrier. `vendorx` is ordinary.
|
||
|
||
The explicit traversal root is opened after following a directory symlink, as
|
||
in Go. It may therefore name a target outside the lexical spelling, but the
|
||
target becomes the canonical traversal boundary. Directory symlinks encountered
|
||
below that root are never followed, so they cannot escape, create cycles, or
|
||
change selection. A cyclic explicit root is rejected while canonicalizing it.
|
||
Relative, absolute, dotted, and explicit-root-symlink spellings that reach one
|
||
physical package collapse to one canonical root. This physical interning is
|
||
WW's stronger command-global identity rule; it deliberately avoids Go's few
|
||
GOPATH cases in which different lexical import paths can retain distinct package
|
||
objects.
|
||
|
||
Raw requested spellings are byte-sorted first and each is then lexically cleaned
|
||
before traversal. Matched source paths are canonicalized, sorted by canonical
|
||
directory and filename, and deduplicated. Products are then byte-sorted by
|
||
canonical directory and variant. Duplicate patterns, overlapping patterns, and
|
||
canonical aliases therefore select one root/product and reuse one action. Go
|
||
itself processes patterns in argv order and suppresses later package objects; WW
|
||
performs the stronger final canonical sort required by its
|
||
request-order-independent command universe. Reversing request or product order
|
||
does not change roots, diagnostics, normalized tool arguments, artifacts, or
|
||
runtime output.
|
||
|
||
An unmatched pattern emits:
|
||
|
||
```text
|
||
ww: warning: "PATTERN" matched no packages
|
||
```
|
||
|
||
`PATTERN` is quoted with the pinned `strconv.Quote` rules, including
|
||
deterministic escapes for quotes, backslashes, controls, non-printing Unicode,
|
||
and malformed UTF-8 bytes. Warnings are emitted in the
|
||
sorted request order. With no remaining roots,
|
||
ordinary build without `-o` succeeds as an empty build, while test reports
|
||
`ww test: no packages to test`. A build with a non-directory `-o` reports no
|
||
packages to build; a directory `-o` reports no main packages to build. A
|
||
non-directory `-o` still requires exactly one production root. An existing
|
||
directory or spelling ending in `/` receives each selected command under its
|
||
canonical directory basename; non-main selected roots receive no named output.
|
||
Two commands with the same destination basename are rejected rather than
|
||
overwriting one another. All selection, canonicalization, package-clause,
|
||
duplicate-output, and unusable-request diagnostics precede producer execution.
|
||
Raw `-o` and `-w` spellings and every derived directory-fan-out output, cold
|
||
scratch name, persistent tool-record name, and package artifact are bounded and
|
||
validated symmetrically before tools; a raw spelling that fits but whose suffix
|
||
or command basename does not fit is rejected with the same Cstage/WWstage
|
||
diagnostic and no filesystem publication.
|
||
For a delegated multi-root or recursive `-S` build, `-w` is required so the
|
||
assembly outputs have caller-owned persistent destinations instead of vanishing
|
||
with the coordinator's temporary plan.
|
||
|
||
Build and test begin with the same eligible canonical directory set. Recursive
|
||
build removes a source-bearing root that has only test files; an explicitly
|
||
selected test-only directory remains an unusable build root and fails. Build
|
||
creates one production product per remaining directory. Test retains test-only
|
||
directories and, after selection, constructs the already specified isolated
|
||
production/no-test, internal production-plus-test, external `_test`, support,
|
||
and generated-main actions. Pattern expansion does not create those variants
|
||
and cannot make one variant visible to another.
|
||
|
||
Vendor selection remains distinct from vendor import resolution. Selecting a
|
||
directory below `vendor`, literally or through an explicitly vendor-rooted
|
||
pattern, keeps its complete canonical local identity; it is never shortened to
|
||
the suffix after `vendor`, and its declaration never renames it. A generic
|
||
recursive pattern does not expose vendored
|
||
descendants as ordinary short command roots. Independently, an allowed real
|
||
source import still searches nearest-first below local `vendor`, creates the
|
||
expanded identity described in section 11.16, supplies the required
|
||
`--import-map`, and performs spelling and visibility checks for that importer
|
||
even when the canonical action already exists.
|
||
|
||
The ownership split is exact:
|
||
|
||
- `cmd/ww/main.c` and `selfhost/cmd/ww/main.ww` recognize recursive and
|
||
multi-root build/test requests symmetrically and delegate them. Their private
|
||
build route consumes production product descriptors in one command-global
|
||
package universe. It records library-root completion without linking and
|
||
links each command product independently. Recursive build forwards `-S`,
|
||
`-L`, and `-l` through the same private product route; assembly-only products
|
||
receive completion markers only after their producer pass succeeds. Both
|
||
stages allocate the same bounded delegation argv and inherit the existing
|
||
environment directly, so delegation adds no WWstage-only environment-copy
|
||
allocation or failure point.
|
||
- `internal/wwpackage/package.ww` owns pattern cleaning and matching, bounded
|
||
directory traversal, source-bearing eligibility, canonical root/product
|
||
sorting and deduplication, build-versus-test product classification, output
|
||
coordination, and deterministic reporting. It does not resolve a source
|
||
import and does not prepend a pattern root to `-I`.
|
||
- `cmd/wwtest` remains a dispatcher. The Cstage and WWstage drivers own package
|
||
enumeration, canonical identity, contextual import resolution, graph loading,
|
||
variants, compilation, archive construction, linking, publication, and
|
||
persistent reuse.
|
||
|
||
Consequently the existing action and tool contracts remain unchanged after
|
||
root selection. Every action unit contains only the owner's byte-sorted source
|
||
files. A compiler receives exactly the sorted, deduplicated `.wwi` exports of
|
||
direct source dependencies and any required vendor import-map binding. Each
|
||
executable link receives its root archive and complete reachable archive
|
||
closure, never a `.wwi`. Canonical duplicate roots reuse the same action; the
|
||
pattern text, declared qualifier, and product receiving an output do not affect
|
||
canonical or persistent action ownership.
|
||
|
||
Selection and package-clause validation finish before the coordinator creates
|
||
its removable temporary plan, and the coordinator never creates persistent
|
||
state. Driver graph, identity, visibility, cycle, closure, and output validation
|
||
finish before a missing delegated work or output directory is created and
|
||
before scratch, voucher, status, tool-state, publication, or producer mutation.
|
||
If later setup of another requested directory fails, both drivers remove every
|
||
empty path prefix created by that setup while preserving all pre-existing caller
|
||
directories. A cold rejected pattern leaves no partial request state. The same
|
||
rejection against an existing caller work root leaves its marker and every
|
||
committed artifact, voucher, and tool record byte-unchanged and invokes no
|
||
producer. A valid equivalent direct or recursive warm request remains a
|
||
package-production no-op; changed exports continue to propagate through direct
|
||
dependencies only.
|
||
|
||
This slice follows only official Go 1.26.5 source at commit
|
||
`c19862e5f8415b4f24b189d065ed739517c548ba`:
|
||
|
||
- local literal versus wildcard handling, the prefix before the first `...`,
|
||
explicit root-symlink following, recursive exclusions, source-empty omission,
|
||
and malformed-directory retention are in
|
||
[`cmd/go/internal/search/search.go`, lines 276–418](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/search/search.go#L276-L418),
|
||
and its quoted unmatched-pattern warning is at
|
||
[lines 424–429](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/search/search.go#L424-L429);
|
||
- wildcard syntax, the empty match for trailing `/...`, and the non-terminal
|
||
`vendor` barrier are in
|
||
[`cmd/internal/pkgpattern/pkgpattern.go`, lines 32–106](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/internal/pkgpattern/pkgpattern.go#L32-L106),
|
||
including invalid-UTF-8 rejection at lines 75–76,
|
||
with non-terminal vendor-element replacement implemented at
|
||
[lines 125–137](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/internal/pkgpattern/pkgpattern.go#L125-L137);
|
||
- manifest-free request expansion, canonical package loading, first-occurrence
|
||
deduplication, and pattern membership are in
|
||
[`cmd/go/internal/load/pkg.go`, lines 2922–2965](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L2922-L2965),
|
||
while source imports alone recurse through package loading at
|
||
[lines 2024–2047](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L2024-L2047);
|
||
- build uses that common matcher, implements single versus directory `-o`, and
|
||
omits wildcard-selected test-only roots at
|
||
[`cmd/go/internal/work/build.go`, lines 459–559](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L459-L559)
|
||
and
|
||
[lines 731–745](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/build.go#L731-L745);
|
||
- test uses the same package request set, reports an empty set, and constructs
|
||
isolated test variants only afterward at
|
||
[`cmd/go/internal/test/test.go`, lines 684–719](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L684-L719)
|
||
and
|
||
[lines 1133–1226](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L1133-L1226);
|
||
- immediate directory ownership and hidden/underscore source-name exclusion
|
||
are in
|
||
[`go/build/build.go`, lines 859–914](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L859-L914)
|
||
(including regular-file source symlink following and symlink-to-directory
|
||
omission at lines 886–900),
|
||
parse-error retention and production/test classification are in
|
||
[lines 931–1036](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L931-L1036),
|
||
and `NoGoError` eligibility is in
|
||
[lines 1076–1082](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L1076-L1082); and
|
||
- child symlinks are skipped by `Lstat`-based directory walking and entries are
|
||
traversed in byte order at
|
||
[`cmd/go/internal/fsys/walk.go`, lines 14–59](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/fsys/walk.go#L14-L59)
|
||
and
|
||
[`os/dir.go`, lines 109–125](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/os/dir.go#L109-L125).
|
||
|
||
Build subcommand option termination is delegated by the Go command at
|
||
[`cmd/go/main.go`, lines 312–321](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/main.go#L312-L321),
|
||
with `--` termination implemented by
|
||
[`flag/flag.go`, lines 1074–1089](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/flag/flag.go#L1074-L1089),
|
||
and the parse loop stops at the first positional at
|
||
[lines 1153–1176](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/flag/flag.go#L1153-L1176).
|
||
|
||
Unmatched-pattern quoting uses the pinned `strconv.Quote` decoder and escape
|
||
rules in
|
||
[`strconv/quote.go`, lines 28–123](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/strconv/quote.go#L28-L123),
|
||
including the exported `Quote` entry at lines 117–123, and the `IsPrint`
|
||
algorithm at
|
||
[`strconv/quote.go`, lines 515–559](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/strconv/quote.go#L515-L559),
|
||
with its generated tables in
|
||
[`strconv/isprint.go`, lines 8–733](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/strconv/isprint.go#L8-L733).
|
||
|
||
No module cutoff, `go.mod`, module vendor mode, `vendor/modules.txt`, importcfg,
|
||
build ID, registry, database, CAS, or network behavior is copied.
|
||
|
||
The native observer `recursive_tree_discovery` generates every tree
|
||
temporarily and runs Cstage and WWstage from independent cold roots. It covers
|
||
ordinary and test-only roots, hidden/underscore/testdata exclusions and direct
|
||
exceptions, terminal and explicit vendor patterns, an imported expanded vendor
|
||
dependency at both package-local and ancestor vendor directories,
|
||
overlapping/reversed/duplicate patterns independently in each stage, root,
|
||
child-directory, regular-file, and directory-target source symlinks,
|
||
canonical aliases, source-empty and malformed directories, build/test variant
|
||
selection with an asserted common production-root set, middle-position and
|
||
valid-UTF-8 wildcard edge cases, byte-sorted quoted unmatched diagnostics,
|
||
multi-command directory output and duplicate-destination rejection, recursive
|
||
assembly/link-flag forwarding, build `--` termination, raw-versus-derived
|
||
output/scratch/work-path boundaries with zero producer calls, owner-only units,
|
||
exact direct exports and import maps, archive-only links, normalized stage-equal tool argv and
|
||
artifacts/binaries/output, warm no-op package production, and cold/warm pre-tool
|
||
rejection-state preservation. The existing
|
||
`vendor_directory_import_resolution` observer supplies the exact compiler
|
||
`--import-map` argv proof for imported vendor dependencies, and the existing
|
||
diamond and long-closure observers independently prove sorted/deduplicated
|
||
direct `.wwi` cardinality and archive-only reachable link closures.
|
||
|
||
### 11.18 Implemented canonical identity, declared-name, and file-import-scope slice
|
||
|
||
Directory-package identity and source naming are now independent throughout
|
||
local build and test. For example:
|
||
|
||
```ww
|
||
// canonical import identity: acme.codec
|
||
package wire;
|
||
```
|
||
|
||
is imported with the existing dotted syntax:
|
||
|
||
```ww
|
||
package main;
|
||
import acme.codec;
|
||
export fn main() i32 = { return wire.value(); };
|
||
```
|
||
|
||
The package action, exports, symbols, archives, dependency edges, persistent
|
||
storage ownership, and link closure remain owned by `acme.codec`. Only the
|
||
source-file binding is named `wire`. The path leaf `codec` is not installed as
|
||
another qualifier, and a sibling source file receives no `wire` binding unless
|
||
that file has its own import.
|
||
|
||
The implemented representation keeps five facts distinct:
|
||
|
||
1. source import spelling, including its source file, line, and column;
|
||
2. contextually expanded canonical import identity;
|
||
3. canonical physical directory;
|
||
4. the one declared package name read from eligible source clauses and `.wwi`
|
||
package markers; and
|
||
5. the source-file-local default qualifier binding from that declaration to
|
||
the canonical target.
|
||
|
||
Both drivers retain one dynamically allocated `sepbind` occurrence for every
|
||
real import site. The occurrence stores the source spelling and position plus a
|
||
stable action index. Contextual `internal` and vendor resolution is therefore
|
||
rechecked for every source import even if its target action already exists. A
|
||
separate package dependency vector unions those occurrences, deduplicates by
|
||
canonical action, and sorts by complete canonical identity/variant/role. That
|
||
vector alone supplies dependency traversal and compiler `--import` arguments.
|
||
Patterns, qualifiers, declared names, export closure facts, and generated edges
|
||
never create source dependency edges.
|
||
|
||
The owner-only composed unit preserves byte-sorted source boundaries with one
|
||
`//ww:module-reset <canonical-owner>` separator per file. Parser nodes carry a
|
||
source-section ID as well as canonical owner and declared package name. Each
|
||
real `N_USE` therefore belongs to one source section. `.wwi` emission repeats
|
||
canonical-owner/package markers as needed for contributing source sections and
|
||
retains imports only with the declarations from the file that owned them.
|
||
Interfaces remain source-like transitional data, but canonical owner and
|
||
declared name are no longer collapsed into one token.
|
||
|
||
`w6c` and `w6c_ww` validate every direct export's leading canonical owner
|
||
against its paired `--import` path. After all direct exports are parsed, they
|
||
build canonical-path-to-declared-name metadata from those interfaces, apply any
|
||
vendor `--import-map` only to canonical identity, and bind the imported
|
||
declaration as the default qualifier of each primary source import. The
|
||
checkers and code generators select bindings by source-section ID and canonical
|
||
owner. Qualified and retained bare-import lookups mark only that file's binding
|
||
used. Two files may consequently bind the same name to different canonical
|
||
packages, while the graph still contains one edge/action for each target.
|
||
|
||
Within one file, two imports that produce the same default qualifier are a
|
||
redeclared binding; the later unused binding is also reported. An unused import
|
||
is reported at its own import position even if a sibling file uses the same
|
||
qualifier or canonical dependency. A package-scope declaration collides with
|
||
an equal import binding from any contributing file, matching Go's reconciliation
|
||
of package and file scopes. Conflicting production package clauses remain a
|
||
loader-owned deterministic error before producers. Compiler-owned scope or use
|
||
errors may invoke the compiler, but the driver removes that action's staged
|
||
`.new` unit/export/assembly/object/archive files instead of renaming them. No
|
||
completion/status marker is written, and no publication occurs after a failed
|
||
compile.
|
||
|
||
Package kind follows the declaration. `package main`, not a path component,
|
||
marks a command. A path ending in `main` remains importable when it declares a
|
||
different name. Any ordinary source import of a package declared `main` is
|
||
rejected as `package <canonical-path> is a program, not an importable package`,
|
||
regardless of its path leaf. The one pinned loader exception is an external
|
||
test's exact same-directory import of the command production: it is rewired to
|
||
the forced-library test copy. WW admits only that canonical colocated edge.
|
||
|
||
Test naming is likewise declaration-based. Production and internal-test
|
||
variants use the production declared name. When production files exist,
|
||
external files must declare `<production-declared-name>_test`; a test-only
|
||
directory may establish one consistent package name from its test files, as in
|
||
pinned `go/build`. External action identity remains the canonical production
|
||
identity plus the existing external variant suffix where that production
|
||
exists. Imports found only in internal or external test files belong only to
|
||
that variant. Support and generated-main actions retain their isolated
|
||
identities and archive closures. A generated dispatcher privately binds its
|
||
tested target through
|
||
`--test-target-package <canonical-path>` so a command variant declared `main`
|
||
does not collide with the dispatcher's own synthesized `main`; this is
|
||
compiler-generated wiring, not source alias syntax. Zero-test dispatchers mark
|
||
their compiler-owned target/support metadata imports consumed.
|
||
|
||
Vendor expansion changes only canonical identity and physical selection. A
|
||
source spelling such as `lib.codec` can resolve to
|
||
`domain.app.vendor.lib.codec`, while the vendored package's declaration, for
|
||
example `package wire`, supplies the file-local qualifier. The driver emits one
|
||
sorted semantic `--import-map lib.codec domain.app.vendor.lib.codec` and one
|
||
direct export input despite repeated import occurrences in separate files.
|
||
The expanded identity continues to own symbols, `.wwi`, archive, voucher, and
|
||
link inputs.
|
||
|
||
Canonical action interning remains the directory/path/variant model of sections
|
||
11.7 and 11.14. Independent file bindings never clone an action, and a declared
|
||
name never enters an artifact basename or storage locator. The name is semantic
|
||
content in the owner unit and export. Changing only a dependency's declaration
|
||
therefore keeps the same action identity but changes its export bytes, causes
|
||
each direct importer to be reconsidered, and stops propagation after an
|
||
importer's regenerated export is unchanged. An identical warm request remains
|
||
a producer no-op. Build workdir format 14 and test format 13 prevent reuse of
|
||
older vouchers that lack these semantics.
|
||
|
||
Compiler argv still contains exactly the sorted, deduplicated `.wwi` exports of
|
||
direct canonical dependencies; no transitive `.wwi` and no qualifier-derived
|
||
path appears. Linker argv still contains only the executable root archive and
|
||
complete reachable archive closure plus runtime/native inputs. Publication and
|
||
completion occur only after the corresponding action/product succeeds.
|
||
|
||
Responsibility is intentionally split as follows:
|
||
|
||
- `internal/wwpackage` classifies production, internal, and external test files
|
||
from their declarations, derives the allowed external name from the
|
||
production declaration, expands request patterns, and submits variant roots.
|
||
It does not resolve imports, choose qualifiers, or create dependency edges.
|
||
- `cmd/ww/main.c` and `selfhost/cmd/ww/main.ww` own per-site parsing and
|
||
contextual resolution, canonical directory/action interning, declared-name
|
||
consistency, imported-command rejection, sorted dependency union, exact tool
|
||
argv, variant/generated-main construction, persistence, linking, and
|
||
publication. Their storage, diagnostics, call positions, and allocation
|
||
failures are isomorphic.
|
||
- `cmd/w6c`/`cmd/wcc` and `selfhost/cmd/w6c`/`selfhost/cmd/wcc` own export
|
||
owner/name reading and writing, file-local binding installation, collision
|
||
and unused-import diagnostics, name/type lookup, canonical symbol ownership,
|
||
and generated-dispatcher private qualification. Cstage and WWstage emit
|
||
byte-identical applicable interfaces, assembly, archives, and binaries.
|
||
- `cmd/wwtest` remains only the test-command dispatcher.
|
||
|
||
The behavior follows pinned official Go 1.26.5 source at commit
|
||
`c19862e5f8415b4f24b189d065ed739517c548ba`:
|
||
|
||
- `go/build.Package` stores `Dir`, `Name`, `ImportPath`, production files, and
|
||
test files independently, while package-clause consistency is checked during
|
||
file classification
|
||
([`go/build/build.go`, lines 436–493](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L436-L493),
|
||
[`go/build/build.go`, lines 939–1049](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/build/build.go#L939-L1049)).
|
||
- The loader interns by resolved `ImportPath`, keeps `ImportPath` and `Name`
|
||
separate, constructs edges from source imports, and rejects an imported
|
||
package whose `Name == "main"` except for the exact same-directory test case
|
||
([`cmd/go/internal/load/pkg.go`, lines 633–636](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L633-L636),
|
||
[lines 757–806](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L757-L806),
|
||
[lines 2024–2047](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/pkg.go#L2024-L2047)).
|
||
- The test loader builds distinct production/internal/external/generated-main
|
||
packages, derives external `Name` from `p.Name + "_test"`, handles the
|
||
same-directory command self-import, and rewrites it to the library-form test
|
||
copy
|
||
([`cmd/go/internal/load/test.go`, lines 144–203](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L144-L203),
|
||
[lines 228–293](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L228-L293),
|
||
[lines 421–472](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/load/test.go#L421-L472)).
|
||
- `go/types.Package` stores independent path and name. The resolver creates one
|
||
child scope per source file, uses the imported package's declared name when
|
||
no alias is present, inserts imports into that file scope, reconciles them
|
||
with package declarations, and reports unused imports
|
||
([`go/types/package.go`, lines 26–40](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/types/package.go#L26-L40),
|
||
[`go/types/resolver.go`, lines 237–350](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/types/resolver.go#L237-L350),
|
||
[lines 463–480](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/types/resolver.go#L463-L480),
|
||
[lines 701–735](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/types/resolver.go#L701-L735)).
|
||
The production compiler mirrors those rules
|
||
([`cmd/compile/internal/types2/resolver.go`, lines 223–335](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/types2/resolver.go#L223-L335),
|
||
[lines 473–486](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/types2/resolver.go#L473-L486),
|
||
[lines 706–740](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/types2/resolver.go#L706-L740)).
|
||
- Compiler export import reconstructs and interns package descriptors by
|
||
canonical path while restoring their independently encoded package names and
|
||
imports
|
||
([`cmd/compile/internal/importer/ureader.go`, lines 152–196](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/compile/internal/importer/ureader.go#L152-L196),
|
||
[`go/internal/gcimporter/ureader.go`, lines 224–244](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/go/internal/gcimporter/ureader.go#L224-L244)).
|
||
- Work actions consume cached canonical package objects and direct imports,
|
||
while test execution builds and links the isolated test action graph
|
||
([`cmd/go/internal/work/action.go`, lines 437–455](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L437-L455),
|
||
[lines 628–659](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/work/action.go#L628-L659),
|
||
[`cmd/go/internal/test/test.go`, lines 1133–1226](https://github.com/golang/go/blob/c19862e5f8415b4f24b189d065ed739517c548ba/src/cmd/go/internal/test/test.go#L1133-L1226)).
|
||
|
||
The deliberately retained source grammar is `import dotted.path;`. Quoted Go
|
||
imports, explicit aliases, dot imports, and blank imports are not implemented.
|
||
Because bindings already retain source ownership and position separately from
|
||
canonical identity and declared name, adding explicit aliases later will not
|
||
require another package-wide namespace redesign.
|
||
|
||
The focused native observer
|
||
`declared_name_identity_and_file_import_scope` generates every tree
|
||
temporarily and exercises both stages from independent cold roots. It proves
|
||
the identity/name/qualifier split, file-local collision and unused behavior,
|
||
one-edge/action reuse, command and imported-command rules, all test variants,
|
||
vendor expansion, recursive/direct selection, owner-only units, exact direct
|
||
exports and archive-only links, warm no-op behavior, declared-name invalidation,
|
||
rejection-state preservation, normalized argv, artifact/binary identity,
|
||
allocation-bearing runtime behavior, and request/product-order independence.
|
||
|
||
## 12. Candidate architectures and hard-gate decision
|
||
|
||
Five candidates were developed as coherent systems, not as feature bins.
|
||
|
||
### 12.1 Candidate A: Go-like integrated language command
|
||
|
||
One `ww` command would infer directory packages/imports, compile/cache/test them,
|
||
and add a small module/lock layer. Native inputs would remain compiler flags or
|
||
toolchain conventions. This preserves the strongest part of Go: explicit imports,
|
||
fast direct export data, and a short ordinary command
|
||
([Go command design](https://go.dev/doc/articles/go_command)). A WW-specific lock
|
||
and no-network build could improve on modern Go module behavior.
|
||
|
||
It still fails as an end-to-end native design. A source-only graph cannot name
|
||
host generators, C header trees, archive order, linker scripts, CRT, SDK, or
|
||
sysroot. Ambient compiler-driver and `pkg-config` behavior would remain outside
|
||
the key, and the build/host/target triad would be incomplete. Adding typed native
|
||
actions, complete toolchains, and content records turns it into Candidate E.
|
||
|
||
### 12.2 Candidate B: Hare/Odin-style local source plus an outer build tool
|
||
|
||
WW would use search roots and directory modules, with source vendored or supplied
|
||
by an OS package manager; Make-like orchestration would own native work. This is
|
||
small locally and avoids a language-owned network resolver. Hare 0.26.0 (released
|
||
2026-02-13) is a useful reference: directory modules, direct textual export data,
|
||
`HAREPATH`, explicit system-library flags, and cross-architecture tooling are
|
||
documented in its official manuals
|
||
([modules](https://harelang.org/documentation/usage/modules.html),
|
||
[project structure](https://harelang.org/documentation/usage/project-structure.html),
|
||
[system libraries](https://harelang.org/documentation/usage/system-libraries.html),
|
||
[cross compilation](https://harelang.org/documentation/usage/cross.html)). Odin's
|
||
named collections are a related local-source convention
|
||
([Odin overview](https://odin-lang.org/docs/overview/)).
|
||
|
||
As a complete WW system it fails: ordered search roots are selection policy
|
||
without locked source identity; ordinary outer recipes expose ambient tools,
|
||
environment, and mtimes; language and native graphs/caches cannot jointly explain
|
||
invalidation; and cross sysroots/ABI providers remain project conventions. Making
|
||
the outer tool hermetic and content-addressed yields Candidate C, not this model.
|
||
|
||
### 12.3 Candidate C: two-layer Plan 9-style builder and orchestrator
|
||
|
||
A strict package builder would compile an import graph. A separate small
|
||
declarative DAG tool would own generators, C/assembly, images, archives, and
|
||
links. With content records, pinned tools, sandboxing, and an exact handoff this
|
||
can pass every hard gate. It reflects `mk`'s valuable complete-graph/parallel-tool
|
||
shape without copying its mtime and ambient-environment assumptions.
|
||
|
||
It loses after the gates because the boundary creates two graph protocols, two
|
||
selection UIs, two explanation namespaces, and either duplicate scheduling/cache
|
||
logic or a coarse “build all packages” action. Ordinary native projects must know
|
||
when to invoke each layer. If both front ends lower into one shared scheduler and
|
||
cache, and `ww` owns the ordinary invocation, the result is the smaller Candidate
|
||
E. Keeping the second production tool after that offers no remaining orthogonal
|
||
concept.
|
||
|
||
### 12.4 Candidate D: Zig/Cargo-like programmable integrated project
|
||
|
||
A manifest would define artifacts and dependencies while a host-executed program
|
||
constructs a flexible native graph. This handles more native cases than a
|
||
language-only command. Zig 0.16.0, pinned here to its 2026-04-13 release metadata,
|
||
provides explicit target/native concepts, hashed package sources, local
|
||
dependency overrides, and generated-file graph edges
|
||
([download metadata](https://ziglang.org/download/index.json),
|
||
[0.16.0 reference](https://ziglang.org/documentation/0.16.0/),
|
||
[build system](https://ziglang.org/learn/build-system/)). Cargo 1.97.1, shipped
|
||
with Rust 1.97.1 on 2026-07-16, provides exact package IDs, workspaces, lock
|
||
checksums, resolver rules, and native `links` collision handling
|
||
([Cargo reference](https://doc.rust-lang.org/1.97.1/cargo/reference/),
|
||
[resolver](https://doc.rust-lang.org/1.97.1/cargo/reference/resolver.html)).
|
||
|
||
Zig's build program and Cargo's `build.rs` execute to decide or report build
|
||
behavior; Cargo explicitly
|
||
documents build-script inputs/outputs and its fingerprint cache
|
||
([build scripts](https://doc.rust-lang.org/1.97.1/cargo/reference/build-scripts.html),
|
||
[build cache](https://doc.rust-lang.org/1.97.1/cargo/reference/build-cache.html)). Those
|
||
reference systems as shipped do not meet WW's complete native/toolchain hard
|
||
gates.
|
||
|
||
The strongest coherent D is not left as a straw man: it content-identifies the
|
||
graph program and host compiler/runtime, declares its whole readable source/tool
|
||
closure, runs it in the denied-by-default sandbox with no network, and requires
|
||
it to emit a closed typed graph before artifact execution. That hardened model
|
||
can pass every gate. It still loses afterward: WW must permanently ship/secure/
|
||
bootstrap an evaluator API and host build-program toolchain, users debug both
|
||
program execution and its emitted graph, dependencies expose framework APIs, and
|
||
routine exceptions accumulate as library features. Finite records buy the same
|
||
WW requirements with less user and implementation machinery.
|
||
|
||
### 12.5 Candidate E: hermetic integrated action build — selected
|
||
|
||
Candidate E retains the Go-like ordinary UX and import-derived language graph,
|
||
then adds only the native/action facts that imports cannot express. Both lower
|
||
to one typed graph and content cache. It borrows declared tools/inputs and action
|
||
results from Bazel's hermetic/remote-execution model, and transparent
|
||
content-derived build records from Nix derivations, without adopting either
|
||
framework, evaluator, daemon topology, or user interface
|
||
([Bazel hermeticity](https://bazel.build/versions/9.2.0/basics/hermeticity),
|
||
[Bazel remote caching](https://bazel.build/remote/caching),
|
||
[remote execution protocol](https://github.com/bazelbuild/remote-apis/blob/master/build/bazel/remote/execution/v2/remote_execution.proto),
|
||
[Nix derivations](https://nix.dev/manual/nix/2.35/store/derivation/)).
|
||
|
||
It passes every hard gate and is selected. Its concepts are exactly package,
|
||
module/source selection, product/action, toolchain/target, artifact/digest, and
|
||
native provider/link plan. There is one graph, one scheduler, one cache key, one
|
||
explanation path, and one ordinary command.
|
||
|
||
### 12.6 Hard-gate matrix
|
||
|
||
Legend: **pass** means the strongest coherent form has a credible end-to-end
|
||
invariant; **fail** means it does not. D denotes the hardened evaluator above,
|
||
not unmodified Zig/Cargo behavior.
|
||
|
||
| Hard gate | A: Go-like | B: local + outer | C: two layer | D: programmable | E: action build |
|
||
|---|---|---|---|---|---|
|
||
| complete inspectable dependency DAG | fail for native work | fail across tools | pass | pass after sandboxed evaluation | **pass** |
|
||
| loud cycles/identity collisions | pass for packages; native incomplete | search shadowing fails identity | pass | pass | **pass** |
|
||
| frozen offline source closure | pass with proposed lock | vendoring can pass | pass | pass | **pass** |
|
||
| correct cache invalidation | fail for native/tool defaults | fail across mtime/outer recipes | pass | pass | **pass** |
|
||
| compiler/linker/runtime/toolchain identity | requires the E native layer | fail | pass | pass | **pass** |
|
||
| explicit B/H/T | fail | fail | pass | pass | **pass** |
|
||
| correct native dependency/link modeling | fail | fail | pass | pass | **pass** |
|
||
| deterministic package selection | pass | fail under search roots | pass | pass with lock | **pass** |
|
||
| reproducible artifacts/boundary | fail for native closure | fail | pass | pass with evaluator certification | **pass** |
|
||
| bounded bootstrap | pass with a seed | pass with enumerated seed/tools | pass | pass, larger evaluator TCB | **pass** |
|
||
| explain cache miss/rebuild | fail | fail | pass across two namespaces | pass across evaluation + graph | **pass** |
|
||
| one simple ordinary path | pass | two commands/configurations | two production layers | pass by convention | **pass** |
|
||
| no build/test network or mutation | pass if changed from Go defaults | outer recipes cannot guarantee | pass | pass | **pass** |
|
||
|
||
C, hardened D, and E survive the gates. E wins the requested post-gate
|
||
comparison:
|
||
|
||
| Criterion | C: two layer | D: hardened evaluator | E: selected |
|
||
|---|---|---|---|
|
||
| conceptual simplicity | two graph/handoff models | evaluator language/API plus emitted graph | one graph protocol; two finite descriptions |
|
||
| daily usability | user chooses builder/orchestrator | ordinary defaults can hide evaluator, exceptions cannot | `ww build/test/run` always |
|
||
| clean speed | parallel orchestration can match | evaluation overhead, then parallel graph | direct complete template plus parallel actions |
|
||
| incremental speed | cross-tool handoff may be coarse | evaluator must rerun/cache its own dependencies | direct exports, lazy content keys, cached link |
|
||
| rebuild blast radius | good only with API digests across handoff | can be good after evaluation | public-output digest propagation is intrinsic |
|
||
| cross-compilation | orchestrator can model it | rich API can model it | roles are built into every typed record |
|
||
| FFI/native honesty | outer layer owns it separately | API can express it | same provider/link records as package objects |
|
||
| implementation maintenance | two adapters/engines or coarse boundary | evaluator, SDK compatibility, sandbox, graph engine | one scheduler/store/adapter set |
|
||
| supply-chain security | two policy surfaces | dependency host code remains an evaluator input | one lock/source/action trust model |
|
||
| ecosystem scalability | two tool ecosystems | framework/API accumulation pressure | closed schemas version only for demonstrated facts |
|
||
| debugging/observability | two explanation namespaces | debug program, evaluator cache, then graph | one causal graph/record diff |
|
||
| bootstrap longevity | small outer interpreter possible | compiler/runtime/evaluator enter seed chain | fixed seed plan; no production evaluator |
|
||
|
||
### 12.7 Why each subsystem is in or out of the core
|
||
|
||
Package parsing/resolution and export data belong in the core because only the
|
||
compiler can state the true language graph and public ABI. Module selection and
|
||
lock verification belong beside it because an import identity must resolve to
|
||
one deterministic source before compilation. The typed action engine, target
|
||
descriptors, native providers, and toolchain closure belong in the core because
|
||
they share invalidation and link correctness with package objects. Test/doc/
|
||
install are thin product selections/materializations over that same graph.
|
||
|
||
Network transport remains a separate operation, though exposed by `ww`, because
|
||
fetching is not building. Credential policy, OS package installation, registry
|
||
hosting, signing authority, remote execution, deployment, and general release
|
||
automation stay outside. The finite `generate` action is the boundary: it lets
|
||
outer domain tools transform declared artifacts without turning WW into their
|
||
framework.
|
||
|
||
## 13. Migration plan
|
||
|
||
Migration cost does not affect the decision. It is nevertheless material: the
|
||
current directly implicated scaffolding is at least 7,969 lines across the
|
||
Makefile, two drivers, two interface writers, package coordinator, and test
|
||
wrapper, plus 4,728 lines in the focused separate-build/package/byte-identity/
|
||
driver tests counted for this audit. Compiler export/import logic, new native
|
||
adapters, and bootstrap work add new scope not represented by those deletion
|
||
counts.
|
||
|
||
The planning estimate is **15–22 engineer-months** for the first production
|
||
Linux/amd64 toolchain, including tests, migration, and deletion, plus **1–2
|
||
engineer-months per materially different additional official target/sysroot**.
|
||
This is an estimate for staffing and sequencing, not a reason to retain a weaker
|
||
architecture.
|
||
|
||
Every phase below ends in a bisect-clean commit. Experimental components are not
|
||
installed as a second user-facing build path. Until the point of no return, the
|
||
old command remains the only production path; after it, the new command is the
|
||
only path.
|
||
|
||
### Phase 0 — conformance corpus and protocol freeze
|
||
|
||
- Land the compact checked-in `protocol/schema/` modules for WWAR framing,
|
||
record/enum/union/default/kind assignments, wrappers, and finite digest byte
|
||
formulas. Freeze the exact schema-file digests. Preserve representation-only
|
||
preimage records without embedding the algorithms that construct them.
|
||
- Generate data-only codec tables deterministically. Keep one schema-aware
|
||
reference codec and a second independent oracle limited to raw WWAR framing,
|
||
hashing, and record identity; neither may implement future compiler or build
|
||
behavior.
|
||
- Gate: strict duplicate-key UTF-8/NFC schema loading, byte-identical generation
|
||
in separate fresh directories, compact valid/invalid vectors, all assignment
|
||
coverage, stable malformed-length precedence, domain separation, action-key
|
||
vectors, and record-kind substitution rejection. No production behavior
|
||
changes beyond adding this conformance gate.
|
||
- Package/API, graph/cache, native B/H/T, and bootstrap measurements remain useful
|
||
fixtures, but become tests in Phases 1, 2, 4, and 6 respectively. They are not
|
||
Phase 0 semantic answer tables.
|
||
|
||
### Phase 1 — compiler export and package protocol
|
||
|
||
- Implement deterministic `.wwe`/`.wwlm` writing and direct-interface reading in
|
||
Cstage and self-hosted compiler paths behind test-only entry points.
|
||
- Add package identity/alias syntax, strict directory enumeration, compiler
|
||
import extraction, target suffix selection, cycle/collision/internal checks,
|
||
and protocol compatibility diagnostics.
|
||
- Construct and test the deep public type closure and public type/ABI digest
|
||
preimages in compiler code with existing standard-library graphs. Reject
|
||
`.wwi` input in the experimental path; do not translate it.
|
||
- Gate: Cstage/WWstage emit byte-identical vectors, importers open only direct
|
||
export files, and API propagation stops on unchanged middle exports.
|
||
|
||
### Phase 2 — one action engine and local CAS
|
||
|
||
- Integrate the frozen WWAR codec into production code and implement typed pure
|
||
action-record/key functions, the action graph, lazy keys, scheduler, atomic
|
||
CAS/results, project index, corruption quarantine, graph JSON, environment and
|
||
sandbox policy, deterministic failure behavior, and causal explanation.
|
||
- Add deterministic package/archive/link adapters using the existing compiler,
|
||
assembler, and linker as explicitly hashed tools. This is a temporary adapter,
|
||
not a compatibility promise.
|
||
- Keep the engine under an internal test binary; the installed `ww` still follows
|
||
the old production path.
|
||
- Gate: all action-input mutation and failure-injection tests pass; identical warm
|
||
builds execute no compiler, assembler, archiver, or linker.
|
||
|
||
### Phase 3 — module, lock, source, and workspace layer
|
||
|
||
- Implement the manifest, lock, work, and vendor text parsers as ordinary typed
|
||
parser code, then implement the closed grammar, monotonic selector, canonical
|
||
lock,
|
||
HTTPS source-index/archive protocol, immutable source store, signatures,
|
||
explicit add/update/lock/fetch, overlays, vendor index, and canonical
|
||
source-tree construction. The schema fixes only the resulting record bytes and
|
||
source-tree digest formula.
|
||
- Build/test/doc/install remain network-denied from their first experimental use.
|
||
- Gate: frozen offline builds work from project source + complete locked source
|
||
closure + lock + installed named toolchain; collision/downgrade/hash/
|
||
path-normalization attacks fail loudly.
|
||
|
||
### Phase 4 — native, target, and external toolchain closure
|
||
|
||
- Implement full target descriptors, B/H/T lowering, C/assembly/native-provider
|
||
records, recursive provider selection/expansion, generated actions/sandbox,
|
||
object sidecars, exact link-plan construction, sysroot,
|
||
libc/CRT/loader/SDK/runtime, shared-library installation, and freestanding
|
||
products.
|
||
- Package supported assembler/linker/archive/C tools as immutable external
|
||
closures. Implement an external-assembler-compatible textual emission backend,
|
||
then have the pinned assembler produce `package.o` inside `ww.package`; port all WW/Plan-9-dialect
|
||
runtime and user assembly to a declared supported external dialect, and pass
|
||
object/link parity. An argv adapter alone cannot consume current `w6a` syntax.
|
||
Stop relying on WW-owned `w6a`/`w6l` before the experimental gate passes.
|
||
- Gate: native conflict, archive group/order, linker-script include, host leak,
|
||
freestanding, shared loader, and at least one real cross-target suite pass.
|
||
|
||
### Phase 5 — repository and consumer conversion rehearsal
|
||
|
||
- Define the final identity/layout conversion in a one-shot checker/rewriter under
|
||
`tools/migrate-build/`: folded files become directory packages; aliases,
|
||
manifests, native providers, products, generated inputs, and locks are emitted.
|
||
It is not an import resolver or runtime compatibility layer.
|
||
- Until cutover, CI applies that tool to a fresh ignored shadow tree. Tracked
|
||
production source stays in old syntax, so the old command remains its only
|
||
production path; the shadow is regenerated, never a second maintained source
|
||
tree or shipped interface.
|
||
- Dogfood the internal engine on the converted shadow of the standard library,
|
||
compiler tools, tests, examples, install layout, CI/package inputs, and sample
|
||
downstream consumers. Compare semantics, diagnostics, performance, artifacts
|
||
where protocols permit, and complete graph explanations.
|
||
- Gate: the regenerated full shadow, distribution/frozen-offline build,
|
||
self-contained installed toolchain outside the build tree, downstream samples,
|
||
and bootstrap inputs pass without an old-format edge.
|
||
|
||
### Phase 6 — new bootstrap and reproducible release
|
||
|
||
- Implement/gate the portable C recovery backend and snapshot generator, then
|
||
generate/check in `bootstrap/ww0.c` and the fixed plan. Produce stages 1/2/3/4,
|
||
rebuild stages with the executable engine, compare the actual bytes of every
|
||
declared output directly, and produce fixed-point and diverse-seed-compilation
|
||
reports, signed toolchain bundles, and recovery documentation on a clean
|
||
machine with no WW compiler.
|
||
- Gate: stage 2 equals stage 3 and stage 3 equals stage 4 by explicit
|
||
byte-for-byte comparison, including raw action/result records, in two
|
||
roots/concurrency levels; project plus complete
|
||
locked source closure and published named tool closure reproduce every release
|
||
artifact.
|
||
|
||
### Phase 7 — point of no return and deletion
|
||
|
||
In one atomic, bisect-clean cutover commit:
|
||
|
||
1. install the new engine as `ww` and make it the sole build/test/bootstrap path;
|
||
2. switch repository imports, manifests, locks, toolchains, CI, installation,
|
||
and release jobs to their final forms;
|
||
3. delete both old drivers, both `.wwi` writers, driver-side `.wwi` concatenation,
|
||
composed-unit/module-wrapper logic, generic C/self-hosted lexer/parser/checker/
|
||
symbol/codegen support for module directives, `-w`,
|
||
old import search/file folding, separate test coordinator,
|
||
duplicated production Make dependency graph, current Cstage bootstrap, and
|
||
retired owned assembler/linker path; and
|
||
4. delete the migration rewriter after all supported consumers have used its
|
||
released standalone copy; keep only a format-error guide.
|
||
|
||
The commit does not accept old `.wwi`, dotted/file imports, `-I`, `-w`, raw
|
||
library searches, or old work directories. No alias, warning period inside the
|
||
compiler, environment switch, or fallback subprocess retains a dual system.
|
||
|
||
### Phase 8 — consolidation
|
||
|
||
- Remove temporary parity fixtures that test deleted artifact bytes while
|
||
retaining semantic, action-key, reproducibility, and bootstrap regression
|
||
tests.
|
||
- Publish migration statistics and archive the old documentation as historical
|
||
release material outside the live manual.
|
||
- Gate: repository search and executable tracing show one graph constructor, one
|
||
package resolver, one cache, one test route, and one bootstrap route.
|
||
|
||
## 14. Validation plan and release gates
|
||
|
||
### 14.1 Unit and format tests
|
||
|
||
- Phase 0 has golden and adversarial vectors only for WWAR representation,
|
||
source-tree digest bytes, action-key/record identity formulas, typed record
|
||
assignments, wrappers, and canonical schema JSON.
|
||
- Phase 0 tests UTF-8/NFC, duplicate schema keys, unknown wire type/schema field,
|
||
oversized declarations, truncation, exact-length mismatch, union shape, and
|
||
record-kind substitution. Case-fold collision, traversal, symlink/device,
|
||
cache collision/corruption, and semantic record tests land with their owning
|
||
executable phases.
|
||
- Phases 1–4 add behavior tests for CAS tree/result objects, `.wwe`, `.wwlm`,
|
||
manifest/lock/work/vendor parsing, and target/toolchain/native processing;
|
||
these are not encoded as Phase 0 vector outcomes.
|
||
- Resolver vectors for minimum selection, incompatible-major identities,
|
||
workspace identity preservation, source origin independence, vendor matching,
|
||
internal packages, aliases, nested module/root-versus-parent-subpackage
|
||
identity collisions (including `/vN`), cycles, and target source specificity.
|
||
|
||
### 14.2 Package/interface tests
|
||
|
||
- One- and multi-file directory membership, file-scoped import use, same/external
|
||
tests, test-only packages, examples/docs, and generated fragments with import/
|
||
package rejection.
|
||
- Direct dependency interface-open counts equal package-graph indegree, never
|
||
transitive closure size.
|
||
- Private dependency edits preserve importer keys; exported but unused additions
|
||
rebuild direct importers; unchanged middle `.wwe` stops propagation; layout,
|
||
calling-convention, runtime ABI, compiler, target, and profile changes rebuild
|
||
the exact affected cone.
|
||
- Parameter renames, comments, private declaration ordering, and absolute source
|
||
paths do not change `.wwe`; semantic/ABI changes do.
|
||
|
||
### 14.3 Native and link tests
|
||
|
||
- C scalar/aggregate/variadic/callback/TLS/unwind ABI probes against an
|
||
independently compiled C harness for every official target.
|
||
- Binding generation changes on header tree, macro map, preprocessor, tool, C
|
||
ABI, and target; undeclared include access is denied.
|
||
- Assembly dialect/CPU mismatch, wrong-format objects, PIC/shared rules, archive
|
||
extraction order, repeated libraries, groups, whole archive, weak/strong
|
||
symbols, version scripts, linker-script includes, and deterministic archives.
|
||
- Exactly-one libc/runtime/provider enforcement; identical coalescing; duplicate
|
||
native ABI/symbol conflict; shared SONAME/loader/runtime installation closure;
|
||
freestanding entry/script with proof that no libc/CRT/loader appears.
|
||
|
||
### 14.4 Cross and sandbox tests
|
||
|
||
- Matrix with `B != H`, compiler product `H != T`, and all three distinct where
|
||
infrastructure permits. A B generator emits an H input; an H binary is never
|
||
executed during build; T objects never enter the H link.
|
||
- Poison host `PATH`, includes, libraries, SDK, locale, time, home, current
|
||
directory, and environment. Every attempted undeclared read/write/network/
|
||
process/tool access fails with its action identity.
|
||
- Cross tests build without a runner, run only with an exact declared runner,
|
||
and fail under `--require-run` when none exists.
|
||
|
||
### 14.5 Cache and failure injection
|
||
|
||
- Change each field in section 6.2 individually and require a key change; change
|
||
each expressly non-semantic observation and require no key change.
|
||
- Bit-flip blobs, trees, results, action mappings, tools, export data, objects,
|
||
and partial files at every publication boundary. Require quarantine/rebuild,
|
||
never acceptance or broad deletion.
|
||
- Concurrent identical publishers, killed compiler/linker, disk full, rename
|
||
failure, read-only output, interrupted materialization, stale project index,
|
||
malicious remote mapping, bad cache signature, and remote outage.
|
||
- Clean, local-hit, explicit remote-import, and no-cache builds must yield the
|
||
same result digests. Test executions still run.
|
||
|
||
### 14.6 Reproducibility and bootstrap
|
||
|
||
- Compare every artifact/action record across two absolute checkouts, source/
|
||
cache/output roots, usernames, locales, time zones, umasks, concurrency levels,
|
||
filesystem enumeration orders, and cold/warm caches.
|
||
- Verify debug/release, static/shared, hosted/freestanding, generated/native, and
|
||
signed/unsigned products. Impure profiles must state exactly why they are
|
||
outside the byte promise and must never enter shared cache.
|
||
- Build `ww0` with each supported host C toolchain, reach stages 2/3/4, run the
|
||
semantic then raw-record fixed point and diverse seed compilation, corrupt each stage input, and recover on a host
|
||
with no WW installation.
|
||
|
||
### 14.7 Performance and migration gates
|
||
|
||
On the audit's fixed eight-CPU reference host, the first release MUST:
|
||
|
||
- run package compilation in parallel and complete the full clean toolchain
|
||
build no slower than the measured 15.515 s `make -j8` baseline;
|
||
- perform a warm 15-package build with no compiler, assembler, archiver, or
|
||
linker process and no slower than the measured 0.053 s driver baseline;
|
||
- read only direct exports and avoid composed-unit duplication;
|
||
- store one CAS copy of duplicate tool/package content across all products;
|
||
- produce fully path-independent official artifacts, including host-side tools;
|
||
and
|
||
- provide a typed explanation for every deliberately induced rebuild.
|
||
|
||
Before cutover, every tracked current package/test/install/bootstrap consumer has
|
||
an assigned new identity and a passing converted test. The cutover gate includes
|
||
a repository-wide search for old forms and executable traces proving no old
|
||
driver, interface, workdir, library search, or test-coordinator path executes.
|
||
|
||
## 15. Evidence appendix
|
||
|
||
### 15.1 Research method and version pins
|
||
|
||
Research used official documentation, standards/manuals, release metadata, and
|
||
current upstream source—not comparison articles or community summaries. The
|
||
evolving-system snapshot was taken 2026-08-09:
|
||
|
||
| System | Pinned snapshot used |
|
||
|---|---|
|
||
| Plan 9 | live official `9p.io` Volume 2 documents, accessed 2026-08-09; pages are not versioned/dated editions |
|
||
| Go | online docs accessed 2026-08-09; Go 1.26.5 `go1.26.5` source tag |
|
||
| Hare | online docs accessed 2026-08-09; Hare 0.26.0 source, released 2026-02-13 |
|
||
| Odin | online docs accessed 2026-08-09; `dev-2026-07a`, commit `819fdc7a80667498b8b365999f1475a66c358640` |
|
||
| Zig | Zig 0.16.0, official metadata release date 2026-04-13; source archive SHA-256 `43186959edc87d5c7a1be7b7d2a25efffd22ce5807c7af99067f86f99641bfdf` |
|
||
| Rust/Cargo | Rust/Cargo 1.97.1, released 2026-07-16; Cargo `0.98.0` commit `c980f4866141969fab6254a680546a277789d6f0` |
|
||
| Bazel | Bazel 9.2.0 documentation/source |
|
||
| Nix | Nix 2.35.2 manual/source |
|
||
|
||
Zig's separate bootstrap-source archive inspected for this decision had SHA-256
|
||
`2a8266a4205772ef40838c8cbdf14875855a515ff3adf89b49c2d2ae93613d10`.
|
||
These pins matter because programmable-build and package behavior changes between
|
||
releases; this document does not generalize an old Zig/Cargo observation to an
|
||
unidentified current version.
|
||
|
||
### 15.2 Primary-source findings
|
||
|
||
**Pike, Plan 9, and early Go.** Pike's sources support explicit computable
|
||
imports, cycle rejection, direct compiled export information, fast compilation,
|
||
and orthogonal concepts. The collective Plan 9 papers add system-wide placement
|
||
of complexity and transparent encodings. Plan 9's namespace papers demonstrate
|
||
contextual filesystem composition; this document infers that contextual location
|
||
must not serve as WW's versioned distribution identity. `mk` shows complete graph scheduling while still relying
|
||
on timestamps, recipes, and environment. The architecture borrows the former
|
||
principles and replaces the latter ambient assumptions.
|
||
|
||
- [The Go Programming Language, 2009](https://go.dev/talks/2009/go_talk-20091030.pdf)
|
||
- [Go at Google: Language Design in the Service of Software Engineering, 2012](https://go.dev/talks/2012/splash.article)
|
||
- [Simplicity is Complicated, 2015](https://go.dev/talks/2015/simplicity-is-complicated.slide)
|
||
- [Go in Go, 2015](https://go.dev/talks/2015/gogo.slide)
|
||
- [Plan 9 overview](https://9p.io/sys/doc/9.html)
|
||
- [The Use of Name Spaces in Plan 9](https://9p.io/sys/doc/names.html)
|
||
- [Maintaining Files on Plan 9 with Mk](https://9p.io/sys/doc/mk.html)
|
||
- [Plan 9 Mkfiles](https://9p.io/sys/doc/mkfiles.html)
|
||
- [Plan 9 compiler suite](https://9p.io/sys/doc/comp.html)
|
||
|
||
**Later Go.** The original Go command demonstrates source-derived package DAGs,
|
||
directory conventions, and compiler-owned dependency work. The modern module
|
||
reference documents module identity, Minimal Version Selection, major-version
|
||
paths, checksums, and commands that may resolve/download modules. Current command
|
||
and source-install documents also separate build cache/toolchain/bootstrap
|
||
behavior. These are evidence, not automatic WW defaults; in particular WW uses
|
||
an exact lock and forbids implicit build-time acquisition.
|
||
|
||
- [About the Go command](https://go.dev/doc/articles/go_command)
|
||
- [Go module reference](https://go.dev/ref/mod)
|
||
- [`go` command reference](https://go.dev/cmd/go/)
|
||
- [Go toolchain selection](https://go.dev/doc/toolchain)
|
||
- [Installing Go from source](https://go.dev/doc/install/source)
|
||
- [Perfectly Reproducible, Verified Go Toolchains](https://go.dev/blog/rebuild)
|
||
- [Go's supply-chain security](https://go.dev/blog/supply-chain)
|
||
- [`go1.26.5` source](https://go.googlesource.com/go/+/refs/tags/go1.26.5/)
|
||
|
||
**Local-source systems.** Hare demonstrates how far a disciplined directory
|
||
module/search-root system can go with little package machinery; its documented
|
||
system-library and cross interfaces also expose why raw host paths and tool
|
||
defaults are insufficient for WW's hard gates. Odin's collections reinforce the
|
||
local namespace option but do not add a locked whole native closure.
|
||
|
||
- [Hare modules](https://harelang.org/documentation/usage/modules.html)
|
||
- [Hare project structure](https://harelang.org/documentation/usage/project-structure.html)
|
||
- [Hare system libraries](https://harelang.org/documentation/usage/system-libraries.html)
|
||
- [Hare cross compilation](https://harelang.org/documentation/usage/cross.html)
|
||
- [Hare 0.26.0 source](https://git.sr.ht/~sircmpwn/hare/tree/0.26.0)
|
||
- [Odin overview](https://odin-lang.org/docs/overview/)
|
||
- [Odin pinned source](https://github.com/odin-lang/Odin/tree/819fdc7a80667498b8b365999f1475a66c358640)
|
||
|
||
**Integrated project systems.** Zig supplies useful target/native vocabulary,
|
||
source hashes, local aliases, declared generated-file edges, and cross-building.
|
||
Cargo supplies package IDs, exact lock checksums, workspace behavior, resolver
|
||
documentation, and a native `links` uniqueness rule. Their programmable build
|
||
program/script model is deliberately rejected; a dependency host program is a
|
||
larger and less inspectable abstraction than WW's finite action record.
|
||
|
||
- [Zig build system](https://ziglang.org/learn/build-system/)
|
||
- [Zig overview](https://ziglang.org/learn/overview/)
|
||
- [Zig 0.16.0 language reference](https://ziglang.org/documentation/0.16.0/)
|
||
- [Zig release metadata](https://ziglang.org/download/index.json)
|
||
- [Zig 0.16.0 source](https://github.com/ziglang/zig/tree/0.16.0)
|
||
- [Cargo 1.97.1 reference](https://doc.rust-lang.org/1.97.1/cargo/reference/)
|
||
- [Cargo current reference entry](https://doc.rust-lang.org/cargo/reference/)
|
||
- [Cargo 1.97.1 resolver](https://doc.rust-lang.org/1.97.1/cargo/reference/resolver.html)
|
||
- [Cargo current resolver entry](https://doc.rust-lang.org/cargo/reference/resolver.html)
|
||
- [Cargo 1.97.1 build scripts](https://doc.rust-lang.org/1.97.1/cargo/reference/build-scripts.html)
|
||
- [Cargo current build-script entry](https://doc.rust-lang.org/cargo/reference/build-scripts.html)
|
||
- [Cargo 1.97.1 build cache](https://doc.rust-lang.org/1.97.1/cargo/reference/build-cache.html)
|
||
- [Cargo current build-cache entry](https://doc.rust-lang.org/cargo/reference/build-cache.html)
|
||
- [Cargo pinned source](https://github.com/rust-lang/cargo/tree/c980f4866141969fab6254a680546a277789d6f0)
|
||
|
||
**Hermetic invariants.** Bazel defines useful distinctions between declared
|
||
actions, execution platforms, action results, and content-addressed remote data.
|
||
Nix derivations demonstrate an inspectable build record whose output depends on
|
||
declared inputs. WW borrows those invariants only. It rejects Bazel's general
|
||
rule/ecosystem machinery and Nix's evaluator/store-as-package-manager as excess
|
||
for one language toolchain.
|
||
|
||
- [Bazel current hermeticity entry](https://bazel.build/concepts/hermeticity)
|
||
- [Bazel 9.2.0 hermeticity](https://bazel.build/versions/9.2.0/basics/hermeticity)
|
||
- [Bazel remote caching](https://bazel.build/remote/caching)
|
||
- [Remote Execution API](https://github.com/bazelbuild/remote-apis/blob/master/build/bazel/remote/execution/v2/remote_execution.proto)
|
||
- [Bazel 9.2.0 source](https://github.com/bazelbuild/bazel/tree/9.2.0)
|
||
- [Nix current derivation entry](https://nix.dev/manual/nix/latest/store/derivation/)
|
||
- [Nix 2.35 derivations](https://nix.dev/manual/nix/2.35/store/derivation/)
|
||
- [Nix build process](https://nix.dev/manual/nix/2.35/store/building.html)
|
||
- [Nix 2.35.2 source](https://github.com/NixOS/nix/tree/2.35.2)
|
||
|
||
**Native and reproducible toolchains.** LLVM and Clang define data-layout and
|
||
cross/toolchain choices that must be explicit for ABI-correct code. GNU manuals
|
||
document build/host/target vocabulary and semantic linker/archive/script
|
||
behavior. The reproducible-builds definition supplies the correct boundary:
|
||
same source, environment, instructions, and dependencies—not merely “same
|
||
compiler source.”
|
||
|
||
- [LLVM language-reference data layout](https://llvm.org/docs/LangRef.html#data-layout)
|
||
- [Clang cross compilation](https://clang.llvm.org/docs/CrossCompilation.html)
|
||
- [Clang toolchain](https://clang.llvm.org/docs/Toolchain.html)
|
||
- [Autoconf target triplets](https://www.gnu.org/software/autoconf/manual/autoconf-2.71/html_node/Specifying-Target-Triplets.html)
|
||
- [GCC language standards/runtime implications](https://gcc.gnu.org/onlinedocs/gcc/Standards.html)
|
||
- [GCC link options](https://gcc.gnu.org/onlinedocs/gcc/Link-Options.html)
|
||
- [GNU linker](https://sourceware.org/binutils/docs/ld.html) and [linker scripts](https://sourceware.org/binutils/docs/ld/Scripts.html)
|
||
- [Reproducible Builds definition](https://reproducible-builds.org/docs/definition/)
|
||
|
||
### 15.3 Repository and empirical evidence record
|
||
|
||
The source audit covered the Makefile; C and self-hosted drivers; compiler entry
|
||
and both interface writers; package/test coordinator; language and test specs;
|
||
bootstrap documents/recipes; and package, separate-compilation, incremental,
|
||
byte-identity, and bootstrap tests. The audit found documentation drift as well
|
||
as code defects: the root instructions count five unit suites while Make lists
|
||
six, and the test-system document describes three pinned data byte divergences
|
||
while the current Make divergence list is empty. Neither drift item influences
|
||
the architecture, but both require cleanup during migration.
|
||
|
||
Raw commands, timings, traces, hashes, fixture logs, environment capture, and
|
||
interpretation for section 11.4 are retained in the session checkpoint under
|
||
`empirical/RESULTS.md` and `empirical/raw/`. The experiments were deliberately
|
||
small and answered only material uncertainties: actual process scheduling,
|
||
direct versus transitive interface consumption, private/public/link-only blast
|
||
radius, nonzero corruption, path identity, Cstage/WWstage symmetry, duplicate
|
||
storage, and hidden-host execution. No toy result is treated as proof that the
|
||
new architecture is complete; sections 13–14 turn each inference into a gate.
|
||
|
||
### 15.4 Assumptions
|
||
|
||
- Breaking import syntax, package layout, compiler flags, cache/workdir format,
|
||
bootstrap artifacts, and consumer builds is authorized.
|
||
- WW can add explicit C ABI/representation annotations and change its compiler
|
||
export protocol without preserving `.wwi` compatibility.
|
||
- Official releases can distribute or name legally usable immutable tool,
|
||
sysroot, runtime, and SDK closures for each supported target.
|
||
- SHA-256 is the version-1 content primitive; every format is domain-separated
|
||
and versioned so a future digest transition can be explicit.
|
||
- A first release may support fewer frozen targets than impure development
|
||
targets; it may not call an impure target “hermetic.”
|
||
- Byte reproducibility covers build artifacts, not identical behavior of an
|
||
external kernel, firmware, network service, or hardware.
|
||
|
||
### 15.5 Remaining risks and bounded experiments
|
||
|
||
These experiments are implementation gates inside the selected architecture;
|
||
they do not reopen its package identity, no-script, one-graph, lock/network,
|
||
direct-export, content-key, native-provider, or stage-zero choices.
|
||
|
||
1. **Deep `.wwe` closure.** Prototype the real standard-library type graph and
|
||
prove that one direct export file contains every transitive layout/type fact
|
||
required without leaking non-semantic source data. Adjust record layout, not
|
||
direct-dependency semantics.
|
||
2. **Generated C seed.** Measure generated `ww0.c` size, C99 portability, host-C
|
||
variance, and fixed-point convergence on at least three unrelated C
|
||
implementations. Restrict/repair the recovery emitter rather than introducing
|
||
a second maintained compiler or opaque-permanent binary seed.
|
||
3. **External linker/tool bundles.** Certify deterministic debug info, build ID,
|
||
archives, scripts, shared-loader metadata, redistribution rights, and resource
|
||
closure. Change tool adapters/bundle membership if needed; do not restore
|
||
ambient driver defaults.
|
||
4. **Sandbox portability.** Implement denial conformance on every official B
|
||
platform, including process children, filesystem race/symlink attacks, clock,
|
||
randomness, and network namespaces. A platform that cannot enforce it remains
|
||
non-frozen rather than gaining an undeclared exception.
|
||
5. **Native-provider coverage.** Exercise ELF first, then Mach-O frameworks/SDKs,
|
||
PE/COFF import libraries, symbol versioning, and kernel image builders. Extend
|
||
the closed typed schema by version where genuinely required; do not add raw
|
||
search or a general build language.
|
||
6. **Registry/private-source protocol.** Test mirror failover, redirects,
|
||
credential isolation, key rotation, provenance, yanked releases, and malicious
|
||
archives. Vendored/exact-origin operation remains the deterministic fallback.
|
||
7. **Performance.** Validate compiler worker strategy and CAS hashing against the
|
||
measured 15-package/full-toolchain budgets. Process topology may change while
|
||
action boundaries and keys remain fixed.
|
||
|
||
### 15.6 Final decision trace
|
||
|
||
The documented Pike evidence shapes the design: computable direct dependencies,
|
||
fast compilation, cycle rejection, package boundaries, and orthogonal concepts;
|
||
the collective Plan 9 papers add transparent encodings and system-wide placement
|
||
of complexity. Later Go work demonstrates one possible module/cache/toolchain
|
||
evolution but does not decide WW's answer. This document then applies those
|
||
principles to requirements early Go's package/build model did not expose
|
||
completely: foreign ABI contracts, C headers, external assembly and objects,
|
||
archive/link ordering, linker scripts, CRT/libc/sysroot identity, host generators,
|
||
and explicit build/host/target closures.
|
||
|
||
The resulting binding conclusion is singular: **replace the current system with
|
||
WW Action Build exactly as specified above**. Do not preserve the old path, and
|
||
do not substitute a programmable project framework or a source-only package
|
||
command during implementation.
|