3917 lines
226 KiB
Markdown
3917 lines
226 KiB
Markdown
# WW package, dependency, build, and bootstrap architecture
|
||
|
||
Status: **binding architecture decision**
|
||
|
||
Decision date: 2026-08-10
|
||
|
||
Implementation status: specified, not yet implemented
|
||
|
||
This document selects the production architecture that replaces WW's current
|
||
package driver, source-like interface protocol, work-directory reuse scheme,
|
||
Make orchestration, test coordinator, and bootstrap chain. It is a greenfield
|
||
decision. Migration effort is recorded only to plan implementation; it did not
|
||
influence the selection.
|
||
|
||
The words **MUST**, **MUST NOT**, **SHOULD**, and **MAY** are normative.
|
||
|
||
## 1. Executive decision
|
||
|
||
WW will have one integrated command backed by one typed, content-addressed
|
||
action graph. Language imports describe the language-package subgraph. A small,
|
||
declarative `ww.mod` file describes only facts that source imports cannot:
|
||
distribution requirements, products, generated inputs, native providers, and
|
||
unusual link steps. Both descriptions lower to the same graph, scheduler,
|
||
sandbox, cache, and explanation machinery. There is no general build language
|
||
and no arbitrary build script. A one-directory, zero-dependency executable
|
||
needs no manifest. Distributed projects use an exact lock file; fetching and
|
||
updating are explicit commands, while build, run, test, documentation, and
|
||
installation never access the network or rewrite project metadata. Packages
|
||
produce target-specific binary export data and one object, importers consume
|
||
only direct export data, and final products link a declared ordered closure.
|
||
Every cached action names its compiler, target, profile, tools, sysroot, runtime,
|
||
native inputs, environment, and content. The official toolchain owns these
|
||
protocols and ships a pinned tool closure, but WW does not permanently own an
|
||
assembler or linker.
|
||
|
||
The decisive insight is that **the package graph and the build graph are not the
|
||
same graph**. Imports are a complete and desirable description of WW-language
|
||
dependencies. They cannot honestly describe a C header tree, a host generator,
|
||
an assembler, a linker script, a CRT, or a sysroot. Making imports pretend to do
|
||
so hides native inputs; making every project use a programmable build framework
|
||
destroys the simple ordinary path. Two small declarative front ends lowering to
|
||
one action engine are simpler as a system than either lie.
|
||
|
||
The architecture is named **WW Action Build** in this document. That is a label,
|
||
not another user-facing product: the command remains `ww`.
|
||
|
||
### 1.1 The Pike lens and attribution
|
||
|
||
Pike explicitly documented and defended these Go design choices and principles:
|
||
|
||
- language-defined imports make dependencies explicit, clear, and mechanically
|
||
computable;
|
||
- unused imports and import cycles are errors; rejecting cycles improves package
|
||
boundaries and independent maintenance;
|
||
- compilation speed and short edit/build cycles are primary design properties;
|
||
- a direct dependency's compiled artifact can carry the deeper public type facts
|
||
needed by its clients, so an importer opens only direct dependency artifacts;
|
||
and
|
||
- orthogonal, predictable concepts, fewer ways to express a construct, and a
|
||
simple user experience are worth substantial implementation work.
|
||
|
||
Those points are stated in Pike's 2009 Go talk, the 2012 SPLASH article, and his
|
||
2015 simplicity talk
|
||
([2009 talk](https://go.dev/talks/2009/go_talk-20091030.pdf),
|
||
[2012 article](https://go.dev/talks/2012/splash.article),
|
||
[Simplicity is Complicated](https://go.dev/talks/2015/simplicity-is-complicated.slide)).
|
||
The collective Plan 9 papers add system-wide placement of complexity, focused
|
||
interfaces, and transparent text or explicitly encoded binary data. `Go in Go`
|
||
documents one contingent case in which owning more of the toolchain simplified
|
||
Go; it does not establish permanent toolchain ownership as a general Pike
|
||
principle ([Go in Go](https://go.dev/talks/2015/gogo.slide)).
|
||
|
||
Plan 9's `mk` constructs
|
||
the dependency graph before execution, rejects cycles and ambiguous recipes,
|
||
and schedules independent work in parallel. Plan 9 used a target-specific
|
||
compiler/assembler/loader family and portably encoded target object conventions
|
||
([mk](https://9p.io/sys/doc/mk.html),
|
||
[mkfiles](https://9p.io/sys/doc/mkfiles.html),
|
||
[compilers](https://9p.io/sys/doc/comp.html)).
|
||
|
||
Modern module-path/version semantics, Minimal Version Selection, `go.sum`, the modern Go build cache,
|
||
automatic toolchain selection, and current supply-chain policy are later Go-team
|
||
designs, not principles uniquely attributable to Pike
|
||
([module reference](https://go.dev/ref/mod),
|
||
[`go` command](https://go.dev/cmd/go/),
|
||
[toolchain selection](https://go.dev/doc/toolchain),
|
||
[toolchain rebuilding](https://go.dev/blog/rebuild),
|
||
[supply-chain policy](https://go.dev/blog/supply-chain)). This decision borrows
|
||
some invariants from those systems but does not attribute them to Pike.
|
||
|
||
The following are this document's inferences from the documented principles:
|
||
|
||
- strict directory packages and direct binary export data are the smallest way
|
||
to keep dependencies computable and compilation fast;
|
||
- a declarative native/action layer is necessary for an unmanaged language,
|
||
because omitting it moves complexity into ambient shell state;
|
||
- one shared action engine is simpler than independent language and outer-build
|
||
caches;
|
||
- given WW's complete-graph, frozen-build, and explainable-key hard gates,
|
||
arbitrary graph-producing programs are rejected; they would require executing
|
||
dependency host code before the graph is inspectable and add another permanent
|
||
user programming model; and
|
||
- WW should specify a toolchain closure but should not maintain an assembler and
|
||
linker forever when pinned external tools make the whole system smaller.
|
||
|
||
Modern native requirements force deliberate departures from historical Plan 9
|
||
and early Go: cryptographic source identities, lock files, explicit build/host/
|
||
target separation, sysroot and SDK identity, hostile dependency acquisition,
|
||
cross-platform sandboxes, and cache-miss explanations were not their complete
|
||
problem statement. WW adopts their architectural restraint, not their ambient
|
||
host assumptions.
|
||
|
||
### 1.2 What follows from being unmanaged and native
|
||
|
||
Being unmanaged/native genuinely requires the build model to know:
|
||
|
||
- the target data layout and C ABI;
|
||
- foreign symbol spelling and visibility;
|
||
- object format, relocation model, CPU features, and assembly dialect;
|
||
- ordered objects, archives, shared libraries, linker scripts, and archive-group
|
||
semantics;
|
||
- the libc, CRT, dynamic loader, SDK, runtime, and sysroot closure;
|
||
- freestanding entry and runtime policy;
|
||
- build-machine tools that generate host- or target-machine inputs; and
|
||
- ABI compatibility among compiler, runtime, native providers, and final link.
|
||
|
||
It does **not** follow that WW needs multiple dependency versions, semantic
|
||
version ranges, a network resolver in every build, programmable build scripts,
|
||
feature unification, a global namespace, or its own linker. In particular, the
|
||
absence of a garbage collector says nothing about version resolution.
|
||
|
||
### 1.3 Binding answers to the critical questions
|
||
|
||
| Question | Binding answer |
|
||
|---|---|
|
||
| Package identity | One canonical import path: the owning module identity for its root package, otherwise that identity plus `/` and the normalized package-relative path. The declared package name is a source qualifier, not identity. |
|
||
| Identity versus location/origin/version/content | All are separate. A resolver record maps identity and selected version to an origin and source-tree digest; a workspace maps identity to a local location. |
|
||
| Directory membership | Exactly one package per directory. Immediate selected source files belong to it. Nested directories never do. |
|
||
| Single-file packages | Deleted. A one-file directory package remains configuration-free. |
|
||
| Language dependency graph | The compiler-parsed imports alone define it. Manifest native/action edges extend the build graph, never the language graph. |
|
||
| Import interfaces | Direct dependencies only. Each direct `.wwe` contains the deep public type information needed to understand its own API. |
|
||
| `.wwi` | Deleted and replaced by deterministic, versioned binary `.wwe` export data. Canonical source prototypes are not an interchange format. |
|
||
| Package invalidation | The package action key changes when its selected own sources/generated inputs, direct export digests, compiler/toolchain, target/profile, declared environment, or protocol changes. A private transitive change does not invalidate it. |
|
||
| Cache key | Domain-separated SHA-256 over the canonical action record defined in section 6.6. |
|
||
| Cache scope | A per-user global local content store plus a project-local graph-history index. Remote import/export is explicit, never part of ordinary build. |
|
||
| Corruption/upgrades | Every object is rehashed on read; corrupt entries are quarantined. Tools and protocol versions are content inputs, so upgrades change keys. |
|
||
| Multiple versions | One selected version of a module identity. Incompatible major releases use distinct module identities ending `/vN`, so those identities may coexist. |
|
||
| Build scripts | Arbitrary scripts are forbidden. A finite declarative action may run a pinned build-machine tool in a denied-by-default sandbox. |
|
||
| Action authority | Exact readable inputs, writable outputs, argv, environment, execution platform, and tool closure. No network, shell, ambient `PATH`, clock, randomness, or undeclared filesystem access. |
|
||
| Acquisition | `add`, `update`, `lock`, `fetch`, `toolchain fetch`, cache transfer, and explicitly authorized remote observation may use the network. Artifact build/analysis and ordinary tests may not; running the finished user program is outside acquisition. |
|
||
| Manifest for a trivial program | No. A standalone directory containing a `main` package is sufficient. A manifest is required for distribution dependencies, multiple products, native providers, or generated inputs. |
|
||
| Configuration placement | Imports in source; identity, requirements, products, native declarations, and actions in `ww.mod`; exact selected closure in `ww.lock`; local paths in `ww.work`; ephemeral target/profile/output choices on the command line. |
|
||
| Local overrides | `ww.work` maps a module identity to a local source tree and records its observed digest. Imports do not change. |
|
||
| Target-specific files | A fixed filename-suffix selection rule; no source-level build expressions and no user-programmable selector. |
|
||
| OS distribution | A distributor may vendor the locked source closure or supply an exact `ww.work`/native-provider map. Substituted files and tools get new digests; frozen mode never silently consults the host. |
|
||
| Reproduction input | For supported official targets: project source, complete locked dependency-source bytes (vendor/CAS export), `ww.lock`, named immutable toolchain bundle, target/profile, and every declared external seed/signing input. Hashes without bytes are insufficient. Impure profiles forfeit the promise. |
|
||
| Assembler/linker ownership | No permanent ownership. The toolchain descriptor pins complete implementations. The current WW tools may bridge migration only. |
|
||
| Stage zero | One release-generated, checked-in portable C99 compiler snapshot plus a tiny declarative bootstrap plan and digest file. |
|
||
|
||
### 1.4 Corrective protocol boundary (2026-08-11)
|
||
|
||
The first Phase 0 experiment over-scoped the protocol freeze. It turned package
|
||
resolution, manifest parsing, compiler projections, action lowering, provider
|
||
recursion, graph traversal, scheduling, cache policy, failure precedence, and
|
||
bootstrap assertions into a declarative expression language. Its checker then
|
||
implemented those operations again. That experiment is preserved as recoverable
|
||
migration evidence, but it is not the production architecture.
|
||
|
||
The correction follows the separation visible in the pinned Go source. Go reads
|
||
imports from source with an imports-only parse and resolves them in ordinary
|
||
loader code ([`go/build/read.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/read.go#272),
|
||
[`cmd/go/internal/load/pkg.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#1775)).
|
||
The compiler writes a narrow export representation in compiler code
|
||
([`cmd/compile/internal/noder/writer.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/writer.go)),
|
||
while `cmd/go` builds and schedules an in-memory action graph with ordinary Go
|
||
functions ([`work/action.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#85),
|
||
[`work/exec.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#73)).
|
||
Action IDs and cache storage/validation are executable hashing and storage
|
||
operations, not schema programs
|
||
([`work.buildActionID`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#260),
|
||
[`internal/cache`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/cache/cache.go#95)).
|
||
Go's `cmd/dist` performs concrete staged builds and checks that the final targets
|
||
are not stale. Separate compiler reproducibility tests compare repeated outputs
|
||
byte-for-byte, while the release process independently rebuilds and compares
|
||
archives bit-for-bit
|
||
([`cmd/dist/build.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/dist/build.go#1404),
|
||
[`reproduciblebuilds_test.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/test/reproduciblebuilds_test.go),
|
||
[`rebuild` account](https://go.dev/blog/rebuild)).
|
||
|
||
WW adopts that division, not Go's module/network/toolchain policy. Normal typed
|
||
C/WW code MUST own source loading, parsing, resolution, compiler behavior,
|
||
lowering, orchestration, storage, and bootstrap execution. Declarative schemas
|
||
MUST describe wire representation only. Tests MUST verify executable behavior;
|
||
a schema or proof-shaped record MUST NOT stand in for running it. A generator is
|
||
permitted only for repetitive codec data and MUST be small, generic,
|
||
deterministic, and byte-for-byte reproducible.
|
||
|
||
Phase 0 therefore freezes only WWAR framing and primitive canonical encoding;
|
||
record/enum/union tags, field order, encoded defaults and record kinds; exact
|
||
domain-separated digest and action-key byte formulas; compact positive and
|
||
malformed-wire vectors; a small reference codec; deterministic data-only codec
|
||
generation; and its repository gate. It does not freeze algorithms for deriving
|
||
the represented records. Every declared record tag remains encoded, including
|
||
an optional field's empty `encoded_default`; absence is not default insertion.
|
||
|
||
The owning implementation phases are binding:
|
||
|
||
| Behavior removed from the Phase 0 experiment | Owning phase |
|
||
|---|---|
|
||
| source imports, package graph/cycles, `.wwe`/`.wwlm`, compiler export and public type/ABI projections | Phase 1 |
|
||
| action construction, graph traversal, scheduling, CAS/cache, environment/sandbox and build failure behavior | Phase 2 |
|
||
| manifest/lock/work/vendor text parsing, module/source resolution, fetching, source-store policy and source-tree construction | Phase 3 |
|
||
| native/provider recursion, lowering, link-plan construction, tool adapters and platform policy | Phase 4 |
|
||
| actual staged bootstrap, fixed-point rebuild and byte comparison | Phase 6 |
|
||
|
||
WW-specific guarantees remain stronger and explicit: frozen artifact builds are
|
||
deterministic and offline, selections are locked, artifacts are content-addressed,
|
||
cached objects are rehashed on read, and bootstrap is established by rebuilding and
|
||
comparing actual bytes. At cutover there is one user-facing build path, as
|
||
already required by the migration plan.
|
||
|
||
## 2. Normative vocabulary
|
||
|
||
| Term | Exact meaning |
|
||
|---|---|
|
||
| **package** | The WW declarations selected from one directory, compiled together under one declared package name and one package identity. |
|
||
| **module** | A distributable, versioned source tree rooted by one `ww.mod`, declaring one globally stable module identity and containing zero or more packages. |
|
||
| **project** | The module or standalone package selected by the user's current command, including its declared products. |
|
||
| **workspace** | A local, non-published set of module-identity-to-directory overlays described by `ww.work`. It changes location, never identity. |
|
||
| **dependency** | A typed directed edge: package import, generated-input edge, tool edge, native-provider edge, runtime edge, ordered link edge, source-input edge, or bootstrap-record edge. A source-input edge is content-rooted and has no producer action; it is valid in template/final action inputs but never in `GraphEdgeV1`. A bootstrap-record edge selects the producer action-record or action-result record for `bootstrap.compare`. The edge kind is never implicit. |
|
||
| **target/platform descriptor** | A canonical architecture/platform/ABI/object/CPU/runtime description. An action labels descriptors by role: execution `B`, product `H`, and optional compiler-output `T`. A target triple is only a short lookup name. |
|
||
| **artifact** | An immutable byte string or canonical directory tree produced by an action and named by a content digest. Materialized files are copies or links, not the artifact's identity. |
|
||
| **toolchain** | An immutable descriptor and content closure containing the compiler, action protocol, export/ABI versions, target descriptors, resource files, runtime implementations, and pinned assembler/linker/archive tools. |
|
||
| **sysroot** | A content-identified target filesystem tree containing the exact headers, libraries, CRT objects, loader metadata, linker scripts, and SDK files exposed to target actions. |
|
||
| **source identity** | `sha256` of the canonical source-tree encoding in section 4.5. It is independent of download URL and checkout path. |
|
||
| **version** | An immutable SemVer release label associated with one module identity and one source identity. It is selection metadata, not package identity. |
|
||
| **product** | A named requested result: executable, static library, shared library, object bundle, test binary, generated tree, documentation tree, or toolchain component. |
|
||
| **action** | A pure, finite build step with a typed canonical record, declared input artifacts, one execution platform, and declared output paths. |
|
||
| **build platform (B)** | The platform on which the build actions execute. |
|
||
| **host platform (H)** | The platform on which the requested product will execute. |
|
||
| **target platform (T)** | For a compiler-like product, the platform for which that product emits code. It is absent for an ordinary executable or library. |
|
||
|
||
Module and package identities are slash-separated ASCII paths. They are
|
||
NFC-normalized, case-sensitive, contain no empty, `.` or `..` segment, and do
|
||
not depend on filesystem case folding. A non-root package identity is written
|
||
`module-id/package/path`; the root package identity is `module-id`. The selected
|
||
module catalog records which module owns each package identity. If two selected
|
||
modules would supply the same package identity, resolution fails rather than
|
||
choosing a longer prefix.
|
||
|
||
A no-manifest invocation gives its sole root package the reserved internal
|
||
identity `@standalone/root`; any external test package gets the reserved identity
|
||
`@test/<first-128-bits-of-SHA256(production-package-identity)>`. These namespaces cannot be
|
||
declared by a module or imported from ordinary source. The default standalone
|
||
executable materializes as `main`, independent of directory basename. A
|
||
standalone package cannot contain/import another local package or be published
|
||
until `ww init` gives it a stable module/package identity.
|
||
|
||
## 3. Source and package rules
|
||
|
||
### 3.1 One directory, one package
|
||
|
||
A package directory contains its immediate regular source files only. The build
|
||
does not follow source symlinks. Nested directories are separate packages. Every
|
||
selected production source MUST begin with the same canonical `package name;`
|
||
clause. The declared name MUST be a valid WW identifier. It need not repeat the
|
||
directory leaf because identity and source qualifier are separate concepts.
|
||
|
||
The following current forms are errors after the migration:
|
||
|
||
- importing a single `.ww` file as if it were a package;
|
||
- placing multiple package blocks in one compilation unit;
|
||
- satisfying an unresolved import from an inline package block;
|
||
- selecting a literal source file as the build root; and
|
||
- finding packages through ordered `-I` search roots.
|
||
|
||
`ww build ./cmd/tool` selects a directory. A directory with one source file is
|
||
still the smallest package and needs no extra metadata.
|
||
|
||
### 3.2 File membership and target selection
|
||
|
||
Production candidates are immediate regular files ending `.ww`, excluding
|
||
`*_test.ww` and generated outputs. Names are byte-sorted after UTF-8 validity and
|
||
normalization checks. A source symlink, duplicate normalized name, case-fold
|
||
collision, or non-regular candidate is a loud error on every host.
|
||
|
||
Target variants use this only convention:
|
||
|
||
```text
|
||
stem[+os][+arch][+environment].ww
|
||
```
|
||
|
||
Recognized tags come from the selected target descriptor, not from the host.
|
||
Files are grouped by `stem`. The matching member with the greatest number of
|
||
tags wins; the untagged member is the fallback. Two equally specific matches are
|
||
an error. Examples are `poll.ww`, `poll+linux.ww`, and
|
||
`poll+linux+amd64+gnu.ww`. This is replacement selection, not additive feature
|
||
selection; additive code uses a distinct stem. Unknown tags are errors. There
|
||
are no boolean selectors, glob expressions, or manifest-defined tag meanings.
|
||
|
||
Test variants put the same tags before the reserved suffix, for example
|
||
`poll+linux_test.ww`; their grammar is `stem[+tags]_test.ww` and the identical
|
||
most-specific rule applies within the test set. A production stem ending
|
||
`_test` is reserved and rejected, preventing a tagged test from being mistaken
|
||
for production source.
|
||
|
||
The selected file-name list is itself an action-key input. Therefore adding or
|
||
removing a more-specific file invalidates the package even when the old files'
|
||
bytes do not change.
|
||
|
||
CPU features and optimization mode always enter the compile key but do not add
|
||
another WW source-selection language. WW-level specialization uses compiler
|
||
intrinsics/runtime dispatch or a distinct package; CPU/float-ABI/PIC-sensitive C
|
||
or assembly uses the finite native `when` constraints in section 8.4. This keeps
|
||
ordinary source membership conventional while still making exceptional native
|
||
selection exact and inspectable.
|
||
|
||
### 3.3 Imports, names, and resolution
|
||
|
||
The canonical forms are:
|
||
|
||
```ww
|
||
import "example.org/codec/hex";
|
||
import wire "example.org/protocol/hex";
|
||
```
|
||
|
||
The quoted string is the package identity. A module package may abbreviate its
|
||
own module prefix with a relative import written `import "./sub/path";`;
|
||
resolution replaces `./` with the importing module identity and normalizes the
|
||
remainder without permitting `..`. Bare dotted imports and filesystem imports
|
||
are deleted. Standalone packages cannot use relative imports.
|
||
|
||
The default source qualifier is the imported package's declared name. An alias
|
||
changes only that qualifier. Two imports producing the same qualifier are an
|
||
error unless one is explicitly aliased. Importing the same identity twice,
|
||
resolving one identity to two sources, or resolving two selected module records
|
||
to the same module identity is a loud collision error.
|
||
|
||
Resolution is exact:
|
||
|
||
1. Build the locked/workspace package catalog by joining every selected module
|
||
identity with its package directories.
|
||
2. Require exactly one catalog owner for the requested package identity; zero is
|
||
unresolved and two is an identity collision, even when one module prefix is
|
||
longer.
|
||
3. Verify that the catalog directory exists in that module source tree and has
|
||
the expected package clause.
|
||
4. Never search another root and never choose by filesystem accident.
|
||
|
||
Imports are parsed by the compiler front end, not a line scanner. Their union
|
||
forms package edges, but name visibility remains file-scoped. Every imported
|
||
qualifier must be used. Package cycles, including self-imports, are reported
|
||
before compilation with one stable identity path through the cycle.
|
||
|
||
If an import's catalog owner is outside the importing module, that owner MUST be
|
||
a direct `require` in the importing module's own `ww.mod`; availability through
|
||
another dependency is not enough. This keeps distribution dependencies as
|
||
explicit as package imports and prevents accidental reliance on a transitive
|
||
selection. `ww add` creates the requirement; source is never rewritten.
|
||
|
||
The sole exception is the toolchain's intrinsic standard module, whose identity
|
||
and source-tree digest are part of the selected toolchain descriptor. It is
|
||
available without a manifest requirement, including to a standalone package;
|
||
it is not searched from an installation directory or upgraded independently.
|
||
|
||
### 3.4 Visibility and internal packages
|
||
|
||
Existing exported-versus-private declaration rules survive. An `internal`
|
||
directory segment adds one resolution rule: a package under
|
||
`M/P/internal/Q` may be imported only by the package `M/P` or a package having
|
||
`M/P/` as a segment prefix. For `M/internal/Q`, the allowed root is module `M`
|
||
and its descendants. This is checked against the catalog's owning module and
|
||
package identities, not checkout paths. There are no friend lists or manifest
|
||
visibility overrides.
|
||
|
||
### 3.5 Tests, examples, documentation, and generated WW
|
||
|
||
Only `*_test.ww` files are test sources. `package p;` tests compile with package
|
||
`p`; `package p_test;` tests compile as a separate external package importing
|
||
the production package normally. Test sources of dependencies are never in a
|
||
consumer graph. `ww test .` tests one package; `ww test ./...` discovers package
|
||
directories under the selected module, excluding hidden, underscore-prefixed,
|
||
vendor, cache, and output directories. Discovery is deterministic and does not
|
||
follow directory symlinks. Test compilation is cached; each selected test binary
|
||
is executed on every command and independent binaries may run in parallel.
|
||
|
||
Each test invocation gets only its declared `test-data` mounted read-only under
|
||
`/data`, its literal `test-env`, a private writable temporary directory, and the
|
||
selected runner/runtime closure. Project/home/host files and network are denied.
|
||
Because execution is an observation rather than an artifact action, real clock,
|
||
process IDs, scheduling, and OS randomness may be exposed and are recorded as
|
||
runner capabilities; their output is never cached or part of byte reproduction.
|
||
|
||
The same-package test action uses the production package identity with the
|
||
non-importable action variant `same-test`; it compiles production and test
|
||
sources together so private names remain visible. An external test uses the
|
||
reserved `@test/<128-bit-production-identity-digest>` package identity and has a
|
||
normal direct import of the production package. Neither identity can collide
|
||
with or be imported by published source.
|
||
|
||
An example is an ordinary package or named product under `examples/`; it has no
|
||
special dependency semantics. Documentation is derived from source comments and
|
||
`.wwe` declarations, not by compiling examples during an ordinary build.
|
||
|
||
Generated WW source MUST be declared as an output of an action and as a generated
|
||
input of exactly one package. It uses the suffix `.wwgen`, does not appear in the
|
||
source tree, may not contain `import` or `package` clauses, and therefore cannot
|
||
discover new graph edges after graph construction. It may refer to built-ins and
|
||
declarations already in its owner package. A generator needing another package
|
||
must have a checked-in owner file that imports it. This restriction keeps the
|
||
complete package graph inspectable before executing generators.
|
||
|
||
A generated fragment may contain a foreign declaration only when the owning
|
||
`package` clause already names its provider slot and the declaration explicitly
|
||
names that slot. It cannot add a native requirement. After generation, the
|
||
compiler verifies imports/package clauses are absent and the observed foreign
|
||
slots exactly match the predeclared set; mismatch is a generator/protocol error.
|
||
|
||
## 4. Dependency distribution model
|
||
|
||
### 4.1 When metadata is required
|
||
|
||
A standalone one-directory executable with no non-toolchain dependency builds
|
||
without metadata. `ww init MODULE` creates a module when the program needs a
|
||
stable import identity, distribution dependencies, multiple packages/products,
|
||
generated inputs, or native declarations.
|
||
|
||
A module tree contains exactly one `ww.mod` at its root; nested manifests are an
|
||
error. A workspace composes separate module roots instead of nesting ownership.
|
||
|
||
`ww.mod` is declarative UTF-8 data. It is not WW code: it has no expressions,
|
||
variables, imports, include files, macros, loops, user functions, or host
|
||
conditionals. Strings use JSON escaping; lists preserve source order and record
|
||
key order is non-semantic. Unknown fields are errors unless a later manifest schema is explicitly
|
||
selected.
|
||
|
||
A minimal module is:
|
||
|
||
```text
|
||
ww-manifest 1
|
||
module = "example.org/hello"
|
||
language = "1"
|
||
toolchain = { id = "ww.org/toolchain", minimum = "v1.4.0" }
|
||
|
||
require "example.org/codec" {
|
||
minimum = "v1.2.3"
|
||
source-index = "https://example.org/codec/.well-known/ww-source"
|
||
}
|
||
```
|
||
|
||
The complete set of top-level clause kinds in schema 1 is `require`, `product`,
|
||
`package`, `action`, and `native`. Global scalar keys are only `module`,
|
||
`language`, and `toolchain`. `require` has required `minimum` and optional
|
||
credential-free `source-index`; the latter maps origin without changing identity.
|
||
The schema tables in section 6.9 close all remaining fields. There is
|
||
deliberately no general `[settings]` escape hatch.
|
||
|
||
The root `main` package is the default executable product, named after the
|
||
module's last segment, with explicit normalized linkage `dynamic` and runtime
|
||
`hosted`. Libraries need no product declaration to be imported.
|
||
Additional or non-default outputs are explicit:
|
||
|
||
```text
|
||
product "inspect" {
|
||
kind = "exe"
|
||
root = "cmd/inspect"
|
||
linkage = "dynamic"
|
||
}
|
||
```
|
||
|
||
### 4.2 Selection rule
|
||
|
||
Versions are `vMAJOR.MINOR.PATCH` SemVer labels with the usual prerelease order.
|
||
A requirement is one minimum version, never a range. Selection chooses the
|
||
greatest minimum requested for each module identity over the complete transitive
|
||
closure and repeats until stable. This is intentionally the small, monotonic
|
||
part of Minimal Version Selection, not every behavior of the Go module command.
|
||
The selected result is written exactly to `ww.lock` by `add`, `update`, or
|
||
`lock`; build never resolves a newer version.
|
||
|
||
Exactly one version of a module identity is selected. A backward-incompatible
|
||
major version `N >= 2` MUST declare a module identity ending `/vN`, and imports
|
||
name that identity. Consequently incompatible releases may coexist as distinct
|
||
identities without an aliasing version resolver. Two versions of the same
|
||
identity cannot coexist.
|
||
|
||
There are no feature sets, optional-dependency activation, target-dependent
|
||
version constraints, upper bounds, wildcard versions, or dependency-wide
|
||
configuration unification. Target variation belongs in source selection and
|
||
native-provider declarations after one source closure is selected.
|
||
|
||
The selected source manifests also contribute one minimum for the same
|
||
toolchain identity. The root lock chooses one exact installed/catalog version at
|
||
least as high as every minimum and records its descriptor/bundle digests. A
|
||
different toolchain identity or unsupported language/export/runtime protocol is
|
||
an error; SemVer alone never overrides protocol compatibility. Dependency
|
||
manifests do not pin the consumer to their development compiler, while an
|
||
application remains exactly reproducible from its lock.
|
||
|
||
### 4.3 Lock file
|
||
|
||
`ww.lock` is generated, canonical, and committed for applications and toolchains.
|
||
Published libraries SHOULD commit it for their own tests, but consumers resolve
|
||
from `ww.mod` requirements. A lock record is:
|
||
|
||
```text
|
||
ww-lock 1
|
||
root-manifest = "sha256:9c..."
|
||
toolchain "ww.org/toolchain" {
|
||
version = "v1.4.0"
|
||
descriptor = "sha256:31..."
|
||
bundle = "sha256:80..."
|
||
origin = "https://dist.wwlang.org/toolchain/v1.4.0/"
|
||
}
|
||
module "example.org/codec" {
|
||
version = "v1.2.3"
|
||
origin = "https://example.org/codec/.ww/v1.2.3.tar.zst"
|
||
archive = "sha256:4a..."
|
||
tree = "sha256:f7..."
|
||
manifest = "sha256:55..."
|
||
signature = "ed25519:key-id:base64..."
|
||
provenance = "https://example.org/codec/.ww/v1.2.3.intoto.jsonl"
|
||
}
|
||
```
|
||
|
||
Records are sorted by identity. Required semantic fields are version, immutable
|
||
origin, archive digest, canonical source-tree digest, and manifest digest.
|
||
Signature and provenance are optional records whose verification policy is
|
||
configured by the user or distributor; hashes are never optional. A signature,
|
||
when present, covers schema, module identity, version, tree digest, and manifest
|
||
digest. Lock files never contain local overlay paths or credentials.
|
||
|
||
### 4.4 Acquisition and network policy
|
||
|
||
`ww add M@V` discovers `M` by the HTTPS convention
|
||
`https://M/.well-known/ww-source`, unless `--from=URL` or a user-configured
|
||
longest-prefix source map supplies an index. The index returns immutable archive
|
||
locations and signed digest records. Redirects and the final URL are recorded.
|
||
The downloaded module manifest MUST declare exactly `M`; the signed index/lock
|
||
record, not source text, binds `V` to its tree digest. Private indexes use the
|
||
same protocol and obtain credentials from the fetch command's credential
|
||
helper; credentials never enter build actions or lock files.
|
||
|
||
`(module identity, version)` is immutable: observing two signed tree digests for
|
||
the same pair is an equivocation error recorded in the source store, never an
|
||
automatic replacement. Yank metadata may prevent new selection but cannot alter
|
||
or invalidate an already locked digest.
|
||
|
||
Only these operations may initiate network requests:
|
||
|
||
- `ww add`, `ww update`, and `ww lock` while selecting metadata;
|
||
- `ww fetch --locked` while materializing the already locked source/toolchain
|
||
closure;
|
||
- `ww toolchain fetch` for an explicitly named toolchain; and
|
||
- explicit `ww cache pull` and `ww cache push`; and
|
||
- explicitly authorized `ww observe` remote execution, which is not a build,
|
||
test-build, or cached artifact action.
|
||
|
||
The build/analysis phases of `ww build`, `run`, `test`, `doc`, `install`,
|
||
`graph`, and `explain` deny network access even when an input is absent. They report the missing source digest and
|
||
the exact `ww fetch --locked` command. They never modify `ww.mod`, `ww.lock`, or
|
||
`ww.work`. `--frozen` additionally requires those files to be present,
|
||
canonical, mutually consistent, and unchanged by selection. `--offline` is an
|
||
explicit assertion of the already mandatory no-network build policy.
|
||
|
||
### 4.5 Source identity and storage
|
||
|
||
A module source tree contains only directories and regular files; symlinks,
|
||
devices, sockets, FIFOs, absolute paths, `..`, duplicate normalized paths, and
|
||
case-fold collisions are rejected. Its identity is:
|
||
|
||
```text
|
||
SHA256("ww-source-tree-v1\0" ||
|
||
for each byte-sorted relative path:
|
||
LP(path) || type || executable-bit || LP(SHA256(file-bytes)))
|
||
```
|
||
|
||
`LP(x)` is an unsigned 64-bit big-endian byte length followed by `x`. Directory
|
||
entries are included with type `dir`; regular files with type `file`. Ownership,
|
||
timestamps, archive compression, checkout path, and non-executable permission
|
||
bits are excluded. Archives are checked both against their blob digest and the
|
||
unpacked tree digest. The source store is immutable and keyed by tree digest.
|
||
|
||
### 4.6 Workspaces, vendoring, and distributors
|
||
|
||
`ww.work` is local, declarative, and normally uncommitted:
|
||
|
||
```text
|
||
ww-work 1
|
||
use "example.org/codec" {
|
||
path = "../codec"
|
||
expect = "sha256:f7..."
|
||
}
|
||
```
|
||
|
||
An overlay replaces only the location for the named module identity. The module
|
||
at that path must declare the same identity. Its current canonical tree digest
|
||
is an action input; `expect` makes accidental drift loud but may be updated by
|
||
`ww work sync`. No import or lock identity changes.
|
||
|
||
`ww vendor` materializes every locked module under `vendor/sha256/<tree>` and
|
||
writes a canonical identity-to-tree `vendor/index.wwv`. Frozen builds may select
|
||
that source store with `--vendor`; the vendored bytes must match the lock. There
|
||
is no flattened import tree and no rewritten import statement.
|
||
|
||
An operating-system distributor has three honest options: ship this vendor
|
||
store, prefill WW's immutable source store, or provide an exact workspace/source
|
||
map to distro-owned trees. Native system libraries are substituted only through
|
||
the explicit provider mechanism in section 8.8. A mutable `/usr` lookup is an
|
||
impure system profile, is local-cache-only, and is rejected by frozen builds.
|
||
|
||
### 4.7 Closed metadata grammar
|
||
|
||
`ww.mod`, `ww.lock`, `ww.work`, vendor indexes, and toolchain/native descriptors
|
||
share this lexical grammar; each schema separately closes its allowed headers,
|
||
clauses, fields, value types, and cardinalities:
|
||
|
||
```text
|
||
document = header newline { statement } EOF
|
||
header = schema-name SP unsigned
|
||
statement = assignment | clause
|
||
assignment = key ws "=" ws value ws newline
|
||
clause = key ws string ws "{" newline
|
||
{ assignment } "}" ws newline
|
||
value = string | unsigned | boolean | list | record
|
||
list = "[" ws [ value { ws "," ws value } [ ws "," ] ] ws "]"
|
||
record = "{" ws [ pair { ws "," ws pair } [ ws "," ] ] ws "}"
|
||
pair = key ws "=" ws value
|
||
key = ALPHA { ALPHA | DIGIT | "_" | "-" }
|
||
string = JSON-string-with-valid-UTF-8
|
||
unsigned = "0" | ("1"…"9" { DIGIT })
|
||
boolean = "true" | "false"
|
||
ws = { SP | TAB | newline | comment }
|
||
comment = "#" { any-character-except-newline }
|
||
```
|
||
|
||
`schema-name` is exactly `ww-manifest`, `ww-lock`, `ww-work`, `ww-vendor`,
|
||
`ww-toolchain`, `ww-native-map`, `ww-native-sidecar`, `ww-install`, or
|
||
`ww-bootstrap`. A clause
|
||
body contains assignments only, so nesting cannot grow into a language.
|
||
Duplicate keys, duplicate singleton clauses, invalid UTF-8, unknown fields,
|
||
integer overflow, and a comment marker inside an unclosed string are errors.
|
||
|
||
Whitespace, comments, assignment order, record-key order, and clause order where
|
||
the schema declares identity keys are non-semantic. List order is semantic.
|
||
Parsing produces a typed record whose canonical semantic encoding is
|
||
`WWAR(record)`, not the original text. Its semantic digest is the applicable
|
||
kind/schema-bound `record_id` from section 6.6. `ww fmt` writes two-space
|
||
canonical text; generated lock/vendor files MUST already equal that rendering
|
||
in frozen mode.
|
||
|
||
## 5. Build model and graph construction
|
||
|
||
### 5.1 One graph, constructed before execution
|
||
|
||
For every command, `ww` constructs a typed graph in these deterministic phases:
|
||
|
||
1. Select the project, manifest, lock, workspace, toolchain descriptor, target,
|
||
and profile. Verify their schemas, canonical identities, and content digests.
|
||
Missing locked inputs are errors; this phase never fetches.
|
||
2. Enumerate selected checked-in package files by section 3.2. Ask the compiler
|
||
front end to parse package clauses, imports, checked-in foreign declarations,
|
||
and test metadata. Generated artifacts are known future input slots but do not
|
||
yet exist. The build driver never scans source lines itself.
|
||
3. Resolve every import by identity, reject collisions/internal violations, and
|
||
compute the complete acyclic package graph.
|
||
4. Add statically declared generated-input, host-tool, native-provider,
|
||
toolchain, runtime, archive, ordered link, and bootstrap-record comparison
|
||
edges. Match target clauses and reject zero or multiple providers. All
|
||
generated output names and consumers are known here.
|
||
5. Lower nodes to the complete **action-template DAG** and report it. A template
|
||
names every edge/input slot, tool, policy, and output, but its final key remains
|
||
unresolved until every predecessor output or selected record content digest
|
||
is known. Only after this point may a cache be read or a tool execute.
|
||
6. As verified cache results or completed predecessors resolve input artifacts,
|
||
finalize ready action records/keys, query the cache, schedule misses, publish
|
||
successful artifacts atomically, then materialize requested products.
|
||
|
||
No executed action may add a node, input, output, import, library, flag, or
|
||
follow-up command. Native C/assembly declarations name complete header/source
|
||
trees rather than learning dependencies from an ambient compiler depfile. This
|
||
may conservatively rebuild for an unused header change, but the graph remains
|
||
complete before execution and the key is correct.
|
||
|
||
`ww graph --actions --format=json` emits the graph after phase 5. Its canonical
|
||
JSON contains node kind, execution platform, typed input/output slots, incoming
|
||
edge kinds, target/profile/toolchain digests, and the exact ordered link plan.
|
||
A resolved node also has its key/cache status; otherwise it has `key: "pending"`,
|
||
`cache: "unknown"`, and a byte-sorted `waiting-on` list. It contains logical
|
||
paths only. A collision, cycle, missing provider,
|
||
undeclared target, or unresolved tool fails graph construction even if a stale
|
||
cache entry might otherwise satisfy the product.
|
||
|
||
### 5.2 Built-in action kinds
|
||
|
||
Schema 1 has this closed set of semantic action kinds:
|
||
|
||
- `ww.package`: compile one package to export data, object code, and link
|
||
metadata;
|
||
- `ww.init`: synthesize one deterministic retained package-initialization
|
||
dispatcher from precomputed package link metadata;
|
||
- `native.compile`: compile one declared C or assembly source unit;
|
||
- `archive`: construct a static library from an ordered object list;
|
||
- `link`: construct an executable or shared library from an ordered link plan;
|
||
- `generate`: execute one declared build-platform tool;
|
||
- `doc`: render a documentation tree from sources and export data;
|
||
- `bootstrap.compare`: compare canonical stage outputs and manifests.
|
||
|
||
Adding an action kind changes the action schema. There is no generic rule engine,
|
||
phony target, implicit suffix rule, command-string target, or shell recipe. An
|
||
archive is made only for an explicit static-library product or native provider;
|
||
WW packages are not automatically wrapped in one-member archives.
|
||
|
||
`run` and `test.run` are non-cacheable invocation nodes, not semantic artifact
|
||
actions. `test.run` consumes the declared data/environment/runner and uses the
|
||
test sandbox above on every request. `ww run`, after its network-denied build,
|
||
launches the user's program with the user's runtime authority/environment unless
|
||
`--sandbox` is explicitly requested; that execution still cannot affect a build
|
||
cache entry. Both store exit status/logs only as observations. Materialization/install is a third category:
|
||
a request-local side effect consuming an immutable artifact/install manifest.
|
||
Neither category can satisfy or poison an artifact-action cache entry.
|
||
|
||
The semantic graph is independent of process boundaries. An implementation MAY
|
||
run compiler workers in-process or in a bounded pool, but each `ww.package`
|
||
action still has an independent canonical record and outputs. There is no
|
||
required background daemon and no daemon state may affect an output.
|
||
|
||
### 5.3 The finite escape hatch
|
||
|
||
Unusual generation and packaging use a declarative `action`, not a build script:
|
||
|
||
```text
|
||
action "protocol-bindings" {
|
||
tool = "product:tools/schema-gen"
|
||
platform = "build"
|
||
inputs = {
|
||
schema = { file = "protocol/schema.idl" }
|
||
}
|
||
outputs = {
|
||
ww = { file = "generated/protocol.wwgen" }
|
||
}
|
||
argv = ["--input", "/in/schema", "--ww", "/out/ww"]
|
||
env = { LANG = "C", TZ = "UTC" }
|
||
}
|
||
|
||
package "protocol" {
|
||
generated = ["action:protocol-bindings:ww"]
|
||
}
|
||
```
|
||
|
||
The tool is either a named executable in the immutable toolchain or a named WW
|
||
product built for `H = B`. `/in/NAME` inputs are read-only mounts and
|
||
`/out/NAME` outputs are initially absent, exclusive writable mounts. The working
|
||
directory is the empty logical `/work`. Output type is exactly `file` or `tree`;
|
||
undeclared files fail the action. `argv` is passed directly, never through a
|
||
shell. Input and output names are identifiers and each path appears through its
|
||
fixed mount, so there is no template language.
|
||
|
||
The sandbox exposes only the declared tool closure at fixed logical `/tool`
|
||
paths, `/in`, `/out`, the literal
|
||
environment map, deterministic locale/time-zone data, and bounded CPU/memory
|
||
resources. Network, process inspection, host devices, user/home directories,
|
||
ambient `PATH`, ambient environment, wall clock, writable source, and filesystem
|
||
paths outside the mounts are denied. Randomness is absent unless a declared
|
||
seed artifact is mounted. A tool may spawn only executable inputs declared in
|
||
its tool closure. These isolation rules apply equally to built-in compiler, C,
|
||
assembler, archiver, linker, documentation, test, and bootstrap executions.
|
||
Strong enforcement is required for frozen/shared-cache builds;
|
||
an unsupported host must fail rather than silently weaken isolation.
|
||
|
||
This action can perform arbitrary computation over finite declared inputs, so it
|
||
is sufficient for code generators, image/file-system builders, binding tools,
|
||
and signing-input preparation. It cannot inspect the project and invent more
|
||
work. Dependency modules may declare actions only for outputs consumed by their
|
||
own packages/products; they cannot register hooks that run merely because the
|
||
module is present.
|
||
|
||
Every artifact-producing action declares `reproducibility = "required"` in a
|
||
frozen or official build. Isolation removes undeclared external inputs, but it
|
||
cannot prove that arbitrary tool internals avoid PIDs, uninitialized memory,
|
||
ASLR-derived values, or race-dependent output. Such variance is a tool/action
|
||
defect. Official releases and first shared-cache publication of a custom
|
||
generator repeat it from clean sandboxes and compare outputs. A node explicitly
|
||
classified `observation` is never result-cached/shared and may not feed an
|
||
artifact action; tests and hardware execution use that class. Impure development
|
||
actions are local-only and outside the byte promise.
|
||
|
||
### 5.4 Build, host, and target
|
||
|
||
WW uses the conventional three-platform meaning rigorously:
|
||
|
||
```text
|
||
B: execution platform on which build actions run
|
||
H: platform ABI of the produced artifact; executable products are intended to run here
|
||
T: output platform of a compiler-like artifact that itself is built for H
|
||
```
|
||
|
||
For an ordinary program `T` is absent. The familiar
|
||
`ww build --target=aarch64-unknown-linux-gnu` spelling sets `H`; it means “build
|
||
the program that runs on this target.” A compiler product may additionally set
|
||
`--host=H --target=T`. Any generator used while producing it still executes on
|
||
`B`; if the generator is itself WW source, its product is compiled with `H = B`.
|
||
|
||
Every action record carries all applicable descriptors, even when two are equal.
|
||
No rule may infer `H` or `T` from the kernel running `ww`. Cross compilation is
|
||
therefore the same graph with a different explicit host/target descriptor, not
|
||
a mode that edits environment variables.
|
||
|
||
In record/JSON field names these are `execution-platform`, `product-platform`,
|
||
and optional `compiler-output-platform`. Source suffix selection, ordinary
|
||
native-provider `when`, sysroot, CRT, runtime, and linker selection always match
|
||
the product platform H. A build-tool dependency instead has its own H equal to
|
||
the parent action's B. CLI `--target` is only the familiar spelling for selecting
|
||
the ordinary product platform; it does not rename the GNU roles internally.
|
||
|
||
### 5.5 Scheduling and failure
|
||
|
||
After graph construction, ready actions run in a deterministic priority order
|
||
with a user-selected concurrency bound. Priority affects latency only; output
|
||
bytes and link order come from records, never completion order. Independent
|
||
actions may finish after another branch fails, but no dependent action starts.
|
||
On the first observed failure WW stops launching work and cancels its owned
|
||
in-flight actions. Which failure triggers cancellation is observational and may
|
||
vary with concurrency; every concurrently observed failure is sorted by stable
|
||
logical node in the report. `--keep-going` instead continues branches whose
|
||
dependency closure remains healthy and reports all failures in that stable
|
||
order. Interrupts cancel only processes owned by this invocation and leave no
|
||
published partial result.
|
||
|
||
Tool stdout and stderr are captured as artifacts and streamed with node labels.
|
||
Diagnostics use module-relative logical paths. `--verbose` may display physical
|
||
mount paths separately, marked non-semantic. A successful action is published
|
||
only after all declared outputs exist, have valid type/mode, are canonicalized
|
||
where required, and have been hashed. A failed action is never entered in the
|
||
action cache.
|
||
|
||
### 5.6 Atomic publication and materialization
|
||
|
||
CAS files and action-result records are written to same-filesystem unique
|
||
temporary names, flushed, rehashed, then atomically renamed to their digest
|
||
locations. For crash durability, WW flushes the parent directory after rename;
|
||
it publishes and flushes every output before the result mapping. A concurrent
|
||
publisher of the same digest verifies equality and
|
||
discards its temporary file. A directory artifact is a canonical tree object
|
||
whose leaves are CAS blobs. An action-result mapping is published last, so no
|
||
reader can observe a result before its outputs.
|
||
|
||
Materialization is a request-local side effect outside the action-template DAG.
|
||
By default requested
|
||
products appear in `out/<target>/<profile>/`; `--out` and `ww install --prefix`
|
||
change only where immutable artifacts are copied or copy-on-write reflinked.
|
||
Hardlinks/symlinks are permitted only when the backing store is enforced
|
||
immutable against the user and mode changes cannot affect its inode. Executable bits are
|
||
set by the artifact record, never by a later ambient `chmod`. Replacement uses
|
||
temporary siblings and atomic rename. `ww clean` removes materialized/project
|
||
state only; `ww cache gc` is the separate explicit global-cache operation.
|
||
|
||
### 5.7 Reproducibility contract
|
||
|
||
For official supported targets, WW promises byte-identical artifacts when every
|
||
artifact action satisfies `reproducibility = "required"` and these are identical:
|
||
|
||
- canonical project source and `ww.lock`;
|
||
- immutable toolchain descriptor and complete bundle;
|
||
- target descriptor and build profile; and
|
||
- all declared action inputs, including generated seeds and signing material.
|
||
|
||
The engine guarantees input isolation, logical paths, and canonical publication;
|
||
repeat-build certification checks arbitrary tool determinism. The resulting promise is independent of absolute checkout, source-store, cache, output,
|
||
and temporary paths; wall time, locale, process order, username, UID, and host
|
||
environment are absent. Logical paths are module/package paths. Debug information
|
||
uses those logical paths and fixed prefix maps. Archive metadata is canonical;
|
||
timestamps and ownership are zeroed; deterministic linker build IDs derive from
|
||
the link key. Official toolchains reject tools that cannot meet this contract.
|
||
|
||
A project source archive plus the complete locked dependency source bytes (a
|
||
vendor/source-CAS export), its lock, the named immutable toolchain bundle, and
|
||
every declared external seed/signing input is therefore a complete offline
|
||
reproduction input. A lock's hashes alone cannot recreate absent bytes. Runtime behavior that depends on a shared library
|
||
outside the pinned runtime/sysroot closure is not covered, and frozen official
|
||
profiles prohibit such a dependency. An explicitly selected impure system
|
||
profile receives no byte-identity promise, cannot publish to a shared cache, and
|
||
prints every ambient input it accepted. `ww verify reproducible` runs isolated
|
||
uncached builds under two physical roots and compares every result artifact and
|
||
action manifest, not just the final executable.
|
||
|
||
## 6. Action records and cache protocol
|
||
|
||
### 6.1 Canonical encoding
|
||
|
||
Action records use **WWAR 1**, this deterministic byte encoding:
|
||
|
||
```text
|
||
WWAR(record) = 0x57 0x57 0x41 0x52 | u16be(1) | value(record)
|
||
value(v) = type:u8 | u64be(payload-length) | payload
|
||
|
||
type 0x01 bytes: payload is the bytes
|
||
type 0x02 string: payload is valid NFC UTF-8 with no NUL
|
||
type 0x03 uint: payload is minimal unsigned big-endian; zero is one 0x00
|
||
type 0x04 bool: payload is exactly 0x00 or 0x01
|
||
type 0x05 list: u32be(count) | each (u64be(value-length) | value)
|
||
type 0x06 map: u32be(count) | each (u64be(key-length) | key-UTF-8 |
|
||
u64be(value-length) | value)
|
||
type 0x07 record: u32be(count) | each (u32be(field-tag) |
|
||
u64be(value-length) | value)
|
||
```
|
||
|
||
Record fields are strictly increasing by numeric tag. Map entries are strictly
|
||
increasing by raw UTF-8 key bytes. Duplicate/out-of-order keys or tags, leading
|
||
zeroes in a uint, invalid booleans/UTF-8/NFC, mismatched counts/lengths, unknown
|
||
schema tags, and trailing bytes are errors. Lists preserve declared order.
|
||
Schema defaults are always encoded, so no semantic field is inferred from
|
||
absence. Floats, signed integers, null, and indefinite lengths do not exist.
|
||
|
||
One byte string/string is at most `2^31-1` bytes, a container has at most
|
||
`2^24-1` members, and nesting depth is at most 64. A content/container-relative
|
||
logical path string uses `/`, is relative, has no NUL/backslash, empty/`.`/`..`
|
||
segment, and passes the schema's ASCII-identity or NFC-source-path rule. A field
|
||
that explicitly permits `.` as its complete root sentinel is the sole exception.
|
||
These are protocol limits, not host `size_t` limits. Human-readable JSON is a
|
||
lossless rendering, not the hashed representation. Phase-0 golden vectors
|
||
include empty/nested records, ordered lists, sorted maps, every rejection, and
|
||
their complete bytes/digests.
|
||
The normative empty-record vector is
|
||
`57574152000107000000000000000400000000`, SHA-256
|
||
`138c6acb7f01e91df73cb1d9c3356d18f19d7b8eb8b0a15426bef32e515d0de0`.
|
||
|
||
Each schema assigns every path field a path class. Artifact, source, generated
|
||
output, install-destination, bundle-relative, sysroot-relative, and
|
||
vendor-relative paths use the relative rule above. Sandbox-execution paths are
|
||
path-independent absolute paths only in the closed virtual namespaces `/work`,
|
||
`/in`, `/out`, `/tool`, and `/data`; schema-1 action working directory is exactly
|
||
`/work`, and exact sandbox path spellings in argv are encoded. Platform-validated
|
||
target-runtime paths are a separate type and may be absolute in H's namespace.
|
||
Workspace locations and observation physical paths are separately typed and
|
||
never enter an artifact action record or key as host locations. Absolute host
|
||
paths, filesystem device/inode numbers, mtimes, cache locations, and command
|
||
process IDs are invalid in action templates and final action records. A physical
|
||
input enters only through a logical name, content digest, type, and semantic
|
||
mode.
|
||
|
||
### 6.2 Required action-record fields
|
||
|
||
Every record contains, in this order:
|
||
|
||
1. WWAR schema and action kind/version;
|
||
2. language edition, compiler protocol, export protocol, object ABI, runtime ABI,
|
||
manifest schema, and lock schema;
|
||
3. B, H, and optional T descriptor digests plus the expanded target fields;
|
||
4. toolchain identity, selected descriptor-slice/closure digest,
|
||
compiler/backend digest, and every executable/shared/resource digest actually
|
||
used by the action; the distribution bundle root/version authenticates
|
||
acquisition but unused targets/tools do not invalidate this action;
|
||
5. profile fields: optimization, debug, assertions, overflow, panic, sanitizers,
|
||
LTO, relocation/code model, symbol stripping, and reproducibility policy;
|
||
6. logical package/product/action identity and sandbox-virtual working directory
|
||
(schema 1 exactly `/work`);
|
||
7. exact argument vector and a sorted literal environment map;
|
||
8. byte-sorted named inputs, each with edge kind, logical path, semantic artifact
|
||
kind, semantic mode, content digest, and—where applicable—origin package identity;
|
||
9. direct export-data inputs byte-sorted by package identity for `ww.package`
|
||
actions;
|
||
10. selected source-membership list and target-selection explanation;
|
||
11. typed native declarations: headers, objects, archives, shared libraries,
|
||
sysroot, SDK, libc, CRT, dynamic loader, linker scripts and their include
|
||
closures, assembler/linker/archive tools, and ABI-provider slots;
|
||
12. the exact ordered link plan, retaining archive groups, whole-archive markers,
|
||
as-needed state, and repeated libraries;
|
||
13. named output paths, types, modes, and canonicalization policies; and
|
||
14. sandbox policy/version, resource bounds, and reproducibility classification.
|
||
|
||
Fields irrelevant to an action are encoded as empty values, not inferred. Native
|
||
flags exist only as typed fields whose meaning is part of a tool adapter. A raw
|
||
flag can be used only in a custom toolchain declaration and then its exact bytes
|
||
are part of the record; the official profile has no hidden default flags.
|
||
|
||
### 6.3 Environment and tool discovery
|
||
|
||
No inherited environment variable is semantic. Built-in actions receive the
|
||
fixed environment specified by their toolchain adapter. A declarative action
|
||
receives only its `env` record. `PATH`, compiler-driver defaults, `pkg-config`,
|
||
shell initialization, host include/library directories, and current directory
|
||
are never consulted to discover an input.
|
||
|
||
User configuration may choose a cache location, concurrency, output directory,
|
||
credential helper, source mirror, or display preference; these do not enter an
|
||
action because they cannot alter output bytes. Choosing a toolchain, target,
|
||
profile, workspace overlay, native provider, environment value, raw option, or
|
||
impure system mapping can alter bytes and therefore always enters the record.
|
||
|
||
### 6.4 Package action inputs and invalidation
|
||
|
||
A `ww.package` action consumes:
|
||
|
||
- the exact selected production or test source files and generated fragments;
|
||
- their ordered membership metadata;
|
||
- only the `.wwe` artifacts of direct imported packages;
|
||
- the compiler/backend and toolchain resources;
|
||
- B/H/T, target descriptor, profile, language/compiler/export/object/runtime
|
||
protocols, and manifest/lock schemas;
|
||
- package-specific predeclared native-provider slots, distinct from selected
|
||
concrete provider declarations; and
|
||
- its literal built-in environment and sandbox policy.
|
||
|
||
It emits `.wwe`, one target object, and canonical link metadata. A private change
|
||
in a dependency changes that dependency's object and the final link key, but not
|
||
the importer's action key. A public change changes the dependency's `.wwe` and
|
||
therefore its direct importers. If a rebuilt importer emits byte-identical
|
||
`.wwe`, the invalidation stops there. Link-only input changes only invalidate
|
||
link/archive actions; materialization merely recopies a newly selected immutable
|
||
artifact when its requested result digest changes.
|
||
|
||
A package's own source bytes remain part of its action key even if a compiler
|
||
could prove a change dead. A target/profile/tool/runtime ABI change creates a
|
||
different key. There is no timestamp freshness shortcut and no “artifact exists”
|
||
predicate.
|
||
|
||
### 6.5 Link action inputs
|
||
|
||
The link action consumes every reachable package/native object digest, explicit
|
||
archive/shared-library digest, CRT object, runtime object, dynamic-loader choice,
|
||
linker script closure, sysroot descriptor, target/profile, exact linker tool and
|
||
resources, and the ordered plan. Objects are ordered by stable package identity;
|
||
an explicit static product's `members` list controls only that archive's member
|
||
order. Native archives retain declared
|
||
order; repeated archives remain repeated; group and whole-archive boundaries are
|
||
semantic. `-L`/`-l` token collections are not an internal representation.
|
||
|
||
The link result is cacheable. A warm identical build need not invoke the linker.
|
||
Changing output/materialization path alone does not change the link key. Changing
|
||
a private package implementation normally preserves importer objects but changes
|
||
the final link key through that package's object digest.
|
||
|
||
### 6.6 Complete cache-key formula
|
||
|
||
Let `R` be the complete WWAR record from sections 6.2–6.5. The action key is:
|
||
|
||
```text
|
||
K = SHA256("WW-ACTION-KEY\0" || uint64be(len(WWAR(R))) || WWAR(R))
|
||
```
|
||
|
||
Input entries contain the SHA-256 digest of their canonical artifact bytes/tree,
|
||
not merely the producer's action key. Thus semantically identical outputs stop
|
||
rebuild propagation even when their producing source/action key changed. Ordered
|
||
fields remain ordered; only fields specified as maps are sorted. The domain
|
||
separator and schemas prevent a digest from one protocol being reinterpreted in
|
||
another.
|
||
|
||
A successful result record contains `K`, result-schema version, output
|
||
name/type/mode/digest tuples only. Stdout, stderr, exit/diagnostic presentation,
|
||
timing, worker identity, resource use, and physical paths go in a separate
|
||
invocation-observation record. Thus one action key has exactly one semantic
|
||
successful result even if its logs differ. Observation records may be
|
||
content-addressed, but never participate in action mapping, cache hits, or
|
||
reproducibility comparison.
|
||
|
||
CAS identities are type-separated. Blob bytes use their own domain; every
|
||
structured object is additionally bound to its top-level record kind and schema:
|
||
|
||
```text
|
||
blob_id = SHA256("WW-BLOB\0" || u64be(length) || bytes)
|
||
|
||
record_id(kind, schema, record) =
|
||
SHA256("WW-RECORD\0" || u32be(kind) || u32be(schema) ||
|
||
u64be(length(WWAR(record))) || WWAR(record))
|
||
```
|
||
|
||
Schema 1 reserves these top-level `kind` numbers: 1 tree, 2 manifest, 3 lock,
|
||
4 workspace, 5 vendor index, 6 native map, 7 platform descriptor, 8 toolchain,
|
||
9 profile, 10 action template, 11 action record, 12 action result,
|
||
13 observation, 14 graph snapshot, 15 native sidecar, 16 native ABI contract,
|
||
17 link plan, 18 export, 19 package link metadata, 20 install manifest,
|
||
21 bootstrap plan, and 22 selected-tool closure. An unknown kind is never
|
||
decoded as another record. `TypedDigestV1` is a record with tag 1 domain
|
||
(`blob` or `record`), tag 2 algorithm (schema 1 only `sha256`), tag 3 record kind
|
||
(`0` for a blob), tag 4 record schema (`0` for a blob), and tag 5 the exact
|
||
32 digest bytes. A field whose type is “typed digest” always means this record;
|
||
a bare hexadecimal string is only a text rendering.
|
||
|
||
A tree record is schema 1 plus a list sorted by entry-name UTF-8 bytes. An entry
|
||
is `(name, kind=file|tree, executable:boolean, typed-child-digest)`. `name` is
|
||
one normalized path segment. Empty directories are explicit tree children;
|
||
symlinks, hardlink identity, devices, xattrs, uid/gid, mtimes, and non-executable
|
||
permission bits do not exist. Duplicate normalized or case-fold-colliding names
|
||
are errors. Validation recursively checks every typed child to a blob; verifying
|
||
only a root digest is insufficient.
|
||
|
||
### 6.7 Storage, sharing, and corruption
|
||
|
||
The default cache is per-user, global across that user's checkouts, local, and
|
||
private to the account:
|
||
|
||
```text
|
||
<cache>/v1/cas/sha256/aa/bb...
|
||
<cache>/v1/actions/sha256/aa/bb...
|
||
<cache>/v1/quarantine/
|
||
```
|
||
|
||
The first path stores blob objects and kind/schema-bound structured records; the
|
||
second maps an action key to `(action-record digest, result-object digest)`. A small ignored
|
||
project index `.ww/state-v1` references the previous successful graph snapshot,
|
||
whose logical nodes point to action-record/key/result digests. It roots that
|
||
history until replacement so explanation can compare records and follow causes;
|
||
it is disposable and never proves freshness. After explicit GC removes history,
|
||
`explain` reports `history-unavailable` rather than inventing “stale.” A
|
||
system-wide cache service requires authenticated isolated writers and the same
|
||
signed-mapping policy as a remote cache.
|
||
|
||
On every cache read, WW verifies the requested object's digest and canonical
|
||
type, decodes the mapped action record, recomputes `K` from it, and requires
|
||
`recomputed K = lookup K = ActionResultV1.tag2` plus
|
||
`ActionResultV1.tag3 = the mapping's typed action-record digest`. It then verifies
|
||
all output objects before use or materialization. A mismatch moves only that
|
||
explicit entry to quarantine, removes its action mapping, reports corruption,
|
||
and rebuilds.
|
||
`ww cache verify` walks the store; `ww cache gc` traces retained action results
|
||
and materializations. A tool upgrade changes its content/descriptor fields and
|
||
cannot reuse the old key.
|
||
|
||
If two executions of one `reproducibility=required` action key produce different
|
||
semantic result digests, WW publishes neither as an authoritative replacement,
|
||
records both observations/artifact sets in quarantine, and fails with a
|
||
nondeterminism diagnostic. Impure actions have no reusable action mapping.
|
||
|
||
Shared caches are opt-in explicit transports. `ww cache pull` imports only
|
||
content-addressed objects and action mappings in an Ed25519 signed envelope over
|
||
`"WW-CACHE-MAP\0"`, cache namespace, action key, action-record digest, result
|
||
digest/schema, and reproducibility/policy classification. The envelope carries
|
||
a signing-key ID; configured trust policy handles rotation/revocation. Hashes
|
||
prove bytes; the trusted cache signing key authorizes the asserted key-to-result
|
||
mapping. All hashes are reverified. An unsigned/untrusted mapping is treated as a miss
|
||
even if its referenced blobs exist. `ww cache push` refuses impure,
|
||
non-reproducible, secret-bearing, or policy-incompatible actions. Literal secret
|
||
environment values are forbidden; a required secret is a classified file input,
|
||
redacted from JSON/explain, and makes the action non-shareable. Ordinary build
|
||
does not contact a shared cache.
|
||
|
||
### 6.8 Explainability
|
||
|
||
For every node, WW retains its last local record and current record. `ww explain
|
||
NODE` reports one of `hit`, `not-built`, `missing-result`, `corrupt-result`,
|
||
`policy-rejected`, or `key-changed`. For `key-changed` it prints the first and,
|
||
with `--all`, every differing typed field, for example:
|
||
|
||
```text
|
||
codec/hex: key changed
|
||
input direct-export example.org/base: 71… -> a4…
|
||
caused by base: exported type Header layout changed
|
||
link hello: key changed
|
||
package-object example.org/codec/hex: 19… -> 27…
|
||
```
|
||
|
||
`ww explain --path NODE` follows the shortest changed-input path to a source,
|
||
tool, target, native provider, or policy root. `--format=json` exposes both WWAR
|
||
renderings and field paths. Export-data differences use the normative `ExportV1`
|
||
semantic field/type diff; if old content was explicitly GC'd the command reports
|
||
`history-unavailable`. Cache misses are never explained merely as “stale.”
|
||
|
||
### 6.9 Version-1 semantic record schemas
|
||
|
||
The following tables freeze schema-1 semantic fields and WWAR numeric tags.
|
||
`1` means exactly one, `0/1` optional, `*` a list, and `map` unique string keys.
|
||
Every absent optional value encodes the stated empty/default. Identity-keyed
|
||
lists are byte-sorted by identity; lists marked `ordered` preserve source/link
|
||
order. Nested records use the field tags in their named table. Enums reject
|
||
unknown values rather than passing them to a tool.
|
||
|
||
#### Project and distribution records
|
||
|
||
| Record/tag | Field | Type/cardinality | Rule/default |
|
||
|---|---|---|---|
|
||
| `ManifestV1/1` | schema | uint/1 | `1` |
|
||
| `/2` | module | string/1 | canonical module identity |
|
||
| `/3` | language | string/1 | language edition |
|
||
| `/4` | toolchain | `ToolchainRef`/1 | compatible ID and minimum |
|
||
| `/5` | requires | `Require`/* | sorted by module |
|
||
| `/6` | products | `Product`/* | sorted by name |
|
||
| `/7` | packages | `PackageConfig`/* | sorted by relative path |
|
||
| `/8` | actions | `GenerateDecl`/* | sorted by name |
|
||
| `/9` | natives | `NativeProvider`/* | sorted by name |
|
||
| `ToolchainRef/1` | id | string/1 | toolchain identity |
|
||
| `/2` | minimum | string/1 | minimum compatible SemVer |
|
||
| `Require/1` | module | string/1 | module identity |
|
||
| `/2` | minimum | string/1 | SemVer minimum |
|
||
| `/3` | source-index | string/0/1 | empty means HTTPS convention |
|
||
| `Product/1` | name | string/1 | unique identifier |
|
||
| `/2` | kind | enum/1 | `exe`, `static`, `shared`, `object`, `generated` |
|
||
| `/3` | root | string/0/1 | package-relative path; required except generated |
|
||
| `/4` | linkage | enum/1 | `dynamic`, `pie`, `static`, `static-pie`, `shared`, `none`; kind-valid |
|
||
| `/5` | runtime | string/1 | `hosted` default, `minimal`, `none`, or slot |
|
||
| `/6` | entry | string/0/1 | empty selects typed toolchain default |
|
||
| `/7` | native | string/* | required slots, sorted |
|
||
| `/8` | linker-script | `ArtifactRef`/0/1 | empty |
|
||
| `/9` | providers | `ProviderSelection`/* | sorted by slot |
|
||
| `/10` | members | string/* ordered | static/archive members; root only by default |
|
||
| `/11` | action | string/0/1 | required only for generated product |
|
||
| `/12` | install-name | string/0/1 | platform-validated, empty |
|
||
| `ProviderSelection/1` | slot | string/1 | ABI slot |
|
||
| `/2` | use | string/1 | `module#native-clause` |
|
||
| `PackageConfig/1` | path | string/1 | normalized relative path; `.` root |
|
||
| `/2` | generated | string/* | sorted `action:NAME:OUTPUT` refs |
|
||
| `/3` | native | string/* | sorted provider slots |
|
||
| `/4` | test-data | `InputDecl`/* | sorted names, read-only under `/data` |
|
||
| `/5` | test-env | map | literal non-secret test environment |
|
||
|
||
`ArtifactRefV1` is permitted in declarative configuration records (including
|
||
toolchain/native records) and action templates, but never in a final action
|
||
record. Its tags are: 1 `ArtifactNamespace` (`source`, `generated`, `package`,
|
||
`toolchain`, `sysroot`, `provider-output`, `cas`, or `graph`); 2 owner identity
|
||
(empty only for a root source);
|
||
3 normalized logical name/path; 4 semantic artifact kind (`file`, `tree`,
|
||
`object`, `archive`, `shared`, `import-library`, `script`, `crt`, `loader`,
|
||
`native-sidecar`, `native-abi`, `export`, `package-link`, `action-record`, or
|
||
`action-result`);
|
||
5 optional expected `TypedDigestV1`; and 6 mode (`data` or `executable`). A local
|
||
source may omit tag 5 because analysis hashes it. A `cas`, external prebuilt,
|
||
toolchain, or sysroot reference must include it. A generated/package/provider
|
||
output gets its digest only from the declared predecessor output. Absolute host
|
||
paths are invalid.
|
||
|
||
`InputSlotRefV1`, `TemplateInputV1`, `ResolvedInputV1`, built-in/template/final
|
||
action-output records, and `ResultOutputV1` use that same closed semantic
|
||
artifact-kind enum; `GenerateDecl.OutputDecl` remains restricted to `file` or
|
||
`tree`. In schema 1, `file`, `object`, `archive`, `shared`, `import-library`,
|
||
`script`, `crt`, and `loader` require a blob digest. `tree` requires record kind
|
||
1, `native-sidecar` kind 15, `native-abi` kind 16, `export` kind 18, and
|
||
`package-link` kind 19, each at record schema 1. Schema-1 artifact-kind values 14
|
||
`action-record` and 15 `action-result` require record kinds 11 and 12,
|
||
respectively, at record schema 1. They are input-only and valid only for
|
||
`bootstrap.compare`; they are invalid in built-in, template, or final action
|
||
outputs and in `ResultOutputV1`. Any other digest domain, record kind, or schema
|
||
is invalid kind substitution.
|
||
|
||
The `graph` namespace has one exact form. Its consumer is `bootstrap.compare`,
|
||
the `TemplateInputV1` edge kind is `bootstrap-record`, and
|
||
`ArtifactRefV1.tag2` is the producer logical node. Tag 3 is the Identifier
|
||
selector `action_record` for artifact kind `action-record` or `action_result`
|
||
for artifact kind `action-result`; tag 5 is absent. `ArtifactRefV1.tag6` and
|
||
`TemplateInputV1.tag5` are `data`, `TemplateInputV1.tag4` repeats the
|
||
corresponding artifact kind, and `TemplateInputV1.tag6` is empty. No other
|
||
consumer, edge kind, selector, kind, expected digest, or mode is valid for this
|
||
namespace. These inputs and edges are bijective: each `graph` template input has
|
||
exactly one `bootstrap-record` `GraphEdgeV1`, and each such edge has exactly one
|
||
`graph` template input. The edge's consumer node is the enclosing template node,
|
||
its consumer input slot equals `TemplateInputV1.tag1`, its producer node equals
|
||
`ArtifactRefV1.tag2`, and its selector equals `ArtifactRefV1.tag3`.
|
||
|
||
`InputSlotRefV1` has tag 1 slot name and tag 2 expected semantic artifact type.
|
||
A final action record contains no `ArtifactRefV1`: every artifact-bearing field is recursively lowered
|
||
to an `InputSlotRefV1`. `TemplateInputV1` tags are 1 unique slot name, 2 edge
|
||
kind, 3 `ArtifactRefV1`, 4 expected semantic type, 5 semantic mode, and 6
|
||
optional origin package identity. `ResolvedInputV1` tags are 1 the same slot
|
||
name, 2 edge kind, 3 normalized logical name/path, 4 semantic artifact type, 5
|
||
semantic mode, 6 the resolved `TypedDigestV1`, and 7 optional originating
|
||
package identity. It contains no producer node, producer action key, action-
|
||
template digest, physical output path, or unresolved filesystem lookup. Thus all
|
||
content that a native plan, link plan, source-selection record, or tool closure can read is
|
||
also present exactly once in action-record tag 10 under a named slot.
|
||
|
||
`GenerateDecl` fields are fixed as follows: tag 1 name; 2 tool artifact/product
|
||
reference; 3 platform enum (schema 1 only `build`); 4 `TargetConstraint` or empty;
|
||
5 input map of `InputDecl`; 6 output map of `OutputDecl`; 7 ordered string argv;
|
||
8 literal string environment map; 9 `ResourcePolicy`; 10 reproducibility enum
|
||
`required` or `impure`. `InputDecl` is tag 1 kind (`file`, `tree`, `artifact`,
|
||
`tool`), 2 logical reference, 3 optional expected typed digest, 4 semantic mode.
|
||
`OutputDecl` is tag 1 kind (`file`, `tree`), 2 logical output path, 3 executable
|
||
boolean. `ResourcePolicy` is tags 1 max CPU count, 2 memory bytes, 3 output bytes,
|
||
4 process count; zero selects the toolchain's recorded bound, never “unlimited.”
|
||
|
||
| Record/tag | Field | Type/cardinality | Rule/default |
|
||
|---|---|---|---|
|
||
| `LockV1/1` | schema | uint/1 | `1` |
|
||
| `/2` | root-manifest | typed digest/1 | semantic manifest record |
|
||
| `/3` | toolchain | `ToolchainLock`/1 | exact closure |
|
||
| `/4` | modules | `ModuleLock`/* | sorted identity |
|
||
| `/5` | native-map | `LockedObject`/0/1 | empty |
|
||
| `ToolchainLock/1…5` | id, version, descriptor, bundle, origin | strings/digests | all required |
|
||
| `ModuleLock/1` | module | string/1 | identity |
|
||
| `/2` | version | string/1 | selected SemVer |
|
||
| `/3` | origin | string/1 | exact final archive URL |
|
||
| `/4` | archive | blob digest/1 | required |
|
||
| `/5` | tree | tree digest/1 | required |
|
||
| `/6` | manifest | record digest/1 | required |
|
||
| `/7` | signature | bytes/0/1 | empty |
|
||
| `/8` | provenance | string/0/1 | empty |
|
||
| `LockedObject/1…3` | origin, digest, signature | string/digest/bytes | origin+digest required |
|
||
|
||
`WorkV1` is tag 1 schema, tag 2 sorted `Use` records, tag 3 sorted local provider
|
||
overrides. `Use` tags are module, path, expected source-tree digest. A provider
|
||
override has slot, provider ID, product-platform constraint, contract digest,
|
||
artifact-tree digest, and provenance in tags 1–6. `VendorV1` is tag 1 schema,
|
||
tag 2 lock-record digest, tag 3 sorted entries `(module, version, source-tree
|
||
digest, vendor-relative path)` in tags 1–4. `NativeMapV1` is tag 1 schema, tag 2
|
||
exact product-platform descriptor digest, tag 3 sorted provider overrides, and
|
||
tag 4 signer/provenance record.
|
||
|
||
#### Profiles, templates, actions, results, and trees
|
||
|
||
A profile is toolchain data, not an open project map:
|
||
|
||
| Tag | `ProfileV1` field | Values |
|
||
|---|---|---|
|
||
| 1 | name | identity |
|
||
| 2 | optimization | `0`, `1`, `2`, `3`, `size` |
|
||
| 3 | debug | `none`, `line`, `full` |
|
||
| 4 | assertions | boolean |
|
||
| 5 | overflow | `trap`, `wrap` |
|
||
| 6 | panic | `abort`, `runtime` |
|
||
| 7 | sanitizers | sorted toolchain capability IDs |
|
||
| 8 | LTO | `none`, `thin`, `full` |
|
||
| 9 | relocation | effective `static`, `pic`, `pie` |
|
||
| 10 | code-model | exact target capability ID |
|
||
| 11 | TLS default | exact target capability ID |
|
||
| 12 | strip | `none`, `debug`, `all` |
|
||
| 13 | reproducibility | `required`, `impure` |
|
||
|
||
| Tag | `ActionTemplateV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | schema | `1` |
|
||
| 2 | kind/version | exact built-in kind or generate version |
|
||
| 3 | protocol record | language/compiler/export/object/runtime/manifest/lock; compiler protocol is distinct from compiler/backend byte identity |
|
||
| 4 | platform roles | B, H, optional T descriptor refs |
|
||
| 5 | selected tool closure | identity plus semantic closure digest/resources |
|
||
| 6 | profile | complete `ProfileV1` |
|
||
| 7 | logical identity/cwd | normalized identity; cwd exactly `/work`, path-independent |
|
||
| 8 | argv | ordered strings |
|
||
| 9 | environment | sorted literal map, no secrets |
|
||
| 10 | input slots | sorted `(name, edge kind, producer/output or source ref, type, mode)` |
|
||
| 11 | source selection | ordered selected membership plus reasons |
|
||
| 12 | native/link template | closed predeclared provider slots, concrete providers, and ordered link records |
|
||
| 13 | outputs | sorted name/path/type/mode/canonicalization |
|
||
| 14 | sandbox/reproducibility | exact policy/version/bounds/class |
|
||
|
||
`ActionRecordV1` has the same top-level tags, but tag 10 contains sorted
|
||
`ResolvedInputV1` records and every `ArtifactRefV1` elsewhere in the template is
|
||
replaced by the corresponding `InputSlotRefV1`. The tag-12 native/link value is
|
||
therefore a resolved plan; it cannot contain an unresolved artifact reference,
|
||
producer, physical path, or readable artifact locator/digest outside tag 10.
|
||
ABI/layout/contract digests embedded in a referenced sidecar or contract are
|
||
semantic verification values, not authority to read another object. No producer
|
||
action key substitutes for a content digest.
|
||
|
||
The tag-12 native/link record separately encodes the sorted predeclared provider
|
||
slots, concrete selected-provider declarations, and the optional ordered link
|
||
policy or resolved plan. A `ww.package` action contains exactly its
|
||
`PackageConfigV1` native-slot list and empty concrete-provider and link-policy/
|
||
plan values; finalization copies that slot list unchanged. Concrete provider
|
||
selection does not enter a package action merely because the provider satisfies
|
||
one of those slots.
|
||
|
||
Finalization interns every source, direct export, package/native object, archive,
|
||
shared library, header/sysroot tree, generated output, tool/resource, CRT,
|
||
loader, script, and init dispatcher into exactly one named template input.
|
||
For `bootstrap.compare`, it also interns every selected raw action record and
|
||
action result as a separate named input. Finalization erases the `graph`
|
||
namespace, producer node, and unresolved selector form. The corresponding
|
||
tag-10 `ResolvedInputV1` retains edge kind `bootstrap-record`, logical selector
|
||
`action_record` or `action_result`, matching artifact kind, `data` mode, and the
|
||
resolved typed digest; that digest is the sole authority to read the raw record.
|
||
Predecessor output and selected-record digests resolve those slots lazily. The
|
||
producer logical node, producer output path, producer key, and template digest
|
||
are graph/provenance facts only and do not enter the consumer's `ActionRecordV1`
|
||
or `K`. Two producers that deliver the same typed bytes to the same semantic
|
||
slot therefore produce the same downstream record and key.
|
||
|
||
`ActionResultV1` tags are: 1 schema, 2 the 32-byte action key, 3 typed
|
||
action-record digest, and 4 sorted `ResultOutputV1` records. `ResultOutputV1`
|
||
tags are 1 unique output name, 2 semantic artifact type, 3 mode (`data` or
|
||
`executable`), and 4 `TypedDigestV1`. `ObservationV1` separately uses tags 1 schema,
|
||
2 logical invocation, 3 optional action key, 4 exit status/signal, 5 stdout blob,
|
||
6 stderr blob, 7 timing/resources, and 8 physical runner metadata; it is never
|
||
an action result.
|
||
|
||
`TreeV1` tags are 1 schema and 2 ordered entries. `TreeEntryV1` tags are 1 name,
|
||
2 kind (`file`, `tree`), 3 executable boolean (false for tree), and 4 typed child
|
||
digest. `GraphSnapshotV1` tags are 1 schema, 2 logical root, 3 sorted
|
||
`GraphNodeV1` records, and 4 sorted `GraphEdgeV1` records. `GraphNodeV1` tags are
|
||
1 logical node ID, 2 typed action-template digest, 3 optional 32-byte action key,
|
||
4 optional typed action-record digest, and 5 optional typed action-result digest.
|
||
`GraphEdgeV1` tags are 1 consumer node ID, 2 consumer input slot, 3 producer node
|
||
ID, 4 producer output name, and 5 edge kind; edges sort by that five-field tuple.
|
||
Tag 4 is an ordinary producer output name except that a `bootstrap-record` edge
|
||
uses selector `action_record` or `action_result`. That branch resolves the
|
||
actual producer `GraphNodeV1.tag4` or tag 5, respectively; it never selects an
|
||
`ActionOutputV1` or `ResultOutputV1`. If the selected producer tag is absent,
|
||
the input remains unresolved and blocks finalization. The `source-input` kind is
|
||
invalid in `GraphEdgeV1`. Non-action source inputs live only in the consumer
|
||
template rather than invented graph nodes. The project index contains only its
|
||
typed graph-snapshot digest.
|
||
|
||
#### Target, toolchain, native, interface, and handoff records
|
||
|
||
| Tag | `PlatformDescriptorV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | schema | `1` |
|
||
| 2 | name | canonical lookup name |
|
||
| 3 | arch/vendor/os/environment/object format | five exact enums |
|
||
| 4 | endian/address spaces/pointers | complete integer-width map |
|
||
| 5 | integer/data-layout | widths, alignments, aggregate algorithm |
|
||
| 6 | C ABI/data model | exact IDs and calling-convention table |
|
||
| 7 | float/variadic/name decoration | exact ABI records |
|
||
| 8 | CPU baseline/features/atomics | baseline plus required/forbidden sets |
|
||
| 9 | minimum OS/SDK | typed version record |
|
||
| 10 | TLS/unwind capabilities | sets plus defaults |
|
||
| 11 | relocation/code capabilities | supported sets plus defaults |
|
||
| 12 | executable/shared/page rules | typed object-format rules |
|
||
| 13 | hosted policy | hosted/freestanding plus capability set |
|
||
| 14 | object/runtime ABI protocols | exact IDs |
|
||
|
||
`TargetConstraintV1` tags 1–17 are, respectively: optional exact descriptor
|
||
digest; arch; vendor; OS; environment; object format; hosted; C ABI; data model;
|
||
float ABI; CPU baseline; required feature set; forbidden feature set; minimum
|
||
SDK; relocation; code model; PIC requirement. Empty scalar/set means no
|
||
constraint. Matching is exactly section 8.4; no expression field exists.
|
||
|
||
`ToolchainV1` tags are: 1 schema; 2 ID; 3 version; 4 the same complete protocol
|
||
record used by action tag 3; 5 sorted `Tool` records; 6 sorted platform
|
||
descriptors; 7 sorted profiles; 8 sorted link policies; 9 runtime/provider
|
||
records; 10 bundle tree digest/signature provenance.
|
||
A `Tool` is `(name, bundle-relative path, executable blob digest, ordered dynamic
|
||
tool dependencies, resource-tree digests, adapter record)` tags 1–6. A link
|
||
policy is `LinkPolicyV1`: tag 1 product-platform descriptor; 2 product kind; 3
|
||
linkage; 4 profile constraint; 5 runtime selector (`hosted`, `minimal`, `none`,
|
||
or an exact provider slot); 6 one ordered link-policy token template; and 7
|
||
output ABI/install policy. Policies sort by the five-field selection key and a
|
||
zero/multiple match is an error. CRTs and scripts are `ArtifactRefV1` tokens,
|
||
not basenames. A dynamic loader is one restricted `dynamic-loader` token holding
|
||
both its artifact and runtime interpreter path; PE/COFF platform-image policy is
|
||
an ordered provider token rather than a fabricated loader artifact.
|
||
`SelectedToolClosureV1` deterministically
|
||
projects only the relevant tools/resources/platform/profile/policy into tags
|
||
1–7; that projection—not unrelated bundle members—is action-key input.
|
||
|
||
#### Native artifact and ABI subrecords
|
||
|
||
Compact manifest paths are lowered to `ArtifactRefV1` before WWAR encoding and
|
||
then to action input-slot references before execution. The native records are:
|
||
|
||
| Tag | `IncludeTreeRefV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | tree | `ArtifactRefV1` of kind `tree` |
|
||
| 2 | class | `quote`, `user`, `system`, or `framework` |
|
||
| 3 | subdirectory | normalized tree-relative path; `.` default |
|
||
|
||
The provider's include list is ordered because header search order is semantic.
|
||
The same tree may occur more than once with another class or subdirectory.
|
||
|
||
| Tag | `NativeSourceV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | name | unique provider-local identity |
|
||
| 2 | source | `ArtifactRefV1` of kind `file` |
|
||
| 3 | language | exact toolchain capability ID, such as `c11` or `gnu-assembly` |
|
||
| 4 | preprocessing | `none` or `c-preprocessor` |
|
||
| 5 | include-indices | ordered indexes into the provider include list; empty means all |
|
||
| 6 | defines | sorted literal macro map; duplicates with provider defines error |
|
||
|
||
Target, profile, relocation/PIC/code/TLS policy, tool, and dialect adapter come
|
||
from the enclosing `native.compile` record. Raw source flags do not exist.
|
||
|
||
| Tag | `PrebuiltObjectV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | name | unique provider-local identity |
|
||
| 2 | object | `ArtifactRefV1` of kind `object` |
|
||
| 3 | sidecar | `ArtifactRefV1` of kind `native-sidecar` |
|
||
| 4 | contract | `ArtifactRefV1` of kind `native-abi` |
|
||
|
||
| Tag | `ArchiveV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | name | unique provider-local identity |
|
||
| 2 | archive | `ArtifactRefV1` of kind `archive` |
|
||
| 3 | sidecar | archive-level `ArtifactRefV1` of kind `native-sidecar` |
|
||
| 4 | members | ordered `ArchiveMemberV1` list in physical order |
|
||
| 5 | contract | `ArtifactRefV1` of kind `native-abi` |
|
||
|
||
`ArchiveMemberV1` tags are 1 member name, 2 member blob `TypedDigestV1`, and 3
|
||
object-sidecar `ArtifactRefV1`. Duplicate names are legal only at distinct
|
||
positions; member order is never sorted. The referenced archive sidecar records
|
||
the ordered member-sidecar record digests as well as the physical member facts.
|
||
|
||
| Tag | `SharedImportLibraryV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | name | unique provider-local identity |
|
||
| 2 | kind | `elf-shared`, `macho-dylib`, or `pe-import` |
|
||
| 3 | link-artifact | shared object/dylib/import-library `ArtifactRefV1` |
|
||
| 4 | link-sidecar | `ArtifactRefV1` of kind `native-sidecar` for tag 3 |
|
||
| 5 | runtime-identity | exact SONAME, install-name, or DLL name |
|
||
| 6 | runtime-artifact | exact deployable shared object/dylib/DLL `ArtifactRefV1` |
|
||
| 7 | runtime-sidecar | `ArtifactRefV1` of kind `native-sidecar` for tag 6 |
|
||
| 8 | contract | `ArtifactRefV1` of kind `native-abi` |
|
||
| 9 | runtime-requires | sorted `NativeRuntimeRequirementV1` list |
|
||
|
||
ELF and Mach-O tags 3 and 6 may resolve to the same bytes. For PE, tag 3 is
|
||
the import library and tag 6 its matching DLL. A platform image still supplies
|
||
tag 6 as a content-identified artifact within that image.
|
||
|
||
`NativeProviderV1` tags are therefore: 1 name; 2 provided slot; 3
|
||
`TargetConstraintV1`; 4 ordered `IncludeTreeRefV1`; 5 ordered `NativeSourceV1`;
|
||
6 sorted provider define map; 7 ordered `PrebuiltObjectV1`; 8 ordered
|
||
`ArchiveV1`; 9 ordered `SharedImportLibraryV1`; 10 sorted required slots; 11
|
||
ordered link-token templates; and 12 an `ArtifactRefV1` of kind `native-abi`.
|
||
Every ABI contract and sidecar is an independently encoded, content-addressed
|
||
record. In a resolved action, those records and every artifact field above are
|
||
`InputSlotRefV1` values; source/action/provider output digests live only in
|
||
action tag 10. A sidecar's internal artifact digest must equal the corresponding
|
||
object/archive/shared input-slot digest, and its contract digest must equal the
|
||
kind-16 `TypedDigestV1` of the referenced native-ABI input record.
|
||
Section/layout/provenance digests inside a sidecar are verification facts, not
|
||
locators from which the action may read undeclared content.
|
||
|
||
| Tag | `NativeABIContractV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | schema | `1` |
|
||
| 2 | slot | exact ABI-provider slot |
|
||
| 3 | platform | `NativeABIPlatformV1` |
|
||
| 4 | symbols | sorted `NativeSymbolContractV1` list |
|
||
| 5 | types | sorted `NativeTypeContractV1` list |
|
||
| 6 | runtime-requires | sorted `NativeRuntimeRequirementV1` list |
|
||
| 7 | features | `NativeFeatureContractV1` |
|
||
| 8 | minimum-platform | optional `PlatformVersionV1` |
|
||
| 9 | code | `NativeCodeContractV1` |
|
||
|
||
Its typed record identity is `record_id(16, 1, contract)` as defined in section
|
||
6.6. Every schema-1 digest identifying a complete `NativeABIContractV1` is the
|
||
corresponding record-domain, kind-16, schema-1 `TypedDigestV1`. WW computes it; a
|
||
supplied digest is never accepted in place of the record. Subordinate layout,
|
||
calling-convention, type-contract, and header-contract digests remain their
|
||
separately specified semantic values.
|
||
|
||
| Tag | `NativeABIPlatformV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | object-format | exact format capability ID |
|
||
| 2 | object-class | exact class/word-size ID |
|
||
| 3 | endian | `little` or `big` |
|
||
| 4 | machine-ABI | exact architecture object ABI ID |
|
||
| 5 | C-ABI | exact C ABI ID |
|
||
| 6 | data-model | exact data-model ID |
|
||
| 7 | data-layout | canonical layout digest |
|
||
| 8 | calling-conventions | canonical convention-table digest |
|
||
| 9 | float-ABI | exact ID |
|
||
| 10 | variadic-ABI | exact ID |
|
||
| 11 | symbol-ABI | exact decoration/versioning ABI ID |
|
||
| 12 | object-ABI | exact object protocol ID |
|
||
| 13 | runtime-ABI | required WW runtime ABI ID or empty |
|
||
|
||
`NativeSymbolContractV1` tags are: 1 exact external name; 2 exact version or
|
||
empty; 3 kind (`function`, `data`, `tls`, `ifunc`); 4 role (`define`, `require`);
|
||
5 binding (`strong`, `weak`); 6 visibility (`default`, `protected`, `hidden`);
|
||
7 calling-convention ID or empty; 8 canonical function/object type-contract
|
||
digest; and 9 optional byte size. Symbols sort by `(name,version,kind,role)`;
|
||
duplicate keys error.
|
||
|
||
`NativeTypeContractV1` tags are: 1 stable binding/header-qualified identity; 2
|
||
kind (`opaque`, `scalar`, `enum`, `struct`, `union`, `function`); 3 exposure
|
||
(`opaque`, `layout`); 4 canonical target-specific layout/signature digest; 5
|
||
optional size; 6 optional alignment; and 7 optional canonical header/macro
|
||
contract digest. Types sort by identity and duplicates error.
|
||
|
||
`NativeRuntimeRequirementV1` tags are 1 provider slot, 2 required ABI-contract
|
||
digest, 3 phase (`link`, `load`, `both`), and 4 optional SONAME/install-name/DLL
|
||
identity. They sort by `(slot,phase,runtime-identity)`; conflicting requirements
|
||
for one slot error. `NativeFeatureContractV1` tags are 1 CPU baseline or empty,
|
||
2 sorted required feature IDs, 3 sorted forbidden feature IDs, and 4 sorted
|
||
atomic-capability IDs; required and forbidden sets must be disjoint.
|
||
|
||
`PlatformVersionV1` tags are 1 version-family ID and unsigned 2 major, 3 minor,
|
||
4 patch, 5 revision. Versions compare lexicographically over tags 2–5 only after
|
||
tag 1 equality. `NativeCodeContractV1` tags are 1 PIC (`any`, `required`,
|
||
`forbidden`); 2 sorted TLS-model IDs; 3 unwind ABI ID or `none`; 4 sorted
|
||
personality/runtime symbols; and 5 sorted required/forbidden relocation records,
|
||
each record being tag 1 capability ID and tag 2 requirement (`required` or
|
||
`forbidden`).
|
||
|
||
`NativeSidecarV1` is evidence, not a second contract. Its tags are: 1 schema; 2
|
||
artifact `TypedDigestV1`; 3 evidenced `NativeABIPlatformV1`; 4 sorted evidenced
|
||
`SectionFactV1`; 5 sorted evidenced `SymbolFactV1`; 6 sorted evidenced
|
||
`RelocationFactV1`; 7 evidenced sorted architecture attribute/notes map; 8
|
||
evidenced `NativeMachineFactsV1`; 9 native ABI-contract digest; 10 sorted
|
||
`NativeRuntimeRequirementV1`; and 11 `ProvenanceV1`. An evidenced value is
|
||
`EvidenceV1`: tag 1 enum (`inspected` or `declared`) and tag 2 the value whose
|
||
type is fixed by the containing field. `NativeMachineFactsV1` tags are 1
|
||
`NativeFeatureContractV1` and 2 `NativeCodeContractV1`.
|
||
|
||
`SectionFactV1` tags are 1 name, 2 format type, 3 flag set, 4 size, 5 alignment,
|
||
6 optional content digest. `SymbolFactV1` tags are the nine
|
||
`NativeSymbolContractV1` fields plus tag 10 section and tag 11 value/offset.
|
||
`RelocationFactV1` tags are 1 section, 2 offset, 3 exact relocation ID, 4 symbol,
|
||
5 signed addend encoded as `(negative:boolean,magnitude:uint)`, and 6 target
|
||
section. `ProvenanceV1` tags are 1 producer/tool typed digest, 2 source/build
|
||
record typed digest, 3 attestation bytes, and 4 signer/policy ID. A frozen opaque
|
||
input requires accepted attestation for every `declared` value.
|
||
|
||
`LinkTokenTemplateV1` and final `LinkTokenV1` share tags: 1 kind; 2 artifact; 3
|
||
provider slot; 4 string value; 5 enabled boolean; 6 validated path; 7 runtime
|
||
path; and 8 ordered nested tokens. Tag 2 is `ArtifactRefV1` in a template and
|
||
`InputSlotRefV1` in a final action. Valid nonempty combinations are exactly:
|
||
object/archive/shared/linker-script/version-script/export-map/def-file use tag 2;
|
||
the template-only provider kind uses tag 3; group/whole use tag 8; as-needed uses tags 5 and 8;
|
||
runtime-search uses tags 4 and 6; install-name, entry, and retain use tag 4; and
|
||
dynamic-loader uses tags 2 and 7. Every other field encodes its empty/default
|
||
value. A final `LinkTokenV1` forbids `provider`; provider selection and its
|
||
ordered fragment are recursively expanded at that exact position before the
|
||
plan is final. Section 8.6 constrains valid nesting. A toolchain `LinkPolicyV1`
|
||
template additionally permits `splice` (tag 4 is exactly `product-objects`,
|
||
`init-dispatch`, `native-providers`, or `product-controls`) and
|
||
`script-slot` (tag 2 is an optional policy-default script). Those two kinds are
|
||
forbidden in project/native link declarations and in a final plan.
|
||
|
||
The selected policy template is flattened in list order. Each splice kind occurs
|
||
exactly once and expands to its already computed ordered product list;
|
||
the single required `script-slot` becomes one `linker-script` at the same list
|
||
position using the product's explicit script when present, otherwise the policy
|
||
default, and disappears only when both are empty. A product script therefore
|
||
replaces, never combines with, the default. Ordinary policy tokens—including CRT
|
||
objects, compiler runtime, dynamic loader, and system-provider slots—stay exactly
|
||
where declared. System providers are ordered template tokens, not a sorted set.
|
||
Each expands to its selected provider's concrete token fragment; dependencies
|
||
expand recursively, and a provider cycle is an analysis error. Every expanded
|
||
artifact, sidecar, and ABI contract is a tag-10 input. A provider such as a PE
|
||
platform image that adds no linker token instead contributes its resolved
|
||
contract slot to the plan's non-link policy field. After expansion no provider
|
||
or splice partition remains.
|
||
|
||
`LinkPlanV1` tags are 1 schema; 2 the exact product-platform/kind/linkage/
|
||
profile/runtime-policy selection record; 3 the single fully expanded ordered
|
||
`LinkTokenV1` stream; 4 selected linker/tool/resource slot refs; 5 selected
|
||
non-link platform/runtime/provider ABI-contract and sidecar slot refs; and 6
|
||
output ABI/install policy. Every file-bearing token and verification record
|
||
references action tag 10. Only
|
||
this resolved flattened plan enters the link action key.
|
||
|
||
`ExportV1` (the WWAR body after `.wwe` magic) tags are: 1 schema; 2 reader
|
||
capabilities; 3 language/type protocol; 4 product-platform/C/object/runtime ABI;
|
||
5 package identity/name; 6 sorted exported-surface origin/type contract table; 7
|
||
canonical type graph; 8 sorted exported declarations/constants/foreign symbols;
|
||
9 public initialization/ABI facts; 10 public-type digest; 11 public-ABI digest. Section 7.2
|
||
defines excluded non-semantic fields. `PackageLinkV1` (`.wwlm`) tags are 1 schema,
|
||
2 package identity, 3 platform/object/runtime ABI, 4 package-object digest, 5
|
||
defined/required foreign symbols, 6 predeclared provider slots/contracts, 7 init
|
||
symbol/dependency facts, 8 install/link requirements. Tag 6 contains exactly one
|
||
compiler-derived `ProviderContractV1` for each package action tag-12 predeclared
|
||
slot, and its sorted slot projection must equal that predeclared list. It cannot
|
||
add a provider slot or graph edge.
|
||
|
||
`InstallManifestV1` tags are 1 schema, 2 product identity/key, 3 sorted entries
|
||
`(artifact digest, mode, relative destination)`, 4 runtime-resolution policy, 5
|
||
sorted shared/runtime closure, 6 target/toolchain/ABI provenance. Absolute prefix
|
||
is deliberately absent. `BootstrapPlanV1` tags are 1 schema, 2 bootstrap-host
|
||
contract, 3 ordered source refs/digests, 4 portable-C compiler/output settings,
|
||
5 recorded host-C command/tool closure, 6 stage-1 outputs, 7 production toolchain
|
||
closure, 8 identical logical stage-2/3/4 action roots, 9 semantic fixed-point
|
||
output roles, and 10 raw-record fixed-point roles. No clause or executable step
|
||
exists beyond that closed plan.
|
||
|
||
These sections fix the architectural fields, but they do not make Phase 0 an
|
||
executable specification of every reference or key transformation. Phase 0
|
||
transcribes only their wire-visible record fields, tags, enum values, union
|
||
discriminants, encoded defaults, field order, record kinds, wrapper framing, and
|
||
digest preimage formulas into the checked-in compact schemas. Cross-field
|
||
validity, construction, resolution, projection, lowering, and failure behavior
|
||
belong to the executable phase that implements them. The Phase 0 generator and
|
||
golden vectors determine bytes, not future build-engine semantics. No phase may
|
||
silently add a wire field, renumber an assignment, change an encoded default, or
|
||
alter a frozen digest formula.
|
||
|
||
## 7. Interface and artifact protocol
|
||
|
||
### 7.1 Package outputs
|
||
|
||
Every `ww.package` action, including a root package, emits the same three named
|
||
artifacts:
|
||
|
||
```text
|
||
export.wwe deterministic binary export data
|
||
package.o one target object
|
||
link.wwlm deterministic package link metadata
|
||
```
|
||
|
||
These are names inside an immutable action result, not globally meaningful
|
||
filenames. The cache is keyed by digests and logical package identity, so there
|
||
is no `__root` special case and no dotted import path used as an artifact
|
||
basename. A root object and dependency object obey identical protocols.
|
||
|
||
`link.wwlm` declares the package object's target, object ABI, defined/required
|
||
foreign symbols, required native-provider slots, initialization ordering, and
|
||
runtime ABI. It attests facts/edges already present in the action-template DAG;
|
||
it may not introduce a provider, dependency, or action after compilation, and a
|
||
mismatch is a compiler/build-protocol error. It does not contain raw linker flags. Ordinary package objects are
|
||
fed directly to the product link. An archive exists only when an explicit
|
||
static-library product asks the `archive` action to combine its ordered declared
|
||
`members` (default: root package only). Transitive package/native dependencies
|
||
remain typed link requirements in the library's install manifest and are not
|
||
silently copied into multiple archives. A deliberately self-contained archive
|
||
must list every member explicitly and pass duplicate-symbol/provider checks.
|
||
|
||
Language initialization never depends on linker input order. `ww.init` consumes
|
||
all reachable `.wwlm` artifacts, topologically orders initialization by package
|
||
imports with byte-sorted ties, diagnoses duplicate/cyclic init facts, and emits
|
||
one dispatcher object. The link plan explicitly roots that dispatcher and every
|
||
referenced init symbol against section garbage collection.
|
||
|
||
### 7.2 WW Export Data 1 (`.wwe`)
|
||
|
||
`.wwe` is a cache/build protocol, not source text and not a long-term binary
|
||
distribution promise. It starts with the eight-byte magic `WWEX\0\0\0\1` and a
|
||
WWAR-encoded body. The magic is fixed framing that is reconstructed and verified
|
||
around the body; it does not create a second blob identity. The typed `.wwe`
|
||
identity is `record_id(18, 1, ExportV1)` over that body. The body contains:
|
||
|
||
1. export schema, language edition, type-system protocol, and required reader
|
||
capabilities;
|
||
2. target descriptor, C ABI, object ABI, and runtime ABI digests;
|
||
3. full package identity and declared package name;
|
||
4. a sorted table of originating package/type identities and declaration-level
|
||
public ABI digests actually referenced by the exported surface;
|
||
5. a canonical type graph sufficient for type checking, layout, calling
|
||
convention, and code generation of every exported declaration;
|
||
6. exported constants, variables, functions, methods, types, and explicit
|
||
foreign symbols; and
|
||
7. the public initialization/ABI facts needed by an importer.
|
||
|
||
Declarations are sorted by `(kind, exported name, stable overload discriminator)`;
|
||
type-graph nodes are assigned by deterministic structural traversal. Integer and
|
||
floating constants use canonical target-independent bit encodings until a
|
||
target conversion is part of their type. Function parameter **names**, source
|
||
locations, comments, unused imports, private function bodies, and declaration
|
||
order are not semantic and are omitted. Documentation/source mapping is a
|
||
separate optional artifact and cannot invalidate an importer.
|
||
|
||
Unmanaged layout sometimes depends on facts that are private at the source
|
||
level. An exported representation therefore records size, alignment, field/base
|
||
offsets, calling convention, niche/tag rules, and relevant private padding or
|
||
opaque-field descriptors without exposing private names. `@repr("c")` types
|
||
also record the exact C data model and layout algorithm version. Opaque types
|
||
record only the operations and layout promises permitted to clients.
|
||
|
||
### 7.3 Direct imports with deep public closure
|
||
|
||
An importer opens one `.wwe` for each direct import and no transitive interface
|
||
file. If a direct dependency's API mentions a type originating in a deeper
|
||
package, its `.wwe` embeds a canonical deep descriptor for the portion of that
|
||
type required to understand and lay out the direct API. The descriptor retains
|
||
the originating package/type identity and digest of that exact referenced
|
||
contract—not the originating package's entire public ABI. It does not pretend
|
||
the type belongs to the middle package.
|
||
|
||
This rule gives both correctness and bounded reads:
|
||
|
||
```text
|
||
source/package imports -> direct .wwe inputs
|
||
direct .wwe -> complete meaning of that direct API
|
||
link graph -> all reachable package objects
|
||
```
|
||
|
||
A public change in a leaf rebuilds direct reverse dependencies. Propagation
|
||
continues only while each rebuilt package's `.wwe` bytes change. A private leaf
|
||
change never enters an importer key. This replaces source-like transitive
|
||
interface prepending and its quadratic composed units.
|
||
|
||
This bounds interface **opens and reparsing**, not necessarily total descriptor
|
||
bytes: heavily re-exported type graphs can duplicate deep descriptors. Phase 1
|
||
measures total `.wwe` size and structural duplication on the real library graph.
|
||
Only if that is material may a later export-schema revision intern immutable
|
||
per-declaration descriptors; direct-import semantics do not change.
|
||
|
||
### 7.4 Public and ABI digests
|
||
|
||
The `.wwe` content digest identifies its complete target-specific bytes. It also
|
||
contains two domain-separated hashes:
|
||
|
||
- **public type digest** over names, types, constants, visibility, and language
|
||
semantics; and
|
||
- **public ABI digest** over target layouts, calling conventions, exported
|
||
symbol contracts, runtime ABI, and representation facts.
|
||
|
||
Compile actions normally depend on the whole `.wwe` content digest. Tools such as
|
||
documentation may depend only on the public type digest when their action kind
|
||
explicitly permits it. Link compatibility checks use the ABI digest. Digest
|
||
subsetting is protocol-defined; callers cannot choose arbitrary ignored fields.
|
||
|
||
### 7.5 Symbols and identity
|
||
|
||
Internal WW symbols are mangled from a protocol-versioned hash of the full
|
||
package identity plus declaration identity, never from a leaf name or artifact
|
||
filename. Resolver rules ensure only one source/version supplies that package
|
||
identity. An explicitly foreign symbol is exactly the source-declared spelling
|
||
and participates in duplicate-provider checks.
|
||
|
||
Package version and distribution origin are absent from mangling because they
|
||
are not identity. An incompatible major version has a different `/vN` module
|
||
identity and therefore different WW symbols. Native C symbols do not gain this
|
||
protection; their provider slots and link collision rules must reject
|
||
incompatible co-selection.
|
||
|
||
### 7.6 Compatibility and deterministic serialization
|
||
|
||
A consumer accepts only the exact export/type/object/runtime protocol combination
|
||
declared compatible by its immutable toolchain descriptor. A new optional record
|
||
still requires a new export schema and reader capability. Unknown records are
|
||
not silently dropped. Target descriptor and ABI mismatches are errors before a
|
||
compiler or linker runs.
|
||
|
||
Because `.wwe` is rebuilt from locked source, WW does not need an indefinitely
|
||
stable compiler-internal export format. A toolchain upgrade changes action keys
|
||
and may rebuild the graph. Public native-library ABI stability is a separate,
|
||
explicit provider contract. Release/bootstrap byte comparisons include `.wwe`,
|
||
objects, link metadata, action records, and executables.
|
||
|
||
## 8. Native integration and cross compilation
|
||
|
||
### 8.1 Complete target descriptors
|
||
|
||
A target triple is a user-facing alias. Before graph construction it expands to
|
||
an immutable target descriptor containing at least:
|
||
|
||
- architecture, vendor, operating system, environment, and object format;
|
||
- endianness, pointer widths/address spaces, integer/long widths, alignment and
|
||
aggregate-layout rules;
|
||
- C ABI/data model, calling conventions, name decoration, variadic convention,
|
||
TLS ABI capabilities/default, floating ABI, and unwind model;
|
||
- baseline CPU, required/forbidden CPU features, minimum OS/SDK version, and
|
||
atomic capability;
|
||
- supported/default relocation and code models, executable/shared-library rules,
|
||
and page constraints;
|
||
- hosted versus freestanding policy; and
|
||
- compatible object ABI and runtime ABI protocol identifiers.
|
||
|
||
The triple `x86_64-unknown-linux-gnu` is insufficient by itself to identify CPU
|
||
features, sysroot, glibc, loader, or linker. Those are separate descriptor/input
|
||
digests. `--cpu` and `--feature` produce a new canonical expanded descriptor;
|
||
the host CPU is never probed to select target features unless the user explicitly
|
||
requests the impure alias `native`, which is rejected by frozen/shared builds.
|
||
|
||
The descriptor supplies ABI invariants plus supported/default policy values. A
|
||
profile/product selects the effective relocation, code, PIC/PIE, and TLS policy
|
||
from those permitted sets; the normalized effective values appear once in the
|
||
action record. A conflicting or unsupported selection is rejected, never
|
||
resolved by precedence between duplicate fields.
|
||
|
||
This follows the native facts exposed by LLVM data layouts and Clang's cross
|
||
compilation/toolchain documentation, while making their often-driver-selected
|
||
inputs explicit
|
||
([LLVM data layout](https://llvm.org/docs/LangRef.html#data-layout),
|
||
[Clang cross compilation](https://clang.llvm.org/docs/CrossCompilation.html),
|
||
[Clang toolchain](https://clang.llvm.org/docs/Toolchain.html)).
|
||
|
||
### 8.2 Toolchain closure (`toolchain.wwt`)
|
||
|
||
An immutable toolchain bundle has a canonical `toolchain.wwt` descriptor with:
|
||
|
||
```text
|
||
ww-toolchain 1
|
||
id = "ww.org/toolchain"
|
||
version = "v1.4.0"
|
||
protocols = { language = "1", compiler = "1", export = "1", object = "1",
|
||
runtime = "1", manifest = 1, lock = 1 }
|
||
|
||
tool "wwc" { path = "bin/wwc", digest = "sha256:..." }
|
||
tool "cc" { path = "bin/clang", digest = "sha256:..." }
|
||
tool "as" { path = "bin/llvm-mc", digest = "sha256:..." }
|
||
tool "ld" { path = "bin/ld.lld", digest = "sha256:..." }
|
||
tool "archive" { path = "bin/llvm-ar", digest = "sha256:..." }
|
||
|
||
target "aarch64-unknown-linux-gnu" {
|
||
descriptor = "targets/aarch64-linux-gnu.wwt"
|
||
sysroot = { tree = "sha256:...", path = "sysroots/aarch64-linux-gnu" }
|
||
runtime = "runtime:aarch64-linux-gnu@1"
|
||
libc = "c:glibc@2.39"
|
||
link-policies = [
|
||
{ kind = "exe", linkage = "dynamic", profile = ["debug", "release"],
|
||
runtime = "hosted",
|
||
tokens = [
|
||
{ kind = "object", artifact = "sysroot:lib/crt1.o" },
|
||
{ kind = "object", artifact = "sysroot:lib/crti.o" },
|
||
{ kind = "splice", value = "product-objects" },
|
||
{ kind = "splice", value = "init-dispatch" },
|
||
{ kind = "splice", value = "native-providers" },
|
||
{ kind = "splice", value = "product-controls" },
|
||
{ kind = "provider", slot = "c:compiler-rt@1" },
|
||
{ kind = "provider", slot = "c:glibc@2.39" },
|
||
{ kind = "dynamic-loader",
|
||
artifact = "sysroot:lib/ld-linux-aarch64.so.1",
|
||
runtime-path = "/lib/ld-linux-aarch64.so.1" },
|
||
{ kind = "script-slot", artifact = "toolchain:lib/ldscripts/elf.lds" },
|
||
{ kind = "object", artifact = "sysroot:lib/crtn.o" }
|
||
] }
|
||
]
|
||
}
|
||
```
|
||
|
||
Every path is bundle-relative and every executable, shared tool dependency,
|
||
resource directory, built-in header tree, target descriptor, runtime, sysroot,
|
||
CRT, default script, and adapter is covered by the bundle's canonical tree
|
||
digest. The engine invokes exact paths and passes explicit target/sysroot/resource
|
||
arguments. A tool's compiled-in search outside the sandbox cannot resolve.
|
||
The compact `sysroot:`/`toolchain:` references in the example normalize to full
|
||
`ArtifactRefV1` records containing the individual typed digest obtained from
|
||
that authenticated tree; the shorthand itself never enters WWAR.
|
||
|
||
The whole bundle digest authenticates acquisition. An action key uses the
|
||
transitive selected descriptor slice/tool/resource/sysroot/runtime closure only;
|
||
adding an unused target or unrelated tool to a republished bundle does not cause
|
||
global recompilation. Changing any selected byte/protocol still changes the key.
|
||
|
||
Official bundles may use LLVM, GNU binutils, or another implementation per
|
||
target; the architecture does not expose that choice as project semantics.
|
||
Tool adapters translate WW's typed record to exact argv and declare all injected
|
||
inputs. A custom bundle must do the same and pass conformance/reproducibility
|
||
tests before frozen mode accepts it.
|
||
|
||
WW permanently owns the language compiler, export/object/runtime ABI protocols,
|
||
target descriptor schema, action engine, and official bundle definitions. It
|
||
does **not** permanently own the assembler, linker, archiver, C compiler, or SDK.
|
||
The current `w6a`/`w6l` may serve as migration inputs, then are removed once a
|
||
pinned external closure passes parity. This is the smaller long-term system.
|
||
|
||
### 8.3 C ABI and foreign declarations
|
||
|
||
Foreign declarations are explicit source contracts:
|
||
|
||
```ww
|
||
@abi("c") @symbol("write")
|
||
@provider("c:libc")
|
||
fn c_write(fd s32, data *u8, count usize) ssize;
|
||
|
||
@repr("c")
|
||
type Header struct { tag u32; length u16; };
|
||
```
|
||
|
||
The compiler checks that every type has a defined representation for the
|
||
selected C ABI and records symbol, calling convention, variadic status, layout,
|
||
and explicitly named provider slot in `.wwe`/`.wwlm`. `@provider` is mandatory
|
||
for each foreign declaration (a group annotation may supply it lexically), and
|
||
the owning package metadata must require that slot; WW never infers it from a
|
||
symbol spelling or link position. A foreign declaration with no provider, two
|
||
definitions of a strong symbol, incompatible calling conventions, or mismatched
|
||
layout digest is a pre-link diagnostic where possible and a mandatory link
|
||
failure otherwise.
|
||
|
||
C headers are not searched or parsed implicitly. Bindings are either checked-in
|
||
WW source produced by an explicit `ww bindgen c` command, or a declared
|
||
`generate` action whose inputs include the exact header trees, target descriptor,
|
||
preprocessor, macro map, include roots, and binding tool. The generated result
|
||
records the declared whole input-tree digests, observed include trace as
|
||
non-semantic audit metadata, and C ABI digest. The trace cannot add an input; a
|
||
future finer-grained scan would require a new built-in action still keyed by the
|
||
complete allowed include-tree digest. No build invokes ambient
|
||
`pkg-config`; `ww native snapshot-pkg-config` is an explicit, impure acquisition
|
||
command that converts one selected host configuration into a reviewable native
|
||
provider record and content snapshot.
|
||
|
||
### 8.4 Native declarations
|
||
|
||
Schema 1 uses two connected declarations. A package lists the ABI slots it
|
||
requires:
|
||
|
||
```text
|
||
package "compress/zlib" {
|
||
native = ["c:zlib@1"]
|
||
generated = ["action:zlib-bindings:ww"]
|
||
}
|
||
```
|
||
|
||
A provider declares exact target artifacts and dependencies:
|
||
|
||
```text
|
||
native "zlib-linux-aarch64" {
|
||
provides = "c:zlib@1"
|
||
when = { os = "linux", arch = "aarch64", environment = "gnu",
|
||
c-abi = "aapcs64", float-abi = "hard",
|
||
requires-features = ["neon"],
|
||
minimum-sdk = "linux:5.10.0.0" }
|
||
include-trees = [
|
||
{ tree = "native/zlib/include", class = "user", subdirectory = "." }
|
||
]
|
||
sources = [
|
||
{ name = "adler32", source = "native/zlib/adler32.c",
|
||
language = "c11", preprocessing = "c-preprocessor",
|
||
include-indices = [0], defines = {} }
|
||
]
|
||
defines = { ZLIB_CONST = "1" }
|
||
objects = []
|
||
archives = []
|
||
shared = []
|
||
requires = ["c:libc"]
|
||
link = [
|
||
{ kind = "object",
|
||
artifact = { namespace = "provider-output",
|
||
owner = "zlib-linux-aarch64",
|
||
name = "adler32", kind = "object", mode = "data" } }
|
||
]
|
||
abi = { file = "native/zlib.wwabi" }
|
||
}
|
||
```
|
||
|
||
Allowed provider fields are exactly `provides`, `when`, `include-trees`,
|
||
`sources`, `defines`, `objects`, `archives`, `shared`, `requires`, `link`, and
|
||
`abi`. `when` is a finite conjunction. It may contain exact
|
||
`descriptor-digest`, `arch`, `vendor`, `os`, `environment`, `object-format`,
|
||
`hosted`, `c-abi`, `data-model`, `float-abi`, `cpu-baseline`, `relocation`,
|
||
`code-model`, and `pic`; `requires-features`/`forbids-features` use subset/
|
||
disjoint-set matching; `minimum-sdk` matches only a product platform whose
|
||
declared minimum is at least that value. It has no general expression. All
|
||
matching candidates are retained: identical provider/artifact digests coalesce,
|
||
while multiple different matches require the product's explicit provider map
|
||
rather than a specificity guess.
|
||
|
||
The textual `abi = { file = PATH }` form parses `PATH` as one complete canonical
|
||
`NativeABIContractV1` subdocument, encodes it as top-level record kind 16, and
|
||
lowers the provider field to an `ArtifactRefV1` containing that typed record
|
||
digest; an inline complete record is equivalent. The local source blob and
|
||
parsed contract are both hashed. The file form is not a digest assertion or a
|
||
build-time include, and partial contracts are invalid.
|
||
|
||
`sources` entries support only toolchain-declared C language editions and
|
||
assembly dialects. Each source has an exact file digest and sees only the listed
|
||
include trees/defines and toolchain headers. `objects`, `archives`, and `shared`
|
||
name content-identified prebuilt artifacts plus their target/object/ABI records.
|
||
A local `path` is content-hashed during analysis; a literal expected digest is
|
||
needed only for an external/prebuilt record. A provider may mix source and prebuilt inputs, but every produced object is its
|
||
own `native.compile` action. There is no filesystem library search.
|
||
|
||
`abi` is not an opaque user assertion. It is the recomputed digest of a canonical
|
||
native ABI contract containing slot, product-platform ABI/data model, calling
|
||
conventions, symbol names/versions/kinds, referenced C layout/header-contract
|
||
digests, required runtime slots, CPU/features, minimum SDK, and PIC/TLS/unwind
|
||
requirements. Foreign bindings carry the expected contract or compatible
|
||
declaration-level subset. WW cross-checks the contract against compiled objects,
|
||
shared/import-library sidecars, and declared providers before link.
|
||
|
||
### 8.5 Assembly and object files
|
||
|
||
Assembly source declares an official external-tool dialect (`gnu`,
|
||
`llvm-integrated`, or another exact toolchain capability), preprocessing mode,
|
||
target, and CPU feature contract. The current `w6a` Plan-9-style dialect is a
|
||
migration input and is not accepted after cutover.
|
||
The selected assembler executable and resources are action inputs. A source for
|
||
one target cannot be selected for another by extension alone.
|
||
|
||
Prebuilt objects carry a sidecar native record with content digest, object
|
||
format, architecture, ABI, required CPU features, defined/undefined symbols,
|
||
PIC/TLS/unwind properties, and producer provenance. WW verifies the object
|
||
format/class/endian/machine header, sections, symbol table, relocations, notes,
|
||
and architecture attributes against every mechanically inferable sidecar fact;
|
||
the same inspection recurses into archive members and shared/import libraries.
|
||
Source-level C contract, libc/runtime compatibility, and provenance are not
|
||
fully inferable from object bytes, so frozen opaque prebuilts additionally need
|
||
a signature/attestation accepted by policy. A missing or contradictory record
|
||
is an error, not permission to ask the host linker what happens.
|
||
|
||
### 8.6 Static/shared libraries and ordered linking
|
||
|
||
Native `link` is an ordered list of typed template tokens:
|
||
|
||
```text
|
||
{ kind = "object", artifact = "object:NAME" }
|
||
{ kind = "archive", artifact = "archive:NAME" }
|
||
{ kind = "shared", artifact = "shared:NAME" }
|
||
{ kind = "provider", slot = "c:zlib@1" }
|
||
{ kind = "group", items = [...] }
|
||
{ kind = "whole", items = [...] }
|
||
{ kind = "as-needed", enabled = true, items = [...] }
|
||
{ kind = "linker-script", artifact = "file:NAME" }
|
||
{ kind = "version-script", artifact = "file:NAME" }
|
||
{ kind = "export-map", artifact = "file:NAME" }
|
||
{ kind = "def-file", artifact = "file:NAME" }
|
||
{ kind = "runtime-search", policy = "origin-relative", path = "lib" }
|
||
{ kind = "install-name", value = "@rpath/libname.so" }
|
||
{ kind = "dynamic-loader", artifact = "file:LOADER", runtime-path = "/lib/ld.so" }
|
||
{ kind = "entry", symbol = "_start" }
|
||
{ kind = "retain", symbol = "ww_init_abcd" }
|
||
```
|
||
|
||
Those are the complete schema-1 project/native template kinds. `provider` is
|
||
recursively replaced at its exact position by the chosen provider fragment;
|
||
the final kinds are every listed kind except `provider`. The toolchain-only
|
||
`splice` and `script-slot` kinds also lower away as specified in section 6.9.
|
||
Every file-bearing final token resolves
|
||
to a declared typed artifact and digest. Nested `items` contain only link tokens;
|
||
repetition is represented by repeating a list entry and is never deduplicated.
|
||
`group` contains archives or provider templates that resolve only to archives,
|
||
`whole` contains archives only, and `as-needed` contains shared inputs or
|
||
provider templates that resolve only to shared inputs; any other expansion or
|
||
nesting is invalid.
|
||
Runtime-search/install-name values are validated by the selected platform
|
||
adapter; frozen bundled policy permits only relocatable origin-relative paths.
|
||
Entry/retain tokens become the pinned linker's typed entry/undefined-root
|
||
mechanism. Raw flags exist only inside a content-identified custom toolchain
|
||
adapter.
|
||
|
||
The plan is not a set and is never alphabetically reordered. The selected
|
||
toolchain policy template determines global position: it can place start CRT
|
||
before the product-object splice and end CRT after runtime providers, rather
|
||
than relying on one universal ordering rule. Inside `product-objects`, WW
|
||
objects use stable package-identity order. Inside `native-providers`, fragments
|
||
use requester-before-provider topological order, which gives `libA` before the
|
||
`libB` it requires. `product-controls` carries the declared entry/retain/install
|
||
tokens; `init-dispatch` carries its single retained object when needed. Cyclic
|
||
static archives must be represented by one explicit `group`; an undeclared
|
||
provider cycle is an error. Repetition, whole-archive, as-needed, export maps,
|
||
and symbol-version scripts remain exact records in the flattened canonical plan.
|
||
|
||
An archive action preserves declared member order and canonicalizes header
|
||
timestamps, ownership, modes, and string tables. A shared-library input includes
|
||
its link-time artifact, SONAME/install-name, ABI digest, transitive runtime
|
||
requirements, and deployable runtime artifact digest. Merely finding the same
|
||
basename in a host directory is never equivalence.
|
||
|
||
The installation manifest also fixes runtime resolution. ELF bundled policy
|
||
copies the exact shared closure under a digest-namespaced relative `lib/` and
|
||
uses an origin-relative RUNPATH; Mach-O uses exact `@rpath`/install names; Windows
|
||
places named DLL artifacts in the declared application directory beside their
|
||
matching import libraries. A platform-image provider may instead bind an exact
|
||
loader/system tree. An OS-managed mutable shared library is an impure runtime
|
||
policy: link bytes can still be recorded, but WW does not promise that execution
|
||
will load a particular digest.
|
||
|
||
Linker scripts are declared content inputs. The adapter resolves/audits only
|
||
file-bearing directives such as `INCLUDE`, `INPUT`, `GROUP`, and `SEARCH_DIR`
|
||
for the pinned linker dialect; included files and permitted sysroot trees are in
|
||
the record. The pinned linker—not WW—interprets section placement, expressions,
|
||
symbols, memory regions, and target semantics inside the closed sandbox.
|
||
Unresolved `SEARCH_DIR`, absolute host paths, and implicit default scripts are
|
||
errors. GNU `ld` documents that scripts and archive order change link semantics;
|
||
WW therefore preserves rather than abstracts them away
|
||
([GNU ld scripts](https://sourceware.org/binutils/docs/ld/Scripts.html),
|
||
[GNU linker](https://sourceware.org/binutils/docs/ld.html)).
|
||
|
||
### 8.7 libc, CRT, SDK, loader, and freestanding products
|
||
|
||
A hosted platform entry names one exact sysroot/SDK and the available libc,
|
||
system, compiler-runtime, and WW-runtime providers; it does not name one
|
||
universal CRT sequence or loader. The selected toolchain link policy for
|
||
`(product-platform descriptor, product kind, linkage, profile, runtime
|
||
selector)` supplies the exact ordered token template containing CRTs, compiler
|
||
runtime, system-provider slots, script slot, and any platform-appropriate loader
|
||
contract. Compiler and linker
|
||
drivers run through no-defaults adapters and receive only that declared closure,
|
||
so they cannot fall back to B's `/usr`.
|
||
|
||
A product chooses a runtime policy:
|
||
|
||
```text
|
||
product "kernel" {
|
||
kind = "exe"
|
||
root = "kernel"
|
||
linkage = "static"
|
||
runtime = "none"
|
||
entry = "_start"
|
||
native = ["freestanding:boot@1"]
|
||
linker-script = "native/kernel.ld"
|
||
}
|
||
```
|
||
|
||
For an executable, `linkage` is exactly `dynamic`, `pie`, `static`, or
|
||
`static-pie`; shared-library products use `shared`. The toolchain maps
|
||
`(product kind, linkage, profile, product-platform, runtime selector)` to one
|
||
exact ordered policy template; selection never ignores `runtime`. Loader
|
||
presence follows the platform's executable rules. On an ABI with
|
||
an explicit program interpreter, `dynamic` and `pie` executables must name its
|
||
runtime path and exact provider artifact; `static` and `static-pie` must not. A
|
||
shared library has a runtime identity and dependencies but no executable program
|
||
interpreter. PE/COFF-style platforms without a separate interpreter bind the
|
||
exact platform-image/loader contract through system-provider policy rather than
|
||
inventing a pathname. Different product policies therefore cannot accidentally
|
||
share one CRT/loader sequence.
|
||
|
||
`runtime` is `hosted`, `minimal`, `none`, or a named provider. `none` supplies no
|
||
libc, CRT, loader, or WW runtime; compiler helper routines must be supplied by a
|
||
declared provider or rejected. `minimal` names an exact freestanding runtime.
|
||
Entry symbol, memory/linker script, relocation/code model, panic/stack policy,
|
||
and any boot image action are explicit. Kernel-style targets never inherit the
|
||
hosted target's defaults.
|
||
|
||
Schema 1 permits `none` only for object/static products and `static` or
|
||
`static-pie` executables. Its selected link template must contain no CRT,
|
||
dynamic-loader, libc, compiler-runtime, or WW-runtime token. `minimal` and named
|
||
providers declare their valid product/linkage set as capabilities; `hosted`
|
||
uses the platform's hosted set. A product/linkage/runtime tuple outside that set
|
||
is rejected during graph construction. The policy's single `script-slot` uses
|
||
the product's `linker-script` when present, replacing the default at the same
|
||
ordered position.
|
||
|
||
The target/toolchain declares the runtime capability required by every
|
||
compiler-emitted helper and language operation. During graph construction a
|
||
`none`/`minimal` product is rejected if selected source operations require an
|
||
unavailable allocation, panic, stack, arithmetic, TLS, unwind, or other runtime
|
||
capability; this is not deferred to an unexplained undefined linker symbol.
|
||
|
||
### 8.8 Provider conflicts and system substitution
|
||
|
||
One link namespace may select exactly one provider digest for an ABI slot such
|
||
as `c:zlib@1`, `c:libc`, or `runtime:ww@1`. Multiple requirements for the same
|
||
slot coalesce only if they resolve to the same provider and ABI digest. Different
|
||
providers, ABI major slots that export colliding unversioned symbols, and two
|
||
native modules claiming the same strong symbols are loud identity-collision
|
||
errors. WW never chooses whichever library appears first.
|
||
|
||
When more than one contract-compatible provider matches, the root product must
|
||
select one explicitly:
|
||
|
||
```text
|
||
product "hello" {
|
||
kind = "exe"
|
||
root = "."
|
||
linkage = "dynamic"
|
||
providers = [
|
||
{ slot = "c:zlib@1", use = "example.org/zlib#zlib-linux-aarch64" }
|
||
]
|
||
}
|
||
```
|
||
|
||
The provider ID is `(declaring module identity, native-clause name)`, rendered
|
||
with `#` only in metadata. Selection cannot change a dependency's required ABI
|
||
contract; the chosen provider must satisfy every declaration-level contract.
|
||
|
||
Two incompatible native versions can coexist only if they use distinct provider
|
||
slots **and** their symbols/runtime names are namespaced or versioned so the link
|
||
record proves no collision. Otherwise the build must adapt one behind a wrapper,
|
||
use dynamic isolation, or fail. Language-level multiple-version selection cannot
|
||
solve a C global-symbol collision.
|
||
|
||
A distro system provider is a normal provider record mapping exact logical
|
||
artifacts to content digests and ABI metadata. If those files live under `/usr`,
|
||
the mapping snapshots/re-hashes them before graph construction and changes the
|
||
key whenever they change. It is marked impure unless the directory tree itself
|
||
is immutable and content-identified. Raw `-L`, `-l`, `LD_LIBRARY_PATH`, compiler
|
||
defaults, and build-time `pkg-config` are not accepted substitutes.
|
||
|
||
A portable distributor substitution uses a closed `ww-native-map 1` file:
|
||
|
||
```text
|
||
ww-native-map 1
|
||
target = "sha256:product-platform-descriptor..."
|
||
provider "c:zlib@1" {
|
||
use = "distro.example/native#zlib"
|
||
contract = "sha256:..."
|
||
artifact-tree = "sha256:..."
|
||
provenance = "https://distro.example/provenance/zlib.jsonl"
|
||
}
|
||
```
|
||
|
||
`ww lock --native-map=FILE` records the map digest/origin in `ww.lock`; frozen
|
||
mode accepts only that exact signed/content-verified map. `ww.work` may contain
|
||
the same `provider` clause for local development, but it is an impure overlay and
|
||
frozen mode rejects it. This gives OS packagers an offline substitution mechanism
|
||
without changing imports or silently consulting `/usr`.
|
||
|
||
### 8.9 Cross-compilation behavior
|
||
|
||
All code-generating tools execute on B. WW/C/assembly compilation for the
|
||
requested ordinary product emits H objects using only H's target descriptor,
|
||
sysroot, headers, runtime, and native providers. A compiler-like product built
|
||
for H may later emit T code, but no T program executes during its own build.
|
||
Object headers and sidecars are checked before linking, so a host object cannot
|
||
silently enter a target product.
|
||
|
||
`ww test --target=H` always builds target test binaries. It executes them only
|
||
when `H = B` or the toolchain declares an explicit content-identified runner
|
||
(local emulator or simulator plus immutable image) as an invocation tool. Otherwise
|
||
it reports “built, not run” unless `--require-run` was requested, in which case
|
||
it fails. The runner and its platform image are action inputs; no ambient emulator
|
||
is discovered.
|
||
|
||
Remote hardware/device testing is a separate explicit
|
||
`ww observe test --runner=NAME` operation with declared endpoint/capability
|
||
authority. It may use network/devices but is a non-build observation: remote
|
||
state is reported, it never populates artifact/shared caches, and it is outside
|
||
byte-reproducibility claims. Ordinary `ww test` retains the no-network policy.
|
||
|
||
This model supports new targets without running a compiler on them: an existing
|
||
host toolchain adds a target descriptor, backend, object adapter, sysroot/runtime,
|
||
and native providers, then builds and tests through a declared runner or hardware
|
||
step. GNU's build/host/target distinction is useful vocabulary, but WW records
|
||
the complete descriptors rather than only triplets
|
||
([Autoconf triplets](https://www.gnu.org/software/autoconf/manual/autoconf-2.71/html_node/Specifying-Target-Triplets.html)).
|
||
|
||
## 9. Command-line design
|
||
|
||
### 9.1 The ordinary path
|
||
|
||
The default workflow is deliberately short:
|
||
|
||
```sh
|
||
ww init example.org/hello
|
||
ww build
|
||
ww run -- argument
|
||
ww test .
|
||
```
|
||
|
||
`ww init MODULE` creates `ww.mod`, a root `main.ww` only when the directory is
|
||
empty, and a lock selecting the currently invoked immutable toolchain. It does
|
||
not add a dependency. In an existing one-directory `main` package, `ww build`
|
||
works without `init`; the invoking toolchain and standalone source identity are
|
||
shown in verbose output.
|
||
|
||
`ww build [DIR|PRODUCT]` builds the default root product or one named product.
|
||
The default profile is the fully specified `debug` profile; `--profile=release`
|
||
selects the toolchain's immutable release profile. `ww run` first performs that
|
||
same build, then runs only a product with `H = B`; arguments after `--` are never
|
||
interpreted by the build.
|
||
|
||
There is no command that means “build and opportunistically download whatever is
|
||
missing.” If a locked source or toolchain is absent, the diagnostic names its
|
||
digest and asks for `ww fetch --locked`.
|
||
|
||
### 9.2 Tests, examples, docs, and installation
|
||
|
||
```sh
|
||
# Exactly one package.
|
||
ww test ./internal/codec
|
||
|
||
# Every package below the current module root.
|
||
ww test ./...
|
||
|
||
# Compile cross-target tests and require a declared runner.
|
||
ww test ./... --target=aarch64-unknown-linux-gnu --require-run
|
||
|
||
# Build an ordinary example product and render documentation.
|
||
ww build example:examples/packet
|
||
ww doc ./... --out=out/doc
|
||
|
||
# Materialize a named release product under a prefix.
|
||
ww install inspect --profile=release --prefix=/opt/ww
|
||
```
|
||
|
||
Test package discovery and result reporting remain deterministic under `-j N`.
|
||
Compilation is cached, but every selected test binary runs. An installation
|
||
manifest lists every copied artifact, digest, mode, runtime dependency, and
|
||
relative destination and is materialized as `<product>.wwinstall`. Installation never discovers libraries in the prefix and
|
||
never mutates the cache artifact.
|
||
|
||
### 9.3 Inspecting the graph and cache
|
||
|
||
```sh
|
||
ww graph --packages
|
||
ww graph --actions
|
||
ww graph --actions --format=json > graph.json
|
||
ww explain example.org/hello/internal/codec
|
||
ww explain --path --all product:hello
|
||
ww cache verify
|
||
ww verify reproducible --profile=release
|
||
```
|
||
|
||
`graph` performs analysis but no build action. A node whose producer output does
|
||
not yet exist has a known action shape and incoming edges but a `pending` key;
|
||
its final key is computed when predecessor content digests become available.
|
||
`explain` compares available current records with the project index and likewise
|
||
does not build. These commands never fetch.
|
||
|
||
### 9.4 Adding, fetching, updating, freezing, and vendoring
|
||
|
||
```sh
|
||
# Query the module's conventional source index, add a minimum, and lock closure.
|
||
ww add example.org/codec@v1.2.3
|
||
|
||
# The same operation with an explicit private or non-conventional origin.
|
||
ww add corp.example/codec@v1.2.3 --from=https://packages.corp/codec/
|
||
|
||
# Materialize exactly the existing lock without changing any project file.
|
||
ww fetch --locked
|
||
|
||
# Change one direct minimum and recompute the complete lock atomically.
|
||
ww update example.org/codec@v1.4.0
|
||
|
||
# Recompute after an intentional manifest edit.
|
||
ww lock
|
||
|
||
# Materialize the exact locked source closure in the project.
|
||
ww vendor --locked
|
||
|
||
# A frozen, offline build from already present locked inputs.
|
||
ww build --frozen --offline --vendor
|
||
```
|
||
|
||
`add` and `update` write `ww.mod` and `ww.lock` together through temporary files
|
||
only after the entire selection and integrity check succeeds. `lock` writes only
|
||
`ww.lock`. `fetch` and `vendor` never change selection. Without `--vendor`, WW
|
||
uses the immutable source store; with it, every vendor digest is reverified.
|
||
`add --from=URL` records that credential-free index as the requirement's
|
||
`source-index`; archive redirects/final origin remain exact lock metadata.
|
||
|
||
An ordinary non-frozen build still never updates an inconsistent lock: it fails
|
||
with the exact `ww lock` command. Frozen mode additionally rejects workspaces,
|
||
impure providers, noncanonical metadata, unlocked tools, and any source digest
|
||
not named by the lock. `--offline` is useful in automation as an assertion but
|
||
does not weaken or strengthen the normal build network denial.
|
||
|
||
### 9.5 Cross compilation and toolchain selection
|
||
|
||
```sh
|
||
# Select a product host platform (the familiar cross-build form).
|
||
ww build --target=aarch64-unknown-linux-gnu --profile=release
|
||
|
||
# Select exact CPU semantics rather than probing the build machine.
|
||
ww build --target=x86_64-unknown-linux-gnu \
|
||
--cpu=x86-64-v3 --feature=-avx512f
|
||
|
||
# Build a compiler that runs on H and emits code for T.
|
||
ww build compiler:wwc \
|
||
--host=aarch64-unknown-linux-gnu \
|
||
--target=riscv64-unknown-none-elf
|
||
|
||
# Acquire and lock an exact toolchain before building; only fetch uses network.
|
||
ww toolchain fetch ww.org/toolchain@v1.4.0
|
||
ww lock --toolchain=ww.org/toolchain@v1.4.0
|
||
ww build --toolchain=ww.org/toolchain@v1.4.0
|
||
```
|
||
|
||
The selected command toolchain must match the common ID, satisfy every minimum,
|
||
and equal the root lock's exact descriptor. A digest
|
||
selection is exact; an ID/version selection resolves only through the lock or
|
||
installed signed catalog and never during build. `ww toolchain list --targets`
|
||
shows descriptors and sysroot/runtime digests, not just triples. `ww lock
|
||
--toolchain` changes only the lock's exact toolchain selection when the chosen
|
||
version satisfies every manifest minimum; it does not rewrite those minimums.
|
||
|
||
### 9.6 Deliberately absent commands/options
|
||
|
||
There is no `ww build --fetch`, build-time package-manager hook, raw `-L`/`-l`,
|
||
global import search path, arbitrary compiler/linker environment injection,
|
||
manifest evaluator, or command that installs dependencies into a mutable global
|
||
source namespace. Expert escape hatches are explicit toolchains, native provider
|
||
records, and finite declarative actions; all remain visible in the graph/key.
|
||
|
||
## 10. Toolchain and bootstrap design
|
||
|
||
### 10.1 The durable stage-zero seed
|
||
|
||
The smallest credible recovery seed is:
|
||
|
||
```text
|
||
bootstrap/ww0.c generated portable C99 snapshot
|
||
bootstrap/bootstrap.plan canonical source/tool/output plan
|
||
bootstrap/SHA256SUMS expected seed and plan digests
|
||
```
|
||
|
||
`ww0.c` is generated at release time from the same compiler sources as the
|
||
production compiler, with a deliberately non-optimizing portable C backend and
|
||
the minimal support routines concatenated into one translation unit. It is not
|
||
a second hand-maintained compiler and is never used in ordinary production
|
||
builds. The resulting `ww0` retains that portable C-emission path: it contains
|
||
only enough compiler, export writer, C output, and fixed-plan execution to emit
|
||
stage-1 C from the enumerated source closure and have the named host C closure
|
||
compile it. It has
|
||
no resolver, network client, general manifest engine, cache, test coordinator,
|
||
or installation framework.
|
||
|
||
The release process regenerates `ww0.c` and fails if its bytes differ from the
|
||
checked-in snapshot. Keeping portable source rather than four opaque host
|
||
binaries permits recovery on a new architecture with a C99 implementation. The
|
||
cost of the simple C emitter is accepted because it bounds and exposes the seed;
|
||
it is a release backend, not another production path.
|
||
|
||
The bootstrap-host contract is narrower than “any C99”: hosted C99 with
|
||
`CHAR_BIT == 8`, exact `uint8_t/uint32_t/uint64_t`, two's-complement signed
|
||
integers, binary file I/O, at least 32-bit address space and the published source/
|
||
object size limits. The snapshot uses no host floating-point result or undefined
|
||
signed overflow. Compile-time assertions plus a tiny I/O/integer conformance
|
||
probe run before compilation. Supported implementation modes and required
|
||
headers are enumerated in `bootstrap.plan`; an implementation outside the
|
||
contract is not silently called portable.
|
||
|
||
### 10.2 Stage transitions and fixed point
|
||
|
||
Recovery uses these exact stages:
|
||
|
||
1. A named host C implementation compiles `ww0.c` to `ww0`. Its executable,
|
||
version output, command, headers, libc, assembler, and linker are recorded in
|
||
`bootstrap-host.wwar`; they are part of the trusted base, not silently blessed.
|
||
2. `ww0 bootstrap/bootstrap.plan` runs on B, emits portable C for the locked
|
||
production compiler/driver, and uses that same recorded host C closure to
|
||
produce **stage 1 with H = B**. Stage 1 must run locally; distributable cross
|
||
compilers are built only after the local fixed point.
|
||
3. Stage 1 uses the pinned production target/tool closure to perform a normal
|
||
frozen build of the same sources for `H = B`, producing **stage 2**.
|
||
4. Stage 2 repeats the identical logical build to produce **stage 3**.
|
||
5. Stage 3 repeats it to produce **stage 4**.
|
||
6. `bootstrap.compare` first requires stage-2 and stage-3 **semantic output
|
||
sets**—compiler/driver executables, export/object/runtime artifacts, and the
|
||
installed semantic bundle tree—to be byte-identical. Producer action records
|
||
and provenance are excluded because stage 2 was built by stage 1 while stage
|
||
3 was built by stage 2. Once those compiler bytes converge, it requires
|
||
stage-3 and stage-4 semantic outputs **and raw action/result records** to be
|
||
byte-identical; their producer compiler digests are then equal. Each raw
|
||
record is a separately named `action-record` or `action-result` input in
|
||
action tag 10; comparison metadata or a digest outside tag 10 is not read
|
||
authority.
|
||
|
||
Stage 1 need not equal stage 2: portable-C and production backends may generate
|
||
different code. Stage 2 MUST equal stage 3 semantically, and stage 3 MUST equal
|
||
stage 4 completely. A canonical stage semantic manifest lists only role/output
|
||
artifact digests, never its producer. A mismatch reports the first differing
|
||
output or action field and is a release failure. Two absolute roots and two
|
||
concurrency levels are used for the official fixed-point job.
|
||
|
||
This fixed point proves self-consistency and path/order reproducibility; it does
|
||
not by itself defeat Ken Thompson's trusting-trust attack. Official releases
|
||
also perform **diverse seed compilation** with two independently sourced host C
|
||
toolchains where available, publish both bootstrap-host records, regenerate the
|
||
C snapshot from the converged WW source/compiler, and compare converged semantic
|
||
outputs before signing. This is additional evidence, not a claim of formal
|
||
diverse-double-compilation proof.
|
||
|
||
### 10.3 Trusted-computing-base accounting
|
||
|
||
The bootstrap report lists, by content digest:
|
||
|
||
- `ww0.c`, `bootstrap.plan`, and their tiny C support layer;
|
||
- host C compiler/preprocessor, headers, libc, assembler, linker, loader, and OS
|
||
kernel used to create/run `ww0`;
|
||
- bootstrap assembler/linker/archive tools and their shared/resource closure;
|
||
- source/lock/toolchain descriptor trees; and
|
||
- the SHA-256 and signature verification implementations/keys.
|
||
|
||
That is the reproducible software/input closure. The operational trusted
|
||
computing base additionally includes CPU, firmware, memory/storage behavior,
|
||
kernel, and execution environment; the report names the available hardware/
|
||
firmware attestations but does not pretend a source hash removes physical trust.
|
||
|
||
Nothing is called “trusted” merely because it was found on `PATH`. A recovery
|
||
build with unpinned system tools may establish a new local fixed point but cannot
|
||
claim byte identity with an official release. An official reconstruction uses a
|
||
published content-identified host/bootstrap closure and compares its advertised
|
||
digests.
|
||
|
||
### 10.4 Toolchain distribution and selection
|
||
|
||
Each release publishes source plus locked-source-closure archive, `ww0.c`, lock
|
||
file, platform toolchain bundles, `toolchain.wwt`, stage-2/3/4 semantic and
|
||
action/result manifests, fixed-point report,
|
||
digests, signatures, and provenance. Binary bundles are conveniences verified
|
||
against the descriptor, not irreplaceable seeds. A user either selects the
|
||
project-locked toolchain or an exact command-line digest; WW never downloads a
|
||
new compiler while building.
|
||
|
||
Runtime ABI, export protocol, object ABI, and action-schema compatibility are
|
||
declared independently. A compiler refuses a runtime whose ABI it does not
|
||
support. Toolchain upgrades can retain source compatibility while intentionally
|
||
invalidating all relevant action keys.
|
||
|
||
### 10.5 New-architecture and no-compiler recovery
|
||
|
||
For a new H satisfying the bootstrap-host contract, compile `ww0.c`; `ww0` emits
|
||
the modified production compiler as portable C and the H C compiler builds the
|
||
locally runnable stage 1. Add/pin H's production object/assembler/linker closure,
|
||
then reach the stage-2/3/4 fixed point. Alternatively an existing B toolchain may
|
||
cross-build that H compiler, but it executes only through an explicit runner.
|
||
For a new T only, no target-local compiler is required: add the backend/target
|
||
descriptor, object tests, sysroot or freestanding runtime, and linker provider
|
||
to an existing B/H toolchain.
|
||
|
||
When no suitable WW compiler exists, the checked-in C seed is sufficient. When
|
||
no C compiler exists either, a platform must provide one previously built `ww0`
|
||
binary plus its exact source/digest, or bootstrap a C implementation; WW does not
|
||
claim a smaller physical trusted base than the machine can execute.
|
||
|
||
## 11. Current-system assessment
|
||
|
||
This section records the verified baseline at commit
|
||
`ca2cadeb4173e8190cd4c8bcc25e7da25bcdb0bc`. Line references describe that
|
||
commit and are implementation evidence, not requirements for the replacement.
|
||
|
||
### 11.1 Current invariants
|
||
|
||
- POSIX Make selects ambient `CC`, `AR`, optional `ccache`, and flags; builds the
|
||
Cstage tools; then uses the Cstage driver to build WWstage tools into separate
|
||
work directories. Make repeats manually enumerated transitive source
|
||
prerequisites for each self-hosted target
|
||
([Makefile](../Makefile), lines 7–20 and 105–279).
|
||
- `ww build` creates fixed-array package nodes keyed by dotted import spelling,
|
||
discovers imports with a hand-written scanner separate from the compiler
|
||
parser, rejects directory-package cycles, visits dependencies in
|
||
DFS postorder, and invokes packages serially. The graph is capped at 256
|
||
packages ([C driver](../cmd/ww/main.c), lines 333–370 and 670–700).
|
||
- An imported directory is a separately compiled package. An imported `.ww`
|
||
file is recursively folded into the importing unit and has no node, identity,
|
||
interface, object, or link artifact of its own. Inline package blocks can also
|
||
satisfy otherwise missing imports (C driver, lines 396–428 and 498–613).
|
||
- A directory package selects all immediate `.ww` files except `*_test.ww`, sorts
|
||
them by bytes, and has no target-specific selection rule (C driver, lines
|
||
239–301). The build path follows source symlinks while the test coordinator
|
||
rejects them.
|
||
- Resolution translates dotted imports to paths and performs a global
|
||
directory-before-file search over the entry root, `-I` roots, and inferred
|
||
source library. Thus a directory in a later root beats a file in an earlier
|
||
root (C driver, lines 152–224 and 966–1003).
|
||
- Every dependency emits source-like `.wwi`. Every importer receives the whole
|
||
transitive `.wwi` closure, tagged with out-of-band module comments and prepended
|
||
to a composed `.unit.ww`; the compiler reparses that unit (C driver, lines
|
||
702–809; [interface writer](../cmd/w6c/wwi.c), lines 1–24 and 505–570).
|
||
- The root compiles without compiler `-I`, emits no `__root.wwi`, bypasses the
|
||
dependency export-signature path, and receives special `main` handling; only
|
||
dependency nodes emit interfaces. `ww test` also injects the resolvable `test`
|
||
package as a synthetic root edge even when source has no such import, while
|
||
runtime is an implicit link edge. Imports do not fully describe even today's
|
||
complete graph (C driver, lines 1066–1083 and 1131–1146).
|
||
- `.wwi` records exported prototypes and direct import text but no compiler,
|
||
format, target, data-layout, object ABI, or runtime ABI identity. It includes
|
||
non-semantic parameter names and import spelling. The compiler's `-I` flag
|
||
both requests interface output and changes `main` symbol handling
|
||
([compiler entry](../cmd/w6c/main.c), lines 25–56 and 83–110).
|
||
- For `P` reachable directory packages including root, a normal driver build
|
||
launches `P` compiler processes, `P` assembler processes, and one linker.
|
||
It writes `P-1` dependency archives itself. The link is root object, dependency
|
||
archives in reverse topological order, runtime, then separately accumulated
|
||
`-L` and `-l` values, losing their original interleaving (C driver, lines
|
||
1095–1249 and 1368–1460).
|
||
- `-w DIR` is a caller-owned mutable reuse directory, not a cache. Freshness is
|
||
exact composed-unit bytes, copied driver/compiler/assembler bytes, a text mode
|
||
stamp, and artifact existence/nonzero size. The graph and units are rebuilt
|
||
in memory and the final executable is relinked on every invocation.
|
||
Publication through `.new` files with the unit committed last is usefully
|
||
atomic (C driver, lines 867–952, 1016–1059, and 1110–1213).
|
||
- Without `-w`, a build creates and deliberately retains
|
||
`<output-stem>.sepwork`; repeating while it exists fails. With `-w`, the caller
|
||
must create and serialize the directory. Normal driver products are always
|
||
root executables linked with the runtime; there is no library-only product
|
||
path, and `-S` merely stops after assembly.
|
||
- `ww test DIR` delegates to a separate coordinator. It groups same-package
|
||
`package p;` and external `package p_test;` tests from `*_test.ww`, excludes
|
||
dependency tests, composes generated source roots, and parallelizes independent
|
||
test binaries with deterministic reporting
|
||
([package coordinator](../internal/wwpackage/package.ww), lines 278–417,
|
||
508–648, and 725–1057).
|
||
- Explicit `ww test FILE` bypasses the directory `*_test.ww` classifier and
|
||
accepts an arbitrarily named source root. Universal directory packages delete
|
||
that distinct test mode.
|
||
- Current bootstrap is mixed C/self-hosted. Make keeps the C driver fixed, uses
|
||
self-hosted compiler stages to produce `ww2`, `ww3`, and `ww4`, and compares
|
||
`ww2 == ww3` and `ww3 == ww4`. The self-hosted driver itself is outside that
|
||
fixed-point chain. The planned four stage-zero binaries are absent; the
|
||
“no C compiler” route still uses host `ar` ([Makefile](../Makefile), lines
|
||
825–883; [bootstrap notes](../BOOTSTRAP.md)).
|
||
|
||
### 11.2 Conflated identities and accidental behavior
|
||
|
||
| Concept that must be separate | Current conflation or accident |
|
||
|---|---|
|
||
| package identity | Dotted import spelling is graph key, module/symbol prefix, artifact basename, and link identity. |
|
||
| declared name | A directory's leaf is checked against it for imports, but root directories and literal file roots receive different validation. `.wwi` itself retains only the leaf package name. |
|
||
| filesystem location | Ordered search roots silently choose/shadow a location; the same physical directory may be compiled under two import identities, while duplicate locations for one spelling produce no collision diagnostic. Paths are lexical, not content identities. |
|
||
| package versus file | Directories create separate-compilation nodes; files disappear into owners. The same `package` syntax means two compilation models. |
|
||
| artifact versus identity | `<dotted-path>.wwi/.s/.o/.a` names artifacts; root aliases to `__root`, which can collide with a real import. |
|
||
| public versus non-semantic interface | Parameter names and AST-preserved type/import spellings influence `.wwi` bytes and reverse rebuilds. Imports/declarations are otherwise canonically sorted; whitespace/comments and original declaration order generally do not. |
|
||
| compiler interface mode versus link identity | `w6c -I` both emits `.wwi` and classifies the package as a dependency for `main` mangling. |
|
||
| native dependency versus linker search | Raw `-L` and `-l` names carry no selected file, ABI, order relationship, target, or content identity. |
|
||
| cache location versus cache key | The explicit `-w` directory is both mutable namespace and freshness state; callers must serialize it. |
|
||
| source root versus command UX | Help historically describes `.` like a basename file, while implementation stats and builds it as a directory. |
|
||
|
||
Other accidental constraints include fixed 256/1024-byte name/path buffers. The
|
||
compiler parser silently truncates dotted full imports beyond 255 bytes, while
|
||
the C driver scanner can stop advancing and hang on an import identifier at that
|
||
limit; the dynamically sized WWstage scanner differs. There is no regression
|
||
test for this stage divergence. Compiler, assembler, and linker launches now use
|
||
structured argument vectors in both stages. Both drivers honor exact executable
|
||
paths in `WW_W6C`, `WW_W6A`, and `WW_W6L` and otherwise select their
|
||
stage-specific sibling tools. Both drivers distinguish the package-source root
|
||
selected by `WW_SRCLIB` from the runtime-artifact root selected by `WW_LIB`, and
|
||
apply the same empty-value and repository/install fallbacks. The two
|
||
implementations remain parallel production algorithms rather than one protocol
|
||
implementation.
|
||
|
||
Build and test disagree about source symlinks. External package tests are built
|
||
from a generated single-file root plus `-I`; an external import of a multi-file
|
||
production package can resolve and fold only its canonical same-named file rather
|
||
than the directory package. Test work-directory names flatten `/` to `_`, so
|
||
distinct lexical paths can collide. These are consequences of routing tests
|
||
around, rather than through, one package model.
|
||
|
||
`make install` copies only `ww`, `wwtest`, and `libwcc.a`, while the driver needs
|
||
sibling compiler/assembler/linker tools and `libwwrt.a`; the installed result is
|
||
not a self-contained functional toolchain outside the build tree.
|
||
|
||
### 11.3 Scaling, invalidation, and hidden inputs
|
||
|
||
The driver stores fixed `deps[256]` arrays and performs linear graph lookup.
|
||
Every package unit reads one interface per transitive dependency, so total
|
||
interface reads and copied interface text are quadratic on deep/dense graphs even
|
||
on a warm build. Package compiler/assembler work within one driver is serial;
|
||
Make gains parallelism only by launching independent top-level driver builds.
|
||
|
||
The observed invalidation rules are:
|
||
|
||
- a private change in a directory dependency rebuilds that package and the
|
||
unconditional final link, but not importers;
|
||
- an exported change changes its `.wwi` and rebuilds the whole reverse-transitive
|
||
ancestor cone, even when an intermediate package's own interface is unchanged;
|
||
- a private change in a folded file import rebuilds its entire owner;
|
||
- a link-only option reruns the always-executed link but not package compiles;
|
||
- changing copied compiler or assembler bytes rebuilds every package; and
|
||
- nonzero corruption of `.s`, `.o`, `.a`, or `.wwi` may be accepted because
|
||
content is not rehashed.
|
||
|
||
Hidden or incompletely modeled inputs include `CC`, `AR`, `PATH`, `ccache`, Make
|
||
flags, compiler built-ins, assembler/linker defaults,
|
||
inferred `argv[0]` library locations, current working directory, file mode, runtime
|
||
archive, linker binary, native-library resolution, host libc/CRT/loader, SDK,
|
||
CPU, target, and environment. Make does not invalidate existing C objects when
|
||
the host compiler or C flags change. Persistent package workdirs now bind the
|
||
exact driver executable, conservatively covering its graph, unit, archive, and
|
||
commit semantics; the remaining ambient inputs are not thereby promoted into a
|
||
cache protocol. WW has no target triple, sysroot, conditional source, generator,
|
||
manifest, lock, source digest, or frozen/offline concept.
|
||
|
||
### 11.4 Measurements and disposable experiments
|
||
|
||
Measurements ran on Linux 6.12.76_1 x86-64 with eight logical CPUs, GCC 14.2.1,
|
||
binutils 2.44, and no `ccache`. They used successful clean builds and isolated
|
||
temporary source fixtures; no production migration was begun. Clean/full means
|
||
use two timed samples and representative cold means three, without CPU isolation
|
||
or OS-cache flushing, so they are observed baselines rather than universal
|
||
performance claims.
|
||
|
||
| Measurement | Result |
|
||
|---|---|
|
||
| clean `make all`, `-j1` | 24.847 s mean |
|
||
| clean `make all`, `-j8` | 15.515 s mean; only 1.60× speedup |
|
||
| warm `make all -j8` | 0.02365 s mean at Make level |
|
||
| clean tool trace | 69 package compiles/assembles in six driver workdirs; 71 `w6c`, 76 `w6a`, six `w6l`, 63 in-driver archives; package work inside each driver stayed serial |
|
||
| small / 8-package / 15-package cold driver build | 0.0103 / 0.1326 / 9.6706 s |
|
||
| same warm driver builds | 0.00814 / 0.01347 / 0.05302 s; each still linked |
|
||
| `out/` | 34,464,015 bytes, 398 files |
|
||
| six WW build workdirs | 28,609,092 bytes, 351 files; 9,954,428 bytes (34.8%) duplicate beyond the first content copy |
|
||
| 15-package root composed unit | 14 `.wwi` sections, 1,464,581 bytes; 62 interface insertions across graph, 1,928,898 composed-unit bytes versus 23,664 distinct interface bytes |
|
||
|
||
A three-package `root -> mid -> leaf` fixture confirmed that root consumes both
|
||
direct and transitive interfaces. A private leaf implementation edit rebuilt
|
||
only leaf plus link. Adding a compatible public leaf API rebuilt all three even
|
||
though `mid` emitted identical `.wwi`. A link-only `-L` change ran only the link.
|
||
|
||
Switching the compiler executable by byte content invalidated all packages.
|
||
Appending data to a nonzero cached `mid.s` was not detected; the old object was
|
||
reused and the build succeeded. This directly rejects artifact-presence caching.
|
||
|
||
Two builds from different absolute source roots produced the same 398 relative
|
||
paths and all sampled WW-generated executables/package artifacts were byte
|
||
identical. The full trees were not: 45 Cstage host objects/copied tools differed,
|
||
including GCC `DW_AT_comp_dir`. Thus current checkout-independent byte identity
|
||
holds for measured WW artifacts, not for the complete build.
|
||
|
||
Cstage and WWstage drivers building the same eight-package graph produced a
|
||
byte-identical final executable and all 38 non-tool artifacts, but took 0.138 s
|
||
and 0.356 s respectively. The audit also found that Cstage formerly returned
|
||
success after an ambient `chmod` lookup failed and left mode 0644, while WWstage
|
||
created mode 0755 directly. Cstage now calls `chmod(2)` on the exact output path
|
||
and reports failure, removing that host-tool and path-splitting asymmetry.
|
||
|
||
### 11.5 What survives and what is deleted
|
||
|
||
The replacement retains these sound concepts:
|
||
|
||
- explicit source imports, package clauses, exported/private declarations, and
|
||
loud directory-package-cycle errors, strengthened to every package after file
|
||
folding (whose cycles were merely visit-deduplicated) is deleted;
|
||
- directory package boundaries, made universal rather than optional;
|
||
- same-package and external `*_test.ww` semantics, dependency-test exclusion,
|
||
deterministic discovery/reporting, and always executing selected tests;
|
||
- deterministic sorting/serialization, atomic artifact publication, fixed-point
|
||
bootstrap checks, and byte-identity tests; and
|
||
- the ordinary `ww build`, `run`, and `test` user experience.
|
||
|
||
The replacement deletes these concepts rather than emulating them indefinitely:
|
||
|
||
- file imports, inline multi-package units, bare dotted imports, `-I` search
|
||
roots, directory-before-file precedence, and the `__root` artifact alias;
|
||
- `.wwi`, `//ww:module` wrappers, `.unit.ww`, transitive interface prepending,
|
||
source-prototype interchange, automatic per-dependency archives, and `-w`;
|
||
- raw ambient `-L`/`-l`, compiler/linker/sysroot defaults, and build-time
|
||
`pkg-config` discovery;
|
||
- separate C and self-hosted production drivers, the production Make graph, and
|
||
the separate source-composing test coordinator; and
|
||
- the current Cstage bootstrap path and permanently owned assembler/linker after
|
||
the generated C seed and pinned external tool closure replace them.
|
||
|
||
There will be no compatibility alias that silently translates an old import,
|
||
interface, workdir, or link search into the new model.
|
||
|
||
### 11.6 Implemented local package slice
|
||
|
||
The first executable package slice is intentionally smaller than the final
|
||
module design above. It is local, offline, and manifest-free. The supported
|
||
form is:
|
||
|
||
```sh
|
||
out/bin/ww build -I /work/acme -o app /work/acme/cmd/app
|
||
```
|
||
|
||
Every selected source uses the existing syntax:
|
||
|
||
```ww
|
||
package main;
|
||
import lib.math;
|
||
```
|
||
|
||
An import is translated from dots to path separators and resolved only as a
|
||
directory package through the entry package's directory, explicit `-I` roots in
|
||
command order, and the toolchain source-library root. A same-named `.ww` file is
|
||
neither a match nor a shadow for an import, so a later root containing the
|
||
directory wins over an earlier file decoy. There is no network, manifest, or
|
||
imported-file fallback. Explicit single-file CLI roots retain their raw-unit
|
||
compatibility path. The loader uses the compiler frontend's imports-only parser,
|
||
unions duplicate imports, byte-sorts direct edges, interns resolved directories
|
||
by filesystem identity, and reports self-imports and stable cycle chains before
|
||
compilation.
|
||
|
||
A directory package consists of its immediate regular non-symlink `.ww` files,
|
||
excluding `*_test.ww`, in byte-sorted filename order. Every selected file must
|
||
declare the same package. An imported directory's declared package must equal
|
||
the final component of its import path; two logical identities for one physical
|
||
directory are rejected rather than compiled twice.
|
||
|
||
Packages compile serially in dependency-first postorder. The compiler emits the
|
||
existing deterministic `.wwi` interface for every directory-package action,
|
||
including an executable root. Its primary section contains that package's
|
||
byte-sorted direct imports and exported declarations. The compiler then appends
|
||
byte-sorted, origin-tagged sections for only the foreign type and constant facts
|
||
recursively reachable from the primary public signatures. Reachable owner-local
|
||
private nominal types are carried without `export`: they make the export
|
||
self-contained for type checking, but qualified source lookup still rejects
|
||
their names. Checked fixed array dimensions are emitted as numeric type facts,
|
||
so a public layout never requires exposing the private constant spelling that
|
||
produced its length.
|
||
|
||
A package compilation unit contains only that package's own byte-sorted sources
|
||
and deterministic `//ww:module-reset` separators. Each **direct** import is a
|
||
separate `--import <canonical-path> <dependency.wwi>` compiler input, sorted by
|
||
canonical path and deduplicated by the loader; no transitive `.wwi` is passed.
|
||
Origin-tagged facts inside those direct artifacts are compiler data, not source
|
||
imports: a source qualifier is visible only when its owning package directly
|
||
imports it, and private members, transitive-only qualifiers, bare values, and
|
||
bare types remain compiler errors. In `-c` package mode the compiler parses each
|
||
export independently, then coalesces repeated exported type/constant facts with
|
||
the same origin, kind, and name, preserving one nominal type identity across
|
||
diamonds; raw non-package `w6c` retains its existing one-source behavior. The
|
||
source-like `.wwi` syntax remains a transitional export encoding pending the
|
||
binary `.wwe` format described above, but the separate direct-input ownership
|
||
boundary is live in production Cstage and WWstage compilers and drivers.
|
||
|
||
An ordinary executable directory root is one normal package action. Its
|
||
declared package identity tags its owner-only unit; it receives only direct
|
||
exports, emits `.wwi`, `.o`, and a deterministic `.a`, and is compiled exactly
|
||
once. The narrow compiler `--entry` flag controls only bare `main` codegen and is
|
||
independent of export production. The linker receives that root archive first,
|
||
then the complete reachable package-archive closure and runtime archive; it
|
||
never receives `.wwi`. The linkers seed `main` before archive selection, so the
|
||
existing WWAR member protocol needs no special root object or format change.
|
||
|
||
`ww build -p -o lib.a DIR` explicitly requests a non-main package product: it
|
||
emits a deterministic archive at `lib.a` and its compiler interface at
|
||
`lib.a.wwi`, without invoking the linker. A logical target retains its full
|
||
identity (`ww build -p -I ROOT -o bar.a foo.bar` emits `foo.bar.*` symbols),
|
||
while a literal directory uses its declared leaf package. Package output
|
||
requires a directory and `-p` cannot be combined with assembly-only `-S`. Two
|
||
cold builds with identical inputs are required to produce byte-identical
|
||
requested products. Compiler intrinsics keep their package-mode runtime ABI
|
||
independent of transitive source interfaces (for example, `alloc` lowers to the
|
||
runtime allocator without requiring an `rt.wwi` compiler input).
|
||
|
||
### 11.7 Implemented directory package-test slice
|
||
|
||
Directory tests now enter that same local package loader and build path. The
|
||
supported manifest-free commands are `ww test DIR`, `ww test DIR/...`, and
|
||
their existing `-run`, `-filter`, `-list`, `-timeout-ms`, `-j`, `-c`, and `-w`
|
||
forms. `ww test -c -o test.bin DIR` names the result when the selected directory
|
||
has one test variant; the coordinator rejects one output name for a multi-variant
|
||
or recursive request. Explicit `ww test FILE` retains its compatibility path.
|
||
|
||
The test coordinator still discovers requested directories, enumerates the
|
||
test package names, selects variants, executes independent binaries, and emits
|
||
captured results in byte-sorted package order. It no longer concatenates a
|
||
generated production/test root, resolves imports, or starts one package graph
|
||
per binary or directory. Instead it sends one ordered build request containing
|
||
every selected directory/variant root to the Cstage or WWstage command. Its
|
||
semantic selections are only the directory and selected variant/package
|
||
identities; it also carries output destinations, coordinator-private completion
|
||
paths, import search roots, and the optional command-scoped work-directory
|
||
policy. The command owns source selection, package loading, compiler inputs,
|
||
archive construction, generated-main construction, and linking for every
|
||
package variant:
|
||
|
||
- The ordinary production action selects the directory's byte-sorted
|
||
non-test files and is reused wherever that canonical package is imported.
|
||
- `same-test` is a distinct internal production-plus-test action. It selects
|
||
the byte-sorted production files followed by the byte-sorted matching
|
||
`package p` test files, so tests can use private production declarations.
|
||
- `external-test` selects only matching `package p_test` files. Its `import p`
|
||
is a direct edge to the canonical production action for that directory.
|
||
That action compiles with module qualifier `p`, selects every production
|
||
file, emits compiler export data and an archive, and exposes no private
|
||
declaration to the external root. If the action is first reached through
|
||
the external product, its collision-proof artifact key is derived from the
|
||
owning root, such as `__ww-test-001-external-production`; a normal import of
|
||
the same `p` and canonical directory reuses that action rather than creating
|
||
a second compilation.
|
||
- Each selected internal or external variant gets a distinct generated-main
|
||
package action. Its generated owner-only unit declares `package main` and
|
||
imports exactly the selected variant and test-support package. It consumes
|
||
exactly those direct `.wwi` artifacts, emits its own `.wwi/.o/.a`, and alone
|
||
receives compiler `-T --entry`.
|
||
|
||
The command loads all variants into one command-scoped package universe.
|
||
Directory variants retain injective artifact keys such as
|
||
`__ww-test-000-same` or `__ww-test-003-external`; their generated mains use
|
||
matching `__ww-test-NNN-main` keys and distinct internal package identities.
|
||
Hyphens keep artifact keys illegal as WW import identities. Repeated requests
|
||
for the same canonical directory and variant reuse one compile action, and
|
||
imports of the same canonical production directory intern to one production
|
||
action across every selected test product. A deterministic dependency-first
|
||
traversal of the complete union therefore invokes the compiler and archiver
|
||
once per package variant, even when many products share it. Generated-main
|
||
actions remain product-specific and are linked separately. The shared plan is
|
||
deliberately package-test-specific: it is not a generalized scheduler, action
|
||
schema, cache, or protocol.
|
||
|
||
Each selected directory retains the ordinary entry-directory-first resolution
|
||
context from the local package slice: its directory, explicit `-I` roots in
|
||
command order, then the toolchain source root. Same and external variants of
|
||
one directory share that context; unrelated directory roots never acquire
|
||
lookup precedence from their request order. When multiple contexts reach one
|
||
canonical production package, the loader verifies that every directory import
|
||
binding is identical before reusing its compile action. A different binding is
|
||
a deterministic package-resolution failure for the roots that reach it, rather
|
||
than a first-root-wins build.
|
||
|
||
A production action failure blocks exactly the roots that reach it. A
|
||
root-local compile or link failure does not suppress a successfully built
|
||
sibling product: the command records completion per product, and the
|
||
coordinator can run and report successful siblings while attributing each
|
||
missing product as a build failure. The single union build and completed test
|
||
products share the coordinator's existing `-j` process bound; captured output
|
||
is still emitted only in byte-sorted directory/package order.
|
||
|
||
Every source-imported dependency is a production variant, so dependency
|
||
`*_test.ww` files never enter the graph. Imports that occur only in selected
|
||
test files add edges only to that internal or external variant. Each compiler
|
||
unit contains only its action's owned source set, while its invocation receives
|
||
only the byte-sorted direct dependency `.wwi` artifacts as separate inputs.
|
||
Variant compiles receive `--test-package`, which validates and retains private
|
||
`@test` declarations as compiler-only export metadata without synthesizing an
|
||
entry point. The distinct generated-main action consumes that metadata from its
|
||
direct variant export and synthesizes the dispatcher with `-T`.
|
||
|
||
The generated-main action, rather than the tested variant, owns the implicit
|
||
direct test-runtime support edge. The command-scoped plan compiles the common
|
||
support production package once for the complete test request. The command
|
||
resolves that edge from the selected toolchain source tree, not the user search
|
||
path; the support package's own imports are also loaded in that toolchain
|
||
context. Normally its graph
|
||
qualifier is `test`, so an explicit source `import test` coalesces with the same
|
||
canonical package. When a real user package occupies that identity, the command
|
||
presents the runtime edge to the compiler under the reserved `__wwtest`
|
||
qualifier. This keeps a production package named `test` available to external
|
||
tests. Explicit raw single-file `ww test FILE` fixtures retain the narrow fused
|
||
`-T` compatibility path because an anonymous multi-package raw unit is not a
|
||
canonical directory package; that path still consumes support as a direct
|
||
export and emits a root `.wwi/.a`.
|
||
|
||
The reserved support action and an ordinary source-imported toolchain `test`
|
||
action may coexist in the command universe because their compiler qualifiers
|
||
are `__wwtest` and `test`. An external-first colocated production action uses
|
||
an owning-root artifact key so it remains collision-proof if the command also
|
||
loads a different physical package bearing the same compiler qualifier.
|
||
Same-qualifier, same-canonical normal and external-production actions always
|
||
coalesce; different logical identities for one ordinary physical directory
|
||
remain an error. These are the
|
||
only narrow action-role exceptions: each product closure is checked to contain
|
||
at most one importable action for a compiler module qualifier, so unrelated
|
||
roles can never introduce duplicate linked package symbols.
|
||
|
||
The selected internal/external variants and a production action reached by
|
||
their imports or test-runtime closure are the sanctioned graph nodes that may
|
||
share a physical directory. This also lets a toolchain package's own tests
|
||
coexist with the production action required by the test runtime. An internal
|
||
variant already defines those production symbols, so the colocated production
|
||
archive is omitted from that product's final link while the production action's
|
||
dependency archives remain in the closure. The external product includes the
|
||
production archive. Variant-only archives are never linked into another
|
||
product. All ordinary logical and physical package-identity collision checks
|
||
remain unchanged.
|
||
|
||
Both stage linkers receive the generated-main archive first, followed by the
|
||
complete reverse-topological reachable package-archive closure, runtime, and
|
||
explicit `-L`/`-l` values through a structured argument vector. No `.wwi` or
|
||
special root `.o` appears in linker argv, and no fixed flattened command buffer
|
||
can truncate a large closure. WWstage emits joined `-Ldir` and `-lname`
|
||
arguments accepted by its native linker, while Cstage preserves the equivalent
|
||
split forms. Generated artifact paths are bounds-checked before any unit is
|
||
opened, so distinct root keys cannot alias by truncation.
|
||
Recursive discovery groups by physical directory before sorting filenames, and
|
||
one stable escaped request key names the persistent command work directory.
|
||
Every `*_test.ww` package variant is built even when a file only
|
||
declares helpers and contains no `@test`, so its package clause and imports are
|
||
still checked by the shared loader. Every built variant is run, and a successful
|
||
compiler-owned dispatcher with empty output is reported as `[no tests]`.
|
||
|
||
Persistent workdirs keep a global driver/compiler/assembler/stamp identity and
|
||
per-action committed units. When that global identity is stale, the command
|
||
first removes every old `.unit.ww` voucher while leaving artifacts recoverable.
|
||
It can then commit the new identity even if one root fails: successful actions
|
||
have freshly committed units, whereas failed or no-longer-requested actions
|
||
cannot be reused. A retry therefore recompiles the failed action without
|
||
discarding unchanged canonical packages built for independent products.
|
||
|
||
This ownership split follows Go 1.26.5's separation of production,
|
||
same-package-test, external-test, and generated test-main inputs in
|
||
[`cmd/go/internal/load/test.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go),
|
||
while retaining WW's compiler-owned dispatcher. Go's loader returns the same
|
||
cached package pointer for repeated imports, and `PackageList` walks arbitrary
|
||
multiple roots with pointer-based deduplication in
|
||
[`cmd/go/internal/load/pkg.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go).
|
||
Its command-scoped builder deduplicates compile actions by package identity. WW
|
||
borrows that one-universe, multi-root ownership boundary without borrowing Go's
|
||
cache machinery. Direct
|
||
compile dependencies and the separately expanded link closure follow the
|
||
boundary in
|
||
[`cmd/go/internal/work/action.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go),
|
||
with compiler and linker import configurations emitted separately in
|
||
[`cmd/go/internal/work/exec.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go).
|
||
|
||
### 11.8 Implemented exact package-tool invocation slice
|
||
|
||
The local package builder now launches the compiler, assembler, and linker as
|
||
an executable plus an argument vector in both Cstage and WWstage. No package
|
||
source path, work-directory artifact path, output path, test-support qualifier,
|
||
or link-closure member is flattened into a shell command. Paths containing
|
||
spaces therefore retain one argument boundary from the package coordinator
|
||
through compilation, assembly, and final executable linking.
|
||
|
||
`WW_W6C`, `WW_W6A`, and `WW_W6L` each name one exact executable path. They are
|
||
not shell fragments and are not searched through `PATH`. With no override,
|
||
Cstage keeps its `w6c`/`w6a`/`w6l` siblings and WWstage keeps its
|
||
`w6c_ww`/`w6a_ww`/`w6l_ww` siblings. The coordinator preserves these variables
|
||
when it starts the one command-scoped package build, so the same contract covers
|
||
ordinary directory builds, same-package tests, external tests, recursive test
|
||
requests, and persistent-workdir tool identity. A failed overridden compiler or
|
||
assembler remains attributed to its owning package in both stages. The Cstage
|
||
linker also sets executable mode with `chmod(2)` on the exact output path rather
|
||
than invoking an ambient command.
|
||
|
||
Source imports still own the graph, each directory is still one production
|
||
package, compiler actions receive direct dependency exports as individual
|
||
arguments beside an owner-only `.unit.ww`, and links receive the complete
|
||
per-root `.a` closure. Repository-native coverage wraps all three real stage
|
||
tools at executable paths containing spaces, records every argument boundary,
|
||
inspects `.unit.ww`, `.wwi`, `.a`, and root archive placement, runs the published
|
||
binary, compares repeated Cstage/WWstage artifacts and traces, and removes one
|
||
direct export at compiler entry to compare package-attributed diagnostics.
|
||
|
||
Go 1.26.5 keeps the same responsibility boundary: its work executor passes the
|
||
selected compiler or linker tool and a constructed argument slice to the
|
||
builder, while package loading and action construction remain separate
|
||
([`cmd/go/internal/work/exec.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go)).
|
||
WW adopts that exact-tool boundary without adding a command schema, generalized
|
||
action graph, scheduler, manifest, cache protocol, or package-manager behavior.
|
||
|
||
### 11.9 Implemented package and runtime library-root parity slice
|
||
|
||
Cstage and WWstage now apply the same two-root contract. A nonempty
|
||
`WW_SRCLIB` selects the toolchain package-source search root. It is appended
|
||
after the entry directory and explicit `-I` roots for every package context,
|
||
and compiler-generated test support is resolved from that toolchain root rather
|
||
than from a user package that happens to have the same name. A nonempty
|
||
`WW_LIB` independently selects runtime artifacts; executable and test links use
|
||
`$WW_LIB/libwwrt.a`. If that archive is absent, both stages pass their sibling
|
||
`start.o` and `syscall.o` runtime objects instead. Missing and explicitly empty
|
||
values use the same deterministic fallbacks in both stages.
|
||
|
||
Bare logical build and run targets search the current directory, explicit
|
||
`-I` roots, then one selected library root. That last root is a nonempty
|
||
`WW_SRCLIB`, else a nonempty `WW_LIB`, else an existing
|
||
`<selfdir>/../../lib`, else an existing `./lib`, else
|
||
`<selfdir>/../lib`. Once a target is resolved, its import graph retains the
|
||
ordinary entry-directory-first context and its separate source-root policy:
|
||
without `WW_SRCLIB`, that policy prefers the repository and `./lib` source
|
||
trees before falling back to the runtime root. This preserves Cstage's
|
||
existing distinction between locating a requested package and selecting the
|
||
toolchain source tree used by that package's imports.
|
||
|
||
The variables select local directories only. They do not add manifests,
|
||
dependency declarations, network lookup, package-manager behavior, or a second
|
||
graph: source `import` declarations still create every language edge, resolved
|
||
directories still intern to canonical package identities, compilers still
|
||
receive only direct `.wwi` exports, and final links still receive the complete
|
||
reachable `.a` closure. Directory tests continue to build same-package and
|
||
external-package variants through the same command-scoped package universe.
|
||
|
||
Repository-native coverage uses distinct source and runtime roots whose paths
|
||
contain spaces, resolves a bare logical target found only through `WW_SRCLIB`,
|
||
places a same-named decoy under `WW_LIB`, records exact compiler, assembler, and
|
||
linker arguments, inspects `.unit.ww`, `.wwi`, `.a`, diagnostics, binaries, and
|
||
runtime results, and compares both stages. It separately exercises the runtime
|
||
object fallback and explicitly empty variables. The package-test observer copies
|
||
the toolchain source library to a selected root, marks the compiler-generated
|
||
test-support source, and proves that both the marker and the selected runtime
|
||
archive reach real same-package test builds through the coordinator.
|
||
|
||
The ownership boundary follows the pinned Go 1.26.5 implementation: source
|
||
headers supply imports in
|
||
[`go/build/read.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/read.go#272),
|
||
the loader canonicalizes and reuses packages in
|
||
[`cmd/go/internal/load/pkg.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#633),
|
||
tests remain real package variants in
|
||
[`cmd/go/internal/load/test.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#85),
|
||
and direct compile dependencies are expanded separately for linking in
|
||
[`cmd/go/internal/work/action.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#628).
|
||
|
||
### 11.10 Implemented persistent-workdir driver identity slice
|
||
|
||
The package driver is now an explicit content input to every persistent
|
||
`-w DIR` package action. Both stages copy the exact invoking executable to
|
||
`.wwtool.ww`, alongside `.wwtool.w6c`, `.wwtool.w6a`, and the mode/format
|
||
stamp. A warm invocation byte-compares all applicable live executables before
|
||
considering any committed unit reusable. A missing or changed driver copy
|
||
invalidates every `.unit.ww` voucher before compilation; old artifacts may
|
||
remain recoverable, but none can be reused without a freshly committed unit.
|
||
The workdir format revisions are 6 for ordinary builds and 7 for tests.
|
||
|
||
This closes a real hidden-input boundary. The driver, rather than `w6c`, owns
|
||
canonical directory interning, source-derived graph construction, owner-only
|
||
unit composition, direct-export argument construction, deterministic dependency
|
||
ordering, single-member package archive serialization, and the artifact commit
|
||
sequence. Unit equality alone
|
||
cannot identify changes to those algorithms, and a manually maintained format
|
||
number can be forgotten. Exact driver bytes conservatively cover them during
|
||
the transitional plain-file reuse scheme. This may rebuild after an unrelated
|
||
driver change, but it cannot falsely reuse a package after a relevant one.
|
||
|
||
The linker remains outside the recorded package identity because persistent
|
||
reuse never skips a final link: each invocation reconstructs the complete
|
||
reachable `.a` closure, selects current runtime inputs and link flags, and runs
|
||
the selected linker. Thus a linker or runtime change affects the requested
|
||
binary immediately without forcing unrelated package compilation.
|
||
|
||
Repository-native coverage runs a three-directory import graph through copied,
|
||
independently mutable Cstage and WWstage drivers. Exact compiler, assembler,
|
||
and linker wrappers prove a cold dependency-first build, an unchanged warm
|
||
compile/assemble skip with a deliberate relink, and full package invalidation
|
||
after only the invoking driver's bytes change. The test inspects owner-only
|
||
`.unit.ww` inputs, separate direct `.wwi` arguments, `.a`, identity files,
|
||
transitive link order, diagnostics before tool execution, published binary
|
||
bytes, runtime exit, and stage equivalence.
|
||
|
||
Go 1.26.5 draws the same semantic line with a richer cache: its build action ID
|
||
binds compiler/assembler tool identities, configuration, selected source
|
||
content, and direct dependency content IDs in
|
||
[`cmd/go/internal/work/exec.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#261),
|
||
while its link action ID separately binds linker configuration and the package
|
||
closure in
|
||
[`cmd/go/internal/work/exec.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#1504).
|
||
WW adopts only the correctness boundary in its existing inspectable `cmp`-based
|
||
workdir. It does not add build IDs, hashes, a CAS, an action graph, a scheduler,
|
||
or a manifest.
|
||
|
||
### 11.11 Implemented directory-only source-import slice
|
||
|
||
Cstage and WWstage now use a directory-only locator for every parsed source
|
||
import, including imports selected only by a package-test variant and the
|
||
compiler-generated test-support edge. The loader makes one ordered pass for
|
||
`<root>/<import-path>/`; it never probes `<root>/<import-path>.ww`. Unit
|
||
composition consequently writes only the owning package's byte-sorted source
|
||
files and never copies a dependency interface or imported source body. Missing
|
||
imports retain the importing source position and the same stable diagnostic in
|
||
both stages.
|
||
|
||
Root selection remains a separate compatibility boundary. A literal `.ww` CLI
|
||
target, or a bare CLI target found as `<root>/<name>.ww` after the global
|
||
directory search, can still create one raw single-file root. Its historical
|
||
inline package clauses may satisfy compiler-fixture bindings inside that raw
|
||
unit. Directory roots cannot use that exemption, and no filesystem source
|
||
import can reach it. This preserves low-level compiler fixtures without
|
||
weakening package-graph identity.
|
||
|
||
The self-hosted tools no longer depend on the removed behavior. `w6a/` and
|
||
`w6l/` are executable `package main` directories whose sorted source sets are
|
||
compiled once. The compiler backend is one `wcc/` directory package with a
|
||
narrow exported check/codegen façade; `w6c` and `wwdump` import that package
|
||
from its parent search root instead of importing its implementation files.
|
||
Legacy test fixtures were converted to directories, except for one intentional
|
||
compiler leaf-collision probe that now invokes `w6c` on an explicitly composed
|
||
raw unit. The Lisp example likewise imports a `lispcore/` directory package.
|
||
|
||
The focused native regression puts only `example/foo.ww` in an earlier import
|
||
root and a two-source `example/foo/` package in a later root. Both stages select
|
||
the directory, emit the exact sorted package-owned unit, consume its direct
|
||
dependency export, produce byte-identical deterministic `.wwi` and `.a`
|
||
artifacts, link and run a transitive archive closure, and repeat the resolution
|
||
through a real directory-package test. With only the file root present, both
|
||
stages reject the import with byte-identical package-attributed stderr. The
|
||
existing exact-tool observer remains the non-duplicated proof that each
|
||
canonical production action compiles once, compiler units contain only owned
|
||
sources, compile argument vectors contain exactly direct `.wwi` inputs, and
|
||
linker vectors contain the complete reachable `.a` closure and no `.wwi` path.
|
||
|
||
### 11.12 Implemented direct compiler-export input slice
|
||
|
||
Cstage and WWstage now share one small package-compiler convention:
|
||
`--import <canonical-import-path> <export.wwi>` may repeat before the one owning
|
||
source unit. It is valid only with `-c`; paths must be nonempty, strictly sorted,
|
||
and unique. Both compilers read and parse every export independently under the
|
||
supplied canonical identity before parsing the owner unit, then pass the merged
|
||
semantic declaration list through the existing checker, deterministic export
|
||
writer, and primary-only code generator. Missing export bytes therefore fail at
|
||
the compiler boundary as `w6c: import <path>: cannot read <file>`, followed by
|
||
the driver's stable owning-package attribution. No import configuration file,
|
||
manifest, schema, package database, or network lookup is involved.
|
||
|
||
Both drivers construct those arguments directly from the package node's sorted,
|
||
deduplicated outgoing edges. They never walk grandchildren for compilation.
|
||
Every `.unit.ww` contains only the node's byte-sorted source files and reset
|
||
separators, while executable linking independently walks the full reachable
|
||
package closure and passes archives, never interfaces. The same path handles an
|
||
ordinary package, the production-plus-internal-test variant, the external test
|
||
package and its reused production package, compiler-generated test main, and
|
||
the reserved test-support package. Persistent workdirs compare a newly emitted
|
||
export with its committed predecessor before allowing a direct importer to
|
||
reuse owner-identical artifacts, retaining correctness without a new cache
|
||
schema or identity record.
|
||
|
||
The exact-argv regression uses the real diamond
|
||
`base -> {left,right} -> root`. It proves one compile per node; no input for
|
||
`base`; only `base.wwi` for each middle node; only sorted `left.wwi` and
|
||
`right.wwi` for `root`; exact owner-only unit bytes; the complete four-package
|
||
link closure including the root archive; no link-time `.wwi`; exit status 42;
|
||
and byte-identical units, exports, archives, executables, and tool argument
|
||
vectors across two clean
|
||
Cstage builds and two clean WWstage builds. The existing directory-package
|
||
variant regression checks separate production, internal, external, and
|
||
generated-main actions, exact generated-main direct variant/support exports,
|
||
the external-production action, and archive-only link closures. Both stages
|
||
compile and run those actions with owner-only units and byte-identical
|
||
artifacts.
|
||
|
||
The pinned official Go 1.26.5 tag (commit
|
||
`c19862e5f8415b4f24b189d065ed739517c548ba`) supplies the design boundary:
|
||
|
||
- Directory identity starts with `ImportDir`'s named directory
|
||
([`go/build/build.go`, lines 521–524](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#521)); directory reads are name-sorted
|
||
([lines 108–111](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#108)), lookup selects directory candidates
|
||
([lines 725–809](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#725)), and the selected directory is enumerated
|
||
([lines 859–900](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#859)). The sorted iteration filters files and appends each source to exactly one ordinary/internal/external-test bucket
|
||
([lines 895–1036](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/build.go#895)).
|
||
- The loader records direct imports
|
||
([`load/pkg.go`, lines 220–225](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#220)), guarantees repeated cache loads return the same package pointer
|
||
([lines 633–636](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#633)), keys reuse by canonical import path
|
||
([lines 757–768](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#757)), and pointer-deduplicates dependency-first package lists
|
||
([lines 2776–2795](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#2776)).
|
||
- Build actions are interned by mode and package identity
|
||
([`work/action.go`, lines 437–447](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#437)); compilation depends only on direct imports
|
||
([lines 628–659](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#628)), and even package `main` receives that normal interned archive-producing compile action
|
||
([lines 641–647](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#641)). A link action places that main compile action at its first dependency and expands transitive dependencies
|
||
([lines 918–957](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#918),
|
||
[lines 1034–1068](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#1034)). The executor derives compiler import inputs from those direct actions
|
||
([`work/exec.go`, lines 864–884](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#864)), passes them separately from source files
|
||
([lines 928–930](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#928)), packs remaining objects into the package archive and records it as the built result
|
||
([lines 1017–1033](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#1017)), then supplies the root archive and all expanded package inputs to linking
|
||
([lines 1592–1624](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#1592),
|
||
[lines 1635–1647](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#1635)).
|
||
- Tests preserve four package roles
|
||
([`load/test.go`, lines 85–102](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#85)): internal production-plus-test
|
||
([lines 175–225](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#175)), external test
|
||
([lines 228–265](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#228)), and generated main
|
||
([lines 272–293](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#272)). Its support imports are loaded for that generated package
|
||
([lines 307–332](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#307)), with the selected internal/external variants attached as distinct direct imports at
|
||
[lines 351–373](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/test.go#351).
|
||
- Unified export writing re-links, re-exports, and prunes facts
|
||
([`noder/unified.go`, lines 152–165](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/unified.go#152)), finalizes self-contained data
|
||
([lines 463–470](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/unified.go#463)), selects export roots, and sorts declaration and body indices before serialization
|
||
([lines 495–570](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/unified.go#495)). Compiler import lookup opens the separately mapped artifact
|
||
([`noder/import.go`, lines 61–101](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/import.go#61)) and decodes its exports independently of source parsing
|
||
([lines 170–225](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/import.go#170)); `ReadPackage` consumes that package decoder
|
||
([`importer/ureader.go`, lines 28–62](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/importer/ureader.go#28)).
|
||
|
||
WW adopts those practical ownership and action semantics while retaining its
|
||
small direct CLI representation and existing self-contained `.wwi` encoding.
|
||
|
||
## 12. Candidate architectures and hard-gate decision
|
||
|
||
Five candidates were developed as coherent systems, not as feature bins.
|
||
|
||
### 12.1 Candidate A: Go-like integrated language command
|
||
|
||
One `ww` command would infer directory packages/imports, compile/cache/test them,
|
||
and add a small module/lock layer. Native inputs would remain compiler flags or
|
||
toolchain conventions. This preserves the strongest part of Go: explicit imports,
|
||
fast direct export data, and a short ordinary command
|
||
([Go command design](https://go.dev/doc/articles/go_command)). A WW-specific lock
|
||
and no-network build could improve on modern Go module behavior.
|
||
|
||
It still fails as an end-to-end native design. A source-only graph cannot name
|
||
host generators, C header trees, archive order, linker scripts, CRT, SDK, or
|
||
sysroot. Ambient compiler-driver and `pkg-config` behavior would remain outside
|
||
the key, and the build/host/target triad would be incomplete. Adding typed native
|
||
actions, complete toolchains, and content records turns it into Candidate E.
|
||
|
||
### 12.2 Candidate B: Hare/Odin-style local source plus an outer build tool
|
||
|
||
WW would use search roots and directory modules, with source vendored or supplied
|
||
by an OS package manager; Make-like orchestration would own native work. This is
|
||
small locally and avoids a language-owned network resolver. Hare 0.26.0 (released
|
||
2026-02-13) is a useful reference: directory modules, direct textual export data,
|
||
`HAREPATH`, explicit system-library flags, and cross-architecture tooling are
|
||
documented in its official manuals
|
||
([modules](https://harelang.org/documentation/usage/modules.html),
|
||
[project structure](https://harelang.org/documentation/usage/project-structure.html),
|
||
[system libraries](https://harelang.org/documentation/usage/system-libraries.html),
|
||
[cross compilation](https://harelang.org/documentation/usage/cross.html)). Odin's
|
||
named collections are a related local-source convention
|
||
([Odin overview](https://odin-lang.org/docs/overview/)).
|
||
|
||
As a complete WW system it fails: ordered search roots are selection policy
|
||
without locked source identity; ordinary outer recipes expose ambient tools,
|
||
environment, and mtimes; language and native graphs/caches cannot jointly explain
|
||
invalidation; and cross sysroots/ABI providers remain project conventions. Making
|
||
the outer tool hermetic and content-addressed yields Candidate C, not this model.
|
||
|
||
### 12.3 Candidate C: two-layer Plan 9-style builder and orchestrator
|
||
|
||
A strict package builder would compile an import graph. A separate small
|
||
declarative DAG tool would own generators, C/assembly, images, archives, and
|
||
links. With content records, pinned tools, sandboxing, and an exact handoff this
|
||
can pass every hard gate. It reflects `mk`'s valuable complete-graph/parallel-tool
|
||
shape without copying its mtime and ambient-environment assumptions.
|
||
|
||
It loses after the gates because the boundary creates two graph protocols, two
|
||
selection UIs, two explanation namespaces, and either duplicate scheduling/cache
|
||
logic or a coarse “build all packages” action. Ordinary native projects must know
|
||
when to invoke each layer. If both front ends lower into one shared scheduler and
|
||
cache, and `ww` owns the ordinary invocation, the result is the smaller Candidate
|
||
E. Keeping the second production tool after that offers no remaining orthogonal
|
||
concept.
|
||
|
||
### 12.4 Candidate D: Zig/Cargo-like programmable integrated project
|
||
|
||
A manifest would define artifacts and dependencies while a host-executed program
|
||
constructs a flexible native graph. This handles more native cases than a
|
||
language-only command. Zig 0.16.0, pinned here to its 2026-04-13 release metadata,
|
||
provides explicit target/native concepts, hashed package sources, local
|
||
dependency overrides, and generated-file graph edges
|
||
([download metadata](https://ziglang.org/download/index.json),
|
||
[0.16.0 reference](https://ziglang.org/documentation/0.16.0/),
|
||
[build system](https://ziglang.org/learn/build-system/)). Cargo 1.97.1, shipped
|
||
with Rust 1.97.1 on 2026-07-16, provides exact package IDs, workspaces, lock
|
||
checksums, resolver rules, and native `links` collision handling
|
||
([Cargo reference](https://doc.rust-lang.org/1.97.1/cargo/reference/),
|
||
[resolver](https://doc.rust-lang.org/1.97.1/cargo/reference/resolver.html)).
|
||
|
||
Zig's build program and Cargo's `build.rs` execute to decide or report build
|
||
behavior; Cargo explicitly
|
||
documents build-script inputs/outputs and its fingerprint cache
|
||
([build scripts](https://doc.rust-lang.org/1.97.1/cargo/reference/build-scripts.html),
|
||
[build cache](https://doc.rust-lang.org/1.97.1/cargo/reference/build-cache.html)). Those
|
||
reference systems as shipped do not meet WW's complete native/toolchain hard
|
||
gates.
|
||
|
||
The strongest coherent D is not left as a straw man: it content-identifies the
|
||
graph program and host compiler/runtime, declares its whole readable source/tool
|
||
closure, runs it in the denied-by-default sandbox with no network, and requires
|
||
it to emit a closed typed graph before artifact execution. That hardened model
|
||
can pass every gate. It still loses afterward: WW must permanently ship/secure/
|
||
bootstrap an evaluator API and host build-program toolchain, users debug both
|
||
program execution and its emitted graph, dependencies expose framework APIs, and
|
||
routine exceptions accumulate as library features. Finite records buy the same
|
||
WW requirements with less user and implementation machinery.
|
||
|
||
### 12.5 Candidate E: hermetic integrated action build — selected
|
||
|
||
Candidate E retains the Go-like ordinary UX and import-derived language graph,
|
||
then adds only the native/action facts that imports cannot express. Both lower
|
||
to one typed graph and content cache. It borrows declared tools/inputs and action
|
||
results from Bazel's hermetic/remote-execution model, and transparent
|
||
content-derived build records from Nix derivations, without adopting either
|
||
framework, evaluator, daemon topology, or user interface
|
||
([Bazel hermeticity](https://bazel.build/versions/9.2.0/basics/hermeticity),
|
||
[Bazel remote caching](https://bazel.build/remote/caching),
|
||
[remote execution protocol](https://github.com/bazelbuild/remote-apis/blob/master/build/bazel/remote/execution/v2/remote_execution.proto),
|
||
[Nix derivations](https://nix.dev/manual/nix/2.35/store/derivation/)).
|
||
|
||
It passes every hard gate and is selected. Its concepts are exactly package,
|
||
module/source selection, product/action, toolchain/target, artifact/digest, and
|
||
native provider/link plan. There is one graph, one scheduler, one cache key, one
|
||
explanation path, and one ordinary command.
|
||
|
||
### 12.6 Hard-gate matrix
|
||
|
||
Legend: **pass** means the strongest coherent form has a credible end-to-end
|
||
invariant; **fail** means it does not. D denotes the hardened evaluator above,
|
||
not unmodified Zig/Cargo behavior.
|
||
|
||
| Hard gate | A: Go-like | B: local + outer | C: two layer | D: programmable | E: action build |
|
||
|---|---|---|---|---|---|
|
||
| complete inspectable dependency DAG | fail for native work | fail across tools | pass | pass after sandboxed evaluation | **pass** |
|
||
| loud cycles/identity collisions | pass for packages; native incomplete | search shadowing fails identity | pass | pass | **pass** |
|
||
| frozen offline source closure | pass with proposed lock | vendoring can pass | pass | pass | **pass** |
|
||
| correct cache invalidation | fail for native/tool defaults | fail across mtime/outer recipes | pass | pass | **pass** |
|
||
| compiler/linker/runtime/toolchain identity | requires the E native layer | fail | pass | pass | **pass** |
|
||
| explicit B/H/T | fail | fail | pass | pass | **pass** |
|
||
| correct native dependency/link modeling | fail | fail | pass | pass | **pass** |
|
||
| deterministic package selection | pass | fail under search roots | pass | pass with lock | **pass** |
|
||
| reproducible artifacts/boundary | fail for native closure | fail | pass | pass with evaluator certification | **pass** |
|
||
| bounded bootstrap | pass with a seed | pass with enumerated seed/tools | pass | pass, larger evaluator TCB | **pass** |
|
||
| explain cache miss/rebuild | fail | fail | pass across two namespaces | pass across evaluation + graph | **pass** |
|
||
| one simple ordinary path | pass | two commands/configurations | two production layers | pass by convention | **pass** |
|
||
| no build/test network or mutation | pass if changed from Go defaults | outer recipes cannot guarantee | pass | pass | **pass** |
|
||
|
||
C, hardened D, and E survive the gates. E wins the requested post-gate
|
||
comparison:
|
||
|
||
| Criterion | C: two layer | D: hardened evaluator | E: selected |
|
||
|---|---|---|---|
|
||
| conceptual simplicity | two graph/handoff models | evaluator language/API plus emitted graph | one graph protocol; two finite descriptions |
|
||
| daily usability | user chooses builder/orchestrator | ordinary defaults can hide evaluator, exceptions cannot | `ww build/test/run` always |
|
||
| clean speed | parallel orchestration can match | evaluation overhead, then parallel graph | direct complete template plus parallel actions |
|
||
| incremental speed | cross-tool handoff may be coarse | evaluator must rerun/cache its own dependencies | direct exports, lazy content keys, cached link |
|
||
| rebuild blast radius | good only with API digests across handoff | can be good after evaluation | public-output digest propagation is intrinsic |
|
||
| cross-compilation | orchestrator can model it | rich API can model it | roles are built into every typed record |
|
||
| FFI/native honesty | outer layer owns it separately | API can express it | same provider/link records as package objects |
|
||
| implementation maintenance | two adapters/engines or coarse boundary | evaluator, SDK compatibility, sandbox, graph engine | one scheduler/store/adapter set |
|
||
| supply-chain security | two policy surfaces | dependency host code remains an evaluator input | one lock/source/action trust model |
|
||
| ecosystem scalability | two tool ecosystems | framework/API accumulation pressure | closed schemas version only for demonstrated facts |
|
||
| debugging/observability | two explanation namespaces | debug program, evaluator cache, then graph | one causal graph/record diff |
|
||
| bootstrap longevity | small outer interpreter possible | compiler/runtime/evaluator enter seed chain | fixed seed plan; no production evaluator |
|
||
|
||
### 12.7 Why each subsystem is in or out of the core
|
||
|
||
Package parsing/resolution and export data belong in the core because only the
|
||
compiler can state the true language graph and public ABI. Module selection and
|
||
lock verification belong beside it because an import identity must resolve to
|
||
one deterministic source before compilation. The typed action engine, target
|
||
descriptors, native providers, and toolchain closure belong in the core because
|
||
they share invalidation and link correctness with package objects. Test/doc/
|
||
install are thin product selections/materializations over that same graph.
|
||
|
||
Network transport remains a separate operation, though exposed by `ww`, because
|
||
fetching is not building. Credential policy, OS package installation, registry
|
||
hosting, signing authority, remote execution, deployment, and general release
|
||
automation stay outside. The finite `generate` action is the boundary: it lets
|
||
outer domain tools transform declared artifacts without turning WW into their
|
||
framework.
|
||
|
||
## 13. Migration plan
|
||
|
||
Migration cost does not affect the decision. It is nevertheless material: the
|
||
current directly implicated scaffolding is at least 7,969 lines across the
|
||
Makefile, two drivers, two interface writers, package coordinator, and test
|
||
wrapper, plus 4,728 lines in the focused separate-build/package/byte-identity/
|
||
driver tests counted for this audit. Compiler export/import logic, new native
|
||
adapters, and bootstrap work add new scope not represented by those deletion
|
||
counts.
|
||
|
||
The planning estimate is **15–22 engineer-months** for the first production
|
||
Linux/amd64 toolchain, including tests, migration, and deletion, plus **1–2
|
||
engineer-months per materially different additional official target/sysroot**.
|
||
This is an estimate for staffing and sequencing, not a reason to retain a weaker
|
||
architecture.
|
||
|
||
Every phase below ends in a bisect-clean commit. Experimental components are not
|
||
installed as a second user-facing build path. Until the point of no return, the
|
||
old command remains the only production path; after it, the new command is the
|
||
only path.
|
||
|
||
### Phase 0 — conformance corpus and protocol freeze
|
||
|
||
- Land the compact checked-in `protocol/schema/` modules for WWAR framing,
|
||
record/enum/union/default/kind assignments, wrappers, and finite digest byte
|
||
formulas. Freeze the exact schema-file digests. Preserve representation-only
|
||
preimage records without embedding the algorithms that construct them.
|
||
- Generate data-only codec tables deterministically. Keep one schema-aware
|
||
reference codec and a second independent oracle limited to raw WWAR framing,
|
||
hashing, and record identity; neither may implement future compiler or build
|
||
behavior.
|
||
- Gate: strict duplicate-key UTF-8/NFC schema loading, byte-identical generation
|
||
in separate fresh directories, compact valid/invalid vectors, all assignment
|
||
coverage, stable malformed-length precedence, domain separation, action-key
|
||
vectors, and record-kind substitution rejection. No production behavior
|
||
changes beyond adding this conformance gate.
|
||
- Package/API, graph/cache, native B/H/T, and bootstrap measurements remain useful
|
||
fixtures, but become tests in Phases 1, 2, 4, and 6 respectively. They are not
|
||
Phase 0 semantic answer tables.
|
||
|
||
### Phase 1 — compiler export and package protocol
|
||
|
||
- Implement deterministic `.wwe`/`.wwlm` writing and direct-interface reading in
|
||
Cstage and self-hosted compiler paths behind test-only entry points.
|
||
- Add package identity/alias syntax, strict directory enumeration, compiler
|
||
import extraction, target suffix selection, cycle/collision/internal checks,
|
||
and protocol compatibility diagnostics.
|
||
- Construct and test the deep public type closure and public type/ABI digest
|
||
preimages in compiler code with existing standard-library graphs. Reject
|
||
`.wwi` input in the experimental path; do not translate it.
|
||
- Gate: Cstage/WWstage emit byte-identical vectors, importers open only direct
|
||
export files, and API propagation stops on unchanged middle exports.
|
||
|
||
### Phase 2 — one action engine and local CAS
|
||
|
||
- Integrate the frozen WWAR codec into production code and implement typed pure
|
||
action-record/key functions, the action graph, lazy keys, scheduler, atomic
|
||
CAS/results, project index, corruption quarantine, graph JSON, environment and
|
||
sandbox policy, deterministic failure behavior, and causal explanation.
|
||
- Add deterministic package/archive/link adapters using the existing compiler,
|
||
assembler, and linker as explicitly hashed tools. This is a temporary adapter,
|
||
not a compatibility promise.
|
||
- Keep the engine under an internal test binary; the installed `ww` still follows
|
||
the old production path.
|
||
- Gate: all action-input mutation and failure-injection tests pass; identical warm
|
||
builds execute no compiler, assembler, archiver, or linker.
|
||
|
||
### Phase 3 — module, lock, source, and workspace layer
|
||
|
||
- Implement the manifest, lock, work, and vendor text parsers as ordinary typed
|
||
parser code, then implement the closed grammar, monotonic selector, canonical
|
||
lock,
|
||
HTTPS source-index/archive protocol, immutable source store, signatures,
|
||
explicit add/update/lock/fetch, overlays, vendor index, and canonical
|
||
source-tree construction. The schema fixes only the resulting record bytes and
|
||
source-tree digest formula.
|
||
- Build/test/doc/install remain network-denied from their first experimental use.
|
||
- Gate: frozen offline builds work from project source + complete locked source
|
||
closure + lock + installed named toolchain; collision/downgrade/hash/
|
||
path-normalization attacks fail loudly.
|
||
|
||
### Phase 4 — native, target, and external toolchain closure
|
||
|
||
- Implement full target descriptors, B/H/T lowering, C/assembly/native-provider
|
||
records, recursive provider selection/expansion, generated actions/sandbox,
|
||
object sidecars, exact link-plan construction, sysroot,
|
||
libc/CRT/loader/SDK/runtime, shared-library installation, and freestanding
|
||
products.
|
||
- Package supported assembler/linker/archive/C tools as immutable external
|
||
closures. Implement an external-assembler-compatible textual emission backend,
|
||
then have the pinned assembler produce `package.o` inside `ww.package`; port all WW/Plan-9-dialect
|
||
runtime and user assembly to a declared supported external dialect, and pass
|
||
object/link parity. An argv adapter alone cannot consume current `w6a` syntax.
|
||
Stop relying on WW-owned `w6a`/`w6l` before the experimental gate passes.
|
||
- Gate: native conflict, archive group/order, linker-script include, host leak,
|
||
freestanding, shared loader, and at least one real cross-target suite pass.
|
||
|
||
### Phase 5 — repository and consumer conversion rehearsal
|
||
|
||
- Define the final identity/layout conversion in a one-shot checker/rewriter under
|
||
`tools/migrate-build/`: folded files become directory packages; aliases,
|
||
manifests, native providers, products, generated inputs, and locks are emitted.
|
||
It is not an import resolver or runtime compatibility layer.
|
||
- Until cutover, CI applies that tool to a fresh ignored shadow tree. Tracked
|
||
production source stays in old syntax, so the old command remains its only
|
||
production path; the shadow is regenerated, never a second maintained source
|
||
tree or shipped interface.
|
||
- Dogfood the internal engine on the converted shadow of the standard library,
|
||
compiler tools, tests, examples, install layout, CI/package inputs, and sample
|
||
downstream consumers. Compare semantics, diagnostics, performance, artifacts
|
||
where protocols permit, and complete graph explanations.
|
||
- Gate: the regenerated full shadow, distribution/frozen-offline build,
|
||
self-contained installed toolchain outside the build tree, downstream samples,
|
||
and bootstrap inputs pass without an old-format edge.
|
||
|
||
### Phase 6 — new bootstrap and reproducible release
|
||
|
||
- Implement/gate the portable C recovery backend and snapshot generator, then
|
||
generate/check in `bootstrap/ww0.c` and the fixed plan. Produce stages 1/2/3/4,
|
||
rebuild stages with the executable engine, compare the actual bytes of every
|
||
declared output directly, and produce fixed-point and diverse-seed-compilation
|
||
reports, signed toolchain bundles, and recovery documentation on a clean
|
||
machine with no WW compiler.
|
||
- Gate: stage 2 equals stage 3 and stage 3 equals stage 4 by explicit
|
||
byte-for-byte comparison, including raw action/result records, in two
|
||
roots/concurrency levels; project plus complete
|
||
locked source closure and published named tool closure reproduce every release
|
||
artifact.
|
||
|
||
### Phase 7 — point of no return and deletion
|
||
|
||
In one atomic, bisect-clean cutover commit:
|
||
|
||
1. install the new engine as `ww` and make it the sole build/test/bootstrap path;
|
||
2. switch repository imports, manifests, locks, toolchains, CI, installation,
|
||
and release jobs to their final forms;
|
||
3. delete both old drivers, both `.wwi` writers, driver-side `.wwi` concatenation,
|
||
composed-unit/module-wrapper logic, generic C/self-hosted lexer/parser/checker/
|
||
symbol/codegen support for module directives, `-w`,
|
||
old import search/file folding, separate test coordinator,
|
||
duplicated production Make dependency graph, current Cstage bootstrap, and
|
||
retired owned assembler/linker path; and
|
||
4. delete the migration rewriter after all supported consumers have used its
|
||
released standalone copy; keep only a format-error guide.
|
||
|
||
The commit does not accept old `.wwi`, dotted/file imports, `-I`, `-w`, raw
|
||
library searches, or old work directories. No alias, warning period inside the
|
||
compiler, environment switch, or fallback subprocess retains a dual system.
|
||
|
||
### Phase 8 — consolidation
|
||
|
||
- Remove temporary parity fixtures that test deleted artifact bytes while
|
||
retaining semantic, action-key, reproducibility, and bootstrap regression
|
||
tests.
|
||
- Publish migration statistics and archive the old documentation as historical
|
||
release material outside the live manual.
|
||
- Gate: repository search and executable tracing show one graph constructor, one
|
||
package resolver, one cache, one test route, and one bootstrap route.
|
||
|
||
## 14. Validation plan and release gates
|
||
|
||
### 14.1 Unit and format tests
|
||
|
||
- Phase 0 has golden and adversarial vectors only for WWAR representation,
|
||
source-tree digest bytes, action-key/record identity formulas, typed record
|
||
assignments, wrappers, and canonical schema JSON.
|
||
- Phase 0 tests UTF-8/NFC, duplicate schema keys, unknown wire type/schema field,
|
||
oversized declarations, truncation, exact-length mismatch, union shape, and
|
||
record-kind substitution. Case-fold collision, traversal, symlink/device,
|
||
cache collision/corruption, and semantic record tests land with their owning
|
||
executable phases.
|
||
- Phases 1–4 add behavior tests for CAS tree/result objects, `.wwe`, `.wwlm`,
|
||
manifest/lock/work/vendor parsing, and target/toolchain/native processing;
|
||
these are not encoded as Phase 0 vector outcomes.
|
||
- Resolver vectors for minimum selection, incompatible-major identities,
|
||
workspace identity preservation, source origin independence, vendor matching,
|
||
internal packages, aliases, nested module/root-versus-parent-subpackage
|
||
identity collisions (including `/vN`), cycles, and target source specificity.
|
||
|
||
### 14.2 Package/interface tests
|
||
|
||
- One- and multi-file directory membership, file-scoped import use, same/external
|
||
tests, test-only packages, examples/docs, and generated fragments with import/
|
||
package rejection.
|
||
- Direct dependency interface-open counts equal package-graph indegree, never
|
||
transitive closure size.
|
||
- Private dependency edits preserve importer keys; exported but unused additions
|
||
rebuild direct importers; unchanged middle `.wwe` stops propagation; layout,
|
||
calling-convention, runtime ABI, compiler, target, and profile changes rebuild
|
||
the exact affected cone.
|
||
- Parameter renames, comments, private declaration ordering, and absolute source
|
||
paths do not change `.wwe`; semantic/ABI changes do.
|
||
|
||
### 14.3 Native and link tests
|
||
|
||
- C scalar/aggregate/variadic/callback/TLS/unwind ABI probes against an
|
||
independently compiled C harness for every official target.
|
||
- Binding generation changes on header tree, macro map, preprocessor, tool, C
|
||
ABI, and target; undeclared include access is denied.
|
||
- Assembly dialect/CPU mismatch, wrong-format objects, PIC/shared rules, archive
|
||
extraction order, repeated libraries, groups, whole archive, weak/strong
|
||
symbols, version scripts, linker-script includes, and deterministic archives.
|
||
- Exactly-one libc/runtime/provider enforcement; identical coalescing; duplicate
|
||
native ABI/symbol conflict; shared SONAME/loader/runtime installation closure;
|
||
freestanding entry/script with proof that no libc/CRT/loader appears.
|
||
|
||
### 14.4 Cross and sandbox tests
|
||
|
||
- Matrix with `B != H`, compiler product `H != T`, and all three distinct where
|
||
infrastructure permits. A B generator emits an H input; an H binary is never
|
||
executed during build; T objects never enter the H link.
|
||
- Poison host `PATH`, includes, libraries, SDK, locale, time, home, current
|
||
directory, and environment. Every attempted undeclared read/write/network/
|
||
process/tool access fails with its action identity.
|
||
- Cross tests build without a runner, run only with an exact declared runner,
|
||
and fail under `--require-run` when none exists.
|
||
|
||
### 14.5 Cache and failure injection
|
||
|
||
- Change each field in section 6.2 individually and require a key change; change
|
||
each expressly non-semantic observation and require no key change.
|
||
- Bit-flip blobs, trees, results, action mappings, tools, export data, objects,
|
||
and partial files at every publication boundary. Require quarantine/rebuild,
|
||
never acceptance or broad deletion.
|
||
- Concurrent identical publishers, killed compiler/linker, disk full, rename
|
||
failure, read-only output, interrupted materialization, stale project index,
|
||
malicious remote mapping, bad cache signature, and remote outage.
|
||
- Clean, local-hit, explicit remote-import, and no-cache builds must yield the
|
||
same result digests. Test executions still run.
|
||
|
||
### 14.6 Reproducibility and bootstrap
|
||
|
||
- Compare every artifact/action record across two absolute checkouts, source/
|
||
cache/output roots, usernames, locales, time zones, umasks, concurrency levels,
|
||
filesystem enumeration orders, and cold/warm caches.
|
||
- Verify debug/release, static/shared, hosted/freestanding, generated/native, and
|
||
signed/unsigned products. Impure profiles must state exactly why they are
|
||
outside the byte promise and must never enter shared cache.
|
||
- Build `ww0` with each supported host C toolchain, reach stages 2/3/4, run the
|
||
semantic then raw-record fixed point and diverse seed compilation, corrupt each stage input, and recover on a host
|
||
with no WW installation.
|
||
|
||
### 14.7 Performance and migration gates
|
||
|
||
On the audit's fixed eight-CPU reference host, the first release MUST:
|
||
|
||
- run package compilation in parallel and complete the full clean toolchain
|
||
build no slower than the measured 15.515 s `make -j8` baseline;
|
||
- perform a warm 15-package build with no compiler, assembler, archiver, or
|
||
linker process and no slower than the measured 0.053 s driver baseline;
|
||
- read only direct exports and avoid composed-unit duplication;
|
||
- store one CAS copy of duplicate tool/package content across all products;
|
||
- produce fully path-independent official artifacts, including host-side tools;
|
||
and
|
||
- provide a typed explanation for every deliberately induced rebuild.
|
||
|
||
Before cutover, every tracked current package/test/install/bootstrap consumer has
|
||
an assigned new identity and a passing converted test. The cutover gate includes
|
||
a repository-wide search for old forms and executable traces proving no old
|
||
driver, interface, workdir, library search, or test-coordinator path executes.
|
||
|
||
## 15. Evidence appendix
|
||
|
||
### 15.1 Research method and version pins
|
||
|
||
Research used official documentation, standards/manuals, release metadata, and
|
||
current upstream source—not comparison articles or community summaries. The
|
||
evolving-system snapshot was taken 2026-08-09:
|
||
|
||
| System | Pinned snapshot used |
|
||
|---|---|
|
||
| Plan 9 | live official `9p.io` Volume 2 documents, accessed 2026-08-09; pages are not versioned/dated editions |
|
||
| Go | online docs accessed 2026-08-09; Go 1.26.5 `go1.26.5` source tag |
|
||
| Hare | online docs accessed 2026-08-09; Hare 0.26.0 source, released 2026-02-13 |
|
||
| Odin | online docs accessed 2026-08-09; `dev-2026-07a`, commit `819fdc7a80667498b8b365999f1475a66c358640` |
|
||
| Zig | Zig 0.16.0, official metadata release date 2026-04-13; source archive SHA-256 `43186959edc87d5c7a1be7b7d2a25efffd22ce5807c7af99067f86f99641bfdf` |
|
||
| Rust/Cargo | Rust/Cargo 1.97.1, released 2026-07-16; Cargo `0.98.0` commit `c980f4866141969fab6254a680546a277789d6f0` |
|
||
| Bazel | Bazel 9.2.0 documentation/source |
|
||
| Nix | Nix 2.35.2 manual/source |
|
||
|
||
Zig's separate bootstrap-source archive inspected for this decision had SHA-256
|
||
`2a8266a4205772ef40838c8cbdf14875855a515ff3adf89b49c2d2ae93613d10`.
|
||
These pins matter because programmable-build and package behavior changes between
|
||
releases; this document does not generalize an old Zig/Cargo observation to an
|
||
unidentified current version.
|
||
|
||
### 15.2 Primary-source findings
|
||
|
||
**Pike, Plan 9, and early Go.** Pike's sources support explicit computable
|
||
imports, cycle rejection, direct compiled export information, fast compilation,
|
||
and orthogonal concepts. The collective Plan 9 papers add system-wide placement
|
||
of complexity and transparent encodings. Plan 9's namespace papers demonstrate
|
||
contextual filesystem composition; this document infers that contextual location
|
||
must not serve as WW's versioned distribution identity. `mk` shows complete graph scheduling while still relying
|
||
on timestamps, recipes, and environment. The architecture borrows the former
|
||
principles and replaces the latter ambient assumptions.
|
||
|
||
- [The Go Programming Language, 2009](https://go.dev/talks/2009/go_talk-20091030.pdf)
|
||
- [Go at Google: Language Design in the Service of Software Engineering, 2012](https://go.dev/talks/2012/splash.article)
|
||
- [Simplicity is Complicated, 2015](https://go.dev/talks/2015/simplicity-is-complicated.slide)
|
||
- [Go in Go, 2015](https://go.dev/talks/2015/gogo.slide)
|
||
- [Plan 9 overview](https://9p.io/sys/doc/9.html)
|
||
- [The Use of Name Spaces in Plan 9](https://9p.io/sys/doc/names.html)
|
||
- [Maintaining Files on Plan 9 with Mk](https://9p.io/sys/doc/mk.html)
|
||
- [Plan 9 Mkfiles](https://9p.io/sys/doc/mkfiles.html)
|
||
- [Plan 9 compiler suite](https://9p.io/sys/doc/comp.html)
|
||
|
||
**Later Go.** The original Go command demonstrates source-derived package DAGs,
|
||
directory conventions, and compiler-owned dependency work. The modern module
|
||
reference documents module identity, Minimal Version Selection, major-version
|
||
paths, checksums, and commands that may resolve/download modules. Current command
|
||
and source-install documents also separate build cache/toolchain/bootstrap
|
||
behavior. These are evidence, not automatic WW defaults; in particular WW uses
|
||
an exact lock and forbids implicit build-time acquisition.
|
||
|
||
- [About the Go command](https://go.dev/doc/articles/go_command)
|
||
- [Go module reference](https://go.dev/ref/mod)
|
||
- [`go` command reference](https://go.dev/cmd/go/)
|
||
- [Go toolchain selection](https://go.dev/doc/toolchain)
|
||
- [Installing Go from source](https://go.dev/doc/install/source)
|
||
- [Perfectly Reproducible, Verified Go Toolchains](https://go.dev/blog/rebuild)
|
||
- [Go's supply-chain security](https://go.dev/blog/supply-chain)
|
||
- [`go1.26.5` source](https://go.googlesource.com/go/+/refs/tags/go1.26.5/)
|
||
|
||
**Local-source systems.** Hare demonstrates how far a disciplined directory
|
||
module/search-root system can go with little package machinery; its documented
|
||
system-library and cross interfaces also expose why raw host paths and tool
|
||
defaults are insufficient for WW's hard gates. Odin's collections reinforce the
|
||
local namespace option but do not add a locked whole native closure.
|
||
|
||
- [Hare modules](https://harelang.org/documentation/usage/modules.html)
|
||
- [Hare project structure](https://harelang.org/documentation/usage/project-structure.html)
|
||
- [Hare system libraries](https://harelang.org/documentation/usage/system-libraries.html)
|
||
- [Hare cross compilation](https://harelang.org/documentation/usage/cross.html)
|
||
- [Hare 0.26.0 source](https://git.sr.ht/~sircmpwn/hare/tree/0.26.0)
|
||
- [Odin overview](https://odin-lang.org/docs/overview/)
|
||
- [Odin pinned source](https://github.com/odin-lang/Odin/tree/819fdc7a80667498b8b365999f1475a66c358640)
|
||
|
||
**Integrated project systems.** Zig supplies useful target/native vocabulary,
|
||
source hashes, local aliases, declared generated-file edges, and cross-building.
|
||
Cargo supplies package IDs, exact lock checksums, workspace behavior, resolver
|
||
documentation, and a native `links` uniqueness rule. Their programmable build
|
||
program/script model is deliberately rejected; a dependency host program is a
|
||
larger and less inspectable abstraction than WW's finite action record.
|
||
|
||
- [Zig build system](https://ziglang.org/learn/build-system/)
|
||
- [Zig overview](https://ziglang.org/learn/overview/)
|
||
- [Zig 0.16.0 language reference](https://ziglang.org/documentation/0.16.0/)
|
||
- [Zig release metadata](https://ziglang.org/download/index.json)
|
||
- [Zig 0.16.0 source](https://github.com/ziglang/zig/tree/0.16.0)
|
||
- [Cargo 1.97.1 reference](https://doc.rust-lang.org/1.97.1/cargo/reference/)
|
||
- [Cargo current reference entry](https://doc.rust-lang.org/cargo/reference/)
|
||
- [Cargo 1.97.1 resolver](https://doc.rust-lang.org/1.97.1/cargo/reference/resolver.html)
|
||
- [Cargo current resolver entry](https://doc.rust-lang.org/cargo/reference/resolver.html)
|
||
- [Cargo 1.97.1 build scripts](https://doc.rust-lang.org/1.97.1/cargo/reference/build-scripts.html)
|
||
- [Cargo current build-script entry](https://doc.rust-lang.org/cargo/reference/build-scripts.html)
|
||
- [Cargo 1.97.1 build cache](https://doc.rust-lang.org/1.97.1/cargo/reference/build-cache.html)
|
||
- [Cargo current build-cache entry](https://doc.rust-lang.org/cargo/reference/build-cache.html)
|
||
- [Cargo pinned source](https://github.com/rust-lang/cargo/tree/c980f4866141969fab6254a680546a277789d6f0)
|
||
|
||
**Hermetic invariants.** Bazel defines useful distinctions between declared
|
||
actions, execution platforms, action results, and content-addressed remote data.
|
||
Nix derivations demonstrate an inspectable build record whose output depends on
|
||
declared inputs. WW borrows those invariants only. It rejects Bazel's general
|
||
rule/ecosystem machinery and Nix's evaluator/store-as-package-manager as excess
|
||
for one language toolchain.
|
||
|
||
- [Bazel current hermeticity entry](https://bazel.build/concepts/hermeticity)
|
||
- [Bazel 9.2.0 hermeticity](https://bazel.build/versions/9.2.0/basics/hermeticity)
|
||
- [Bazel remote caching](https://bazel.build/remote/caching)
|
||
- [Remote Execution API](https://github.com/bazelbuild/remote-apis/blob/master/build/bazel/remote/execution/v2/remote_execution.proto)
|
||
- [Bazel 9.2.0 source](https://github.com/bazelbuild/bazel/tree/9.2.0)
|
||
- [Nix current derivation entry](https://nix.dev/manual/nix/latest/store/derivation/)
|
||
- [Nix 2.35 derivations](https://nix.dev/manual/nix/2.35/store/derivation/)
|
||
- [Nix build process](https://nix.dev/manual/nix/2.35/store/building.html)
|
||
- [Nix 2.35.2 source](https://github.com/NixOS/nix/tree/2.35.2)
|
||
|
||
**Native and reproducible toolchains.** LLVM and Clang define data-layout and
|
||
cross/toolchain choices that must be explicit for ABI-correct code. GNU manuals
|
||
document build/host/target vocabulary and semantic linker/archive/script
|
||
behavior. The reproducible-builds definition supplies the correct boundary:
|
||
same source, environment, instructions, and dependencies—not merely “same
|
||
compiler source.”
|
||
|
||
- [LLVM language-reference data layout](https://llvm.org/docs/LangRef.html#data-layout)
|
||
- [Clang cross compilation](https://clang.llvm.org/docs/CrossCompilation.html)
|
||
- [Clang toolchain](https://clang.llvm.org/docs/Toolchain.html)
|
||
- [Autoconf target triplets](https://www.gnu.org/software/autoconf/manual/autoconf-2.71/html_node/Specifying-Target-Triplets.html)
|
||
- [GCC language standards/runtime implications](https://gcc.gnu.org/onlinedocs/gcc/Standards.html)
|
||
- [GCC link options](https://gcc.gnu.org/onlinedocs/gcc/Link-Options.html)
|
||
- [GNU linker](https://sourceware.org/binutils/docs/ld.html) and [linker scripts](https://sourceware.org/binutils/docs/ld/Scripts.html)
|
||
- [Reproducible Builds definition](https://reproducible-builds.org/docs/definition/)
|
||
|
||
### 15.3 Repository and empirical evidence record
|
||
|
||
The source audit covered the Makefile; C and self-hosted drivers; compiler entry
|
||
and both interface writers; package/test coordinator; language and test specs;
|
||
bootstrap documents/recipes; and package, separate-compilation, incremental,
|
||
byte-identity, and bootstrap tests. The audit found documentation drift as well
|
||
as code defects: the root instructions count five unit suites while Make lists
|
||
six, and the test-system document describes three pinned data byte divergences
|
||
while the current Make divergence list is empty. Neither drift item influences
|
||
the architecture, but both require cleanup during migration.
|
||
|
||
Raw commands, timings, traces, hashes, fixture logs, environment capture, and
|
||
interpretation for section 11.4 are retained in the session checkpoint under
|
||
`empirical/RESULTS.md` and `empirical/raw/`. The experiments were deliberately
|
||
small and answered only material uncertainties: actual process scheduling,
|
||
direct versus transitive interface consumption, private/public/link-only blast
|
||
radius, nonzero corruption, path identity, Cstage/WWstage symmetry, duplicate
|
||
storage, and hidden-host execution. No toy result is treated as proof that the
|
||
new architecture is complete; sections 13–14 turn each inference into a gate.
|
||
|
||
### 15.4 Assumptions
|
||
|
||
- Breaking import syntax, package layout, compiler flags, cache/workdir format,
|
||
bootstrap artifacts, and consumer builds is authorized.
|
||
- WW can add explicit C ABI/representation annotations and change its compiler
|
||
export protocol without preserving `.wwi` compatibility.
|
||
- Official releases can distribute or name legally usable immutable tool,
|
||
sysroot, runtime, and SDK closures for each supported target.
|
||
- SHA-256 is the version-1 content primitive; every format is domain-separated
|
||
and versioned so a future digest transition can be explicit.
|
||
- A first release may support fewer frozen targets than impure development
|
||
targets; it may not call an impure target “hermetic.”
|
||
- Byte reproducibility covers build artifacts, not identical behavior of an
|
||
external kernel, firmware, network service, or hardware.
|
||
|
||
### 15.5 Remaining risks and bounded experiments
|
||
|
||
These experiments are implementation gates inside the selected architecture;
|
||
they do not reopen its package identity, no-script, one-graph, lock/network,
|
||
direct-export, content-key, native-provider, or stage-zero choices.
|
||
|
||
1. **Deep `.wwe` closure.** Prototype the real standard-library type graph and
|
||
prove that one direct export file contains every transitive layout/type fact
|
||
required without leaking non-semantic source data. Adjust record layout, not
|
||
direct-dependency semantics.
|
||
2. **Generated C seed.** Measure generated `ww0.c` size, C99 portability, host-C
|
||
variance, and fixed-point convergence on at least three unrelated C
|
||
implementations. Restrict/repair the recovery emitter rather than introducing
|
||
a second maintained compiler or opaque-permanent binary seed.
|
||
3. **External linker/tool bundles.** Certify deterministic debug info, build ID,
|
||
archives, scripts, shared-loader metadata, redistribution rights, and resource
|
||
closure. Change tool adapters/bundle membership if needed; do not restore
|
||
ambient driver defaults.
|
||
4. **Sandbox portability.** Implement denial conformance on every official B
|
||
platform, including process children, filesystem race/symlink attacks, clock,
|
||
randomness, and network namespaces. A platform that cannot enforce it remains
|
||
non-frozen rather than gaining an undeclared exception.
|
||
5. **Native-provider coverage.** Exercise ELF first, then Mach-O frameworks/SDKs,
|
||
PE/COFF import libraries, symbol versioning, and kernel image builders. Extend
|
||
the closed typed schema by version where genuinely required; do not add raw
|
||
search or a general build language.
|
||
6. **Registry/private-source protocol.** Test mirror failover, redirects,
|
||
credential isolation, key rotation, provenance, yanked releases, and malicious
|
||
archives. Vendored/exact-origin operation remains the deterministic fallback.
|
||
7. **Performance.** Validate compiler worker strategy and CAS hashing against the
|
||
measured 15-package/full-toolchain budgets. Process topology may change while
|
||
action boundaries and keys remain fixed.
|
||
|
||
### 15.6 Final decision trace
|
||
|
||
The documented Pike evidence shapes the design: computable direct dependencies,
|
||
fast compilation, cycle rejection, package boundaries, and orthogonal concepts;
|
||
the collective Plan 9 papers add transparent encodings and system-wide placement
|
||
of complexity. Later Go work demonstrates one possible module/cache/toolchain
|
||
evolution but does not decide WW's answer. This document then applies those
|
||
principles to requirements early Go's package/build model did not expose
|
||
completely: foreign ABI contracts, C headers, external assembly and objects,
|
||
archive/link ordering, linker scripts, CRT/libc/sysroot identity, host generators,
|
||
and explicit build/host/target closures.
|
||
|
||
The resulting binding conclusion is singular: **replace the current system with
|
||
WW Action Build exactly as specified above**. Do not preserve the old path, and
|
||
do not substitute a programmable project framework or a source-only package
|
||
command during implementation.
|