Make the schema-1 transcription mechanically consistent before the protocol freeze: restore compiler-protocol identity, use the typed native-contract record identity, distinguish encoding and path classes, and close the package artifact kinds. These corrections must land together because one exhaustive IDL cannot encode the prior conflicting rules.
3282 lines
183 KiB
Markdown
3282 lines
183 KiB
Markdown
# WW package, dependency, build, and bootstrap architecture
|
||
|
||
Status: **binding architecture decision**
|
||
|
||
Decision date: 2026-08-10
|
||
|
||
Implementation status: specified, not yet implemented
|
||
|
||
This document selects the production architecture that replaces WW's current
|
||
package driver, source-like interface protocol, work-directory reuse scheme,
|
||
Make orchestration, test coordinator, and bootstrap chain. It is a greenfield
|
||
decision. Migration effort is recorded only to plan implementation; it did not
|
||
influence the selection.
|
||
|
||
The words **MUST**, **MUST NOT**, **SHOULD**, and **MAY** are normative.
|
||
|
||
## 1. Executive decision
|
||
|
||
WW will have one integrated command backed by one typed, content-addressed
|
||
action graph. Language imports describe the language-package subgraph. A small,
|
||
declarative `ww.mod` file describes only facts that source imports cannot:
|
||
distribution requirements, products, generated inputs, native providers, and
|
||
unusual link steps. Both descriptions lower to the same graph, scheduler,
|
||
sandbox, cache, and explanation machinery. There is no general build language
|
||
and no arbitrary build script. A one-directory, zero-dependency executable
|
||
needs no manifest. Distributed projects use an exact lock file; fetching and
|
||
updating are explicit commands, while build, run, test, documentation, and
|
||
installation never access the network or rewrite project metadata. Packages
|
||
produce target-specific binary export data and one object, importers consume
|
||
only direct export data, and final products link a declared ordered closure.
|
||
Every cached action names its compiler, target, profile, tools, sysroot, runtime,
|
||
native inputs, environment, and content. The official toolchain owns these
|
||
protocols and ships a pinned tool closure, but WW does not permanently own an
|
||
assembler or linker.
|
||
|
||
The decisive insight is that **the package graph and the build graph are not the
|
||
same graph**. Imports are a complete and desirable description of WW-language
|
||
dependencies. They cannot honestly describe a C header tree, a host generator,
|
||
an assembler, a linker script, a CRT, or a sysroot. Making imports pretend to do
|
||
so hides native inputs; making every project use a programmable build framework
|
||
destroys the simple ordinary path. Two small declarative front ends lowering to
|
||
one action engine are simpler as a system than either lie.
|
||
|
||
The architecture is named **WW Action Build** in this document. That is a label,
|
||
not another user-facing product: the command remains `ww`.
|
||
|
||
### 1.1 The Pike lens and attribution
|
||
|
||
Pike explicitly documented and defended these Go design choices and principles:
|
||
|
||
- language-defined imports make dependencies explicit, clear, and mechanically
|
||
computable;
|
||
- unused imports and import cycles are errors; rejecting cycles improves package
|
||
boundaries and independent maintenance;
|
||
- compilation speed and short edit/build cycles are primary design properties;
|
||
- a direct dependency's compiled artifact can carry the deeper public type facts
|
||
needed by its clients, so an importer opens only direct dependency artifacts;
|
||
and
|
||
- orthogonal, predictable concepts, fewer ways to express a construct, and a
|
||
simple user experience are worth substantial implementation work.
|
||
|
||
Those points are stated in Pike's 2009 Go talk, the 2012 SPLASH article, and his
|
||
2015 simplicity talk
|
||
([2009 talk](https://go.dev/talks/2009/go_talk-20091030.pdf),
|
||
[2012 article](https://go.dev/talks/2012/splash.article),
|
||
[Simplicity is Complicated](https://go.dev/talks/2015/simplicity-is-complicated.slide)).
|
||
The collective Plan 9 papers add system-wide placement of complexity, focused
|
||
interfaces, and transparent text or explicitly encoded binary data. `Go in Go`
|
||
documents one contingent case in which owning more of the toolchain simplified
|
||
Go; it does not establish permanent toolchain ownership as a general Pike
|
||
principle ([Go in Go](https://go.dev/talks/2015/gogo.slide)).
|
||
|
||
Plan 9's `mk` constructs
|
||
the dependency graph before execution, rejects cycles and ambiguous recipes,
|
||
and schedules independent work in parallel. Plan 9 used a target-specific
|
||
compiler/assembler/loader family and portably encoded target object conventions
|
||
([mk](https://9p.io/sys/doc/mk.html),
|
||
[mkfiles](https://9p.io/sys/doc/mkfiles.html),
|
||
[compilers](https://9p.io/sys/doc/comp.html)).
|
||
|
||
Modern module-path/version semantics, Minimal Version Selection, `go.sum`, the modern Go build cache,
|
||
automatic toolchain selection, and current supply-chain policy are later Go-team
|
||
designs, not principles uniquely attributable to Pike
|
||
([module reference](https://go.dev/ref/mod),
|
||
[`go` command](https://go.dev/cmd/go/),
|
||
[toolchain selection](https://go.dev/doc/toolchain),
|
||
[toolchain rebuilding](https://go.dev/blog/rebuild),
|
||
[supply-chain policy](https://go.dev/blog/supply-chain)). This decision borrows
|
||
some invariants from those systems but does not attribute them to Pike.
|
||
|
||
The following are this document's inferences from the documented principles:
|
||
|
||
- strict directory packages and direct binary export data are the smallest way
|
||
to keep dependencies computable and compilation fast;
|
||
- a declarative native/action layer is necessary for an unmanaged language,
|
||
because omitting it moves complexity into ambient shell state;
|
||
- one shared action engine is simpler than independent language and outer-build
|
||
caches;
|
||
- given WW's complete-graph, frozen-build, and explainable-key hard gates,
|
||
arbitrary graph-producing programs are rejected; they would require executing
|
||
dependency host code before the graph is inspectable and add another permanent
|
||
user programming model; and
|
||
- WW should specify a toolchain closure but should not maintain an assembler and
|
||
linker forever when pinned external tools make the whole system smaller.
|
||
|
||
Modern native requirements force deliberate departures from historical Plan 9
|
||
and early Go: cryptographic source identities, lock files, explicit build/host/
|
||
target separation, sysroot and SDK identity, hostile dependency acquisition,
|
||
cross-platform sandboxes, and cache-miss explanations were not their complete
|
||
problem statement. WW adopts their architectural restraint, not their ambient
|
||
host assumptions.
|
||
|
||
### 1.2 What follows from being unmanaged and native
|
||
|
||
Being unmanaged/native genuinely requires the build model to know:
|
||
|
||
- the target data layout and C ABI;
|
||
- foreign symbol spelling and visibility;
|
||
- object format, relocation model, CPU features, and assembly dialect;
|
||
- ordered objects, archives, shared libraries, linker scripts, and archive-group
|
||
semantics;
|
||
- the libc, CRT, dynamic loader, SDK, runtime, and sysroot closure;
|
||
- freestanding entry and runtime policy;
|
||
- build-machine tools that generate host- or target-machine inputs; and
|
||
- ABI compatibility among compiler, runtime, native providers, and final link.
|
||
|
||
It does **not** follow that WW needs multiple dependency versions, semantic
|
||
version ranges, a network resolver in every build, programmable build scripts,
|
||
feature unification, a global namespace, or its own linker. In particular, the
|
||
absence of a garbage collector says nothing about version resolution.
|
||
|
||
### 1.3 Binding answers to the critical questions
|
||
|
||
| Question | Binding answer |
|
||
|---|---|
|
||
| Package identity | One canonical import path: the owning module identity for its root package, otherwise that identity plus `/` and the normalized package-relative path. The declared package name is a source qualifier, not identity. |
|
||
| Identity versus location/origin/version/content | All are separate. A resolver record maps identity and selected version to an origin and source-tree digest; a workspace maps identity to a local location. |
|
||
| Directory membership | Exactly one package per directory. Immediate selected source files belong to it. Nested directories never do. |
|
||
| Single-file packages | Deleted. A one-file directory package remains configuration-free. |
|
||
| Language dependency graph | The compiler-parsed imports alone define it. Manifest native/action edges extend the build graph, never the language graph. |
|
||
| Import interfaces | Direct dependencies only. Each direct `.wwe` contains the deep public type information needed to understand its own API. |
|
||
| `.wwi` | Deleted and replaced by deterministic, versioned binary `.wwe` export data. Canonical source prototypes are not an interchange format. |
|
||
| Package invalidation | The package action key changes when its selected own sources/generated inputs, direct export digests, compiler/toolchain, target/profile, declared environment, or protocol changes. A private transitive change does not invalidate it. |
|
||
| Cache key | Domain-separated SHA-256 over the canonical action record defined in section 6.6. |
|
||
| Cache scope | A per-user global local content store plus a project-local graph-history index. Remote import/export is explicit, never part of ordinary build. |
|
||
| Corruption/upgrades | Every object is rehashed on read; corrupt entries are quarantined. Tools and protocol versions are content inputs, so upgrades change keys. |
|
||
| Multiple versions | One selected version of a module identity. Incompatible major releases use distinct module identities ending `/vN`, so those identities may coexist. |
|
||
| Build scripts | Arbitrary scripts are forbidden. A finite declarative action may run a pinned build-machine tool in a denied-by-default sandbox. |
|
||
| Action authority | Exact readable inputs, writable outputs, argv, environment, execution platform, and tool closure. No network, shell, ambient `PATH`, clock, randomness, or undeclared filesystem access. |
|
||
| Acquisition | `add`, `update`, `lock`, `fetch`, `toolchain fetch`, cache transfer, and explicitly authorized remote observation may use the network. Artifact build/analysis and ordinary tests may not; running the finished user program is outside acquisition. |
|
||
| Manifest for a trivial program | No. A standalone directory containing a `main` package is sufficient. A manifest is required for distribution dependencies, multiple products, native providers, or generated inputs. |
|
||
| Configuration placement | Imports in source; identity, requirements, products, native declarations, and actions in `ww.mod`; exact selected closure in `ww.lock`; local paths in `ww.work`; ephemeral target/profile/output choices on the command line. |
|
||
| Local overrides | `ww.work` maps a module identity to a local source tree and records its observed digest. Imports do not change. |
|
||
| Target-specific files | A fixed filename-suffix selection rule; no source-level build expressions and no user-programmable selector. |
|
||
| OS distribution | A distributor may vendor the locked source closure or supply an exact `ww.work`/native-provider map. Substituted files and tools get new digests; frozen mode never silently consults the host. |
|
||
| Reproduction input | For supported official targets: project source, complete locked dependency-source bytes (vendor/CAS export), `ww.lock`, named immutable toolchain bundle, target/profile, and every declared external seed/signing input. Hashes without bytes are insufficient. Impure profiles forfeit the promise. |
|
||
| Assembler/linker ownership | No permanent ownership. The toolchain descriptor pins complete implementations. The current WW tools may bridge migration only. |
|
||
| Stage zero | One release-generated, checked-in portable C99 compiler snapshot plus a tiny declarative bootstrap plan and digest file. |
|
||
|
||
## 2. Normative vocabulary
|
||
|
||
| Term | Exact meaning |
|
||
|---|---|
|
||
| **package** | The WW declarations selected from one directory, compiled together under one declared package name and one package identity. |
|
||
| **module** | A distributable, versioned source tree rooted by one `ww.mod`, declaring one globally stable module identity and containing zero or more packages. |
|
||
| **project** | The module or standalone package selected by the user's current command, including its declared products. |
|
||
| **workspace** | A local, non-published set of module-identity-to-directory overlays described by `ww.work`. It changes location, never identity. |
|
||
| **dependency** | A typed directed edge: package import, generated-input edge, tool edge, native-provider edge, runtime edge, or ordered link edge. The edge kind is never implicit. |
|
||
| **target/platform descriptor** | A canonical architecture/platform/ABI/object/CPU/runtime description. An action labels descriptors by role: execution `B`, product `H`, and optional compiler-output `T`. A target triple is only a short lookup name. |
|
||
| **artifact** | An immutable byte string or canonical directory tree produced by an action and named by a content digest. Materialized files are copies or links, not the artifact's identity. |
|
||
| **toolchain** | An immutable descriptor and content closure containing the compiler, action protocol, export/ABI versions, target descriptors, resource files, runtime implementations, and pinned assembler/linker/archive tools. |
|
||
| **sysroot** | A content-identified target filesystem tree containing the exact headers, libraries, CRT objects, loader metadata, linker scripts, and SDK files exposed to target actions. |
|
||
| **source identity** | `sha256` of the canonical source-tree encoding in section 4.5. It is independent of download URL and checkout path. |
|
||
| **version** | An immutable SemVer release label associated with one module identity and one source identity. It is selection metadata, not package identity. |
|
||
| **product** | A named requested result: executable, static library, shared library, object bundle, test binary, generated tree, documentation tree, or toolchain component. |
|
||
| **action** | A pure, finite build step with a typed canonical record, declared input artifacts, one execution platform, and declared output paths. |
|
||
| **build platform (B)** | The platform on which the build actions execute. |
|
||
| **host platform (H)** | The platform on which the requested product will execute. |
|
||
| **target platform (T)** | For a compiler-like product, the platform for which that product emits code. It is absent for an ordinary executable or library. |
|
||
|
||
Module and package identities are slash-separated ASCII paths. They are
|
||
NFC-normalized, case-sensitive, contain no empty, `.` or `..` segment, and do
|
||
not depend on filesystem case folding. A non-root package identity is written
|
||
`module-id/package/path`; the root package identity is `module-id`. The selected
|
||
module catalog records which module owns each package identity. If two selected
|
||
modules would supply the same package identity, resolution fails rather than
|
||
choosing a longer prefix.
|
||
|
||
A no-manifest invocation gives its sole root package the reserved internal
|
||
identity `@standalone/root`; any external test package gets the reserved identity
|
||
`@test/<first-128-bits-of-SHA256(production-package-identity)>`. These namespaces cannot be
|
||
declared by a module or imported from ordinary source. The default standalone
|
||
executable materializes as `main`, independent of directory basename. A
|
||
standalone package cannot contain/import another local package or be published
|
||
until `ww init` gives it a stable module/package identity.
|
||
|
||
## 3. Source and package rules
|
||
|
||
### 3.1 One directory, one package
|
||
|
||
A package directory contains its immediate regular source files only. The build
|
||
does not follow source symlinks. Nested directories are separate packages. Every
|
||
selected production source MUST begin with the same canonical `package name;`
|
||
clause. The declared name MUST be a valid WW identifier. It need not repeat the
|
||
directory leaf because identity and source qualifier are separate concepts.
|
||
|
||
The following current forms are errors after the migration:
|
||
|
||
- importing a single `.ww` file as if it were a package;
|
||
- placing multiple package blocks in one compilation unit;
|
||
- satisfying an unresolved import from an inline package block;
|
||
- selecting a literal source file as the build root; and
|
||
- finding packages through ordered `-I` search roots.
|
||
|
||
`ww build ./cmd/tool` selects a directory. A directory with one source file is
|
||
still the smallest package and needs no extra metadata.
|
||
|
||
### 3.2 File membership and target selection
|
||
|
||
Production candidates are immediate regular files ending `.ww`, excluding
|
||
`*_test.ww` and generated outputs. Names are byte-sorted after UTF-8 validity and
|
||
normalization checks. A source symlink, duplicate normalized name, case-fold
|
||
collision, or non-regular candidate is a loud error on every host.
|
||
|
||
Target variants use this only convention:
|
||
|
||
```text
|
||
stem[+os][+arch][+environment].ww
|
||
```
|
||
|
||
Recognized tags come from the selected target descriptor, not from the host.
|
||
Files are grouped by `stem`. The matching member with the greatest number of
|
||
tags wins; the untagged member is the fallback. Two equally specific matches are
|
||
an error. Examples are `poll.ww`, `poll+linux.ww`, and
|
||
`poll+linux+amd64+gnu.ww`. This is replacement selection, not additive feature
|
||
selection; additive code uses a distinct stem. Unknown tags are errors. There
|
||
are no boolean selectors, glob expressions, or manifest-defined tag meanings.
|
||
|
||
Test variants put the same tags before the reserved suffix, for example
|
||
`poll+linux_test.ww`; their grammar is `stem[+tags]_test.ww` and the identical
|
||
most-specific rule applies within the test set. A production stem ending
|
||
`_test` is reserved and rejected, preventing a tagged test from being mistaken
|
||
for production source.
|
||
|
||
The selected file-name list is itself an action-key input. Therefore adding or
|
||
removing a more-specific file invalidates the package even when the old files'
|
||
bytes do not change.
|
||
|
||
CPU features and optimization mode always enter the compile key but do not add
|
||
another WW source-selection language. WW-level specialization uses compiler
|
||
intrinsics/runtime dispatch or a distinct package; CPU/float-ABI/PIC-sensitive C
|
||
or assembly uses the finite native `when` constraints in section 8.4. This keeps
|
||
ordinary source membership conventional while still making exceptional native
|
||
selection exact and inspectable.
|
||
|
||
### 3.3 Imports, names, and resolution
|
||
|
||
The canonical forms are:
|
||
|
||
```ww
|
||
import "example.org/codec/hex";
|
||
import wire "example.org/protocol/hex";
|
||
```
|
||
|
||
The quoted string is the package identity. A module package may abbreviate its
|
||
own module prefix with a relative import written `import "./sub/path";`;
|
||
resolution replaces `./` with the importing module identity and normalizes the
|
||
remainder without permitting `..`. Bare dotted imports and filesystem imports
|
||
are deleted. Standalone packages cannot use relative imports.
|
||
|
||
The default source qualifier is the imported package's declared name. An alias
|
||
changes only that qualifier. Two imports producing the same qualifier are an
|
||
error unless one is explicitly aliased. Importing the same identity twice,
|
||
resolving one identity to two sources, or resolving two selected module records
|
||
to the same module identity is a loud collision error.
|
||
|
||
Resolution is exact:
|
||
|
||
1. Build the locked/workspace package catalog by joining every selected module
|
||
identity with its package directories.
|
||
2. Require exactly one catalog owner for the requested package identity; zero is
|
||
unresolved and two is an identity collision, even when one module prefix is
|
||
longer.
|
||
3. Verify that the catalog directory exists in that module source tree and has
|
||
the expected package clause.
|
||
4. Never search another root and never choose by filesystem accident.
|
||
|
||
Imports are parsed by the compiler front end, not a line scanner. Their union
|
||
forms package edges, but name visibility remains file-scoped. Every imported
|
||
qualifier must be used. Package cycles, including self-imports, are reported
|
||
before compilation with one stable identity path through the cycle.
|
||
|
||
If an import's catalog owner is outside the importing module, that owner MUST be
|
||
a direct `require` in the importing module's own `ww.mod`; availability through
|
||
another dependency is not enough. This keeps distribution dependencies as
|
||
explicit as package imports and prevents accidental reliance on a transitive
|
||
selection. `ww add` creates the requirement; source is never rewritten.
|
||
|
||
The sole exception is the toolchain's intrinsic standard module, whose identity
|
||
and source-tree digest are part of the selected toolchain descriptor. It is
|
||
available without a manifest requirement, including to a standalone package;
|
||
it is not searched from an installation directory or upgraded independently.
|
||
|
||
### 3.4 Visibility and internal packages
|
||
|
||
Existing exported-versus-private declaration rules survive. An `internal`
|
||
directory segment adds one resolution rule: a package under
|
||
`M/P/internal/Q` may be imported only by the package `M/P` or a package having
|
||
`M/P/` as a segment prefix. For `M/internal/Q`, the allowed root is module `M`
|
||
and its descendants. This is checked against the catalog's owning module and
|
||
package identities, not checkout paths. There are no friend lists or manifest
|
||
visibility overrides.
|
||
|
||
### 3.5 Tests, examples, documentation, and generated WW
|
||
|
||
Only `*_test.ww` files are test sources. `package p;` tests compile with package
|
||
`p`; `package p_test;` tests compile as a separate external package importing
|
||
the production package normally. Test sources of dependencies are never in a
|
||
consumer graph. `ww test .` tests one package; `ww test ./...` discovers package
|
||
directories under the selected module, excluding hidden, underscore-prefixed,
|
||
vendor, cache, and output directories. Discovery is deterministic and does not
|
||
follow directory symlinks. Test compilation is cached; each selected test binary
|
||
is executed on every command and independent binaries may run in parallel.
|
||
|
||
Each test invocation gets only its declared `test-data` mounted read-only under
|
||
`/data`, its literal `test-env`, a private writable temporary directory, and the
|
||
selected runner/runtime closure. Project/home/host files and network are denied.
|
||
Because execution is an observation rather than an artifact action, real clock,
|
||
process IDs, scheduling, and OS randomness may be exposed and are recorded as
|
||
runner capabilities; their output is never cached or part of byte reproduction.
|
||
|
||
The same-package test action uses the production package identity with the
|
||
non-importable action variant `same-test`; it compiles production and test
|
||
sources together so private names remain visible. An external test uses the
|
||
reserved `@test/<128-bit-production-identity-digest>` package identity and has a
|
||
normal direct import of the production package. Neither identity can collide
|
||
with or be imported by published source.
|
||
|
||
An example is an ordinary package or named product under `examples/`; it has no
|
||
special dependency semantics. Documentation is derived from source comments and
|
||
`.wwe` declarations, not by compiling examples during an ordinary build.
|
||
|
||
Generated WW source MUST be declared as an output of an action and as a generated
|
||
input of exactly one package. It uses the suffix `.wwgen`, does not appear in the
|
||
source tree, may not contain `import` or `package` clauses, and therefore cannot
|
||
discover new graph edges after graph construction. It may refer to built-ins and
|
||
declarations already in its owner package. A generator needing another package
|
||
must have a checked-in owner file that imports it. This restriction keeps the
|
||
complete package graph inspectable before executing generators.
|
||
|
||
A generated fragment may contain a foreign declaration only when the owning
|
||
`package` clause already names its provider slot and the declaration explicitly
|
||
names that slot. It cannot add a native requirement. After generation, the
|
||
compiler verifies imports/package clauses are absent and the observed foreign
|
||
slots exactly match the predeclared set; mismatch is a generator/protocol error.
|
||
|
||
## 4. Dependency distribution model
|
||
|
||
### 4.1 When metadata is required
|
||
|
||
A standalone one-directory executable with no non-toolchain dependency builds
|
||
without metadata. `ww init MODULE` creates a module when the program needs a
|
||
stable import identity, distribution dependencies, multiple packages/products,
|
||
generated inputs, or native declarations.
|
||
|
||
A module tree contains exactly one `ww.mod` at its root; nested manifests are an
|
||
error. A workspace composes separate module roots instead of nesting ownership.
|
||
|
||
`ww.mod` is declarative UTF-8 data. It is not WW code: it has no expressions,
|
||
variables, imports, include files, macros, loops, user functions, or host
|
||
conditionals. Strings use JSON escaping; lists preserve source order and record
|
||
key order is non-semantic. Unknown fields are errors unless a later manifest schema is explicitly
|
||
selected.
|
||
|
||
A minimal module is:
|
||
|
||
```text
|
||
ww-manifest 1
|
||
module = "example.org/hello"
|
||
language = "1"
|
||
toolchain = { id = "ww.org/toolchain", minimum = "v1.4.0" }
|
||
|
||
require "example.org/codec" {
|
||
minimum = "v1.2.3"
|
||
source-index = "https://example.org/codec/.well-known/ww-source"
|
||
}
|
||
```
|
||
|
||
The complete set of top-level clause kinds in schema 1 is `require`, `product`,
|
||
`package`, `action`, and `native`. Global scalar keys are only `module`,
|
||
`language`, and `toolchain`. `require` has required `minimum` and optional
|
||
credential-free `source-index`; the latter maps origin without changing identity.
|
||
The schema tables in section 6.9 close all remaining fields. There is
|
||
deliberately no general `[settings]` escape hatch.
|
||
|
||
The root `main` package is the default executable product, named after the
|
||
module's last segment. Libraries need no product declaration to be imported.
|
||
Additional or non-default outputs are explicit:
|
||
|
||
```text
|
||
product "inspect" {
|
||
kind = "exe"
|
||
root = "cmd/inspect"
|
||
}
|
||
```
|
||
|
||
### 4.2 Selection rule
|
||
|
||
Versions are `vMAJOR.MINOR.PATCH` SemVer labels with the usual prerelease order.
|
||
A requirement is one minimum version, never a range. Selection chooses the
|
||
greatest minimum requested for each module identity over the complete transitive
|
||
closure and repeats until stable. This is intentionally the small, monotonic
|
||
part of Minimal Version Selection, not every behavior of the Go module command.
|
||
The selected result is written exactly to `ww.lock` by `add`, `update`, or
|
||
`lock`; build never resolves a newer version.
|
||
|
||
Exactly one version of a module identity is selected. A backward-incompatible
|
||
major version `N >= 2` MUST declare a module identity ending `/vN`, and imports
|
||
name that identity. Consequently incompatible releases may coexist as distinct
|
||
identities without an aliasing version resolver. Two versions of the same
|
||
identity cannot coexist.
|
||
|
||
There are no feature sets, optional-dependency activation, target-dependent
|
||
version constraints, upper bounds, wildcard versions, or dependency-wide
|
||
configuration unification. Target variation belongs in source selection and
|
||
native-provider declarations after one source closure is selected.
|
||
|
||
The selected source manifests also contribute one minimum for the same
|
||
toolchain identity. The root lock chooses one exact installed/catalog version at
|
||
least as high as every minimum and records its descriptor/bundle digests. A
|
||
different toolchain identity or unsupported language/export/runtime protocol is
|
||
an error; SemVer alone never overrides protocol compatibility. Dependency
|
||
manifests do not pin the consumer to their development compiler, while an
|
||
application remains exactly reproducible from its lock.
|
||
|
||
### 4.3 Lock file
|
||
|
||
`ww.lock` is generated, canonical, and committed for applications and toolchains.
|
||
Published libraries SHOULD commit it for their own tests, but consumers resolve
|
||
from `ww.mod` requirements. A lock record is:
|
||
|
||
```text
|
||
ww-lock 1
|
||
root-manifest = "sha256:9c..."
|
||
toolchain "ww.org/toolchain" {
|
||
version = "v1.4.0"
|
||
descriptor = "sha256:31..."
|
||
bundle = "sha256:80..."
|
||
origin = "https://dist.wwlang.org/toolchain/v1.4.0/"
|
||
}
|
||
module "example.org/codec" {
|
||
version = "v1.2.3"
|
||
origin = "https://example.org/codec/.ww/v1.2.3.tar.zst"
|
||
archive = "sha256:4a..."
|
||
tree = "sha256:f7..."
|
||
manifest = "sha256:55..."
|
||
signature = "ed25519:key-id:base64..."
|
||
provenance = "https://example.org/codec/.ww/v1.2.3.intoto.jsonl"
|
||
}
|
||
```
|
||
|
||
Records are sorted by identity. Required semantic fields are version, immutable
|
||
origin, archive digest, canonical source-tree digest, and manifest digest.
|
||
Signature and provenance are optional records whose verification policy is
|
||
configured by the user or distributor; hashes are never optional. A signature,
|
||
when present, covers schema, module identity, version, tree digest, and manifest
|
||
digest. Lock files never contain local overlay paths or credentials.
|
||
|
||
### 4.4 Acquisition and network policy
|
||
|
||
`ww add M@V` discovers `M` by the HTTPS convention
|
||
`https://M/.well-known/ww-source`, unless `--from=URL` or a user-configured
|
||
longest-prefix source map supplies an index. The index returns immutable archive
|
||
locations and signed digest records. Redirects and the final URL are recorded.
|
||
The downloaded module manifest MUST declare exactly `M`; the signed index/lock
|
||
record, not source text, binds `V` to its tree digest. Private indexes use the
|
||
same protocol and obtain credentials from the fetch command's credential
|
||
helper; credentials never enter build actions or lock files.
|
||
|
||
`(module identity, version)` is immutable: observing two signed tree digests for
|
||
the same pair is an equivocation error recorded in the source store, never an
|
||
automatic replacement. Yank metadata may prevent new selection but cannot alter
|
||
or invalidate an already locked digest.
|
||
|
||
Only these operations may initiate network requests:
|
||
|
||
- `ww add`, `ww update`, and `ww lock` while selecting metadata;
|
||
- `ww fetch --locked` while materializing the already locked source/toolchain
|
||
closure;
|
||
- `ww toolchain fetch` for an explicitly named toolchain; and
|
||
- explicit `ww cache pull` and `ww cache push`; and
|
||
- explicitly authorized `ww observe` remote execution, which is not a build,
|
||
test-build, or cached artifact action.
|
||
|
||
The build/analysis phases of `ww build`, `run`, `test`, `doc`, `install`,
|
||
`graph`, and `explain` deny network access even when an input is absent. They report the missing source digest and
|
||
the exact `ww fetch --locked` command. They never modify `ww.mod`, `ww.lock`, or
|
||
`ww.work`. `--frozen` additionally requires those files to be present,
|
||
canonical, mutually consistent, and unchanged by selection. `--offline` is an
|
||
explicit assertion of the already mandatory no-network build policy.
|
||
|
||
### 4.5 Source identity and storage
|
||
|
||
A module source tree contains only directories and regular files; symlinks,
|
||
devices, sockets, FIFOs, absolute paths, `..`, duplicate normalized paths, and
|
||
case-fold collisions are rejected. Its identity is:
|
||
|
||
```text
|
||
SHA256("ww-source-tree-v1\0" ||
|
||
for each byte-sorted relative path:
|
||
LP(path) || type || executable-bit || LP(SHA256(file-bytes)))
|
||
```
|
||
|
||
`LP(x)` is an unsigned 64-bit big-endian byte length followed by `x`. Directory
|
||
entries are included with type `dir`; regular files with type `file`. Ownership,
|
||
timestamps, archive compression, checkout path, and non-executable permission
|
||
bits are excluded. Archives are checked both against their blob digest and the
|
||
unpacked tree digest. The source store is immutable and keyed by tree digest.
|
||
|
||
### 4.6 Workspaces, vendoring, and distributors
|
||
|
||
`ww.work` is local, declarative, and normally uncommitted:
|
||
|
||
```text
|
||
ww-work 1
|
||
use "example.org/codec" {
|
||
path = "../codec"
|
||
expect = "sha256:f7..."
|
||
}
|
||
```
|
||
|
||
An overlay replaces only the location for the named module identity. The module
|
||
at that path must declare the same identity. Its current canonical tree digest
|
||
is an action input; `expect` makes accidental drift loud but may be updated by
|
||
`ww work sync`. No import or lock identity changes.
|
||
|
||
`ww vendor` materializes every locked module under `vendor/sha256/<tree>` and
|
||
writes a canonical identity-to-tree `vendor/index.wwv`. Frozen builds may select
|
||
that source store with `--vendor`; the vendored bytes must match the lock. There
|
||
is no flattened import tree and no rewritten import statement.
|
||
|
||
An operating-system distributor has three honest options: ship this vendor
|
||
store, prefill WW's immutable source store, or provide an exact workspace/source
|
||
map to distro-owned trees. Native system libraries are substituted only through
|
||
the explicit provider mechanism in section 8.8. A mutable `/usr` lookup is an
|
||
impure system profile, is local-cache-only, and is rejected by frozen builds.
|
||
|
||
### 4.7 Closed metadata grammar
|
||
|
||
`ww.mod`, `ww.lock`, `ww.work`, vendor indexes, and toolchain/native descriptors
|
||
share this lexical grammar; each schema separately closes its allowed headers,
|
||
clauses, fields, value types, and cardinalities:
|
||
|
||
```text
|
||
document = header newline { statement } EOF
|
||
header = schema-name SP unsigned
|
||
statement = assignment | clause
|
||
assignment = key ws "=" ws value ws newline
|
||
clause = key ws string ws "{" newline
|
||
{ assignment } "}" ws newline
|
||
value = string | unsigned | boolean | list | record
|
||
list = "[" ws [ value { ws "," ws value } [ ws "," ] ] ws "]"
|
||
record = "{" ws [ pair { ws "," ws pair } [ ws "," ] ] ws "}"
|
||
pair = key ws "=" ws value
|
||
key = ALPHA { ALPHA | DIGIT | "_" | "-" }
|
||
string = JSON-string-with-valid-UTF-8
|
||
unsigned = "0" | ("1"…"9" { DIGIT })
|
||
boolean = "true" | "false"
|
||
ws = { SP | TAB | newline | comment }
|
||
comment = "#" { any-character-except-newline }
|
||
```
|
||
|
||
`schema-name` is exactly `ww-manifest`, `ww-lock`, `ww-work`, `ww-vendor`,
|
||
`ww-toolchain`, `ww-native-map`, `ww-native-sidecar`, `ww-install`, or
|
||
`ww-bootstrap`. A clause
|
||
body contains assignments only, so nesting cannot grow into a language.
|
||
Duplicate keys, duplicate singleton clauses, invalid UTF-8, unknown fields,
|
||
integer overflow, and a comment marker inside an unclosed string are errors.
|
||
|
||
Whitespace, comments, assignment order, record-key order, and clause order where
|
||
the schema declares identity keys are non-semantic. List order is semantic.
|
||
Parsing produces a typed record whose canonical semantic encoding is
|
||
`WWAR(record)`, not the original text. Its semantic digest is the applicable
|
||
kind/schema-bound `record_id` from section 6.6. `ww fmt` writes two-space
|
||
canonical text; generated lock/vendor files MUST already equal that rendering
|
||
in frozen mode.
|
||
|
||
## 5. Build model and graph construction
|
||
|
||
### 5.1 One graph, constructed before execution
|
||
|
||
For every command, `ww` constructs a typed graph in these deterministic phases:
|
||
|
||
1. Select the project, manifest, lock, workspace, toolchain descriptor, target,
|
||
and profile. Verify their schemas, canonical identities, and content digests.
|
||
Missing locked inputs are errors; this phase never fetches.
|
||
2. Enumerate selected checked-in package files by section 3.2. Ask the compiler
|
||
front end to parse package clauses, imports, checked-in foreign declarations,
|
||
and test metadata. Generated artifacts are known future input slots but do not
|
||
yet exist. The build driver never scans source lines itself.
|
||
3. Resolve every import by identity, reject collisions/internal violations, and
|
||
compute the complete acyclic package graph.
|
||
4. Add statically declared generated-input, host-tool, native-provider,
|
||
toolchain, runtime, archive, and ordered link edges. Match target clauses and
|
||
reject zero or multiple providers. All generated output names and consumers
|
||
are known here.
|
||
5. Lower nodes to the complete **action-template DAG** and report it. A template
|
||
names every edge/input slot, tool, policy, and output, but its final key remains
|
||
unresolved until every predecessor output content digest is known. Only after
|
||
this point may a cache be read or a tool execute.
|
||
6. As verified cache results or completed predecessors resolve input artifacts,
|
||
finalize ready action records/keys, query the cache, schedule misses, publish
|
||
successful artifacts atomically, then materialize requested products.
|
||
|
||
No executed action may add a node, input, output, import, library, flag, or
|
||
follow-up command. Native C/assembly declarations name complete header/source
|
||
trees rather than learning dependencies from an ambient compiler depfile. This
|
||
may conservatively rebuild for an unused header change, but the graph remains
|
||
complete before execution and the key is correct.
|
||
|
||
`ww graph --actions --format=json` emits the graph after phase 5. Its canonical
|
||
JSON contains node kind, execution platform, typed input/output slots, incoming
|
||
edge kinds, target/profile/toolchain digests, and the exact ordered link plan.
|
||
A resolved node also has its key/cache status; otherwise it has `key: "pending"`,
|
||
`cache: "unknown"`, and a byte-sorted `waiting-on` list. It contains logical
|
||
paths only. A collision, cycle, missing provider,
|
||
undeclared target, or unresolved tool fails graph construction even if a stale
|
||
cache entry might otherwise satisfy the product.
|
||
|
||
### 5.2 Built-in action kinds
|
||
|
||
Schema 1 has this closed set of semantic action kinds:
|
||
|
||
- `ww.package`: compile one package to export data, object code, and link
|
||
metadata;
|
||
- `ww.init`: synthesize one deterministic retained package-initialization
|
||
dispatcher from precomputed package link metadata;
|
||
- `native.compile`: compile one declared C or assembly source unit;
|
||
- `archive`: construct a static library from an ordered object list;
|
||
- `link`: construct an executable or shared library from an ordered link plan;
|
||
- `generate`: execute one declared build-platform tool;
|
||
- `doc`: render a documentation tree from sources and export data;
|
||
- `bootstrap.compare`: compare canonical stage outputs and manifests.
|
||
|
||
Adding an action kind changes the action schema. There is no generic rule engine,
|
||
phony target, implicit suffix rule, command-string target, or shell recipe. An
|
||
archive is made only for an explicit static-library product or native provider;
|
||
WW packages are not automatically wrapped in one-member archives.
|
||
|
||
`run` and `test.run` are non-cacheable invocation nodes, not semantic artifact
|
||
actions. `test.run` consumes the declared data/environment/runner and uses the
|
||
test sandbox above on every request. `ww run`, after its network-denied build,
|
||
launches the user's program with the user's runtime authority/environment unless
|
||
`--sandbox` is explicitly requested; that execution still cannot affect a build
|
||
cache entry. Both store exit status/logs only as observations. Materialization/install is a third category:
|
||
a request-local side effect consuming an immutable artifact/install manifest.
|
||
Neither category can satisfy or poison an artifact-action cache entry.
|
||
|
||
The semantic graph is independent of process boundaries. An implementation MAY
|
||
run compiler workers in-process or in a bounded pool, but each `ww.package`
|
||
action still has an independent canonical record and outputs. There is no
|
||
required background daemon and no daemon state may affect an output.
|
||
|
||
### 5.3 The finite escape hatch
|
||
|
||
Unusual generation and packaging use a declarative `action`, not a build script:
|
||
|
||
```text
|
||
action "protocol-bindings" {
|
||
tool = "product:tools/schema-gen"
|
||
platform = "build"
|
||
inputs = {
|
||
schema = { file = "protocol/schema.idl" }
|
||
}
|
||
outputs = {
|
||
ww = { file = "generated/protocol.wwgen" }
|
||
}
|
||
argv = ["--input", "/in/schema", "--ww", "/out/ww"]
|
||
env = { LANG = "C", TZ = "UTC" }
|
||
}
|
||
|
||
package "protocol" {
|
||
generated = ["action:protocol-bindings:ww"]
|
||
}
|
||
```
|
||
|
||
The tool is either a named executable in the immutable toolchain or a named WW
|
||
product built for `H = B`. `/in/NAME` inputs are read-only mounts and
|
||
`/out/NAME` outputs are initially absent, exclusive writable mounts. The working
|
||
directory is the empty logical `/work`. Output type is exactly `file` or `tree`;
|
||
undeclared files fail the action. `argv` is passed directly, never through a
|
||
shell. Input and output names are identifiers and each path appears through its
|
||
fixed mount, so there is no template language.
|
||
|
||
The sandbox exposes only the declared tool closure at fixed logical `/tool`
|
||
paths, `/in`, `/out`, the literal
|
||
environment map, deterministic locale/time-zone data, and bounded CPU/memory
|
||
resources. Network, process inspection, host devices, user/home directories,
|
||
ambient `PATH`, ambient environment, wall clock, writable source, and filesystem
|
||
paths outside the mounts are denied. Randomness is absent unless a declared
|
||
seed artifact is mounted. A tool may spawn only executable inputs declared in
|
||
its tool closure. These isolation rules apply equally to built-in compiler, C,
|
||
assembler, archiver, linker, documentation, test, and bootstrap executions.
|
||
Strong enforcement is required for frozen/shared-cache builds;
|
||
an unsupported host must fail rather than silently weaken isolation.
|
||
|
||
This action can perform arbitrary computation over finite declared inputs, so it
|
||
is sufficient for code generators, image/file-system builders, binding tools,
|
||
and signing-input preparation. It cannot inspect the project and invent more
|
||
work. Dependency modules may declare actions only for outputs consumed by their
|
||
own packages/products; they cannot register hooks that run merely because the
|
||
module is present.
|
||
|
||
Every artifact-producing action declares `reproducibility = "required"` in a
|
||
frozen or official build. Isolation removes undeclared external inputs, but it
|
||
cannot prove that arbitrary tool internals avoid PIDs, uninitialized memory,
|
||
ASLR-derived values, or race-dependent output. Such variance is a tool/action
|
||
defect. Official releases and first shared-cache publication of a custom
|
||
generator repeat it from clean sandboxes and compare outputs. A node explicitly
|
||
classified `observation` is never result-cached/shared and may not feed an
|
||
artifact action; tests and hardware execution use that class. Impure development
|
||
actions are local-only and outside the byte promise.
|
||
|
||
### 5.4 Build, host, and target
|
||
|
||
WW uses the conventional three-platform meaning rigorously:
|
||
|
||
```text
|
||
B: execution platform on which build actions run
|
||
H: platform ABI of the produced artifact; executable products are intended to run here
|
||
T: output platform of a compiler-like artifact that itself is built for H
|
||
```
|
||
|
||
For an ordinary program `T` is absent. The familiar
|
||
`ww build --target=aarch64-unknown-linux-gnu` spelling sets `H`; it means “build
|
||
the program that runs on this target.” A compiler product may additionally set
|
||
`--host=H --target=T`. Any generator used while producing it still executes on
|
||
`B`; if the generator is itself WW source, its product is compiled with `H = B`.
|
||
|
||
Every action record carries all applicable descriptors, even when two are equal.
|
||
No rule may infer `H` or `T` from the kernel running `ww`. Cross compilation is
|
||
therefore the same graph with a different explicit host/target descriptor, not
|
||
a mode that edits environment variables.
|
||
|
||
In record/JSON field names these are `execution-platform`, `product-platform`,
|
||
and optional `compiler-output-platform`. Source suffix selection, ordinary
|
||
native-provider `when`, sysroot, CRT, runtime, and linker selection always match
|
||
the product platform H. A build-tool dependency instead has its own H equal to
|
||
the parent action's B. CLI `--target` is only the familiar spelling for selecting
|
||
the ordinary product platform; it does not rename the GNU roles internally.
|
||
|
||
### 5.5 Scheduling and failure
|
||
|
||
After graph construction, ready actions run in a deterministic priority order
|
||
with a user-selected concurrency bound. Priority affects latency only; output
|
||
bytes and link order come from records, never completion order. Independent
|
||
actions may finish after another branch fails, but no dependent action starts.
|
||
On the first observed failure WW stops launching work and cancels its owned
|
||
in-flight actions. Which failure triggers cancellation is observational and may
|
||
vary with concurrency; every concurrently observed failure is sorted by stable
|
||
logical node in the report. `--keep-going` instead continues branches whose
|
||
dependency closure remains healthy and reports all failures in that stable
|
||
order. Interrupts cancel only processes owned by this invocation and leave no
|
||
published partial result.
|
||
|
||
Tool stdout and stderr are captured as artifacts and streamed with node labels.
|
||
Diagnostics use module-relative logical paths. `--verbose` may display physical
|
||
mount paths separately, marked non-semantic. A successful action is published
|
||
only after all declared outputs exist, have valid type/mode, are canonicalized
|
||
where required, and have been hashed. A failed action is never entered in the
|
||
action cache.
|
||
|
||
### 5.6 Atomic publication and materialization
|
||
|
||
CAS files and action-result records are written to same-filesystem unique
|
||
temporary names, flushed, rehashed, then atomically renamed to their digest
|
||
locations. For crash durability, WW flushes the parent directory after rename;
|
||
it publishes and flushes every output before the result mapping. A concurrent
|
||
publisher of the same digest verifies equality and
|
||
discards its temporary file. A directory artifact is a canonical tree object
|
||
whose leaves are CAS blobs. An action-result mapping is published last, so no
|
||
reader can observe a result before its outputs.
|
||
|
||
Materialization is a request-local side effect outside the action-template DAG.
|
||
By default requested
|
||
products appear in `out/<target>/<profile>/`; `--out` and `ww install --prefix`
|
||
change only where immutable artifacts are copied or copy-on-write reflinked.
|
||
Hardlinks/symlinks are permitted only when the backing store is enforced
|
||
immutable against the user and mode changes cannot affect its inode. Executable bits are
|
||
set by the artifact record, never by a later ambient `chmod`. Replacement uses
|
||
temporary siblings and atomic rename. `ww clean` removes materialized/project
|
||
state only; `ww cache gc` is the separate explicit global-cache operation.
|
||
|
||
### 5.7 Reproducibility contract
|
||
|
||
For official supported targets, WW promises byte-identical artifacts when every
|
||
artifact action satisfies `reproducibility = "required"` and these are identical:
|
||
|
||
- canonical project source and `ww.lock`;
|
||
- immutable toolchain descriptor and complete bundle;
|
||
- target descriptor and build profile; and
|
||
- all declared action inputs, including generated seeds and signing material.
|
||
|
||
The engine guarantees input isolation, logical paths, and canonical publication;
|
||
repeat-build certification checks arbitrary tool determinism. The resulting promise is independent of absolute checkout, source-store, cache, output,
|
||
and temporary paths; wall time, locale, process order, username, UID, and host
|
||
environment are absent. Logical paths are module/package paths. Debug information
|
||
uses those logical paths and fixed prefix maps. Archive metadata is canonical;
|
||
timestamps and ownership are zeroed; deterministic linker build IDs derive from
|
||
the link key. Official toolchains reject tools that cannot meet this contract.
|
||
|
||
A project source archive plus the complete locked dependency source bytes (a
|
||
vendor/source-CAS export), its lock, the named immutable toolchain bundle, and
|
||
every declared external seed/signing input is therefore a complete offline
|
||
reproduction input. A lock's hashes alone cannot recreate absent bytes. Runtime behavior that depends on a shared library
|
||
outside the pinned runtime/sysroot closure is not covered, and frozen official
|
||
profiles prohibit such a dependency. An explicitly selected impure system
|
||
profile receives no byte-identity promise, cannot publish to a shared cache, and
|
||
prints every ambient input it accepted. `ww verify reproducible` runs isolated
|
||
uncached builds under two physical roots and compares every result artifact and
|
||
action manifest, not just the final executable.
|
||
|
||
## 6. Action records and cache protocol
|
||
|
||
### 6.1 Canonical encoding
|
||
|
||
Action records use **WWAR 1**, this deterministic byte encoding:
|
||
|
||
```text
|
||
WWAR(record) = 0x57 0x57 0x41 0x52 | u16be(1) | value(record)
|
||
value(v) = type:u8 | u64be(payload-length) | payload
|
||
|
||
type 0x01 bytes: payload is the bytes
|
||
type 0x02 string: payload is valid NFC UTF-8 with no NUL
|
||
type 0x03 uint: payload is minimal unsigned big-endian; zero is one 0x00
|
||
type 0x04 bool: payload is exactly 0x00 or 0x01
|
||
type 0x05 list: u32be(count) | each (u64be(value-length) | value)
|
||
type 0x06 map: u32be(count) | each (u64be(key-length) | key-UTF-8 |
|
||
u64be(value-length) | value)
|
||
type 0x07 record: u32be(count) | each (u32be(field-tag) |
|
||
u64be(value-length) | value)
|
||
```
|
||
|
||
Record fields are strictly increasing by numeric tag. Map entries are strictly
|
||
increasing by raw UTF-8 key bytes. Duplicate/out-of-order keys or tags, leading
|
||
zeroes in a uint, invalid booleans/UTF-8/NFC, mismatched counts/lengths, unknown
|
||
schema tags, and trailing bytes are errors. Lists preserve declared order.
|
||
Schema defaults are always encoded, so no semantic field is inferred from
|
||
absence. Floats, signed integers, null, and indefinite lengths do not exist.
|
||
|
||
One byte string/string is at most `2^31-1` bytes, a container has at most
|
||
`2^24-1` members, and nesting depth is at most 64. A content/container-relative
|
||
logical path string uses `/`, is relative, has no NUL/backslash, empty/`.`/`..`
|
||
segment, and passes the schema's ASCII-identity or NFC-source-path rule. A field
|
||
that explicitly permits `.` as its complete root sentinel is the sole exception.
|
||
These are protocol limits, not host `size_t` limits. Human-readable JSON is a
|
||
lossless rendering, not the hashed representation. Phase-0 golden vectors
|
||
include empty/nested records, ordered lists, sorted maps, every rejection, and
|
||
their complete bytes/digests.
|
||
The normative empty-record vector is
|
||
`57574152000107000000000000000400000000`, SHA-256
|
||
`138c6acb7f01e91df73cb1d9c3356d18f19d7b8eb8b0a15426bef32e515d0de0`.
|
||
|
||
Each schema assigns every path field a path class. Artifact, source, generated
|
||
output, install-destination, bundle-relative, sysroot-relative, and
|
||
vendor-relative paths use the relative rule above. Sandbox-execution paths are
|
||
path-independent absolute paths only in the closed virtual namespaces `/work`,
|
||
`/in`, `/out`, `/tool`, and `/data`; schema-1 action working directory is exactly
|
||
`/work`, and exact sandbox path spellings in argv are encoded. Platform-validated
|
||
target-runtime paths are a separate type and may be absolute in H's namespace.
|
||
Workspace locations and observation physical paths are separately typed and
|
||
never enter an artifact action record or key as host locations. Absolute host
|
||
paths, filesystem device/inode numbers, mtimes, cache locations, and command
|
||
process IDs are invalid in action templates and final action records. A physical
|
||
input enters only through a logical name, content digest, type, and semantic
|
||
mode.
|
||
|
||
### 6.2 Required action-record fields
|
||
|
||
Every record contains, in this order:
|
||
|
||
1. WWAR schema and action kind/version;
|
||
2. language edition, compiler protocol, export protocol, object ABI, runtime ABI,
|
||
manifest schema, and lock schema;
|
||
3. B, H, and optional T descriptor digests plus the expanded target fields;
|
||
4. toolchain identity, selected descriptor-slice/closure digest,
|
||
compiler/backend digest, and every executable/shared/resource digest actually
|
||
used by the action; the distribution bundle root/version authenticates
|
||
acquisition but unused targets/tools do not invalidate this action;
|
||
5. profile fields: optimization, debug, assertions, overflow, panic, sanitizers,
|
||
LTO, relocation/code model, symbol stripping, and reproducibility policy;
|
||
6. logical package/product/action identity and sandbox-virtual working directory
|
||
(schema 1 exactly `/work`);
|
||
7. exact argument vector and a sorted literal environment map;
|
||
8. byte-sorted named inputs, each with edge kind, logical path, semantic artifact
|
||
kind, semantic mode, content digest, and—where applicable—origin package identity;
|
||
9. direct export-data inputs byte-sorted by package identity for `ww.package`
|
||
actions;
|
||
10. selected source-membership list and target-selection explanation;
|
||
11. typed native declarations: headers, objects, archives, shared libraries,
|
||
sysroot, SDK, libc, CRT, dynamic loader, linker scripts and their include
|
||
closures, assembler/linker/archive tools, and ABI-provider slots;
|
||
12. the exact ordered link plan, retaining archive groups, whole-archive markers,
|
||
as-needed state, and repeated libraries;
|
||
13. named output paths, types, modes, and canonicalization policies; and
|
||
14. sandbox policy/version, resource bounds, and reproducibility classification.
|
||
|
||
Fields irrelevant to an action are encoded as empty values, not inferred. Native
|
||
flags exist only as typed fields whose meaning is part of a tool adapter. A raw
|
||
flag can be used only in a custom toolchain declaration and then its exact bytes
|
||
are part of the record; the official profile has no hidden default flags.
|
||
|
||
### 6.3 Environment and tool discovery
|
||
|
||
No inherited environment variable is semantic. Built-in actions receive the
|
||
fixed environment specified by their toolchain adapter. A declarative action
|
||
receives only its `env` record. `PATH`, compiler-driver defaults, `pkg-config`,
|
||
shell initialization, host include/library directories, and current directory
|
||
are never consulted to discover an input.
|
||
|
||
User configuration may choose a cache location, concurrency, output directory,
|
||
credential helper, source mirror, or display preference; these do not enter an
|
||
action because they cannot alter output bytes. Choosing a toolchain, target,
|
||
profile, workspace overlay, native provider, environment value, raw option, or
|
||
impure system mapping can alter bytes and therefore always enters the record.
|
||
|
||
### 6.4 Package action inputs and invalidation
|
||
|
||
A `ww.package` action consumes:
|
||
|
||
- the exact selected production or test source files and generated fragments;
|
||
- their ordered membership metadata;
|
||
- only the `.wwe` artifacts of direct imported packages;
|
||
- the compiler/backend and toolchain resources;
|
||
- B/H/T, target descriptor, profile, language/compiler/export/object/runtime
|
||
protocols, and manifest/lock schemas;
|
||
- package-specific native-provider declarations; and
|
||
- its literal built-in environment and sandbox policy.
|
||
|
||
It emits `.wwe`, one target object, and canonical link metadata. A private change
|
||
in a dependency changes that dependency's object and the final link key, but not
|
||
the importer's action key. A public change changes the dependency's `.wwe` and
|
||
therefore its direct importers. If a rebuilt importer emits byte-identical
|
||
`.wwe`, the invalidation stops there. Link-only input changes only invalidate
|
||
link/archive actions; materialization merely recopies a newly selected immutable
|
||
artifact when its requested result digest changes.
|
||
|
||
A package's own source bytes remain part of its action key even if a compiler
|
||
could prove a change dead. A target/profile/tool/runtime ABI change creates a
|
||
different key. There is no timestamp freshness shortcut and no “artifact exists”
|
||
predicate.
|
||
|
||
### 6.5 Link action inputs
|
||
|
||
The link action consumes every reachable package/native object digest, explicit
|
||
archive/shared-library digest, CRT object, runtime object, dynamic-loader choice,
|
||
linker script closure, sysroot descriptor, target/profile, exact linker tool and
|
||
resources, and the ordered plan. Objects are ordered by stable package identity;
|
||
an explicit static product's `members` list controls only that archive's member
|
||
order. Native archives retain declared
|
||
order; repeated archives remain repeated; group and whole-archive boundaries are
|
||
semantic. `-L`/`-l` token collections are not an internal representation.
|
||
|
||
The link result is cacheable. A warm identical build need not invoke the linker.
|
||
Changing output/materialization path alone does not change the link key. Changing
|
||
a private package implementation normally preserves importer objects but changes
|
||
the final link key through that package's object digest.
|
||
|
||
### 6.6 Complete cache-key formula
|
||
|
||
Let `R` be the complete WWAR record from sections 6.2–6.5. The action key is:
|
||
|
||
```text
|
||
K = SHA256("WW-ACTION-KEY\0" || uint64be(len(WWAR(R))) || WWAR(R))
|
||
```
|
||
|
||
Input entries contain the SHA-256 digest of their canonical artifact bytes/tree,
|
||
not merely the producer's action key. Thus semantically identical outputs stop
|
||
rebuild propagation even when their producing source/action key changed. Ordered
|
||
fields remain ordered; only fields specified as maps are sorted. The domain
|
||
separator and schemas prevent a digest from one protocol being reinterpreted in
|
||
another.
|
||
|
||
A successful result record contains `K`, result-schema version, output
|
||
name/type/mode/digest tuples only. Stdout, stderr, exit/diagnostic presentation,
|
||
timing, worker identity, resource use, and physical paths go in a separate
|
||
invocation-observation record. Thus one action key has exactly one semantic
|
||
successful result even if its logs differ. Observation records may be
|
||
content-addressed, but never participate in action mapping, cache hits, or
|
||
reproducibility comparison.
|
||
|
||
CAS identities are type-separated. Blob bytes use their own domain; every
|
||
structured object is additionally bound to its top-level record kind and schema:
|
||
|
||
```text
|
||
blob_id = SHA256("WW-BLOB\0" || u64be(length) || bytes)
|
||
|
||
record_id(kind, schema, record) =
|
||
SHA256("WW-RECORD\0" || u32be(kind) || u32be(schema) ||
|
||
u64be(length(WWAR(record))) || WWAR(record))
|
||
```
|
||
|
||
Schema 1 reserves these top-level `kind` numbers: 1 tree, 2 manifest, 3 lock,
|
||
4 workspace, 5 vendor index, 6 native map, 7 platform descriptor, 8 toolchain,
|
||
9 profile, 10 action template, 11 action record, 12 action result,
|
||
13 observation, 14 graph snapshot, 15 native sidecar, 16 native ABI contract,
|
||
17 link plan, 18 export, 19 package link metadata, 20 install manifest,
|
||
21 bootstrap plan, and 22 selected-tool closure. An unknown kind is never
|
||
decoded as another record. `TypedDigestV1` is a record with tag 1 domain
|
||
(`blob` or `record`), tag 2 algorithm (schema 1 only `sha256`), tag 3 record kind
|
||
(`0` for a blob), tag 4 record schema (`0` for a blob), and tag 5 the exact
|
||
32 digest bytes. A field whose type is “typed digest” always means this record;
|
||
a bare hexadecimal string is only a text rendering.
|
||
|
||
A tree record is schema 1 plus a list sorted by entry-name UTF-8 bytes. An entry
|
||
is `(name, kind=file|tree, executable:boolean, typed-child-digest)`. `name` is
|
||
one normalized path segment. Empty directories are explicit tree children;
|
||
symlinks, hardlink identity, devices, xattrs, uid/gid, mtimes, and non-executable
|
||
permission bits do not exist. Duplicate normalized or case-fold-colliding names
|
||
are errors. Validation recursively checks every typed child to a blob; verifying
|
||
only a root digest is insufficient.
|
||
|
||
### 6.7 Storage, sharing, and corruption
|
||
|
||
The default cache is per-user, global across that user's checkouts, local, and
|
||
private to the account:
|
||
|
||
```text
|
||
<cache>/v1/cas/sha256/aa/bb...
|
||
<cache>/v1/actions/sha256/aa/bb...
|
||
<cache>/v1/quarantine/
|
||
```
|
||
|
||
The first path stores blob objects and kind/schema-bound structured records; the
|
||
second maps an action key to `(action-record digest, result-object digest)`. A small ignored
|
||
project index `.ww/state-v1` references the previous successful graph snapshot,
|
||
whose logical nodes point to action-record/key/result digests. It roots that
|
||
history until replacement so explanation can compare records and follow causes;
|
||
it is disposable and never proves freshness. After explicit GC removes history,
|
||
`explain` reports `history-unavailable` rather than inventing “stale.” A
|
||
system-wide cache service requires authenticated isolated writers and the same
|
||
signed-mapping policy as a remote cache.
|
||
|
||
On every cache read, WW verifies the requested object's digest and canonical
|
||
type, decodes the mapped action record, recomputes `K` from it, and requires
|
||
`recomputed K = lookup K = ActionResultV1.tag2` plus
|
||
`ActionResultV1.tag3 = the mapping's typed action-record digest`. It then verifies
|
||
all output objects before use or materialization. A mismatch moves only that
|
||
explicit entry to quarantine, removes its action mapping, reports corruption,
|
||
and rebuilds.
|
||
`ww cache verify` walks the store; `ww cache gc` traces retained action results
|
||
and materializations. A tool upgrade changes its content/descriptor fields and
|
||
cannot reuse the old key.
|
||
|
||
If two executions of one `reproducibility=required` action key produce different
|
||
semantic result digests, WW publishes neither as an authoritative replacement,
|
||
records both observations/artifact sets in quarantine, and fails with a
|
||
nondeterminism diagnostic. Impure actions have no reusable action mapping.
|
||
|
||
Shared caches are opt-in explicit transports. `ww cache pull` imports only
|
||
content-addressed objects and action mappings in an Ed25519 signed envelope over
|
||
`"WW-CACHE-MAP\0"`, cache namespace, action key, action-record digest, result
|
||
digest/schema, and reproducibility/policy classification. The envelope carries
|
||
a signing-key ID; configured trust policy handles rotation/revocation. Hashes
|
||
prove bytes; the trusted cache signing key authorizes the asserted key-to-result
|
||
mapping. All hashes are reverified. An unsigned/untrusted mapping is treated as a miss
|
||
even if its referenced blobs exist. `ww cache push` refuses impure,
|
||
non-reproducible, secret-bearing, or policy-incompatible actions. Literal secret
|
||
environment values are forbidden; a required secret is a classified file input,
|
||
redacted from JSON/explain, and makes the action non-shareable. Ordinary build
|
||
does not contact a shared cache.
|
||
|
||
### 6.8 Explainability
|
||
|
||
For every node, WW retains its last local record and current record. `ww explain
|
||
NODE` reports one of `hit`, `not-built`, `missing-result`, `corrupt-result`,
|
||
`policy-rejected`, or `key-changed`. For `key-changed` it prints the first and,
|
||
with `--all`, every differing typed field, for example:
|
||
|
||
```text
|
||
codec/hex: key changed
|
||
input direct-export example.org/base: 71… -> a4…
|
||
caused by base: exported type Header layout changed
|
||
link hello: key changed
|
||
package-object example.org/codec/hex: 19… -> 27…
|
||
```
|
||
|
||
`ww explain --path NODE` follows the shortest changed-input path to a source,
|
||
tool, target, native provider, or policy root. `--format=json` exposes both WWAR
|
||
renderings and field paths. Export-data differences use the normative `ExportV1`
|
||
semantic field/type diff; if old content was explicitly GC'd the command reports
|
||
`history-unavailable`. Cache misses are never explained merely as “stale.”
|
||
|
||
### 6.9 Version-1 semantic record schemas
|
||
|
||
The following tables freeze schema-1 semantic fields and WWAR numeric tags.
|
||
`1` means exactly one, `0/1` optional, `*` a list, and `map` unique string keys.
|
||
Every absent optional value encodes the stated empty/default. Identity-keyed
|
||
lists are byte-sorted by identity; lists marked `ordered` preserve source/link
|
||
order. Nested records use the field tags in their named table. Enums reject
|
||
unknown values rather than passing them to a tool.
|
||
|
||
#### Project and distribution records
|
||
|
||
| Record/tag | Field | Type/cardinality | Rule/default |
|
||
|---|---|---|---|
|
||
| `ManifestV1/1` | schema | uint/1 | `1` |
|
||
| `/2` | module | string/1 | canonical module identity |
|
||
| `/3` | language | string/1 | language edition |
|
||
| `/4` | toolchain | `ToolchainRef`/1 | compatible ID and minimum |
|
||
| `/5` | requires | `Require`/* | sorted by module |
|
||
| `/6` | products | `Product`/* | sorted by name |
|
||
| `/7` | packages | `PackageConfig`/* | sorted by relative path |
|
||
| `/8` | actions | `GenerateDecl`/* | sorted by name |
|
||
| `/9` | natives | `NativeProvider`/* | sorted by name |
|
||
| `ToolchainRef/1` | id | string/1 | toolchain identity |
|
||
| `/2` | minimum | string/1 | minimum compatible SemVer |
|
||
| `Require/1` | module | string/1 | module identity |
|
||
| `/2` | minimum | string/1 | SemVer minimum |
|
||
| `/3` | source-index | string/0/1 | empty means HTTPS convention |
|
||
| `Product/1` | name | string/1 | unique identifier |
|
||
| `/2` | kind | enum/1 | `exe`, `static`, `shared`, `object`, `generated` |
|
||
| `/3` | root | string/0/1 | package-relative path; required except generated |
|
||
| `/4` | linkage | enum/1 | `dynamic`, `pie`, `static`, `static-pie`, `shared`, `none`; kind-valid |
|
||
| `/5` | runtime | string/1 | `hosted` default, `minimal`, `none`, or slot |
|
||
| `/6` | entry | string/0/1 | empty selects typed toolchain default |
|
||
| `/7` | native | string/* | required slots, sorted |
|
||
| `/8` | linker-script | `ArtifactRef`/0/1 | empty |
|
||
| `/9` | providers | `ProviderSelection`/* | sorted by slot |
|
||
| `/10` | members | string/* ordered | static/archive members; root only by default |
|
||
| `/11` | action | string/0/1 | required only for generated product |
|
||
| `/12` | install-name | string/0/1 | platform-validated, empty |
|
||
| `ProviderSelection/1` | slot | string/1 | ABI slot |
|
||
| `/2` | use | string/1 | `module#native-clause` |
|
||
| `PackageConfig/1` | path | string/1 | normalized relative path; `.` root |
|
||
| `/2` | generated | string/* | sorted `action:NAME:OUTPUT` refs |
|
||
| `/3` | native | string/* | sorted provider slots |
|
||
| `/4` | test-data | `InputDecl`/* | sorted names, read-only under `/data` |
|
||
| `/5` | test-env | map | literal non-secret test environment |
|
||
|
||
`ArtifactRefV1` is permitted in declarative configuration records (including
|
||
toolchain/native records) and action templates, but never in a final action
|
||
record. Its tags are: 1 namespace (`source`, `generated`, `package`, `toolchain`, `sysroot`,
|
||
`provider-output`, or `cas`); 2 owner identity (empty only for a root source);
|
||
3 normalized logical name/path; 4 semantic artifact kind (`file`, `tree`,
|
||
`object`, `archive`, `shared`, `import-library`, `script`, `crt`, `loader`,
|
||
`native-sidecar`, `native-abi`, `export`, or `package-link`);
|
||
5 optional expected `TypedDigestV1`; and 6 mode (`data` or `executable`). A local
|
||
source may omit tag 5 because analysis hashes it. A `cas`, external prebuilt,
|
||
toolchain, or sysroot reference must include it. A generated/package/provider
|
||
output gets its digest only from the declared predecessor output. Absolute host
|
||
paths are invalid.
|
||
|
||
`InputSlotRefV1`, `TemplateInputV1`, `ResolvedInputV1`, built-in/template/final
|
||
action-output records, and `ResultOutputV1` use that same closed semantic
|
||
artifact-kind enum; `GenerateDecl.OutputDecl` remains restricted to `file` or
|
||
`tree`. In schema 1, `file`, `object`, `archive`, `shared`, `import-library`,
|
||
`script`, `crt`, and `loader` require a blob digest. `tree` requires record kind
|
||
1, `native-sidecar` kind 15, `native-abi` kind 16, `export` kind 18, and
|
||
`package-link` kind 19, each at record schema 1. Any other digest domain, record
|
||
kind, or schema is invalid kind substitution.
|
||
|
||
`InputSlotRefV1` has tag 1 slot name and tag 2 expected semantic artifact type.
|
||
A final action record contains no `ArtifactRefV1`: every artifact-bearing field is recursively lowered
|
||
to an `InputSlotRefV1`. `TemplateInputV1` tags are 1 unique slot name, 2 edge
|
||
kind, 3 `ArtifactRefV1`, 4 expected semantic type, 5 semantic mode, and 6
|
||
optional origin package identity. `ResolvedInputV1` tags are 1 the same slot
|
||
name, 2 edge kind, 3 normalized logical name/path, 4 semantic artifact type, 5
|
||
semantic mode, 6 the resolved `TypedDigestV1`, and 7 optional originating
|
||
package identity. It contains no producer node, producer action key, action-
|
||
template digest, physical output path, or unresolved filesystem lookup. Thus all
|
||
content that a native plan, link plan, source-selection record, or tool closure can read is
|
||
also present exactly once in action-record tag 10 under a named slot.
|
||
|
||
`GenerateDecl` fields are fixed as follows: tag 1 name; 2 tool artifact/product
|
||
reference; 3 platform enum (schema 1 only `build`); 4 `TargetConstraint` or empty;
|
||
5 input map of `InputDecl`; 6 output map of `OutputDecl`; 7 ordered string argv;
|
||
8 literal string environment map; 9 `ResourcePolicy`; 10 reproducibility enum
|
||
`required` or `impure`. `InputDecl` is tag 1 kind (`file`, `tree`, `artifact`,
|
||
`tool`), 2 logical reference, 3 optional expected typed digest, 4 semantic mode.
|
||
`OutputDecl` is tag 1 kind (`file`, `tree`), 2 logical output path, 3 executable
|
||
boolean. `ResourcePolicy` is tags 1 max CPU count, 2 memory bytes, 3 output bytes,
|
||
4 process count; zero selects the toolchain's recorded bound, never “unlimited.”
|
||
|
||
| Record/tag | Field | Type/cardinality | Rule/default |
|
||
|---|---|---|---|
|
||
| `LockV1/1` | schema | uint/1 | `1` |
|
||
| `/2` | root-manifest | typed digest/1 | semantic manifest record |
|
||
| `/3` | toolchain | `ToolchainLock`/1 | exact closure |
|
||
| `/4` | modules | `ModuleLock`/* | sorted identity |
|
||
| `/5` | native-map | `LockedObject`/0/1 | empty |
|
||
| `ToolchainLock/1…5` | id, version, descriptor, bundle, origin | strings/digests | all required |
|
||
| `ModuleLock/1` | module | string/1 | identity |
|
||
| `/2` | version | string/1 | selected SemVer |
|
||
| `/3` | origin | string/1 | exact final archive URL |
|
||
| `/4` | archive | blob digest/1 | required |
|
||
| `/5` | tree | tree digest/1 | required |
|
||
| `/6` | manifest | record digest/1 | required |
|
||
| `/7` | signature | bytes/0/1 | empty |
|
||
| `/8` | provenance | string/0/1 | empty |
|
||
| `LockedObject/1…3` | origin, digest, signature | string/digest/bytes | origin+digest required |
|
||
|
||
`WorkV1` is tag 1 schema, tag 2 sorted `Use` records, tag 3 sorted local provider
|
||
overrides. `Use` tags are module, path, expected source-tree digest. A provider
|
||
override has slot, provider ID, product-platform constraint, contract digest,
|
||
artifact-tree digest, and provenance in tags 1–6. `VendorV1` is tag 1 schema,
|
||
tag 2 lock-record digest, tag 3 sorted entries `(module, version, source-tree
|
||
digest, vendor-relative path)` in tags 1–4. `NativeMapV1` is tag 1 schema, tag 2
|
||
exact product-platform descriptor digest, tag 3 sorted provider overrides, and
|
||
tag 4 signer/provenance record.
|
||
|
||
#### Profiles, templates, actions, results, and trees
|
||
|
||
A profile is toolchain data, not an open project map:
|
||
|
||
| Tag | `ProfileV1` field | Values |
|
||
|---|---|---|
|
||
| 1 | name | identity |
|
||
| 2 | optimization | `0`, `1`, `2`, `3`, `size` |
|
||
| 3 | debug | `none`, `line`, `full` |
|
||
| 4 | assertions | boolean |
|
||
| 5 | overflow | `trap`, `wrap` |
|
||
| 6 | panic | `abort`, `runtime` |
|
||
| 7 | sanitizers | sorted toolchain capability IDs |
|
||
| 8 | LTO | `none`, `thin`, `full` |
|
||
| 9 | relocation | effective `static`, `pic`, `pie` |
|
||
| 10 | code-model | exact target capability ID |
|
||
| 11 | TLS default | exact target capability ID |
|
||
| 12 | strip | `none`, `debug`, `all` |
|
||
| 13 | reproducibility | `required`, `impure` |
|
||
|
||
| Tag | `ActionTemplateV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | schema | `1` |
|
||
| 2 | kind/version | exact built-in kind or generate version |
|
||
| 3 | protocol record | language/compiler/export/object/runtime/manifest/lock; compiler protocol is distinct from compiler/backend byte identity |
|
||
| 4 | platform roles | B, H, optional T descriptor refs |
|
||
| 5 | selected tool closure | identity plus semantic closure digest/resources |
|
||
| 6 | profile | complete `ProfileV1` |
|
||
| 7 | logical identity/cwd | normalized identity; cwd exactly `/work`, path-independent |
|
||
| 8 | argv | ordered strings |
|
||
| 9 | environment | sorted literal map, no secrets |
|
||
| 10 | input slots | sorted `(name, edge kind, producer/output or source ref, type, mode)` |
|
||
| 11 | source selection | ordered selected membership plus reasons |
|
||
| 12 | native/link template | closed provider and ordered link records |
|
||
| 13 | outputs | sorted name/path/type/mode/canonicalization |
|
||
| 14 | sandbox/reproducibility | exact policy/version/bounds/class |
|
||
|
||
`ActionRecordV1` has the same top-level tags, but tag 10 contains sorted
|
||
`ResolvedInputV1` records and every `ArtifactRefV1` elsewhere in the template is
|
||
replaced by the corresponding `InputSlotRefV1`. The tag-12 native/link value is
|
||
therefore a resolved plan; it cannot contain an unresolved artifact reference,
|
||
producer, physical path, or readable artifact locator/digest outside tag 10.
|
||
ABI/layout/contract digests embedded in a referenced sidecar or contract are
|
||
semantic verification values, not authority to read another object. No producer
|
||
action key substitutes for a content digest.
|
||
|
||
Finalization interns every source, direct export, package/native object, archive,
|
||
shared library, header/sysroot tree, generated output, tool/resource, CRT,
|
||
loader, script, and init dispatcher into exactly one named template input.
|
||
Predecessor result digests resolve those slots lazily. The producer logical node,
|
||
producer output path, producer key, and template digest are graph/provenance
|
||
facts only and do not enter the consumer's `ActionRecordV1` or `K`. Two producers
|
||
that deliver the same typed bytes to the same semantic slot therefore produce
|
||
the same downstream record and key.
|
||
|
||
`ActionResultV1` tags are: 1 schema, 2 the 32-byte action key, 3 typed
|
||
action-record digest, and 4 sorted `ResultOutputV1` records. `ResultOutputV1`
|
||
tags are 1 unique output name, 2 semantic artifact type, 3 mode (`data` or
|
||
`executable`), and 4 `TypedDigestV1`. `ObservationV1` separately uses tags 1 schema,
|
||
2 logical invocation, 3 optional action key, 4 exit status/signal, 5 stdout blob,
|
||
6 stderr blob, 7 timing/resources, and 8 physical runner metadata; it is never
|
||
an action result.
|
||
|
||
`TreeV1` tags are 1 schema and 2 ordered entries. `TreeEntryV1` tags are 1 name,
|
||
2 kind (`file`, `tree`), 3 executable boolean (false for tree), and 4 typed child
|
||
digest. `GraphSnapshotV1` tags are 1 schema, 2 logical root, 3 sorted
|
||
`GraphNodeV1` records, and 4 sorted `GraphEdgeV1` records. `GraphNodeV1` tags are
|
||
1 logical node ID, 2 typed action-template digest, 3 optional 32-byte action key,
|
||
4 optional typed action-record digest, and 5 optional typed action-result digest.
|
||
`GraphEdgeV1` tags are 1 consumer node ID, 2 consumer input slot, 3 producer node
|
||
ID, 4 producer output name, and 5 edge kind; edges sort by that five-field tuple.
|
||
Non-action source inputs live only in the consumer template rather than invented
|
||
graph nodes. The project index contains only its typed graph-snapshot digest.
|
||
|
||
#### Target, toolchain, native, interface, and handoff records
|
||
|
||
| Tag | `PlatformDescriptorV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | schema | `1` |
|
||
| 2 | name | canonical lookup name |
|
||
| 3 | arch/vendor/os/environment/object format | five exact enums |
|
||
| 4 | endian/address spaces/pointers | complete integer-width map |
|
||
| 5 | integer/data-layout | widths, alignments, aggregate algorithm |
|
||
| 6 | C ABI/data model | exact IDs and calling-convention table |
|
||
| 7 | float/variadic/name decoration | exact ABI records |
|
||
| 8 | CPU baseline/features/atomics | baseline plus required/forbidden sets |
|
||
| 9 | minimum OS/SDK | typed version record |
|
||
| 10 | TLS/unwind capabilities | sets plus defaults |
|
||
| 11 | relocation/code capabilities | supported sets plus defaults |
|
||
| 12 | executable/shared/page rules | typed object-format rules |
|
||
| 13 | hosted policy | hosted/freestanding plus capability set |
|
||
| 14 | object/runtime ABI protocols | exact IDs |
|
||
|
||
`TargetConstraintV1` tags 1–17 are, respectively: optional exact descriptor
|
||
digest; arch; vendor; OS; environment; object format; hosted; C ABI; data model;
|
||
float ABI; CPU baseline; required feature set; forbidden feature set; minimum
|
||
SDK; relocation; code model; PIC requirement. Empty scalar/set means no
|
||
constraint. Matching is exactly section 8.4; no expression field exists.
|
||
|
||
`ToolchainV1` tags are: 1 schema; 2 ID; 3 version; 4 the same complete protocol
|
||
record used by action tag 3; 5 sorted `Tool` records; 6 sorted platform
|
||
descriptors; 7 sorted profiles; 8 sorted link policies; 9 runtime/provider
|
||
records; 10 bundle tree digest/signature provenance.
|
||
A `Tool` is `(name, bundle-relative path, executable blob digest, ordered dynamic
|
||
tool dependencies, resource-tree digests, adapter record)` tags 1–6. A link
|
||
policy is `LinkPolicyV1`: tag 1 product-platform descriptor; 2 product kind; 3
|
||
linkage; 4 profile constraint; 5 runtime selector (`hosted`, `minimal`, `none`,
|
||
or an exact provider slot); 6 one ordered link-policy token template; and 7
|
||
output ABI/install policy. Policies sort by the five-field selection key and a
|
||
zero/multiple match is an error. CRTs and scripts are `ArtifactRefV1` tokens,
|
||
not basenames. A dynamic loader is one restricted `dynamic-loader` token holding
|
||
both its artifact and runtime interpreter path; PE/COFF platform-image policy is
|
||
an ordered provider token rather than a fabricated loader artifact.
|
||
`SelectedToolClosureV1` deterministically
|
||
projects only the relevant tools/resources/platform/profile/policy into tags
|
||
1–7; that projection—not unrelated bundle members—is action-key input.
|
||
|
||
#### Native artifact and ABI subrecords
|
||
|
||
Compact manifest paths are lowered to `ArtifactRefV1` before WWAR encoding and
|
||
then to action input-slot references before execution. The native records are:
|
||
|
||
| Tag | `IncludeTreeRefV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | tree | `ArtifactRefV1` of kind `tree` |
|
||
| 2 | class | `quote`, `user`, `system`, or `framework` |
|
||
| 3 | subdirectory | normalized tree-relative path; `.` default |
|
||
|
||
The provider's include list is ordered because header search order is semantic.
|
||
The same tree may occur more than once with another class or subdirectory.
|
||
|
||
| Tag | `NativeSourceV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | name | unique provider-local identity |
|
||
| 2 | source | `ArtifactRefV1` of kind `file` |
|
||
| 3 | language | exact toolchain capability ID, such as `c11` or `gnu-assembly` |
|
||
| 4 | preprocessing | `none` or `c-preprocessor` |
|
||
| 5 | include-indices | ordered indexes into the provider include list; empty means all |
|
||
| 6 | defines | sorted literal macro map; duplicates with provider defines error |
|
||
|
||
Target, profile, relocation/PIC/code/TLS policy, tool, and dialect adapter come
|
||
from the enclosing `native.compile` record. Raw source flags do not exist.
|
||
|
||
| Tag | `PrebuiltObjectV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | name | unique provider-local identity |
|
||
| 2 | object | `ArtifactRefV1` of kind `object` |
|
||
| 3 | sidecar | `ArtifactRefV1` of kind `native-sidecar` |
|
||
| 4 | contract | `ArtifactRefV1` of kind `native-abi` |
|
||
|
||
| Tag | `ArchiveV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | name | unique provider-local identity |
|
||
| 2 | archive | `ArtifactRefV1` of kind `archive` |
|
||
| 3 | sidecar | archive-level `ArtifactRefV1` of kind `native-sidecar` |
|
||
| 4 | members | ordered `ArchiveMemberV1` list in physical order |
|
||
| 5 | contract | `ArtifactRefV1` of kind `native-abi` |
|
||
|
||
`ArchiveMemberV1` tags are 1 member name, 2 member blob `TypedDigestV1`, and 3
|
||
object-sidecar `ArtifactRefV1`. Duplicate names are legal only at distinct
|
||
positions; member order is never sorted. The referenced archive sidecar records
|
||
the ordered member-sidecar record digests as well as the physical member facts.
|
||
|
||
| Tag | `SharedImportLibraryV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | name | unique provider-local identity |
|
||
| 2 | kind | `elf-shared`, `macho-dylib`, or `pe-import` |
|
||
| 3 | link-artifact | shared object/dylib/import-library `ArtifactRefV1` |
|
||
| 4 | link-sidecar | `ArtifactRefV1` of kind `native-sidecar` for tag 3 |
|
||
| 5 | runtime-identity | exact SONAME, install-name, or DLL name |
|
||
| 6 | runtime-artifact | exact deployable shared object/dylib/DLL `ArtifactRefV1` |
|
||
| 7 | runtime-sidecar | `ArtifactRefV1` of kind `native-sidecar` for tag 6 |
|
||
| 8 | contract | `ArtifactRefV1` of kind `native-abi` |
|
||
| 9 | runtime-requires | sorted `NativeRuntimeRequirementV1` list |
|
||
|
||
ELF and Mach-O tags 3 and 6 may resolve to the same bytes. For PE, tag 3 is
|
||
the import library and tag 6 its matching DLL. A platform image still supplies
|
||
tag 6 as a content-identified artifact within that image.
|
||
|
||
`NativeProviderV1` tags are therefore: 1 name; 2 provided slot; 3
|
||
`TargetConstraintV1`; 4 ordered `IncludeTreeRefV1`; 5 ordered `NativeSourceV1`;
|
||
6 sorted provider define map; 7 ordered `PrebuiltObjectV1`; 8 ordered
|
||
`ArchiveV1`; 9 ordered `SharedImportLibraryV1`; 10 sorted required slots; 11
|
||
ordered link-token templates; and 12 an `ArtifactRefV1` of kind `native-abi`.
|
||
Every ABI contract and sidecar is an independently encoded, content-addressed
|
||
record. In a resolved action, those records and every artifact field above are
|
||
`InputSlotRefV1` values; source/action/provider output digests live only in
|
||
action tag 10. A sidecar's internal artifact digest must equal the corresponding
|
||
object/archive/shared input-slot digest, and its contract digest must equal the
|
||
kind-16 `TypedDigestV1` of the referenced native-ABI input record.
|
||
Section/layout/provenance digests inside a sidecar are verification facts, not
|
||
locators from which the action may read undeclared content.
|
||
|
||
| Tag | `NativeABIContractV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | schema | `1` |
|
||
| 2 | slot | exact ABI-provider slot |
|
||
| 3 | platform | `NativeABIPlatformV1` |
|
||
| 4 | symbols | sorted `NativeSymbolContractV1` list |
|
||
| 5 | types | sorted `NativeTypeContractV1` list |
|
||
| 6 | runtime-requires | sorted `NativeRuntimeRequirementV1` list |
|
||
| 7 | features | `NativeFeatureContractV1` |
|
||
| 8 | minimum-platform | optional `PlatformVersionV1` |
|
||
| 9 | code | `NativeCodeContractV1` |
|
||
|
||
Its typed record identity is `record_id(16, 1, contract)` as defined in section
|
||
6.6. Every schema-1 digest identifying a complete `NativeABIContractV1` is the
|
||
corresponding record-domain, kind-16, schema-1 `TypedDigestV1`. WW computes it; a
|
||
supplied digest is never accepted in place of the record. Subordinate layout,
|
||
calling-convention, type-contract, and header-contract digests remain their
|
||
separately specified semantic values.
|
||
|
||
| Tag | `NativeABIPlatformV1` field | Rule |
|
||
|---|---|---|
|
||
| 1 | object-format | exact format capability ID |
|
||
| 2 | object-class | exact class/word-size ID |
|
||
| 3 | endian | `little` or `big` |
|
||
| 4 | machine-ABI | exact architecture object ABI ID |
|
||
| 5 | C-ABI | exact C ABI ID |
|
||
| 6 | data-model | exact data-model ID |
|
||
| 7 | data-layout | canonical layout digest |
|
||
| 8 | calling-conventions | canonical convention-table digest |
|
||
| 9 | float-ABI | exact ID |
|
||
| 10 | variadic-ABI | exact ID |
|
||
| 11 | symbol-ABI | exact decoration/versioning ABI ID |
|
||
| 12 | object-ABI | exact object protocol ID |
|
||
| 13 | runtime-ABI | required WW runtime ABI ID or empty |
|
||
|
||
`NativeSymbolContractV1` tags are: 1 exact external name; 2 exact version or
|
||
empty; 3 kind (`function`, `data`, `tls`, `ifunc`); 4 role (`define`, `require`);
|
||
5 binding (`strong`, `weak`); 6 visibility (`default`, `protected`, `hidden`);
|
||
7 calling-convention ID or empty; 8 canonical function/object type-contract
|
||
digest; and 9 optional byte size. Symbols sort by `(name,version,kind,role)`;
|
||
duplicate keys error.
|
||
|
||
`NativeTypeContractV1` tags are: 1 stable binding/header-qualified identity; 2
|
||
kind (`opaque`, `scalar`, `enum`, `struct`, `union`, `function`); 3 exposure
|
||
(`opaque`, `layout`); 4 canonical target-specific layout/signature digest; 5
|
||
optional size; 6 optional alignment; and 7 optional canonical header/macro
|
||
contract digest. Types sort by identity and duplicates error.
|
||
|
||
`NativeRuntimeRequirementV1` tags are 1 provider slot, 2 required ABI-contract
|
||
digest, 3 phase (`link`, `load`, `both`), and 4 optional SONAME/install-name/DLL
|
||
identity. They sort by `(slot,phase,runtime-identity)`; conflicting requirements
|
||
for one slot error. `NativeFeatureContractV1` tags are 1 CPU baseline or empty,
|
||
2 sorted required feature IDs, 3 sorted forbidden feature IDs, and 4 sorted
|
||
atomic-capability IDs; required and forbidden sets must be disjoint.
|
||
|
||
`PlatformVersionV1` tags are 1 version-family ID and unsigned 2 major, 3 minor,
|
||
4 patch, 5 revision. Versions compare lexicographically over tags 2–5 only after
|
||
tag 1 equality. `NativeCodeContractV1` tags are 1 PIC (`any`, `required`,
|
||
`forbidden`); 2 sorted TLS-model IDs; 3 unwind ABI ID or `none`; 4 sorted
|
||
personality/runtime symbols; and 5 sorted required/forbidden relocation records,
|
||
each record being tag 1 capability ID and tag 2 requirement (`required` or
|
||
`forbidden`).
|
||
|
||
`NativeSidecarV1` is evidence, not a second contract. Its tags are: 1 schema; 2
|
||
artifact `TypedDigestV1`; 3 evidenced `NativeABIPlatformV1`; 4 sorted evidenced
|
||
`SectionFactV1`; 5 sorted evidenced `SymbolFactV1`; 6 sorted evidenced
|
||
`RelocationFactV1`; 7 evidenced sorted architecture attribute/notes map; 8
|
||
evidenced `NativeMachineFactsV1`; 9 native ABI-contract digest; 10 sorted
|
||
`NativeRuntimeRequirementV1`; and 11 `ProvenanceV1`. An evidenced value is
|
||
`EvidenceV1`: tag 1 enum (`inspected` or `declared`) and tag 2 the value whose
|
||
type is fixed by the containing field. `NativeMachineFactsV1` tags are 1
|
||
`NativeFeatureContractV1` and 2 `NativeCodeContractV1`.
|
||
|
||
`SectionFactV1` tags are 1 name, 2 format type, 3 flag set, 4 size, 5 alignment,
|
||
6 optional content digest. `SymbolFactV1` tags are the nine
|
||
`NativeSymbolContractV1` fields plus tag 10 section and tag 11 value/offset.
|
||
`RelocationFactV1` tags are 1 section, 2 offset, 3 exact relocation ID, 4 symbol,
|
||
5 signed addend encoded as `(negative:boolean,magnitude:uint)`, and 6 target
|
||
section. `ProvenanceV1` tags are 1 producer/tool typed digest, 2 source/build
|
||
record typed digest, 3 attestation bytes, and 4 signer/policy ID. A frozen opaque
|
||
input requires accepted attestation for every `declared` value.
|
||
|
||
`LinkTokenTemplateV1` and final `LinkTokenV1` share tags: 1 kind; 2 artifact; 3
|
||
provider slot; 4 string value; 5 enabled boolean; 6 validated path; 7 runtime
|
||
path; and 8 ordered nested tokens. Tag 2 is `ArtifactRefV1` in a template and
|
||
`InputSlotRefV1` in a final action. Valid nonempty combinations are exactly:
|
||
object/archive/shared/linker-script/version-script/export-map/def-file use tag 2;
|
||
the template-only provider kind uses tag 3; group/whole use tag 8; as-needed uses tags 5 and 8;
|
||
runtime-search uses tags 4 and 6; install-name, entry, and retain use tag 4; and
|
||
dynamic-loader uses tags 2 and 7. Every other field encodes its empty/default
|
||
value. A final `LinkTokenV1` forbids `provider`; provider selection and its
|
||
ordered fragment are recursively expanded at that exact position before the
|
||
plan is final. Section 8.6 constrains valid nesting. A toolchain `LinkPolicyV1`
|
||
template additionally permits `splice` (tag 4 is exactly `product-objects`,
|
||
`init-dispatch`, `native-providers`, or `product-controls`) and
|
||
`script-slot` (tag 2 is an optional policy-default script). Those two kinds are
|
||
forbidden in project/native link declarations and in a final plan.
|
||
|
||
The selected policy template is flattened in list order. Each splice kind occurs
|
||
exactly once and expands to its already computed ordered product list;
|
||
the single required `script-slot` becomes one `linker-script` at the same list
|
||
position using the product's explicit script when present, otherwise the policy
|
||
default, and disappears only when both are empty. A product script therefore
|
||
replaces, never combines with, the default. Ordinary policy tokens—including CRT
|
||
objects, compiler runtime, dynamic loader, and system-provider slots—stay exactly
|
||
where declared. System providers are ordered template tokens, not a sorted set.
|
||
Each expands to its selected provider's concrete token fragment; dependencies
|
||
expand recursively, and a provider cycle is an analysis error. Every expanded
|
||
artifact, sidecar, and ABI contract is a tag-10 input. A provider such as a PE
|
||
platform image that adds no linker token instead contributes its resolved
|
||
contract slot to the plan's non-link policy field. After expansion no provider
|
||
or splice partition remains.
|
||
|
||
`LinkPlanV1` tags are 1 schema; 2 the exact product-platform/kind/linkage/
|
||
profile/runtime-policy selection record; 3 the single fully expanded ordered
|
||
`LinkTokenV1` stream; 4 selected linker/tool/resource slot refs; 5 selected
|
||
non-link platform/runtime/provider ABI-contract and sidecar slot refs; and 6
|
||
output ABI/install policy. Every file-bearing token and verification record
|
||
references action tag 10. Only
|
||
this resolved flattened plan enters the link action key.
|
||
|
||
`ExportV1` (the WWAR body after `.wwe` magic) tags are: 1 schema; 2 reader
|
||
capabilities; 3 language/type protocol; 4 product-platform/C/object/runtime ABI;
|
||
5 package identity/name; 6 sorted exported-surface origin/type contract table; 7
|
||
canonical type graph; 8 sorted exported declarations/constants/foreign symbols;
|
||
9 public initialization/ABI facts; 10 public-type digest; 11 public-ABI digest. Section 7.2
|
||
defines excluded non-semantic fields. `PackageLinkV1` (`.wwlm`) tags are 1 schema,
|
||
2 package identity, 3 platform/object/runtime ABI, 4 package-object digest, 5
|
||
defined/required foreign symbols, 6 predeclared provider slots/contracts, 7 init
|
||
symbol/dependency facts, 8 install/link requirements. It cannot add a graph edge.
|
||
|
||
`InstallManifestV1` tags are 1 schema, 2 product identity/key, 3 sorted entries
|
||
`(artifact digest, mode, relative destination)`, 4 runtime-resolution policy, 5
|
||
sorted shared/runtime closure, 6 target/toolchain/ABI provenance. Absolute prefix
|
||
is deliberately absent. `BootstrapPlanV1` tags are 1 schema, 2 bootstrap-host
|
||
contract, 3 ordered source refs/digests, 4 portable-C compiler/output settings,
|
||
5 recorded host-C command/tool closure, 6 stage-1 outputs, 7 production toolchain
|
||
closure, 8 identical logical stage-2/3/4 action roots, 9 semantic fixed-point
|
||
output roles, and 10 raw-record fixed-point roles. No clause or executable step
|
||
exists beyond that closed plan.
|
||
|
||
These sections fix the architectural fields and every reference/key
|
||
transformation. Phase 0 transcribes every named and nested record into the
|
||
normative checked-in `protocol/wwar-v1.idl`, assigns any remaining mechanical
|
||
numeric enum values and field tags, and freezes its digest before Phase 1. The
|
||
IDL must state every cardinality, default, ordering rule, enum, discriminator,
|
||
cross-field constraint, and top-level record kind; generated codecs/tables and
|
||
golden vectors derive from it. The IDL, WWAR 1, and those vectors jointly
|
||
determine bytes. Phase 0 may not add a semantic field, change a default/order
|
||
rule, retain producer identity in a final action, or alter action-key behavior.
|
||
|
||
## 7. Interface and artifact protocol
|
||
|
||
### 7.1 Package outputs
|
||
|
||
Every `ww.package` action, including a root package, emits the same three named
|
||
artifacts:
|
||
|
||
```text
|
||
export.wwe deterministic binary export data
|
||
package.o one target object
|
||
link.wwlm deterministic package link metadata
|
||
```
|
||
|
||
These are names inside an immutable action result, not globally meaningful
|
||
filenames. The cache is keyed by digests and logical package identity, so there
|
||
is no `__root` special case and no dotted import path used as an artifact
|
||
basename. A root object and dependency object obey identical protocols.
|
||
|
||
`link.wwlm` declares the package object's target, object ABI, defined/required
|
||
foreign symbols, required native-provider slots, initialization ordering, and
|
||
runtime ABI. It attests facts/edges already present in the action-template DAG;
|
||
it may not introduce a provider, dependency, or action after compilation, and a
|
||
mismatch is a compiler/build-protocol error. It does not contain raw linker flags. Ordinary package objects are
|
||
fed directly to the product link. An archive exists only when an explicit
|
||
static-library product asks the `archive` action to combine its ordered declared
|
||
`members` (default: root package only). Transitive package/native dependencies
|
||
remain typed link requirements in the library's install manifest and are not
|
||
silently copied into multiple archives. A deliberately self-contained archive
|
||
must list every member explicitly and pass duplicate-symbol/provider checks.
|
||
|
||
Language initialization never depends on linker input order. `ww.init` consumes
|
||
all reachable `.wwlm` artifacts, topologically orders initialization by package
|
||
imports with byte-sorted ties, diagnoses duplicate/cyclic init facts, and emits
|
||
one dispatcher object. The link plan explicitly roots that dispatcher and every
|
||
referenced init symbol against section garbage collection.
|
||
|
||
### 7.2 WW Export Data 1 (`.wwe`)
|
||
|
||
`.wwe` is a cache/build protocol, not source text and not a long-term binary
|
||
distribution promise. It starts with the eight-byte magic `WWEX\0\0\0\1` and a
|
||
WWAR-encoded body. The magic is fixed framing that is reconstructed and verified
|
||
around the body; it does not create a second blob identity. The typed `.wwe`
|
||
identity is `record_id(18, 1, ExportV1)` over that body. The body contains:
|
||
|
||
1. export schema, language edition, type-system protocol, and required reader
|
||
capabilities;
|
||
2. target descriptor, C ABI, object ABI, and runtime ABI digests;
|
||
3. full package identity and declared package name;
|
||
4. a sorted table of originating package/type identities and declaration-level
|
||
public ABI digests actually referenced by the exported surface;
|
||
5. a canonical type graph sufficient for type checking, layout, calling
|
||
convention, and code generation of every exported declaration;
|
||
6. exported constants, variables, functions, methods, types, and explicit
|
||
foreign symbols; and
|
||
7. the public initialization/ABI facts needed by an importer.
|
||
|
||
Declarations are sorted by `(kind, exported name, stable overload discriminator)`;
|
||
type-graph nodes are assigned by deterministic structural traversal. Integer and
|
||
floating constants use canonical target-independent bit encodings until a
|
||
target conversion is part of their type. Function parameter **names**, source
|
||
locations, comments, unused imports, private function bodies, and declaration
|
||
order are not semantic and are omitted. Documentation/source mapping is a
|
||
separate optional artifact and cannot invalidate an importer.
|
||
|
||
Unmanaged layout sometimes depends on facts that are private at the source
|
||
level. An exported representation therefore records size, alignment, field/base
|
||
offsets, calling convention, niche/tag rules, and relevant private padding or
|
||
opaque-field descriptors without exposing private names. `@repr("c")` types
|
||
also record the exact C data model and layout algorithm version. Opaque types
|
||
record only the operations and layout promises permitted to clients.
|
||
|
||
### 7.3 Direct imports with deep public closure
|
||
|
||
An importer opens one `.wwe` for each direct import and no transitive interface
|
||
file. If a direct dependency's API mentions a type originating in a deeper
|
||
package, its `.wwe` embeds a canonical deep descriptor for the portion of that
|
||
type required to understand and lay out the direct API. The descriptor retains
|
||
the originating package/type identity and digest of that exact referenced
|
||
contract—not the originating package's entire public ABI. It does not pretend
|
||
the type belongs to the middle package.
|
||
|
||
This rule gives both correctness and bounded reads:
|
||
|
||
```text
|
||
source/package imports -> direct .wwe inputs
|
||
direct .wwe -> complete meaning of that direct API
|
||
link graph -> all reachable package objects
|
||
```
|
||
|
||
A public change in a leaf rebuilds direct reverse dependencies. Propagation
|
||
continues only while each rebuilt package's `.wwe` bytes change. A private leaf
|
||
change never enters an importer key. This replaces source-like transitive
|
||
interface prepending and its quadratic composed units.
|
||
|
||
This bounds interface **opens and reparsing**, not necessarily total descriptor
|
||
bytes: heavily re-exported type graphs can duplicate deep descriptors. Phase 1
|
||
measures total `.wwe` size and structural duplication on the real library graph.
|
||
Only if that is material may a later export-schema revision intern immutable
|
||
per-declaration descriptors; direct-import semantics do not change.
|
||
|
||
### 7.4 Public and ABI digests
|
||
|
||
The `.wwe` content digest identifies its complete target-specific bytes. It also
|
||
contains two domain-separated hashes:
|
||
|
||
- **public type digest** over names, types, constants, visibility, and language
|
||
semantics; and
|
||
- **public ABI digest** over target layouts, calling conventions, exported
|
||
symbol contracts, runtime ABI, and representation facts.
|
||
|
||
Compile actions normally depend on the whole `.wwe` content digest. Tools such as
|
||
documentation may depend only on the public type digest when their action kind
|
||
explicitly permits it. Link compatibility checks use the ABI digest. Digest
|
||
subsetting is protocol-defined; callers cannot choose arbitrary ignored fields.
|
||
|
||
### 7.5 Symbols and identity
|
||
|
||
Internal WW symbols are mangled from a protocol-versioned hash of the full
|
||
package identity plus declaration identity, never from a leaf name or artifact
|
||
filename. Resolver rules ensure only one source/version supplies that package
|
||
identity. An explicitly foreign symbol is exactly the source-declared spelling
|
||
and participates in duplicate-provider checks.
|
||
|
||
Package version and distribution origin are absent from mangling because they
|
||
are not identity. An incompatible major version has a different `/vN` module
|
||
identity and therefore different WW symbols. Native C symbols do not gain this
|
||
protection; their provider slots and link collision rules must reject
|
||
incompatible co-selection.
|
||
|
||
### 7.6 Compatibility and deterministic serialization
|
||
|
||
A consumer accepts only the exact export/type/object/runtime protocol combination
|
||
declared compatible by its immutable toolchain descriptor. A new optional record
|
||
still requires a new export schema and reader capability. Unknown records are
|
||
not silently dropped. Target descriptor and ABI mismatches are errors before a
|
||
compiler or linker runs.
|
||
|
||
Because `.wwe` is rebuilt from locked source, WW does not need an indefinitely
|
||
stable compiler-internal export format. A toolchain upgrade changes action keys
|
||
and may rebuild the graph. Public native-library ABI stability is a separate,
|
||
explicit provider contract. Release/bootstrap byte comparisons include `.wwe`,
|
||
objects, link metadata, action records, and executables.
|
||
|
||
## 8. Native integration and cross compilation
|
||
|
||
### 8.1 Complete target descriptors
|
||
|
||
A target triple is a user-facing alias. Before graph construction it expands to
|
||
an immutable target descriptor containing at least:
|
||
|
||
- architecture, vendor, operating system, environment, and object format;
|
||
- endianness, pointer widths/address spaces, integer/long widths, alignment and
|
||
aggregate-layout rules;
|
||
- C ABI/data model, calling conventions, name decoration, variadic convention,
|
||
TLS ABI capabilities/default, floating ABI, and unwind model;
|
||
- baseline CPU, required/forbidden CPU features, minimum OS/SDK version, and
|
||
atomic capability;
|
||
- supported/default relocation and code models, executable/shared-library rules,
|
||
and page constraints;
|
||
- hosted versus freestanding policy; and
|
||
- compatible object ABI and runtime ABI protocol identifiers.
|
||
|
||
The triple `x86_64-unknown-linux-gnu` is insufficient by itself to identify CPU
|
||
features, sysroot, glibc, loader, or linker. Those are separate descriptor/input
|
||
digests. `--cpu` and `--feature` produce a new canonical expanded descriptor;
|
||
the host CPU is never probed to select target features unless the user explicitly
|
||
requests the impure alias `native`, which is rejected by frozen/shared builds.
|
||
|
||
The descriptor supplies ABI invariants plus supported/default policy values. A
|
||
profile/product selects the effective relocation, code, PIC/PIE, and TLS policy
|
||
from those permitted sets; the normalized effective values appear once in the
|
||
action record. A conflicting or unsupported selection is rejected, never
|
||
resolved by precedence between duplicate fields.
|
||
|
||
This follows the native facts exposed by LLVM data layouts and Clang's cross
|
||
compilation/toolchain documentation, while making their often-driver-selected
|
||
inputs explicit
|
||
([LLVM data layout](https://llvm.org/docs/LangRef.html#data-layout),
|
||
[Clang cross compilation](https://clang.llvm.org/docs/CrossCompilation.html),
|
||
[Clang toolchain](https://clang.llvm.org/docs/Toolchain.html)).
|
||
|
||
### 8.2 Toolchain closure (`toolchain.wwt`)
|
||
|
||
An immutable toolchain bundle has a canonical `toolchain.wwt` descriptor with:
|
||
|
||
```text
|
||
ww-toolchain 1
|
||
id = "ww.org/toolchain"
|
||
version = "v1.4.0"
|
||
protocols = { language = "1", compiler = "1", export = "1", object = "1",
|
||
runtime = "1", manifest = 1, lock = 1 }
|
||
|
||
tool "wwc" { path = "bin/wwc", digest = "sha256:..." }
|
||
tool "cc" { path = "bin/clang", digest = "sha256:..." }
|
||
tool "as" { path = "bin/llvm-mc", digest = "sha256:..." }
|
||
tool "ld" { path = "bin/ld.lld", digest = "sha256:..." }
|
||
tool "archive" { path = "bin/llvm-ar", digest = "sha256:..." }
|
||
|
||
target "aarch64-unknown-linux-gnu" {
|
||
descriptor = "targets/aarch64-linux-gnu.wwt"
|
||
sysroot = { tree = "sha256:...", path = "sysroots/aarch64-linux-gnu" }
|
||
runtime = "runtime:aarch64-linux-gnu@1"
|
||
libc = "c:glibc@2.39"
|
||
link-policies = [
|
||
{ kind = "exe", linkage = "dynamic", profile = ["debug", "release"],
|
||
runtime = "hosted",
|
||
tokens = [
|
||
{ kind = "object", artifact = "sysroot:lib/crt1.o" },
|
||
{ kind = "object", artifact = "sysroot:lib/crti.o" },
|
||
{ kind = "splice", value = "product-objects" },
|
||
{ kind = "splice", value = "init-dispatch" },
|
||
{ kind = "splice", value = "native-providers" },
|
||
{ kind = "splice", value = "product-controls" },
|
||
{ kind = "provider", slot = "c:compiler-rt@1" },
|
||
{ kind = "provider", slot = "c:glibc@2.39" },
|
||
{ kind = "dynamic-loader",
|
||
artifact = "sysroot:lib/ld-linux-aarch64.so.1",
|
||
runtime-path = "/lib/ld-linux-aarch64.so.1" },
|
||
{ kind = "script-slot", artifact = "toolchain:lib/ldscripts/elf.lds" },
|
||
{ kind = "object", artifact = "sysroot:lib/crtn.o" }
|
||
] }
|
||
]
|
||
}
|
||
```
|
||
|
||
Every path is bundle-relative and every executable, shared tool dependency,
|
||
resource directory, built-in header tree, target descriptor, runtime, sysroot,
|
||
CRT, default script, and adapter is covered by the bundle's canonical tree
|
||
digest. The engine invokes exact paths and passes explicit target/sysroot/resource
|
||
arguments. A tool's compiled-in search outside the sandbox cannot resolve.
|
||
The compact `sysroot:`/`toolchain:` references in the example normalize to full
|
||
`ArtifactRefV1` records containing the individual typed digest obtained from
|
||
that authenticated tree; the shorthand itself never enters WWAR.
|
||
|
||
The whole bundle digest authenticates acquisition. An action key uses the
|
||
transitive selected descriptor slice/tool/resource/sysroot/runtime closure only;
|
||
adding an unused target or unrelated tool to a republished bundle does not cause
|
||
global recompilation. Changing any selected byte/protocol still changes the key.
|
||
|
||
Official bundles may use LLVM, GNU binutils, or another implementation per
|
||
target; the architecture does not expose that choice as project semantics.
|
||
Tool adapters translate WW's typed record to exact argv and declare all injected
|
||
inputs. A custom bundle must do the same and pass conformance/reproducibility
|
||
tests before frozen mode accepts it.
|
||
|
||
WW permanently owns the language compiler, export/object/runtime ABI protocols,
|
||
target descriptor schema, action engine, and official bundle definitions. It
|
||
does **not** permanently own the assembler, linker, archiver, C compiler, or SDK.
|
||
The current `w6a`/`w6l` may serve as migration inputs, then are removed once a
|
||
pinned external closure passes parity. This is the smaller long-term system.
|
||
|
||
### 8.3 C ABI and foreign declarations
|
||
|
||
Foreign declarations are explicit source contracts:
|
||
|
||
```ww
|
||
@abi("c") @symbol("write")
|
||
@provider("c:libc")
|
||
fn c_write(fd s32, data *u8, count usize) ssize;
|
||
|
||
@repr("c")
|
||
type Header struct { tag u32; length u16; };
|
||
```
|
||
|
||
The compiler checks that every type has a defined representation for the
|
||
selected C ABI and records symbol, calling convention, variadic status, layout,
|
||
and explicitly named provider slot in `.wwe`/`.wwlm`. `@provider` is mandatory
|
||
for each foreign declaration (a group annotation may supply it lexically), and
|
||
the owning package metadata must require that slot; WW never infers it from a
|
||
symbol spelling or link position. A foreign declaration with no provider, two
|
||
definitions of a strong symbol, incompatible calling conventions, or mismatched
|
||
layout digest is a pre-link diagnostic where possible and a mandatory link
|
||
failure otherwise.
|
||
|
||
C headers are not searched or parsed implicitly. Bindings are either checked-in
|
||
WW source produced by an explicit `ww bindgen c` command, or a declared
|
||
`generate` action whose inputs include the exact header trees, target descriptor,
|
||
preprocessor, macro map, include roots, and binding tool. The generated result
|
||
records the declared whole input-tree digests, observed include trace as
|
||
non-semantic audit metadata, and C ABI digest. The trace cannot add an input; a
|
||
future finer-grained scan would require a new built-in action still keyed by the
|
||
complete allowed include-tree digest. No build invokes ambient
|
||
`pkg-config`; `ww native snapshot-pkg-config` is an explicit, impure acquisition
|
||
command that converts one selected host configuration into a reviewable native
|
||
provider record and content snapshot.
|
||
|
||
### 8.4 Native declarations
|
||
|
||
Schema 1 uses two connected declarations. A package lists the ABI slots it
|
||
requires:
|
||
|
||
```text
|
||
package "compress/zlib" {
|
||
native = ["c:zlib@1"]
|
||
generated = ["action:zlib-bindings:ww"]
|
||
}
|
||
```
|
||
|
||
A provider declares exact target artifacts and dependencies:
|
||
|
||
```text
|
||
native "zlib-linux-aarch64" {
|
||
provides = "c:zlib@1"
|
||
when = { os = "linux", arch = "aarch64", environment = "gnu",
|
||
c-abi = "aapcs64", float-abi = "hard",
|
||
requires-features = ["neon"],
|
||
minimum-sdk = "linux:5.10.0.0" }
|
||
include-trees = [
|
||
{ tree = "native/zlib/include", class = "user", subdirectory = "." }
|
||
]
|
||
sources = [
|
||
{ name = "adler32", source = "native/zlib/adler32.c",
|
||
language = "c11", preprocessing = "c-preprocessor",
|
||
include-indices = [0], defines = {} }
|
||
]
|
||
defines = { ZLIB_CONST = "1" }
|
||
objects = []
|
||
archives = []
|
||
shared = []
|
||
requires = ["c:libc"]
|
||
link = [
|
||
{ kind = "object",
|
||
artifact = { namespace = "provider-output",
|
||
owner = "zlib-linux-aarch64",
|
||
name = "adler32", kind = "object", mode = "data" } }
|
||
]
|
||
abi = { file = "native/zlib.wwabi" }
|
||
}
|
||
```
|
||
|
||
Allowed provider fields are exactly `provides`, `when`, `include-trees`,
|
||
`sources`, `defines`, `objects`, `archives`, `shared`, `requires`, `link`, and
|
||
`abi`. `when` is a finite conjunction. It may contain exact
|
||
`descriptor-digest`, `arch`, `vendor`, `os`, `environment`, `object-format`,
|
||
`hosted`, `c-abi`, `data-model`, `float-abi`, `cpu-baseline`, `relocation`,
|
||
`code-model`, and `pic`; `requires-features`/`forbids-features` use subset/
|
||
disjoint-set matching; `minimum-sdk` matches only a product platform whose
|
||
declared minimum is at least that value. It has no general expression. All
|
||
matching candidates are retained: identical provider/artifact digests coalesce,
|
||
while multiple different matches require the product's explicit provider map
|
||
rather than a specificity guess.
|
||
|
||
The textual `abi = { file = PATH }` form parses `PATH` as one complete canonical
|
||
`NativeABIContractV1` subdocument, encodes it as top-level record kind 16, and
|
||
lowers the provider field to an `ArtifactRefV1` containing that typed record
|
||
digest; an inline complete record is equivalent. The local source blob and
|
||
parsed contract are both hashed. The file form is not a digest assertion or a
|
||
build-time include, and partial contracts are invalid.
|
||
|
||
`sources` entries support only toolchain-declared C language editions and
|
||
assembly dialects. Each source has an exact file digest and sees only the listed
|
||
include trees/defines and toolchain headers. `objects`, `archives`, and `shared`
|
||
name content-identified prebuilt artifacts plus their target/object/ABI records.
|
||
A local `path` is content-hashed during analysis; a literal expected digest is
|
||
needed only for an external/prebuilt record. A provider may mix source and prebuilt inputs, but every produced object is its
|
||
own `native.compile` action. There is no filesystem library search.
|
||
|
||
`abi` is not an opaque user assertion. It is the recomputed digest of a canonical
|
||
native ABI contract containing slot, product-platform ABI/data model, calling
|
||
conventions, symbol names/versions/kinds, referenced C layout/header-contract
|
||
digests, required runtime slots, CPU/features, minimum SDK, and PIC/TLS/unwind
|
||
requirements. Foreign bindings carry the expected contract or compatible
|
||
declaration-level subset. WW cross-checks the contract against compiled objects,
|
||
shared/import-library sidecars, and declared providers before link.
|
||
|
||
### 8.5 Assembly and object files
|
||
|
||
Assembly source declares an official external-tool dialect (`gnu`,
|
||
`llvm-integrated`, or another exact toolchain capability), preprocessing mode,
|
||
target, and CPU feature contract. The current `w6a` Plan-9-style dialect is a
|
||
migration input and is not accepted after cutover.
|
||
The selected assembler executable and resources are action inputs. A source for
|
||
one target cannot be selected for another by extension alone.
|
||
|
||
Prebuilt objects carry a sidecar native record with content digest, object
|
||
format, architecture, ABI, required CPU features, defined/undefined symbols,
|
||
PIC/TLS/unwind properties, and producer provenance. WW verifies the object
|
||
format/class/endian/machine header, sections, symbol table, relocations, notes,
|
||
and architecture attributes against every mechanically inferable sidecar fact;
|
||
the same inspection recurses into archive members and shared/import libraries.
|
||
Source-level C contract, libc/runtime compatibility, and provenance are not
|
||
fully inferable from object bytes, so frozen opaque prebuilts additionally need
|
||
a signature/attestation accepted by policy. A missing or contradictory record
|
||
is an error, not permission to ask the host linker what happens.
|
||
|
||
### 8.6 Static/shared libraries and ordered linking
|
||
|
||
Native `link` is an ordered list of typed template tokens:
|
||
|
||
```text
|
||
{ kind = "object", artifact = "object:NAME" }
|
||
{ kind = "archive", artifact = "archive:NAME" }
|
||
{ kind = "shared", artifact = "shared:NAME" }
|
||
{ kind = "provider", slot = "c:zlib@1" }
|
||
{ kind = "group", items = [...] }
|
||
{ kind = "whole", items = [...] }
|
||
{ kind = "as-needed", enabled = true, items = [...] }
|
||
{ kind = "linker-script", artifact = "file:NAME" }
|
||
{ kind = "version-script", artifact = "file:NAME" }
|
||
{ kind = "export-map", artifact = "file:NAME" }
|
||
{ kind = "def-file", artifact = "file:NAME" }
|
||
{ kind = "runtime-search", policy = "origin-relative", path = "lib" }
|
||
{ kind = "install-name", value = "@rpath/libname.so" }
|
||
{ kind = "dynamic-loader", artifact = "file:LOADER", runtime-path = "/lib/ld.so" }
|
||
{ kind = "entry", symbol = "_start" }
|
||
{ kind = "retain", symbol = "ww_init_abcd" }
|
||
```
|
||
|
||
Those are the complete schema-1 project/native template kinds. `provider` is
|
||
recursively replaced at its exact position by the chosen provider fragment;
|
||
the final kinds are every listed kind except `provider`. The toolchain-only
|
||
`splice` and `script-slot` kinds also lower away as specified in section 6.9.
|
||
Every file-bearing final token resolves
|
||
to a declared typed artifact and digest. Nested `items` contain only link tokens;
|
||
repetition is represented by repeating a list entry and is never deduplicated.
|
||
`group` contains archives or provider templates that resolve only to archives,
|
||
`whole` contains archives only, and `as-needed` contains shared inputs or
|
||
provider templates that resolve only to shared inputs; any other expansion or
|
||
nesting is invalid.
|
||
Runtime-search/install-name values are validated by the selected platform
|
||
adapter; frozen bundled policy permits only relocatable origin-relative paths.
|
||
Entry/retain tokens become the pinned linker's typed entry/undefined-root
|
||
mechanism. Raw flags exist only inside a content-identified custom toolchain
|
||
adapter.
|
||
|
||
The plan is not a set and is never alphabetically reordered. The selected
|
||
toolchain policy template determines global position: it can place start CRT
|
||
before the product-object splice and end CRT after runtime providers, rather
|
||
than relying on one universal ordering rule. Inside `product-objects`, WW
|
||
objects use stable package-identity order. Inside `native-providers`, fragments
|
||
use requester-before-provider topological order, which gives `libA` before the
|
||
`libB` it requires. `product-controls` carries the declared entry/retain/install
|
||
tokens; `init-dispatch` carries its single retained object when needed. Cyclic
|
||
static archives must be represented by one explicit `group`; an undeclared
|
||
provider cycle is an error. Repetition, whole-archive, as-needed, export maps,
|
||
and symbol-version scripts remain exact records in the flattened canonical plan.
|
||
|
||
An archive action preserves declared member order and canonicalizes header
|
||
timestamps, ownership, modes, and string tables. A shared-library input includes
|
||
its link-time artifact, SONAME/install-name, ABI digest, transitive runtime
|
||
requirements, and deployable runtime artifact digest. Merely finding the same
|
||
basename in a host directory is never equivalence.
|
||
|
||
The installation manifest also fixes runtime resolution. ELF bundled policy
|
||
copies the exact shared closure under a digest-namespaced relative `lib/` and
|
||
uses an origin-relative RUNPATH; Mach-O uses exact `@rpath`/install names; Windows
|
||
places named DLL artifacts in the declared application directory beside their
|
||
matching import libraries. A platform-image provider may instead bind an exact
|
||
loader/system tree. An OS-managed mutable shared library is an impure runtime
|
||
policy: link bytes can still be recorded, but WW does not promise that execution
|
||
will load a particular digest.
|
||
|
||
Linker scripts are declared content inputs. The adapter resolves/audits only
|
||
file-bearing directives such as `INCLUDE`, `INPUT`, `GROUP`, and `SEARCH_DIR`
|
||
for the pinned linker dialect; included files and permitted sysroot trees are in
|
||
the record. The pinned linker—not WW—interprets section placement, expressions,
|
||
symbols, memory regions, and target semantics inside the closed sandbox.
|
||
Unresolved `SEARCH_DIR`, absolute host paths, and implicit default scripts are
|
||
errors. GNU `ld` documents that scripts and archive order change link semantics;
|
||
WW therefore preserves rather than abstracts them away
|
||
([GNU ld scripts](https://sourceware.org/binutils/docs/ld/Scripts.html),
|
||
[GNU linker](https://sourceware.org/binutils/docs/ld.html)).
|
||
|
||
### 8.7 libc, CRT, SDK, loader, and freestanding products
|
||
|
||
A hosted platform entry names one exact sysroot/SDK and the available libc,
|
||
system, compiler-runtime, and WW-runtime providers; it does not name one
|
||
universal CRT sequence or loader. The selected toolchain link policy for
|
||
`(product-platform descriptor, product kind, linkage, profile, runtime
|
||
selector)` supplies the exact ordered token template containing CRTs, compiler
|
||
runtime, system-provider slots, script slot, and any platform-appropriate loader
|
||
contract. Compiler and linker
|
||
drivers run through no-defaults adapters and receive only that declared closure,
|
||
so they cannot fall back to B's `/usr`.
|
||
|
||
A product chooses a runtime policy:
|
||
|
||
```text
|
||
product "kernel" {
|
||
kind = "exe"
|
||
root = "kernel"
|
||
linkage = "static"
|
||
runtime = "none"
|
||
entry = "_start"
|
||
native = ["freestanding:boot@1"]
|
||
linker-script = "native/kernel.ld"
|
||
}
|
||
```
|
||
|
||
For an executable, `linkage` is exactly `dynamic`, `pie`, `static`, or
|
||
`static-pie`; shared-library products use `shared`. The toolchain maps
|
||
`(product kind, linkage, profile, product-platform, runtime selector)` to one
|
||
exact ordered policy template; selection never ignores `runtime`. Loader
|
||
presence follows the platform's executable rules. On an ABI with
|
||
an explicit program interpreter, `dynamic` and `pie` executables must name its
|
||
runtime path and exact provider artifact; `static` and `static-pie` must not. A
|
||
shared library has a runtime identity and dependencies but no executable program
|
||
interpreter. PE/COFF-style platforms without a separate interpreter bind the
|
||
exact platform-image/loader contract through system-provider policy rather than
|
||
inventing a pathname. Different product policies therefore cannot accidentally
|
||
share one CRT/loader sequence.
|
||
|
||
`runtime` is `hosted`, `minimal`, `none`, or a named provider. `none` supplies no
|
||
libc, CRT, loader, or WW runtime; compiler helper routines must be supplied by a
|
||
declared provider or rejected. `minimal` names an exact freestanding runtime.
|
||
Entry symbol, memory/linker script, relocation/code model, panic/stack policy,
|
||
and any boot image action are explicit. Kernel-style targets never inherit the
|
||
hosted target's defaults.
|
||
|
||
Schema 1 permits `none` only for object/static products and `static` or
|
||
`static-pie` executables. Its selected link template must contain no CRT,
|
||
dynamic-loader, libc, compiler-runtime, or WW-runtime token. `minimal` and named
|
||
providers declare their valid product/linkage set as capabilities; `hosted`
|
||
uses the platform's hosted set. A product/linkage/runtime tuple outside that set
|
||
is rejected during graph construction. The policy's single `script-slot` uses
|
||
the product's `linker-script` when present, replacing the default at the same
|
||
ordered position.
|
||
|
||
The target/toolchain declares the runtime capability required by every
|
||
compiler-emitted helper and language operation. During graph construction a
|
||
`none`/`minimal` product is rejected if selected source operations require an
|
||
unavailable allocation, panic, stack, arithmetic, TLS, unwind, or other runtime
|
||
capability; this is not deferred to an unexplained undefined linker symbol.
|
||
|
||
### 8.8 Provider conflicts and system substitution
|
||
|
||
One link namespace may select exactly one provider digest for an ABI slot such
|
||
as `c:zlib@1`, `c:libc`, or `runtime:ww@1`. Multiple requirements for the same
|
||
slot coalesce only if they resolve to the same provider and ABI digest. Different
|
||
providers, ABI major slots that export colliding unversioned symbols, and two
|
||
native modules claiming the same strong symbols are loud identity-collision
|
||
errors. WW never chooses whichever library appears first.
|
||
|
||
When more than one contract-compatible provider matches, the root product must
|
||
select one explicitly:
|
||
|
||
```text
|
||
product "hello" {
|
||
kind = "exe"
|
||
root = "."
|
||
providers = [
|
||
{ slot = "c:zlib@1", use = "example.org/zlib#zlib-linux-aarch64" }
|
||
]
|
||
}
|
||
```
|
||
|
||
The provider ID is `(declaring module identity, native-clause name)`, rendered
|
||
with `#` only in metadata. Selection cannot change a dependency's required ABI
|
||
contract; the chosen provider must satisfy every declaration-level contract.
|
||
|
||
Two incompatible native versions can coexist only if they use distinct provider
|
||
slots **and** their symbols/runtime names are namespaced or versioned so the link
|
||
record proves no collision. Otherwise the build must adapt one behind a wrapper,
|
||
use dynamic isolation, or fail. Language-level multiple-version selection cannot
|
||
solve a C global-symbol collision.
|
||
|
||
A distro system provider is a normal provider record mapping exact logical
|
||
artifacts to content digests and ABI metadata. If those files live under `/usr`,
|
||
the mapping snapshots/re-hashes them before graph construction and changes the
|
||
key whenever they change. It is marked impure unless the directory tree itself
|
||
is immutable and content-identified. Raw `-L`, `-l`, `LD_LIBRARY_PATH`, compiler
|
||
defaults, and build-time `pkg-config` are not accepted substitutes.
|
||
|
||
A portable distributor substitution uses a closed `ww-native-map 1` file:
|
||
|
||
```text
|
||
ww-native-map 1
|
||
target = "sha256:product-platform-descriptor..."
|
||
provider "c:zlib@1" {
|
||
use = "distro.example/native#zlib"
|
||
contract = "sha256:..."
|
||
artifact-tree = "sha256:..."
|
||
provenance = "https://distro.example/provenance/zlib.jsonl"
|
||
}
|
||
```
|
||
|
||
`ww lock --native-map=FILE` records the map digest/origin in `ww.lock`; frozen
|
||
mode accepts only that exact signed/content-verified map. `ww.work` may contain
|
||
the same `provider` clause for local development, but it is an impure overlay and
|
||
frozen mode rejects it. This gives OS packagers an offline substitution mechanism
|
||
without changing imports or silently consulting `/usr`.
|
||
|
||
### 8.9 Cross-compilation behavior
|
||
|
||
All code-generating tools execute on B. WW/C/assembly compilation for the
|
||
requested ordinary product emits H objects using only H's target descriptor,
|
||
sysroot, headers, runtime, and native providers. A compiler-like product built
|
||
for H may later emit T code, but no T program executes during its own build.
|
||
Object headers and sidecars are checked before linking, so a host object cannot
|
||
silently enter a target product.
|
||
|
||
`ww test --target=H` always builds target test binaries. It executes them only
|
||
when `H = B` or the toolchain declares an explicit content-identified runner
|
||
(local emulator or simulator plus immutable image) as an invocation tool. Otherwise
|
||
it reports “built, not run” unless `--require-run` was requested, in which case
|
||
it fails. The runner and its platform image are action inputs; no ambient emulator
|
||
is discovered.
|
||
|
||
Remote hardware/device testing is a separate explicit
|
||
`ww observe test --runner=NAME` operation with declared endpoint/capability
|
||
authority. It may use network/devices but is a non-build observation: remote
|
||
state is reported, it never populates artifact/shared caches, and it is outside
|
||
byte-reproducibility claims. Ordinary `ww test` retains the no-network policy.
|
||
|
||
This model supports new targets without running a compiler on them: an existing
|
||
host toolchain adds a target descriptor, backend, object adapter, sysroot/runtime,
|
||
and native providers, then builds and tests through a declared runner or hardware
|
||
step. GNU's build/host/target distinction is useful vocabulary, but WW records
|
||
the complete descriptors rather than only triplets
|
||
([Autoconf triplets](https://www.gnu.org/software/autoconf/manual/autoconf-2.71/html_node/Specifying-Target-Triplets.html)).
|
||
|
||
## 9. Command-line design
|
||
|
||
### 9.1 The ordinary path
|
||
|
||
The default workflow is deliberately short:
|
||
|
||
```sh
|
||
ww init example.org/hello
|
||
ww build
|
||
ww run -- argument
|
||
ww test .
|
||
```
|
||
|
||
`ww init MODULE` creates `ww.mod`, a root `main.ww` only when the directory is
|
||
empty, and a lock selecting the currently invoked immutable toolchain. It does
|
||
not add a dependency. In an existing one-directory `main` package, `ww build`
|
||
works without `init`; the invoking toolchain and standalone source identity are
|
||
shown in verbose output.
|
||
|
||
`ww build [DIR|PRODUCT]` builds the default root product or one named product.
|
||
The default profile is the fully specified `debug` profile; `--profile=release`
|
||
selects the toolchain's immutable release profile. `ww run` first performs that
|
||
same build, then runs only a product with `H = B`; arguments after `--` are never
|
||
interpreted by the build.
|
||
|
||
There is no command that means “build and opportunistically download whatever is
|
||
missing.” If a locked source or toolchain is absent, the diagnostic names its
|
||
digest and asks for `ww fetch --locked`.
|
||
|
||
### 9.2 Tests, examples, docs, and installation
|
||
|
||
```sh
|
||
# Exactly one package.
|
||
ww test ./internal/codec
|
||
|
||
# Every package below the current module root.
|
||
ww test ./...
|
||
|
||
# Compile cross-target tests and require a declared runner.
|
||
ww test ./... --target=aarch64-unknown-linux-gnu --require-run
|
||
|
||
# Build an ordinary example product and render documentation.
|
||
ww build example:examples/packet
|
||
ww doc ./... --out=out/doc
|
||
|
||
# Materialize a named release product under a prefix.
|
||
ww install inspect --profile=release --prefix=/opt/ww
|
||
```
|
||
|
||
Test package discovery and result reporting remain deterministic under `-j N`.
|
||
Compilation is cached, but every selected test binary runs. An installation
|
||
manifest lists every copied artifact, digest, mode, runtime dependency, and
|
||
relative destination and is materialized as `<product>.wwinstall`. Installation never discovers libraries in the prefix and
|
||
never mutates the cache artifact.
|
||
|
||
### 9.3 Inspecting the graph and cache
|
||
|
||
```sh
|
||
ww graph --packages
|
||
ww graph --actions
|
||
ww graph --actions --format=json > graph.json
|
||
ww explain example.org/hello/internal/codec
|
||
ww explain --path --all product:hello
|
||
ww cache verify
|
||
ww verify reproducible --profile=release
|
||
```
|
||
|
||
`graph` performs analysis but no build action. A node whose producer output does
|
||
not yet exist has a known action shape and incoming edges but a `pending` key;
|
||
its final key is computed when predecessor content digests become available.
|
||
`explain` compares available current records with the project index and likewise
|
||
does not build. These commands never fetch.
|
||
|
||
### 9.4 Adding, fetching, updating, freezing, and vendoring
|
||
|
||
```sh
|
||
# Query the module's conventional source index, add a minimum, and lock closure.
|
||
ww add example.org/codec@v1.2.3
|
||
|
||
# The same operation with an explicit private or non-conventional origin.
|
||
ww add corp.example/codec@v1.2.3 --from=https://packages.corp/codec/
|
||
|
||
# Materialize exactly the existing lock without changing any project file.
|
||
ww fetch --locked
|
||
|
||
# Change one direct minimum and recompute the complete lock atomically.
|
||
ww update example.org/codec@v1.4.0
|
||
|
||
# Recompute after an intentional manifest edit.
|
||
ww lock
|
||
|
||
# Materialize the exact locked source closure in the project.
|
||
ww vendor --locked
|
||
|
||
# A frozen, offline build from already present locked inputs.
|
||
ww build --frozen --offline --vendor
|
||
```
|
||
|
||
`add` and `update` write `ww.mod` and `ww.lock` together through temporary files
|
||
only after the entire selection and integrity check succeeds. `lock` writes only
|
||
`ww.lock`. `fetch` and `vendor` never change selection. Without `--vendor`, WW
|
||
uses the immutable source store; with it, every vendor digest is reverified.
|
||
`add --from=URL` records that credential-free index as the requirement's
|
||
`source-index`; archive redirects/final origin remain exact lock metadata.
|
||
|
||
An ordinary non-frozen build still never updates an inconsistent lock: it fails
|
||
with the exact `ww lock` command. Frozen mode additionally rejects workspaces,
|
||
impure providers, noncanonical metadata, unlocked tools, and any source digest
|
||
not named by the lock. `--offline` is useful in automation as an assertion but
|
||
does not weaken or strengthen the normal build network denial.
|
||
|
||
### 9.5 Cross compilation and toolchain selection
|
||
|
||
```sh
|
||
# Select a product host platform (the familiar cross-build form).
|
||
ww build --target=aarch64-unknown-linux-gnu --profile=release
|
||
|
||
# Select exact CPU semantics rather than probing the build machine.
|
||
ww build --target=x86_64-unknown-linux-gnu \
|
||
--cpu=x86-64-v3 --feature=-avx512f
|
||
|
||
# Build a compiler that runs on H and emits code for T.
|
||
ww build compiler:wwc \
|
||
--host=aarch64-unknown-linux-gnu \
|
||
--target=riscv64-unknown-none-elf
|
||
|
||
# Acquire and lock an exact toolchain before building; only fetch uses network.
|
||
ww toolchain fetch ww.org/toolchain@v1.4.0
|
||
ww lock --toolchain=ww.org/toolchain@v1.4.0
|
||
ww build --toolchain=ww.org/toolchain@v1.4.0
|
||
```
|
||
|
||
The selected command toolchain must match the common ID, satisfy every minimum,
|
||
and equal the root lock's exact descriptor. A digest
|
||
selection is exact; an ID/version selection resolves only through the lock or
|
||
installed signed catalog and never during build. `ww toolchain list --targets`
|
||
shows descriptors and sysroot/runtime digests, not just triples. `ww lock
|
||
--toolchain` changes only the lock's exact toolchain selection when the chosen
|
||
version satisfies every manifest minimum; it does not rewrite those minimums.
|
||
|
||
### 9.6 Deliberately absent commands/options
|
||
|
||
There is no `ww build --fetch`, build-time package-manager hook, raw `-L`/`-l`,
|
||
global import search path, arbitrary compiler/linker environment injection,
|
||
manifest evaluator, or command that installs dependencies into a mutable global
|
||
source namespace. Expert escape hatches are explicit toolchains, native provider
|
||
records, and finite declarative actions; all remain visible in the graph/key.
|
||
|
||
## 10. Toolchain and bootstrap design
|
||
|
||
### 10.1 The durable stage-zero seed
|
||
|
||
The smallest credible recovery seed is:
|
||
|
||
```text
|
||
bootstrap/ww0.c generated portable C99 snapshot
|
||
bootstrap/bootstrap.plan canonical source/tool/output plan
|
||
bootstrap/SHA256SUMS expected seed and plan digests
|
||
```
|
||
|
||
`ww0.c` is generated at release time from the same compiler sources as the
|
||
production compiler, with a deliberately non-optimizing portable C backend and
|
||
the minimal support routines concatenated into one translation unit. It is not
|
||
a second hand-maintained compiler and is never used in ordinary production
|
||
builds. The resulting `ww0` retains that portable C-emission path: it contains
|
||
only enough compiler, export writer, C output, and fixed-plan execution to emit
|
||
stage-1 C from the enumerated source closure and have the named host C closure
|
||
compile it. It has
|
||
no resolver, network client, general manifest engine, cache, test coordinator,
|
||
or installation framework.
|
||
|
||
The release process regenerates `ww0.c` and fails if its bytes differ from the
|
||
checked-in snapshot. Keeping portable source rather than four opaque host
|
||
binaries permits recovery on a new architecture with a C99 implementation. The
|
||
cost of the simple C emitter is accepted because it bounds and exposes the seed;
|
||
it is a release backend, not another production path.
|
||
|
||
The bootstrap-host contract is narrower than “any C99”: hosted C99 with
|
||
`CHAR_BIT == 8`, exact `uint8_t/uint32_t/uint64_t`, two's-complement signed
|
||
integers, binary file I/O, at least 32-bit address space and the published source/
|
||
object size limits. The snapshot uses no host floating-point result or undefined
|
||
signed overflow. Compile-time assertions plus a tiny I/O/integer conformance
|
||
probe run before compilation. Supported implementation modes and required
|
||
headers are enumerated in `bootstrap.plan`; an implementation outside the
|
||
contract is not silently called portable.
|
||
|
||
### 10.2 Stage transitions and fixed point
|
||
|
||
Recovery uses these exact stages:
|
||
|
||
1. A named host C implementation compiles `ww0.c` to `ww0`. Its executable,
|
||
version output, command, headers, libc, assembler, and linker are recorded in
|
||
`bootstrap-host.wwar`; they are part of the trusted base, not silently blessed.
|
||
2. `ww0 bootstrap/bootstrap.plan` runs on B, emits portable C for the locked
|
||
production compiler/driver, and uses that same recorded host C closure to
|
||
produce **stage 1 with H = B**. Stage 1 must run locally; distributable cross
|
||
compilers are built only after the local fixed point.
|
||
3. Stage 1 uses the pinned production target/tool closure to perform a normal
|
||
frozen build of the same sources for `H = B`, producing **stage 2**.
|
||
4. Stage 2 repeats the identical logical build to produce **stage 3**.
|
||
5. Stage 3 repeats it to produce **stage 4**.
|
||
6. `bootstrap.compare` first requires stage-2 and stage-3 **semantic output
|
||
sets**—compiler/driver executables, export/object/runtime artifacts, and the
|
||
installed semantic bundle tree—to be byte-identical. Producer action records
|
||
and provenance are excluded because stage 2 was built by stage 1 while stage
|
||
3 was built by stage 2. Once those compiler bytes converge, it requires
|
||
stage-3 and stage-4 semantic outputs **and raw action/result records** to be
|
||
byte-identical; their producer compiler digests are then equal.
|
||
|
||
Stage 1 need not equal stage 2: portable-C and production backends may generate
|
||
different code. Stage 2 MUST equal stage 3 semantically, and stage 3 MUST equal
|
||
stage 4 completely. A canonical stage semantic manifest lists only role/output
|
||
artifact digests, never its producer. A mismatch reports the first differing
|
||
output or action field and is a release failure. Two absolute roots and two
|
||
concurrency levels are used for the official fixed-point job.
|
||
|
||
This fixed point proves self-consistency and path/order reproducibility; it does
|
||
not by itself defeat Ken Thompson's trusting-trust attack. Official releases
|
||
also perform **diverse seed compilation** with two independently sourced host C
|
||
toolchains where available, publish both bootstrap-host records, regenerate the
|
||
C snapshot from the converged WW source/compiler, and compare converged semantic
|
||
outputs before signing. This is additional evidence, not a claim of formal
|
||
diverse-double-compilation proof.
|
||
|
||
### 10.3 Trusted-computing-base accounting
|
||
|
||
The bootstrap report lists, by content digest:
|
||
|
||
- `ww0.c`, `bootstrap.plan`, and their tiny C support layer;
|
||
- host C compiler/preprocessor, headers, libc, assembler, linker, loader, and OS
|
||
kernel used to create/run `ww0`;
|
||
- bootstrap assembler/linker/archive tools and their shared/resource closure;
|
||
- source/lock/toolchain descriptor trees; and
|
||
- the SHA-256 and signature verification implementations/keys.
|
||
|
||
That is the reproducible software/input closure. The operational trusted
|
||
computing base additionally includes CPU, firmware, memory/storage behavior,
|
||
kernel, and execution environment; the report names the available hardware/
|
||
firmware attestations but does not pretend a source hash removes physical trust.
|
||
|
||
Nothing is called “trusted” merely because it was found on `PATH`. A recovery
|
||
build with unpinned system tools may establish a new local fixed point but cannot
|
||
claim byte identity with an official release. An official reconstruction uses a
|
||
published content-identified host/bootstrap closure and compares its advertised
|
||
digests.
|
||
|
||
### 10.4 Toolchain distribution and selection
|
||
|
||
Each release publishes source plus locked-source-closure archive, `ww0.c`, lock
|
||
file, platform toolchain bundles, `toolchain.wwt`, stage-2/3/4 semantic and
|
||
action/result manifests, fixed-point report,
|
||
digests, signatures, and provenance. Binary bundles are conveniences verified
|
||
against the descriptor, not irreplaceable seeds. A user either selects the
|
||
project-locked toolchain or an exact command-line digest; WW never downloads a
|
||
new compiler while building.
|
||
|
||
Runtime ABI, export protocol, object ABI, and action-schema compatibility are
|
||
declared independently. A compiler refuses a runtime whose ABI it does not
|
||
support. Toolchain upgrades can retain source compatibility while intentionally
|
||
invalidating all relevant action keys.
|
||
|
||
### 10.5 New-architecture and no-compiler recovery
|
||
|
||
For a new H satisfying the bootstrap-host contract, compile `ww0.c`; `ww0` emits
|
||
the modified production compiler as portable C and the H C compiler builds the
|
||
locally runnable stage 1. Add/pin H's production object/assembler/linker closure,
|
||
then reach the stage-2/3/4 fixed point. Alternatively an existing B toolchain may
|
||
cross-build that H compiler, but it executes only through an explicit runner.
|
||
For a new T only, no target-local compiler is required: add the backend/target
|
||
descriptor, object tests, sysroot or freestanding runtime, and linker provider
|
||
to an existing B/H toolchain.
|
||
|
||
When no suitable WW compiler exists, the checked-in C seed is sufficient. When
|
||
no C compiler exists either, a platform must provide one previously built `ww0`
|
||
binary plus its exact source/digest, or bootstrap a C implementation; WW does not
|
||
claim a smaller physical trusted base than the machine can execute.
|
||
|
||
## 11. Current-system assessment
|
||
|
||
This section records the verified baseline at commit
|
||
`ca2cadeb4173e8190cd4c8bcc25e7da25bcdb0bc`. Line references describe that
|
||
commit and are implementation evidence, not requirements for the replacement.
|
||
|
||
### 11.1 Current invariants
|
||
|
||
- POSIX Make selects ambient `CC`, `AR`, optional `ccache`, and flags; builds the
|
||
Cstage tools; then uses the Cstage driver to build WWstage tools into separate
|
||
work directories. Make repeats manually enumerated transitive source
|
||
prerequisites for each self-hosted target
|
||
([Makefile](../Makefile), lines 7–20 and 105–279).
|
||
- `ww build` creates fixed-array package nodes keyed by dotted import spelling,
|
||
discovers imports with a hand-written scanner separate from the compiler
|
||
parser, rejects directory-package cycles, visits dependencies in
|
||
DFS postorder, and invokes packages serially. The graph is capped at 256
|
||
packages ([C driver](../cmd/ww/main.c), lines 333–370 and 670–700).
|
||
- An imported directory is a separately compiled package. An imported `.ww`
|
||
file is recursively folded into the importing unit and has no node, identity,
|
||
interface, object, or link artifact of its own. Inline package blocks can also
|
||
satisfy otherwise missing imports (C driver, lines 396–428 and 498–613).
|
||
- A directory package selects all immediate `.ww` files except `*_test.ww`, sorts
|
||
them by bytes, and has no target-specific selection rule (C driver, lines
|
||
239–301). The build path follows source symlinks while the test coordinator
|
||
rejects them.
|
||
- Resolution translates dotted imports to paths and performs a global
|
||
directory-before-file search over the entry root, `-I` roots, and inferred
|
||
source library. Thus a directory in a later root beats a file in an earlier
|
||
root (C driver, lines 152–224 and 966–1003).
|
||
- Every dependency emits source-like `.wwi`. Every importer receives the whole
|
||
transitive `.wwi` closure, tagged with out-of-band module comments and prepended
|
||
to a composed `.unit.ww`; the compiler reparses that unit (C driver, lines
|
||
702–809; [interface writer](../cmd/w6c/wwi.c), lines 1–24 and 505–570).
|
||
- The root compiles without compiler `-I`, emits no `__root.wwi`, bypasses the
|
||
dependency export-signature path, and receives special `main` handling; only
|
||
dependency nodes emit interfaces. `ww test` also injects the resolvable `test`
|
||
package as a synthetic root edge even when source has no such import, while
|
||
runtime is an implicit link edge. Imports do not fully describe even today's
|
||
complete graph (C driver, lines 1066–1083 and 1131–1146).
|
||
- `.wwi` records exported prototypes and direct import text but no compiler,
|
||
format, target, data-layout, object ABI, or runtime ABI identity. It includes
|
||
non-semantic parameter names and import spelling. The compiler's `-I` flag
|
||
both requests interface output and changes `main` symbol handling
|
||
([compiler entry](../cmd/w6c/main.c), lines 25–56 and 83–110).
|
||
- For `P` reachable directory packages including root, a normal driver build
|
||
launches `P` compiler processes, `P` assembler processes, and one linker.
|
||
It writes `P-1` dependency archives itself. The link is root object, dependency
|
||
archives in reverse topological order, runtime, then separately accumulated
|
||
`-L` and `-l` values, losing their original interleaving (C driver, lines
|
||
1095–1249 and 1368–1460).
|
||
- `-w DIR` is a caller-owned mutable reuse directory, not a cache. Freshness is
|
||
exact composed-unit bytes, copied compiler/assembler bytes, a text mode stamp,
|
||
and artifact existence/nonzero size. The graph and units are rebuilt in
|
||
memory and the final executable is relinked on every invocation. Publication
|
||
through `.new` files with the unit committed last is usefully atomic (C driver,
|
||
lines 867–952, 1016–1059, and 1110–1213).
|
||
- Without `-w`, a build creates and deliberately retains
|
||
`<output-stem>.sepwork`; repeating while it exists fails. With `-w`, the caller
|
||
must create and serialize the directory. Normal driver products are always
|
||
root executables linked with the runtime; there is no library-only product
|
||
path, and `-S` merely stops after assembly.
|
||
- `ww test DIR` delegates to a separate coordinator. It groups same-package
|
||
`package p;` and external `package p_test;` tests from `*_test.ww`, excludes
|
||
dependency tests, composes generated source roots, and parallelizes independent
|
||
test binaries with deterministic reporting
|
||
([package coordinator](../internal/wwpackage/package.ww), lines 278–417,
|
||
508–648, and 725–1057).
|
||
- Explicit `ww test FILE` bypasses the directory `*_test.ww` classifier and
|
||
accepts an arbitrarily named source root. Universal directory packages delete
|
||
that distinct test mode.
|
||
- Current bootstrap is mixed C/self-hosted. Make keeps the C driver fixed, uses
|
||
self-hosted compiler stages to produce `ww2`, `ww3`, and `ww4`, and compares
|
||
`ww2 == ww3` and `ww3 == ww4`. The self-hosted driver itself is outside that
|
||
fixed-point chain. The planned four stage-zero binaries are absent; the
|
||
“no C compiler” route still uses host `ar` ([Makefile](../Makefile), lines
|
||
825–883; [bootstrap notes](../BOOTSTRAP.md)).
|
||
|
||
### 11.2 Conflated identities and accidental behavior
|
||
|
||
| Concept that must be separate | Current conflation or accident |
|
||
|---|---|
|
||
| package identity | Dotted import spelling is graph key, module/symbol prefix, artifact basename, and link identity. |
|
||
| declared name | A directory's leaf is checked against it for imports, but root directories and literal file roots receive different validation. `.wwi` itself retains only the leaf package name. |
|
||
| filesystem location | Ordered search roots silently choose/shadow a location; the same physical directory may be compiled under two import identities, while duplicate locations for one spelling produce no collision diagnostic. Paths are lexical, not content identities. |
|
||
| package versus file | Directories create separate-compilation nodes; files disappear into owners. The same `package` syntax means two compilation models. |
|
||
| artifact versus identity | `<dotted-path>.wwi/.s/.o/.a` names artifacts; root aliases to `__root`, which can collide with a real import. |
|
||
| public versus non-semantic interface | Parameter names and AST-preserved type/import spellings influence `.wwi` bytes and reverse rebuilds. Imports/declarations are otherwise canonically sorted; whitespace/comments and original declaration order generally do not. |
|
||
| compiler interface mode versus link identity | `w6c -I` both emits `.wwi` and classifies the package as a dependency for `main` mangling. |
|
||
| native dependency versus linker search | Raw `-L` and `-l` names carry no selected file, ABI, order relationship, target, or content identity. |
|
||
| cache location versus cache key | The explicit `-w` directory is both mutable namespace and freshness state; callers must serialize it. |
|
||
| source root versus command UX | Help historically describes `.` like a basename file, while implementation stats and builds it as a directory. |
|
||
|
||
Other accidental constraints include fixed 256/1024-byte name/path buffers. The
|
||
compiler parser silently truncates dotted full imports beyond 255 bytes, while
|
||
the C driver scanner can stop advancing and hang on an import identifier at that
|
||
limit; the dynamically sized WWstage scanner differs. There is no regression
|
||
test for this stage divergence. Other defects include unquoted Cstage `system()` command
|
||
construction, and dependence on `/bin/sh`. Cstage honors `WW_W6C`, `WW_W6A`,
|
||
`WW_W6L`, `WW_LIB`, and source-library fallbacks; the self-hosted driver instead
|
||
hardcodes sibling tools and relative libraries. The two implementations are
|
||
therefore similar algorithms, not one protocol implementation.
|
||
|
||
Build and test disagree about source symlinks. External package tests are built
|
||
from a generated single-file root plus `-I`; an external import of a multi-file
|
||
production package can resolve and fold only its canonical same-named file rather
|
||
than the directory package. Test work-directory names flatten `/` to `_`, so
|
||
distinct lexical paths can collide. These are consequences of routing tests
|
||
around, rather than through, one package model.
|
||
|
||
`make install` copies only `ww`, `wwtest`, and `libwcc.a`, while the driver needs
|
||
sibling compiler/assembler/linker tools and `libwwrt.a`; the installed result is
|
||
not a self-contained functional toolchain outside the build tree.
|
||
|
||
### 11.3 Scaling, invalidation, and hidden inputs
|
||
|
||
The driver stores fixed `deps[256]` arrays and performs linear graph lookup.
|
||
Every package unit reads one interface per transitive dependency, so total
|
||
interface reads and copied interface text are quadratic on deep/dense graphs even
|
||
on a warm build. Package compiler/assembler work within one driver is serial;
|
||
Make gains parallelism only by launching independent top-level driver builds.
|
||
|
||
The observed invalidation rules are:
|
||
|
||
- a private change in a directory dependency rebuilds that package and the
|
||
unconditional final link, but not importers;
|
||
- an exported change changes its `.wwi` and rebuilds the whole reverse-transitive
|
||
ancestor cone, even when an intermediate package's own interface is unchanged;
|
||
- a private change in a folded file import rebuilds its entire owner;
|
||
- a link-only option reruns the always-executed link but not package compiles;
|
||
- changing copied compiler or assembler bytes rebuilds every package; and
|
||
- nonzero corruption of `.s`, `.o`, `.a`, or `.wwi` may be accepted because
|
||
content is not rehashed.
|
||
|
||
Hidden or incompletely modeled inputs include `CC`, `AR`, `PATH`, `ccache`, Make
|
||
flags, shell quoting/behavior, compiler built-ins, assembler/linker defaults,
|
||
inferred `argv[0]` library locations, current working directory, `chmod`, runtime
|
||
archive, linker binary, native-library resolution, host libc/CRT/loader, SDK,
|
||
CPU, target, and environment. Make does not invalidate existing C objects when
|
||
the host compiler or C flags change. Workdir reuse also omits the driver binary/
|
||
graph and archive semantics, relying on a manually bumped text format stamp; a
|
||
driver change can therefore reuse stale outputs. WW has no target triple, sysroot, conditional
|
||
source, generator, manifest, lock, source digest, or frozen/offline concept.
|
||
|
||
### 11.4 Measurements and disposable experiments
|
||
|
||
Measurements ran on Linux 6.12.76_1 x86-64 with eight logical CPUs, GCC 14.2.1,
|
||
binutils 2.44, and no `ccache`. They used successful clean builds and isolated
|
||
temporary source fixtures; no production migration was begun. Clean/full means
|
||
use two timed samples and representative cold means three, without CPU isolation
|
||
or OS-cache flushing, so they are observed baselines rather than universal
|
||
performance claims.
|
||
|
||
| Measurement | Result |
|
||
|---|---|
|
||
| clean `make all`, `-j1` | 24.847 s mean |
|
||
| clean `make all`, `-j8` | 15.515 s mean; only 1.60× speedup |
|
||
| warm `make all -j8` | 0.02365 s mean at Make level |
|
||
| clean tool trace | 69 package compiles/assembles in six driver workdirs; 71 `w6c`, 76 `w6a`, six `w6l`, 63 in-driver archives; package work inside each driver stayed serial |
|
||
| small / 8-package / 15-package cold driver build | 0.0103 / 0.1326 / 9.6706 s |
|
||
| same warm driver builds | 0.00814 / 0.01347 / 0.05302 s; each still linked |
|
||
| `out/` | 34,464,015 bytes, 398 files |
|
||
| six WW build workdirs | 28,609,092 bytes, 351 files; 9,954,428 bytes (34.8%) duplicate beyond the first content copy |
|
||
| 15-package root composed unit | 14 `.wwi` sections, 1,464,581 bytes; 62 interface insertions across graph, 1,928,898 composed-unit bytes versus 23,664 distinct interface bytes |
|
||
|
||
A three-package `root -> mid -> leaf` fixture confirmed that root consumes both
|
||
direct and transitive interfaces. A private leaf implementation edit rebuilt
|
||
only leaf plus link. Adding a compatible public leaf API rebuilt all three even
|
||
though `mid` emitted identical `.wwi`. A link-only `-L` change ran only the link.
|
||
|
||
Switching the compiler executable by byte content invalidated all packages.
|
||
Appending data to a nonzero cached `mid.s` was not detected; the old object was
|
||
reused and the build succeeded. This directly rejects artifact-presence caching.
|
||
|
||
Two builds from different absolute source roots produced the same 398 relative
|
||
paths and all sampled WW-generated executables/package artifacts were byte
|
||
identical. The full trees were not: 45 Cstage host objects/copied tools differed,
|
||
including GCC `DW_AT_comp_dir`. Thus current checkout-independent byte identity
|
||
holds for measured WW artifacts, not for the complete build.
|
||
|
||
Cstage and WWstage drivers building the same eight-package graph produced a
|
||
byte-identical final executable and all 38 non-tool artifacts, but took 0.138 s
|
||
and 0.356 s respectively. Under `env -i PATH=/nonexistent`, Cstage returned
|
||
success after ambient `chmod` failed and left mode 0644; WWstage returned success
|
||
with mode 0755. This exposes implementation asymmetry and an undeclared host tool.
|
||
|
||
### 11.5 What survives and what is deleted
|
||
|
||
The replacement retains these sound concepts:
|
||
|
||
- explicit source imports, package clauses, exported/private declarations, and
|
||
loud directory-package-cycle errors, strengthened to every package after file
|
||
folding (whose cycles were merely visit-deduplicated) is deleted;
|
||
- directory package boundaries, made universal rather than optional;
|
||
- same-package and external `*_test.ww` semantics, dependency-test exclusion,
|
||
deterministic discovery/reporting, and always executing selected tests;
|
||
- deterministic sorting/serialization, atomic artifact publication, fixed-point
|
||
bootstrap checks, and byte-identity tests; and
|
||
- the ordinary `ww build`, `run`, and `test` user experience.
|
||
|
||
The replacement deletes these concepts rather than emulating them indefinitely:
|
||
|
||
- file imports, inline multi-package units, bare dotted imports, `-I` search
|
||
roots, directory-before-file precedence, and the `__root` artifact alias;
|
||
- `.wwi`, `//ww:module` wrappers, `.unit.ww`, transitive interface prepending,
|
||
source-prototype interchange, automatic per-dependency archives, and `-w`;
|
||
- raw ambient `-L`/`-l`, compiler/linker/sysroot defaults, and build-time
|
||
`pkg-config` discovery;
|
||
- separate C and self-hosted production drivers, the production Make graph, and
|
||
the separate source-composing test coordinator; and
|
||
- the current Cstage bootstrap path and permanently owned assembler/linker after
|
||
the generated C seed and pinned external tool closure replace them.
|
||
|
||
There will be no compatibility alias that silently translates an old import,
|
||
interface, workdir, or link search into the new model.
|
||
|
||
## 12. Candidate architectures and hard-gate decision
|
||
|
||
Five candidates were developed as coherent systems, not as feature bins.
|
||
|
||
### 12.1 Candidate A: Go-like integrated language command
|
||
|
||
One `ww` command would infer directory packages/imports, compile/cache/test them,
|
||
and add a small module/lock layer. Native inputs would remain compiler flags or
|
||
toolchain conventions. This preserves the strongest part of Go: explicit imports,
|
||
fast direct export data, and a short ordinary command
|
||
([Go command design](https://go.dev/doc/articles/go_command)). A WW-specific lock
|
||
and no-network build could improve on modern Go module behavior.
|
||
|
||
It still fails as an end-to-end native design. A source-only graph cannot name
|
||
host generators, C header trees, archive order, linker scripts, CRT, SDK, or
|
||
sysroot. Ambient compiler-driver and `pkg-config` behavior would remain outside
|
||
the key, and the build/host/target triad would be incomplete. Adding typed native
|
||
actions, complete toolchains, and content records turns it into Candidate E.
|
||
|
||
### 12.2 Candidate B: Hare/Odin-style local source plus an outer build tool
|
||
|
||
WW would use search roots and directory modules, with source vendored or supplied
|
||
by an OS package manager; Make-like orchestration would own native work. This is
|
||
small locally and avoids a language-owned network resolver. Hare 0.26.0 (released
|
||
2026-02-13) is a useful reference: directory modules, direct textual export data,
|
||
`HAREPATH`, explicit system-library flags, and cross-architecture tooling are
|
||
documented in its official manuals
|
||
([modules](https://harelang.org/documentation/usage/modules.html),
|
||
[project structure](https://harelang.org/documentation/usage/project-structure.html),
|
||
[system libraries](https://harelang.org/documentation/usage/system-libraries.html),
|
||
[cross compilation](https://harelang.org/documentation/usage/cross.html)). Odin's
|
||
named collections are a related local-source convention
|
||
([Odin overview](https://odin-lang.org/docs/overview/)).
|
||
|
||
As a complete WW system it fails: ordered search roots are selection policy
|
||
without locked source identity; ordinary outer recipes expose ambient tools,
|
||
environment, and mtimes; language and native graphs/caches cannot jointly explain
|
||
invalidation; and cross sysroots/ABI providers remain project conventions. Making
|
||
the outer tool hermetic and content-addressed yields Candidate C, not this model.
|
||
|
||
### 12.3 Candidate C: two-layer Plan 9-style builder and orchestrator
|
||
|
||
A strict package builder would compile an import graph. A separate small
|
||
declarative DAG tool would own generators, C/assembly, images, archives, and
|
||
links. With content records, pinned tools, sandboxing, and an exact handoff this
|
||
can pass every hard gate. It reflects `mk`'s valuable complete-graph/parallel-tool
|
||
shape without copying its mtime and ambient-environment assumptions.
|
||
|
||
It loses after the gates because the boundary creates two graph protocols, two
|
||
selection UIs, two explanation namespaces, and either duplicate scheduling/cache
|
||
logic or a coarse “build all packages” action. Ordinary native projects must know
|
||
when to invoke each layer. If both front ends lower into one shared scheduler and
|
||
cache, and `ww` owns the ordinary invocation, the result is the smaller Candidate
|
||
E. Keeping the second production tool after that offers no remaining orthogonal
|
||
concept.
|
||
|
||
### 12.4 Candidate D: Zig/Cargo-like programmable integrated project
|
||
|
||
A manifest would define artifacts and dependencies while a host-executed program
|
||
constructs a flexible native graph. This handles more native cases than a
|
||
language-only command. Zig 0.16.0, pinned here to its 2026-04-13 release metadata,
|
||
provides explicit target/native concepts, hashed package sources, local
|
||
dependency overrides, and generated-file graph edges
|
||
([download metadata](https://ziglang.org/download/index.json),
|
||
[0.16.0 reference](https://ziglang.org/documentation/0.16.0/),
|
||
[build system](https://ziglang.org/learn/build-system/)). Cargo 1.97.1, shipped
|
||
with Rust 1.97.1 on 2026-07-16, provides exact package IDs, workspaces, lock
|
||
checksums, resolver rules, and native `links` collision handling
|
||
([Cargo reference](https://doc.rust-lang.org/1.97.1/cargo/reference/),
|
||
[resolver](https://doc.rust-lang.org/1.97.1/cargo/reference/resolver.html)).
|
||
|
||
Zig's build program and Cargo's `build.rs` execute to decide or report build
|
||
behavior; Cargo explicitly
|
||
documents build-script inputs/outputs and its fingerprint cache
|
||
([build scripts](https://doc.rust-lang.org/1.97.1/cargo/reference/build-scripts.html),
|
||
[build cache](https://doc.rust-lang.org/1.97.1/cargo/reference/build-cache.html)). Those
|
||
reference systems as shipped do not meet WW's complete native/toolchain hard
|
||
gates.
|
||
|
||
The strongest coherent D is not left as a straw man: it content-identifies the
|
||
graph program and host compiler/runtime, declares its whole readable source/tool
|
||
closure, runs it in the denied-by-default sandbox with no network, and requires
|
||
it to emit a closed typed graph before artifact execution. That hardened model
|
||
can pass every gate. It still loses afterward: WW must permanently ship/secure/
|
||
bootstrap an evaluator API and host build-program toolchain, users debug both
|
||
program execution and its emitted graph, dependencies expose framework APIs, and
|
||
routine exceptions accumulate as library features. Finite records buy the same
|
||
WW requirements with less user and implementation machinery.
|
||
|
||
### 12.5 Candidate E: hermetic integrated action build — selected
|
||
|
||
Candidate E retains the Go-like ordinary UX and import-derived language graph,
|
||
then adds only the native/action facts that imports cannot express. Both lower
|
||
to one typed graph and content cache. It borrows declared tools/inputs and action
|
||
results from Bazel's hermetic/remote-execution model, and transparent
|
||
content-derived build records from Nix derivations, without adopting either
|
||
framework, evaluator, daemon topology, or user interface
|
||
([Bazel hermeticity](https://bazel.build/versions/9.2.0/basics/hermeticity),
|
||
[Bazel remote caching](https://bazel.build/remote/caching),
|
||
[remote execution protocol](https://github.com/bazelbuild/remote-apis/blob/master/build/bazel/remote/execution/v2/remote_execution.proto),
|
||
[Nix derivations](https://nix.dev/manual/nix/2.35/store/derivation/)).
|
||
|
||
It passes every hard gate and is selected. Its concepts are exactly package,
|
||
module/source selection, product/action, toolchain/target, artifact/digest, and
|
||
native provider/link plan. There is one graph, one scheduler, one cache key, one
|
||
explanation path, and one ordinary command.
|
||
|
||
### 12.6 Hard-gate matrix
|
||
|
||
Legend: **pass** means the strongest coherent form has a credible end-to-end
|
||
invariant; **fail** means it does not. D denotes the hardened evaluator above,
|
||
not unmodified Zig/Cargo behavior.
|
||
|
||
| Hard gate | A: Go-like | B: local + outer | C: two layer | D: programmable | E: action build |
|
||
|---|---|---|---|---|---|
|
||
| complete inspectable dependency DAG | fail for native work | fail across tools | pass | pass after sandboxed evaluation | **pass** |
|
||
| loud cycles/identity collisions | pass for packages; native incomplete | search shadowing fails identity | pass | pass | **pass** |
|
||
| frozen offline source closure | pass with proposed lock | vendoring can pass | pass | pass | **pass** |
|
||
| correct cache invalidation | fail for native/tool defaults | fail across mtime/outer recipes | pass | pass | **pass** |
|
||
| compiler/linker/runtime/toolchain identity | requires the E native layer | fail | pass | pass | **pass** |
|
||
| explicit B/H/T | fail | fail | pass | pass | **pass** |
|
||
| correct native dependency/link modeling | fail | fail | pass | pass | **pass** |
|
||
| deterministic package selection | pass | fail under search roots | pass | pass with lock | **pass** |
|
||
| reproducible artifacts/boundary | fail for native closure | fail | pass | pass with evaluator certification | **pass** |
|
||
| bounded bootstrap | pass with a seed | pass with enumerated seed/tools | pass | pass, larger evaluator TCB | **pass** |
|
||
| explain cache miss/rebuild | fail | fail | pass across two namespaces | pass across evaluation + graph | **pass** |
|
||
| one simple ordinary path | pass | two commands/configurations | two production layers | pass by convention | **pass** |
|
||
| no build/test network or mutation | pass if changed from Go defaults | outer recipes cannot guarantee | pass | pass | **pass** |
|
||
|
||
C, hardened D, and E survive the gates. E wins the requested post-gate
|
||
comparison:
|
||
|
||
| Criterion | C: two layer | D: hardened evaluator | E: selected |
|
||
|---|---|---|---|
|
||
| conceptual simplicity | two graph/handoff models | evaluator language/API plus emitted graph | one graph protocol; two finite descriptions |
|
||
| daily usability | user chooses builder/orchestrator | ordinary defaults can hide evaluator, exceptions cannot | `ww build/test/run` always |
|
||
| clean speed | parallel orchestration can match | evaluation overhead, then parallel graph | direct complete template plus parallel actions |
|
||
| incremental speed | cross-tool handoff may be coarse | evaluator must rerun/cache its own dependencies | direct exports, lazy content keys, cached link |
|
||
| rebuild blast radius | good only with API digests across handoff | can be good after evaluation | public-output digest propagation is intrinsic |
|
||
| cross-compilation | orchestrator can model it | rich API can model it | roles are built into every typed record |
|
||
| FFI/native honesty | outer layer owns it separately | API can express it | same provider/link records as package objects |
|
||
| implementation maintenance | two adapters/engines or coarse boundary | evaluator, SDK compatibility, sandbox, graph engine | one scheduler/store/adapter set |
|
||
| supply-chain security | two policy surfaces | dependency host code remains an evaluator input | one lock/source/action trust model |
|
||
| ecosystem scalability | two tool ecosystems | framework/API accumulation pressure | closed schemas version only for demonstrated facts |
|
||
| debugging/observability | two explanation namespaces | debug program, evaluator cache, then graph | one causal graph/record diff |
|
||
| bootstrap longevity | small outer interpreter possible | compiler/runtime/evaluator enter seed chain | fixed seed plan; no production evaluator |
|
||
|
||
### 12.7 Why each subsystem is in or out of the core
|
||
|
||
Package parsing/resolution and export data belong in the core because only the
|
||
compiler can state the true language graph and public ABI. Module selection and
|
||
lock verification belong beside it because an import identity must resolve to
|
||
one deterministic source before compilation. The typed action engine, target
|
||
descriptors, native providers, and toolchain closure belong in the core because
|
||
they share invalidation and link correctness with package objects. Test/doc/
|
||
install are thin product selections/materializations over that same graph.
|
||
|
||
Network transport remains a separate operation, though exposed by `ww`, because
|
||
fetching is not building. Credential policy, OS package installation, registry
|
||
hosting, signing authority, remote execution, deployment, and general release
|
||
automation stay outside. The finite `generate` action is the boundary: it lets
|
||
outer domain tools transform declared artifacts without turning WW into their
|
||
framework.
|
||
|
||
## 13. Migration plan
|
||
|
||
Migration cost does not affect the decision. It is nevertheless material: the
|
||
current directly implicated scaffolding is at least 7,969 lines across the
|
||
Makefile, two drivers, two interface writers, package coordinator, and test
|
||
wrapper, plus 4,728 lines in the focused separate-build/package/byte-identity/
|
||
driver tests counted for this audit. Compiler export/import logic, new native
|
||
adapters, and bootstrap work add new scope not represented by those deletion
|
||
counts.
|
||
|
||
The planning estimate is **15–22 engineer-months** for the first production
|
||
Linux/amd64 toolchain, including tests, migration, and deletion, plus **1–2
|
||
engineer-months per materially different additional official target/sysroot**.
|
||
This is an estimate for staffing and sequencing, not a reason to retain a weaker
|
||
architecture.
|
||
|
||
Every phase below ends in a bisect-clean commit. Experimental components are not
|
||
installed as a second user-facing build path. Until the point of no return, the
|
||
old command remains the only production path; after it, the new command is the
|
||
only path.
|
||
|
||
### Phase 0 — conformance corpus and protocol freeze
|
||
|
||
- Land this decision and the exhaustive checked-in `protocol/wwar-v1.idl`. The
|
||
IDL mechanically assigns every remaining nested tag/enum and is rejected if
|
||
it adds or changes any semantic field, ordering/default, reference lowering,
|
||
or key rule specified here. Generate codecs plus canonical vectors for tree
|
||
hashing/WWAR/action keys, target descriptors, manifest/lock grammar, native ABI
|
||
records, link plans, and failure diagnostics.
|
||
- Turn current measurement fixtures into versioned benchmarks: 1/8/15-package
|
||
graphs, direct/transitive API changes, corrupted cache artifacts, two absolute
|
||
roots, C/WW stage equivalence, and B/H/T generator traps.
|
||
- Gate: freeze the IDL digest; independent generated and hand-written readers
|
||
reproduce every digest from the vectors and reject every malformed union,
|
||
unresolved final reference, and record-kind substitution; no production
|
||
behavior changes.
|
||
|
||
### Phase 1 — compiler export and package protocol
|
||
|
||
- Implement deterministic `.wwe`/`.wwlm` writing and direct-interface reading in
|
||
Cstage and self-hosted compiler paths behind test-only entry points.
|
||
- Add package identity/alias syntax, strict directory enumeration, compiler
|
||
import extraction, target suffix selection, cycle/collision/internal checks,
|
||
and protocol compatibility diagnostics.
|
||
- Prove the deep public type closure and API digest with existing standard-library
|
||
graphs. Reject `.wwi` input in the experimental path; do not translate it.
|
||
- Gate: Cstage/WWstage emit byte-identical vectors, importers open only direct
|
||
export files, and API propagation stops on unchanged middle exports.
|
||
|
||
### Phase 2 — one action engine and local CAS
|
||
|
||
- Implement WWAR, typed action graph, lazy keys, scheduler, atomic CAS/results,
|
||
project index, corruption quarantine, graph JSON, and causal explanation.
|
||
- Add deterministic package/archive/link adapters using the existing compiler,
|
||
assembler, and linker as explicitly hashed tools. This is a temporary adapter,
|
||
not a compatibility promise.
|
||
- Keep the engine under an internal test binary; the installed `ww` still follows
|
||
the old production path.
|
||
- Gate: all action-input mutation and failure-injection tests pass; identical warm
|
||
builds execute no compiler, assembler, archiver, or linker.
|
||
|
||
### Phase 3 — module, lock, source, and workspace layer
|
||
|
||
- Implement the closed manifest grammar, monotonic selector, canonical lock,
|
||
HTTPS source-index/archive protocol, immutable source store, signatures,
|
||
explicit add/update/lock/fetch, overlays, and vendor index.
|
||
- Build/test/doc/install remain network-denied from their first experimental use.
|
||
- Gate: frozen offline builds work from project source + complete locked source
|
||
closure + lock + installed named toolchain; collision/downgrade/hash/
|
||
path-normalization attacks fail loudly.
|
||
|
||
### Phase 4 — native, target, and external toolchain closure
|
||
|
||
- Implement full target descriptors, B/H/T lowering, C/assembly/native-provider
|
||
records, generated actions/sandbox, object sidecars, exact link plans, sysroot,
|
||
libc/CRT/loader/SDK/runtime, shared-library installation, and freestanding
|
||
products.
|
||
- Package supported assembler/linker/archive/C tools as immutable external
|
||
closures. Implement an external-assembler-compatible textual emission backend,
|
||
then have the pinned assembler produce `package.o` inside `ww.package`; port all WW/Plan-9-dialect
|
||
runtime and user assembly to a declared supported external dialect, and pass
|
||
object/link parity. An argv adapter alone cannot consume current `w6a` syntax.
|
||
Stop relying on WW-owned `w6a`/`w6l` before the experimental gate passes.
|
||
- Gate: native conflict, archive group/order, linker-script include, host leak,
|
||
freestanding, shared loader, and at least one real cross-target suite pass.
|
||
|
||
### Phase 5 — repository and consumer conversion rehearsal
|
||
|
||
- Define the final identity/layout conversion in a one-shot checker/rewriter under
|
||
`tools/migrate-build/`: folded files become directory packages; aliases,
|
||
manifests, native providers, products, generated inputs, and locks are emitted.
|
||
It is not an import resolver or runtime compatibility layer.
|
||
- Until cutover, CI applies that tool to a fresh ignored shadow tree. Tracked
|
||
production source stays in old syntax, so the old command remains its only
|
||
production path; the shadow is regenerated, never a second maintained source
|
||
tree or shipped interface.
|
||
- Dogfood the internal engine on the converted shadow of the standard library,
|
||
compiler tools, tests, examples, install layout, CI/package inputs, and sample
|
||
downstream consumers. Compare semantics, diagnostics, performance, artifacts
|
||
where protocols permit, and complete graph explanations.
|
||
- Gate: the regenerated full shadow, distribution/frozen-offline build,
|
||
self-contained installed toolchain outside the build tree, downstream samples,
|
||
and bootstrap inputs pass without an old-format edge.
|
||
|
||
### Phase 6 — new bootstrap and reproducible release
|
||
|
||
- Implement/gate the portable C recovery backend and snapshot generator, then
|
||
generate/check in `bootstrap/ww0.c` and the fixed plan. Produce stages 1/2/3/4,
|
||
fixed-point and diverse-seed-compilation reports, signed toolchain
|
||
bundles, and recovery documentation on a clean machine with no WW compiler.
|
||
- Gate: stage 2 equals stage 3 semantically and stage 3 equals stage 4 including
|
||
raw action/result records in two roots/concurrency levels; project plus complete
|
||
locked source closure and published named tool closure reproduce every release
|
||
artifact.
|
||
|
||
### Phase 7 — point of no return and deletion
|
||
|
||
In one atomic, bisect-clean cutover commit:
|
||
|
||
1. install the new engine as `ww` and make it the sole build/test/bootstrap path;
|
||
2. switch repository imports, manifests, locks, toolchains, CI, installation,
|
||
and release jobs to their final forms;
|
||
3. delete both old drivers, both `.wwi` writers, driver-side `.wwi` concatenation,
|
||
composed-unit/module-wrapper logic, generic C/self-hosted lexer/parser/checker/
|
||
symbol/codegen support for module directives, `-w`,
|
||
old import search/file folding, separate test coordinator,
|
||
duplicated production Make dependency graph, current Cstage bootstrap, and
|
||
retired owned assembler/linker path; and
|
||
4. delete the migration rewriter after all supported consumers have used its
|
||
released standalone copy; keep only a format-error guide.
|
||
|
||
The commit does not accept old `.wwi`, dotted/file imports, `-I`, `-w`, raw
|
||
library searches, or old work directories. No alias, warning period inside the
|
||
compiler, environment switch, or fallback subprocess retains a dual system.
|
||
|
||
### Phase 8 — consolidation
|
||
|
||
- Remove temporary parity fixtures that test deleted artifact bytes while
|
||
retaining semantic, action-key, reproducibility, and bootstrap regression
|
||
tests.
|
||
- Publish migration statistics and archive the old documentation as historical
|
||
release material outside the live manual.
|
||
- Gate: repository search and executable tracing show one graph constructor, one
|
||
package resolver, one cache, one test route, and one bootstrap route.
|
||
|
||
## 14. Validation plan and release gates
|
||
|
||
### 14.1 Unit and format tests
|
||
|
||
- Golden and adversarial vectors for WWAR, source-tree digest, action key, CAS
|
||
tree/result objects, `.wwe`, `.wwlm`, manifest, lock, work, vendor index,
|
||
target/toolchain/native records, and canonical JSON.
|
||
- Unicode normalization, case folding, traversal, symlink/device, duplicate key,
|
||
unknown schema/field, oversized input, truncation, hash collision simulation,
|
||
and malformed binary tests.
|
||
- Resolver vectors for minimum selection, incompatible-major identities,
|
||
workspace identity preservation, source origin independence, vendor matching,
|
||
internal packages, aliases, nested module/root-versus-parent-subpackage
|
||
identity collisions (including `/vN`), cycles, and target source specificity.
|
||
|
||
### 14.2 Package/interface tests
|
||
|
||
- One- and multi-file directory membership, file-scoped import use, same/external
|
||
tests, test-only packages, examples/docs, and generated fragments with import/
|
||
package rejection.
|
||
- Direct dependency interface-open counts equal package-graph indegree, never
|
||
transitive closure size.
|
||
- Private dependency edits preserve importer keys; exported but unused additions
|
||
rebuild direct importers; unchanged middle `.wwe` stops propagation; layout,
|
||
calling-convention, runtime ABI, compiler, target, and profile changes rebuild
|
||
the exact affected cone.
|
||
- Parameter renames, comments, private declaration ordering, and absolute source
|
||
paths do not change `.wwe`; semantic/ABI changes do.
|
||
|
||
### 14.3 Native and link tests
|
||
|
||
- C scalar/aggregate/variadic/callback/TLS/unwind ABI probes against an
|
||
independently compiled C harness for every official target.
|
||
- Binding generation changes on header tree, macro map, preprocessor, tool, C
|
||
ABI, and target; undeclared include access is denied.
|
||
- Assembly dialect/CPU mismatch, wrong-format objects, PIC/shared rules, archive
|
||
extraction order, repeated libraries, groups, whole archive, weak/strong
|
||
symbols, version scripts, linker-script includes, and deterministic archives.
|
||
- Exactly-one libc/runtime/provider enforcement; identical coalescing; duplicate
|
||
native ABI/symbol conflict; shared SONAME/loader/runtime installation closure;
|
||
freestanding entry/script with proof that no libc/CRT/loader appears.
|
||
|
||
### 14.4 Cross and sandbox tests
|
||
|
||
- Matrix with `B != H`, compiler product `H != T`, and all three distinct where
|
||
infrastructure permits. A B generator emits an H input; an H binary is never
|
||
executed during build; T objects never enter the H link.
|
||
- Poison host `PATH`, includes, libraries, SDK, locale, time, home, current
|
||
directory, and environment. Every attempted undeclared read/write/network/
|
||
process/tool access fails with its action identity.
|
||
- Cross tests build without a runner, run only with an exact declared runner,
|
||
and fail under `--require-run` when none exists.
|
||
|
||
### 14.5 Cache and failure injection
|
||
|
||
- Change each field in section 6.2 individually and require a key change; change
|
||
each expressly non-semantic observation and require no key change.
|
||
- Bit-flip blobs, trees, results, action mappings, tools, export data, objects,
|
||
and partial files at every publication boundary. Require quarantine/rebuild,
|
||
never acceptance or broad deletion.
|
||
- Concurrent identical publishers, killed compiler/linker, disk full, rename
|
||
failure, read-only output, interrupted materialization, stale project index,
|
||
malicious remote mapping, bad cache signature, and remote outage.
|
||
- Clean, local-hit, explicit remote-import, and no-cache builds must yield the
|
||
same result digests. Test executions still run.
|
||
|
||
### 14.6 Reproducibility and bootstrap
|
||
|
||
- Compare every artifact/action record across two absolute checkouts, source/
|
||
cache/output roots, usernames, locales, time zones, umasks, concurrency levels,
|
||
filesystem enumeration orders, and cold/warm caches.
|
||
- Verify debug/release, static/shared, hosted/freestanding, generated/native, and
|
||
signed/unsigned products. Impure profiles must state exactly why they are
|
||
outside the byte promise and must never enter shared cache.
|
||
- Build `ww0` with each supported host C toolchain, reach stages 2/3/4, run the
|
||
semantic then raw-record fixed point and diverse seed compilation, corrupt each stage input, and recover on a host
|
||
with no WW installation.
|
||
|
||
### 14.7 Performance and migration gates
|
||
|
||
On the audit's fixed eight-CPU reference host, the first release MUST:
|
||
|
||
- run package compilation in parallel and complete the full clean toolchain
|
||
build no slower than the measured 15.515 s `make -j8` baseline;
|
||
- perform a warm 15-package build with no compiler, assembler, archiver, or
|
||
linker process and no slower than the measured 0.053 s driver baseline;
|
||
- read only direct exports and avoid composed-unit duplication;
|
||
- store one CAS copy of duplicate tool/package content across all products;
|
||
- produce fully path-independent official artifacts, including host-side tools;
|
||
and
|
||
- provide a typed explanation for every deliberately induced rebuild.
|
||
|
||
Before cutover, every tracked current package/test/install/bootstrap consumer has
|
||
an assigned new identity and a passing converted test. The cutover gate includes
|
||
a repository-wide search for old forms and executable traces proving no old
|
||
driver, interface, workdir, library search, or test-coordinator path executes.
|
||
|
||
## 15. Evidence appendix
|
||
|
||
### 15.1 Research method and version pins
|
||
|
||
Research used official documentation, standards/manuals, release metadata, and
|
||
current upstream source—not comparison articles or community summaries. The
|
||
evolving-system snapshot was taken 2026-08-09:
|
||
|
||
| System | Pinned snapshot used |
|
||
|---|---|
|
||
| Plan 9 | live official `9p.io` Volume 2 documents, accessed 2026-08-09; pages are not versioned/dated editions |
|
||
| Go | online docs accessed 2026-08-09; Go 1.26.5 `go1.26.5` source tag |
|
||
| Hare | online docs accessed 2026-08-09; Hare 0.26.0 source, released 2026-02-13 |
|
||
| Odin | online docs accessed 2026-08-09; `dev-2026-07a`, commit `819fdc7a80667498b8b365999f1475a66c358640` |
|
||
| Zig | Zig 0.16.0, official metadata release date 2026-04-13; source archive SHA-256 `43186959edc87d5c7a1be7b7d2a25efffd22ce5807c7af99067f86f99641bfdf` |
|
||
| Rust/Cargo | Rust/Cargo 1.97.1, released 2026-07-16; Cargo `0.98.0` commit `c980f4866141969fab6254a680546a277789d6f0` |
|
||
| Bazel | Bazel 9.2.0 documentation/source |
|
||
| Nix | Nix 2.35.2 manual/source |
|
||
|
||
Zig's separate bootstrap-source archive inspected for this decision had SHA-256
|
||
`2a8266a4205772ef40838c8cbdf14875855a515ff3adf89b49c2d2ae93613d10`.
|
||
These pins matter because programmable-build and package behavior changes between
|
||
releases; this document does not generalize an old Zig/Cargo observation to an
|
||
unidentified current version.
|
||
|
||
### 15.2 Primary-source findings
|
||
|
||
**Pike, Plan 9, and early Go.** Pike's sources support explicit computable
|
||
imports, cycle rejection, direct compiled export information, fast compilation,
|
||
and orthogonal concepts. The collective Plan 9 papers add system-wide placement
|
||
of complexity and transparent encodings. Plan 9's namespace papers demonstrate
|
||
contextual filesystem composition; this document infers that contextual location
|
||
must not serve as WW's versioned distribution identity. `mk` shows complete graph scheduling while still relying
|
||
on timestamps, recipes, and environment. The architecture borrows the former
|
||
principles and replaces the latter ambient assumptions.
|
||
|
||
- [The Go Programming Language, 2009](https://go.dev/talks/2009/go_talk-20091030.pdf)
|
||
- [Go at Google: Language Design in the Service of Software Engineering, 2012](https://go.dev/talks/2012/splash.article)
|
||
- [Simplicity is Complicated, 2015](https://go.dev/talks/2015/simplicity-is-complicated.slide)
|
||
- [Go in Go, 2015](https://go.dev/talks/2015/gogo.slide)
|
||
- [Plan 9 overview](https://9p.io/sys/doc/9.html)
|
||
- [The Use of Name Spaces in Plan 9](https://9p.io/sys/doc/names.html)
|
||
- [Maintaining Files on Plan 9 with Mk](https://9p.io/sys/doc/mk.html)
|
||
- [Plan 9 Mkfiles](https://9p.io/sys/doc/mkfiles.html)
|
||
- [Plan 9 compiler suite](https://9p.io/sys/doc/comp.html)
|
||
|
||
**Later Go.** The original Go command demonstrates source-derived package DAGs,
|
||
directory conventions, and compiler-owned dependency work. The modern module
|
||
reference documents module identity, Minimal Version Selection, major-version
|
||
paths, checksums, and commands that may resolve/download modules. Current command
|
||
and source-install documents also separate build cache/toolchain/bootstrap
|
||
behavior. These are evidence, not automatic WW defaults; in particular WW uses
|
||
an exact lock and forbids implicit build-time acquisition.
|
||
|
||
- [About the Go command](https://go.dev/doc/articles/go_command)
|
||
- [Go module reference](https://go.dev/ref/mod)
|
||
- [`go` command reference](https://go.dev/cmd/go/)
|
||
- [Go toolchain selection](https://go.dev/doc/toolchain)
|
||
- [Installing Go from source](https://go.dev/doc/install/source)
|
||
- [Perfectly Reproducible, Verified Go Toolchains](https://go.dev/blog/rebuild)
|
||
- [Go's supply-chain security](https://go.dev/blog/supply-chain)
|
||
- [`go1.26.5` source](https://go.googlesource.com/go/+/refs/tags/go1.26.5/)
|
||
|
||
**Local-source systems.** Hare demonstrates how far a disciplined directory
|
||
module/search-root system can go with little package machinery; its documented
|
||
system-library and cross interfaces also expose why raw host paths and tool
|
||
defaults are insufficient for WW's hard gates. Odin's collections reinforce the
|
||
local namespace option but do not add a locked whole native closure.
|
||
|
||
- [Hare modules](https://harelang.org/documentation/usage/modules.html)
|
||
- [Hare project structure](https://harelang.org/documentation/usage/project-structure.html)
|
||
- [Hare system libraries](https://harelang.org/documentation/usage/system-libraries.html)
|
||
- [Hare cross compilation](https://harelang.org/documentation/usage/cross.html)
|
||
- [Hare 0.26.0 source](https://git.sr.ht/~sircmpwn/hare/tree/0.26.0)
|
||
- [Odin overview](https://odin-lang.org/docs/overview/)
|
||
- [Odin pinned source](https://github.com/odin-lang/Odin/tree/819fdc7a80667498b8b365999f1475a66c358640)
|
||
|
||
**Integrated project systems.** Zig supplies useful target/native vocabulary,
|
||
source hashes, local aliases, declared generated-file edges, and cross-building.
|
||
Cargo supplies package IDs, exact lock checksums, workspace behavior, resolver
|
||
documentation, and a native `links` uniqueness rule. Their programmable build
|
||
program/script model is deliberately rejected; a dependency host program is a
|
||
larger and less inspectable abstraction than WW's finite action record.
|
||
|
||
- [Zig build system](https://ziglang.org/learn/build-system/)
|
||
- [Zig overview](https://ziglang.org/learn/overview/)
|
||
- [Zig 0.16.0 language reference](https://ziglang.org/documentation/0.16.0/)
|
||
- [Zig release metadata](https://ziglang.org/download/index.json)
|
||
- [Zig 0.16.0 source](https://github.com/ziglang/zig/tree/0.16.0)
|
||
- [Cargo 1.97.1 reference](https://doc.rust-lang.org/1.97.1/cargo/reference/)
|
||
- [Cargo current reference entry](https://doc.rust-lang.org/cargo/reference/)
|
||
- [Cargo 1.97.1 resolver](https://doc.rust-lang.org/1.97.1/cargo/reference/resolver.html)
|
||
- [Cargo current resolver entry](https://doc.rust-lang.org/cargo/reference/resolver.html)
|
||
- [Cargo 1.97.1 build scripts](https://doc.rust-lang.org/1.97.1/cargo/reference/build-scripts.html)
|
||
- [Cargo current build-script entry](https://doc.rust-lang.org/cargo/reference/build-scripts.html)
|
||
- [Cargo 1.97.1 build cache](https://doc.rust-lang.org/1.97.1/cargo/reference/build-cache.html)
|
||
- [Cargo current build-cache entry](https://doc.rust-lang.org/cargo/reference/build-cache.html)
|
||
- [Cargo pinned source](https://github.com/rust-lang/cargo/tree/c980f4866141969fab6254a680546a277789d6f0)
|
||
|
||
**Hermetic invariants.** Bazel defines useful distinctions between declared
|
||
actions, execution platforms, action results, and content-addressed remote data.
|
||
Nix derivations demonstrate an inspectable build record whose output depends on
|
||
declared inputs. WW borrows those invariants only. It rejects Bazel's general
|
||
rule/ecosystem machinery and Nix's evaluator/store-as-package-manager as excess
|
||
for one language toolchain.
|
||
|
||
- [Bazel current hermeticity entry](https://bazel.build/concepts/hermeticity)
|
||
- [Bazel 9.2.0 hermeticity](https://bazel.build/versions/9.2.0/basics/hermeticity)
|
||
- [Bazel remote caching](https://bazel.build/remote/caching)
|
||
- [Remote Execution API](https://github.com/bazelbuild/remote-apis/blob/master/build/bazel/remote/execution/v2/remote_execution.proto)
|
||
- [Bazel 9.2.0 source](https://github.com/bazelbuild/bazel/tree/9.2.0)
|
||
- [Nix current derivation entry](https://nix.dev/manual/nix/latest/store/derivation/)
|
||
- [Nix 2.35 derivations](https://nix.dev/manual/nix/2.35/store/derivation/)
|
||
- [Nix build process](https://nix.dev/manual/nix/2.35/store/building.html)
|
||
- [Nix 2.35.2 source](https://github.com/NixOS/nix/tree/2.35.2)
|
||
|
||
**Native and reproducible toolchains.** LLVM and Clang define data-layout and
|
||
cross/toolchain choices that must be explicit for ABI-correct code. GNU manuals
|
||
document build/host/target vocabulary and semantic linker/archive/script
|
||
behavior. The reproducible-builds definition supplies the correct boundary:
|
||
same source, environment, instructions, and dependencies—not merely “same
|
||
compiler source.”
|
||
|
||
- [LLVM language-reference data layout](https://llvm.org/docs/LangRef.html#data-layout)
|
||
- [Clang cross compilation](https://clang.llvm.org/docs/CrossCompilation.html)
|
||
- [Clang toolchain](https://clang.llvm.org/docs/Toolchain.html)
|
||
- [Autoconf target triplets](https://www.gnu.org/software/autoconf/manual/autoconf-2.71/html_node/Specifying-Target-Triplets.html)
|
||
- [GCC language standards/runtime implications](https://gcc.gnu.org/onlinedocs/gcc/Standards.html)
|
||
- [GCC link options](https://gcc.gnu.org/onlinedocs/gcc/Link-Options.html)
|
||
- [GNU linker](https://sourceware.org/binutils/docs/ld.html) and [linker scripts](https://sourceware.org/binutils/docs/ld/Scripts.html)
|
||
- [Reproducible Builds definition](https://reproducible-builds.org/docs/definition/)
|
||
|
||
### 15.3 Repository and empirical evidence record
|
||
|
||
The source audit covered the Makefile; C and self-hosted drivers; compiler entry
|
||
and both interface writers; package/test coordinator; language and test specs;
|
||
bootstrap documents/recipes; and package, separate-compilation, incremental,
|
||
byte-identity, and bootstrap tests. The audit found documentation drift as well
|
||
as code defects: the root instructions count five unit suites while Make lists
|
||
six, and the test-system document describes three pinned data byte divergences
|
||
while the current Make divergence list is empty. Neither drift item influences
|
||
the architecture, but both require cleanup during migration.
|
||
|
||
Raw commands, timings, traces, hashes, fixture logs, environment capture, and
|
||
interpretation for section 11.4 are retained in the session checkpoint under
|
||
`empirical/RESULTS.md` and `empirical/raw/`. The experiments were deliberately
|
||
small and answered only material uncertainties: actual process scheduling,
|
||
direct versus transitive interface consumption, private/public/link-only blast
|
||
radius, nonzero corruption, path identity, Cstage/WWstage symmetry, duplicate
|
||
storage, and hidden-host execution. No toy result is treated as proof that the
|
||
new architecture is complete; sections 13–14 turn each inference into a gate.
|
||
|
||
### 15.4 Assumptions
|
||
|
||
- Breaking import syntax, package layout, compiler flags, cache/workdir format,
|
||
bootstrap artifacts, and consumer builds is authorized.
|
||
- WW can add explicit C ABI/representation annotations and change its compiler
|
||
export protocol without preserving `.wwi` compatibility.
|
||
- Official releases can distribute or name legally usable immutable tool,
|
||
sysroot, runtime, and SDK closures for each supported target.
|
||
- SHA-256 is the version-1 content primitive; every format is domain-separated
|
||
and versioned so a future digest transition can be explicit.
|
||
- A first release may support fewer frozen targets than impure development
|
||
targets; it may not call an impure target “hermetic.”
|
||
- Byte reproducibility covers build artifacts, not identical behavior of an
|
||
external kernel, firmware, network service, or hardware.
|
||
|
||
### 15.5 Remaining risks and bounded experiments
|
||
|
||
These experiments are implementation gates inside the selected architecture;
|
||
they do not reopen its package identity, no-script, one-graph, lock/network,
|
||
direct-export, content-key, native-provider, or stage-zero choices.
|
||
|
||
1. **Deep `.wwe` closure.** Prototype the real standard-library type graph and
|
||
prove that one direct export file contains every transitive layout/type fact
|
||
required without leaking non-semantic source data. Adjust record layout, not
|
||
direct-dependency semantics.
|
||
2. **Generated C seed.** Measure generated `ww0.c` size, C99 portability, host-C
|
||
variance, and fixed-point convergence on at least three unrelated C
|
||
implementations. Restrict/repair the recovery emitter rather than introducing
|
||
a second maintained compiler or opaque-permanent binary seed.
|
||
3. **External linker/tool bundles.** Certify deterministic debug info, build ID,
|
||
archives, scripts, shared-loader metadata, redistribution rights, and resource
|
||
closure. Change tool adapters/bundle membership if needed; do not restore
|
||
ambient driver defaults.
|
||
4. **Sandbox portability.** Implement denial conformance on every official B
|
||
platform, including process children, filesystem race/symlink attacks, clock,
|
||
randomness, and network namespaces. A platform that cannot enforce it remains
|
||
non-frozen rather than gaining an undeclared exception.
|
||
5. **Native-provider coverage.** Exercise ELF first, then Mach-O frameworks/SDKs,
|
||
PE/COFF import libraries, symbol versioning, and kernel image builders. Extend
|
||
the closed typed schema by version where genuinely required; do not add raw
|
||
search or a general build language.
|
||
6. **Registry/private-source protocol.** Test mirror failover, redirects,
|
||
credential isolation, key rotation, provenance, yanked releases, and malicious
|
||
archives. Vendored/exact-origin operation remains the deterministic fallback.
|
||
7. **Performance.** Validate compiler worker strategy and CAS hashing against the
|
||
measured 15-package/full-toolchain budgets. Process topology may change while
|
||
action boundaries and keys remain fixed.
|
||
|
||
### 15.6 Final decision trace
|
||
|
||
The documented Pike evidence shapes the design: computable direct dependencies,
|
||
fast compilation, cycle rejection, package boundaries, and orthogonal concepts;
|
||
the collective Plan 9 papers add transparent encodings and system-wide placement
|
||
of complexity. Later Go work demonstrates one possible module/cache/toolchain
|
||
evolution but does not decide WW's answer. This document then applies those
|
||
principles to requirements early Go's package/build model did not expose
|
||
completely: foreign ABI contracts, C headers, external assembly and objects,
|
||
archive/link ordering, linker scripts, CRT/libc/sysroot identity, host generators,
|
||
and explicit build/host/target closures.
|
||
|
||
The resulting binding conclusion is singular: **replace the current system with
|
||
WW Action Build exactly as specified above**. Do not preserve the old path, and
|
||
do not substitute a programmable project framework or a source-only package
|
||
command during implementation.
|