From 3157e1d6887565b2551e050a8c4c067408566ea6 Mon Sep 17 00:00:00 2001 From: Hojun-Cho Date: Tue, 11 Aug 2026 19:36:58 +0900 Subject: [PATCH] docs: simplify WWAR phase boundaries --- docs/build-system.md | 160 ++++++++++++++++++++++++++++++++----------- 1 file changed, 121 insertions(+), 39 deletions(-) diff --git a/docs/build-system.md b/docs/build-system.md index 27ada257..f1e26c45 100644 --- a/docs/build-system.md +++ b/docs/build-system.md @@ -157,6 +157,68 @@ absence of a garbage collector says nothing about version resolution. | Assembler/linker ownership | No permanent ownership. The toolchain descriptor pins complete implementations. The current WW tools may bridge migration only. | | Stage zero | One release-generated, checked-in portable C99 compiler snapshot plus a tiny declarative bootstrap plan and digest file. | +### 1.4 Corrective protocol boundary (2026-08-11) + +The first Phase 0 experiment over-scoped the protocol freeze. It turned package +resolution, manifest parsing, compiler projections, action lowering, provider +recursion, graph traversal, scheduling, cache policy, failure precedence, and +bootstrap assertions into a declarative expression language. Its checker then +implemented those operations again. That experiment is preserved as recoverable +migration evidence, but it is not the production architecture. + +The correction follows the separation visible in the pinned Go source. Go reads +imports from source with an imports-only parse and resolves them in ordinary +loader code ([`go/build/read.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/go/build/read.go#272), +[`cmd/go/internal/load/pkg.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/load/pkg.go#1775)). +The compiler writes a narrow export representation in compiler code +([`cmd/compile/internal/noder/writer.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/noder/writer.go)), +while `cmd/go` builds and schedules an in-memory action graph with ordinary Go +functions ([`work/action.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/action.go#85), +[`work/exec.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#73)). +Action IDs and cache storage/validation are executable hashing and storage +operations, not schema programs +([`work.buildActionID`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/work/exec.go#260), +[`internal/cache`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/go/internal/cache/cache.go#95)). +Go's `cmd/dist` performs concrete staged builds and checks that the final targets +are not stale. Separate compiler reproducibility tests compare repeated outputs +byte-for-byte, while the release process independently rebuilds and compares +archives bit-for-bit +([`cmd/dist/build.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/dist/build.go#1404), +[`reproduciblebuilds_test.go`](https://go.googlesource.com/go/+/refs/tags/go1.26.5/src/cmd/compile/internal/test/reproduciblebuilds_test.go), +[`rebuild` account](https://go.dev/blog/rebuild)). + +WW adopts that division, not Go's module/network/toolchain policy. Normal typed +C/WW code MUST own source loading, parsing, resolution, compiler behavior, +lowering, orchestration, storage, and bootstrap execution. Declarative schemas +MUST describe wire representation only. Tests MUST verify executable behavior; +a schema or proof-shaped record MUST NOT stand in for running it. A generator is +permitted only for repetitive codec data and MUST be small, generic, +deterministic, and byte-for-byte reproducible. + +Phase 0 therefore freezes only WWAR framing and primitive canonical encoding; +record/enum/union tags, field order, encoded defaults and record kinds; exact +domain-separated digest and action-key byte formulas; compact positive and +malformed-wire vectors; a small reference codec; deterministic data-only codec +generation; and its repository gate. It does not freeze algorithms for deriving +the represented records. Every declared record tag remains encoded, including +an optional field's empty `encoded_default`; absence is not default insertion. + +The owning implementation phases are binding: + +| Behavior removed from the Phase 0 experiment | Owning phase | +|---|---| +| source imports, package graph/cycles, `.wwe`/`.wwlm`, compiler export and public type/ABI projections | Phase 1 | +| action construction, graph traversal, scheduling, CAS/cache, environment/sandbox and build failure behavior | Phase 2 | +| manifest/lock/work/vendor text parsing, module/source resolution, fetching, source-store policy and source-tree construction | Phase 3 | +| native/provider recursion, lowering, link-plan construction, tool adapters and platform policy | Phase 4 | +| actual staged bootstrap, fixed-point rebuild and byte comparison | Phase 6 | + +WW-specific guarantees remain stronger and explicit: frozen artifact builds are +deterministic and offline, selections are locked, artifacts are content-addressed, +cached objects are rehashed on read, and bootstrap is established by rebuilding and +comparing actual bytes. At cutover there is one user-facing build path, as +already required by the migration plan. + ## 2. Normative vocabulary | Term | Exact meaning | @@ -1589,15 +1651,16 @@ closure, 8 identical logical stage-2/3/4 action roots, 9 semantic fixed-point output roles, and 10 raw-record fixed-point roles. No clause or executable step exists beyond that closed plan. -These sections fix the architectural fields and every reference/key -transformation. Phase 0 transcribes every named and nested record into the -normative checked-in `protocol/wwar-v1.idl`, assigns any remaining mechanical -numeric enum values and field tags, and freezes its digest before Phase 1. The -IDL must state every cardinality, default, ordering rule, enum, discriminator, -cross-field constraint, and top-level record kind; generated codecs/tables and -golden vectors derive from it. The IDL, WWAR 1, and those vectors jointly -determine bytes. Phase 0 may not add a semantic field, change a default/order -rule, retain producer identity in a final action, or alter action-key behavior. +These sections fix the architectural fields, but they do not make Phase 0 an +executable specification of every reference or key transformation. Phase 0 +transcribes only their wire-visible record fields, tags, enum values, union +discriminants, encoded defaults, field order, record kinds, wrapper framing, and +digest preimage formulas into the checked-in compact schemas. Cross-field +validity, construction, resolution, projection, lowering, and failure behavior +belong to the executable phase that implements them. The Phase 0 generator and +golden vectors determine bytes, not future build-engine semantics. No phase may +silently add a wire field, renumber an assignment, change an encoded default, or +alter a frozen digest formula. ## 7. Interface and artifact protocol @@ -2894,19 +2957,22 @@ only path. ### Phase 0 — conformance corpus and protocol freeze -- Land this decision and the exhaustive checked-in `protocol/wwar-v1.idl`. The - IDL mechanically assigns every remaining nested tag/enum and is rejected if - it adds or changes any semantic field, ordering/default, reference lowering, - or key rule specified here. Generate codecs plus canonical vectors for tree - hashing/WWAR/action keys, target descriptors, manifest/lock grammar, native ABI - records, link plans, and failure diagnostics. -- Turn current measurement fixtures into versioned benchmarks: 1/8/15-package - graphs, direct/transitive API changes, corrupted cache artifacts, two absolute - roots, C/WW stage equivalence, and B/H/T generator traps. -- Gate: freeze the IDL digest; independent generated and hand-written readers - reproduce every digest from the vectors and reject every malformed union, - unresolved final reference, and record-kind substitution; no production - behavior changes. +- Land the compact checked-in `protocol/schema/` modules for WWAR framing, + record/enum/union/default/kind assignments, wrappers, and finite digest byte + formulas. Freeze the exact schema-file digests. Preserve representation-only + preimage records without embedding the algorithms that construct them. +- Generate data-only codec tables deterministically. Keep one schema-aware + reference codec and a second independent oracle limited to raw WWAR framing, + hashing, and record identity; neither may implement future compiler or build + behavior. +- Gate: strict duplicate-key UTF-8/NFC schema loading, byte-identical generation + in separate fresh directories, compact valid/invalid vectors, all assignment + coverage, stable malformed-length precedence, domain separation, action-key + vectors, and record-kind substitution rejection. No production behavior + changes beyond adding this conformance gate. +- Package/API, graph/cache, native B/H/T, and bootstrap measurements remain useful + fixtures, but become tests in Phases 1, 2, 4, and 6 respectively. They are not + Phase 0 semantic answer tables. ### Phase 1 — compiler export and package protocol @@ -2915,15 +2981,18 @@ only path. - Add package identity/alias syntax, strict directory enumeration, compiler import extraction, target suffix selection, cycle/collision/internal checks, and protocol compatibility diagnostics. -- Prove the deep public type closure and API digest with existing standard-library - graphs. Reject `.wwi` input in the experimental path; do not translate it. +- Construct and test the deep public type closure and public type/ABI digest + preimages in compiler code with existing standard-library graphs. Reject + `.wwi` input in the experimental path; do not translate it. - Gate: Cstage/WWstage emit byte-identical vectors, importers open only direct export files, and API propagation stops on unchanged middle exports. ### Phase 2 — one action engine and local CAS -- Implement WWAR, typed action graph, lazy keys, scheduler, atomic CAS/results, - project index, corruption quarantine, graph JSON, and causal explanation. +- Integrate the frozen WWAR codec into production code and implement typed pure + action-record/key functions, the action graph, lazy keys, scheduler, atomic + CAS/results, project index, corruption quarantine, graph JSON, environment and + sandbox policy, deterministic failure behavior, and causal explanation. - Add deterministic package/archive/link adapters using the existing compiler, assembler, and linker as explicitly hashed tools. This is a temporary adapter, not a compatibility promise. @@ -2934,9 +3003,13 @@ only path. ### Phase 3 — module, lock, source, and workspace layer -- Implement the closed manifest grammar, monotonic selector, canonical lock, +- Implement the manifest, lock, work, and vendor text parsers as ordinary typed + parser code, then implement the closed grammar, monotonic selector, canonical + lock, HTTPS source-index/archive protocol, immutable source store, signatures, - explicit add/update/lock/fetch, overlays, and vendor index. + explicit add/update/lock/fetch, overlays, vendor index, and canonical + source-tree construction. The schema fixes only the resulting record bytes and + source-tree digest formula. - Build/test/doc/install remain network-denied from their first experimental use. - Gate: frozen offline builds work from project source + complete locked source closure + lock + installed named toolchain; collision/downgrade/hash/ @@ -2945,7 +3018,8 @@ only path. ### Phase 4 — native, target, and external toolchain closure - Implement full target descriptors, B/H/T lowering, C/assembly/native-provider - records, generated actions/sandbox, object sidecars, exact link plans, sysroot, + records, recursive provider selection/expansion, generated actions/sandbox, + object sidecars, exact link-plan construction, sysroot, libc/CRT/loader/SDK/runtime, shared-library installation, and freestanding products. - Package supported assembler/linker/archive/C tools as immutable external @@ -2979,10 +3053,13 @@ only path. - Implement/gate the portable C recovery backend and snapshot generator, then generate/check in `bootstrap/ww0.c` and the fixed plan. Produce stages 1/2/3/4, - fixed-point and diverse-seed-compilation reports, signed toolchain - bundles, and recovery documentation on a clean machine with no WW compiler. -- Gate: stage 2 equals stage 3 semantically and stage 3 equals stage 4 including - raw action/result records in two roots/concurrency levels; project plus complete + rebuild stages with the executable engine, compare the actual bytes of every + declared output directly, and produce fixed-point and diverse-seed-compilation + reports, signed toolchain bundles, and recovery documentation on a clean + machine with no WW compiler. +- Gate: stage 2 equals stage 3 and stage 3 equals stage 4 by explicit + byte-for-byte comparison, including raw action/result records, in two + roots/concurrency levels; project plus complete locked source closure and published named tool closure reproduce every release artifact. @@ -3020,12 +3097,17 @@ compiler, environment switch, or fallback subprocess retains a dual system. ### 14.1 Unit and format tests -- Golden and adversarial vectors for WWAR, source-tree digest, action key, CAS - tree/result objects, `.wwe`, `.wwlm`, manifest, lock, work, vendor index, - target/toolchain/native records, and canonical JSON. -- Unicode normalization, case folding, traversal, symlink/device, duplicate key, - unknown schema/field, oversized input, truncation, hash collision simulation, - and malformed binary tests. +- Phase 0 has golden and adversarial vectors only for WWAR representation, + source-tree digest bytes, action-key/record identity formulas, typed record + assignments, wrappers, and canonical schema JSON. +- Phase 0 tests UTF-8/NFC, duplicate schema keys, unknown wire type/schema field, + oversized declarations, truncation, exact-length mismatch, union shape, and + record-kind substitution. Case-fold collision, traversal, symlink/device, + cache collision/corruption, and semantic record tests land with their owning + executable phases. +- Phases 1–4 add behavior tests for CAS tree/result objects, `.wwe`, `.wwlm`, + manifest/lock/work/vendor parsing, and target/toolchain/native processing; + these are not encoded as Phase 0 vector outcomes. - Resolver vectors for minimum selection, incompatible-major identities, workspace identity preservation, source origin independence, vendor matching, internal packages, aliases, nested module/root-versus-parent-subpackage