Separate import lookup from raw CLI target compatibility in both production drivers. Package scans and test-support edges now create directory nodes only, while unit composition emits only the owning package's sorted sources after direct export data. Document the Go 1.26.5 ownership evidence and retained single-file root boundary.
Three same-cluster robustness holes. enumeratedir treated a failed
getdents64 (r<0) as EOF: a mid-walk error silently truncated the
package source list, and a first-read error was misdiagnosed as
"directory contains no WW package sources" -- it now routes the
caller's "cannot read directory" arm. run_test_bin's waitpid had no
EINTR retry and its return was unchecked, so an interrupted wait
left status==0 and reported a false test PASS (the do_run twin
already retried). The coordinator's pkgskipspace now also skips
/* */ before the package clause like the driver's sep_skip_space --
a source opening with a block comment built under ww but failed
coordinator discovery.
selfhost/, cmd/, internal/ join the tree-wide sweep: every section
banner dies (91 selfhost + the cmd C-style dividers -> 0); narration
and stale contracts deleted (pre-#22 bundler notes, retired
single-PT_LOAD and no-archive claims, superseded ABI tables); every
ref/harec/qbe cite, task cite, encoding/ELF contract, and rule-10
twin pointer kept; lost lifetime/rationale lines restored where the
sweep over-cut (elf_globals ownership, kwtab linear-scan). Comment-
only proven: all five wwstage tool binaries byte-identical across
the sweep; test-commit, test-byteid (161+1399, 0 pinned-divergent),
and test-bootstrap (fixed point + 991-995 byte-id) all exit 0.
The read-through banked 66 latent-bug leads (checkpoint).
Go compiles only _test.go files as tests; discovery now keys on the
_test.ww suffix alone. The line-leading-@test compatibility allowance
(noncanonical filenames admitted as test sources) is removed from both
driver stages and the coordinator. An @test declaration outside a
*_test.ww file is rejected loudly ("@test declaration outside
*_test.ww", wording byte-identical cs/ww) instead of silently running
under compose or silently dropping in a non-T build (#6). Tree audit
found zero real carriers; the two allowance fixtures flip canonical
(dep_test.ww, widget_test.ww). New pins: direnum attest-noncanon
reject row (both-stage stderr parity) and the coordinator
noncanonical_attest_rejected package row.
Go has no per-file test targets: the PACKAGE is the unit of testing
(`go test ./...`) and same-package test files compose together. The
59-target LIBRARY_TESTS fan-out (per-file -w workdirs, one Make rule
per suite) collapses to one line: `ww test -j $(JOBS) -w
out/wwbuild/wwtest-lib -I lib/ww lib/...` — the regex dir-route
precedent generalized. The lib/regex line and the per-file pattern
rule instance dissolve with it (test/lang keeps its own rule).
Measured before committing (-j4, strings edit row): old per-file
warm 4.7-4.9s; bare walk 5.2-5.6s — a real regression, so the
coordinator first gained the brief's persistent per-package workdir:
`-w DIR` on a package target forwards to wwpackage, which keys
DIR/<dir>_<pkg> per group and hands it to each inner `ww test -c`
build. Reuse stays entirely with the driver's existing
content-identity contract — the coordinator adds pure path policy,
no cache machinery. Both driver stages drop their package-target -w
rejects (forward instead); -w with -c stays rejected at the
coordinator (two ownership contracts). After: 3.9-4.0s on the edit
row, 1.0s warm no-op, 2.5s cold — faster than the old flow on every
row.
package_test's tree -w reject row becomes the positive contract
(cold+warm byte-stable stream, cs/ww same) plus the -c conflict
reject. libbyteid roster shape DECIDED: per-file fx entries stay —
every enrolled file is still standalone-buildable, so coverage is
byte-for-byte unchanged; the dir-mode entry form arrives only with
the B3 shared-helper split that first needs it. Docs: owner table,
target table, -w contract paragraph.
The last Open-driver-work bullet. `-c -o <name>` replaces the fixed
<package>.test stem for exactly one package; the coordinator rejects a
multi-package fan-out ("cannot use -o with multiple packages", Go's
`go test -o` rule) and -o without -c is rejected at the driver
("needs -c for a package target" — a plain run executes from the temp
root, so a caller-owned name has nothing to name). Both driver stages
byte-identical wording; -S keeps its single-file-only reject.
package_test gains the contract row (naming, fixed-stem absence,
artifact runs, both rejects, both drivers); the tree-mode -o row's
pinned wording follows the contract.
A -w DIR workdir replaces the fresh .sepwork scratch with a caller-owned
persistent package-artifact tree. A package is reused only when its
freshly composed unit byte-equals the committed unit and byte copies of
the compiler/assembler recorded in the dir equal the live tools — pure
content identity, no mtimes, no hashes, every decision reproducible
with cmp against plain files. Recompiles stage at .new names and commit
by rename, unit strictly last, so an interrupted build forces a
recompile and can never leave a committed unit vouching for uncommitted
artifacts; .o/.a additionally reject zero size (ELF/ar are never
empty), while .s/.wwi accept legitimate empties (FFI-only rt). A mode
stamp pins the -T/-S shape and the artifact protocol revision. Classic
scratch keeps its exact acquire/refuse/cleanup contract; run rejects
-w; dir-mode test rejects -w; both driver stages implement identical
behavior and wording.
PROJECT_PLAN.md was a historical disclaimer wrapping five still-open
T1 driver items — those move to docs/test-system-v2.md (Open driver
work); the rest is dated 2026-08-05 design history, archived in git.
Also retired-mechanism cleanup: both drivers drop the *.combined.ww
enumeration skip (the amalgamator is gone; leftover debris now fails
loudly instead of being silently accommodated), the consumer-less
m4_combined_epoch.md5 pin is deleted, and the wwi.ww/ww.h/check.ww
comments that still described combined.ww as the live path are
re-cited to the sep reality.
Scratch acquisition is one bare mkdir that refuses a pre-existing
.sepwork; the path reaches cleanup only after acquisition succeeds.
ww build and ww test -o keep the tree for the caller; ww run and
no-output ww test build under an owned /tmp/ww_{run,test}_<pid> dir
and remove it on every result, escalating rc on cleanup failure.
The content-keyed package cache is deleted from both stages; -S stops
after per-package .s emission; --sep is gone and unknown flags are
rejected; directory ww test delegates to the wwtest coordinator; the
wwstage driver routes every subprocess through os.exec.
The wwstage Makefile recipes move off the persistent <tool>_ww.d
build-in-place shape to a per-invocation mktemp workspace: the old
recipes relied on the retired pre-wipe semantics and any rebuild
would trip the new refusal on their leftover main.sepwork.
The three 989_pkgcache_* carriers lose their subject with the cache
and go with it; the rest of the wave's carrier adaptation follows in
its own commits.
A torn producer write (e.g. disk-full mid-copy) could leave a 0-byte P.wwi or
P.o in out/.pkgcache under a self-consistent key; cache_lookup checked only
existence, so every later build HIT and served the empty artifact forever
(silent serve-wrong). Reject size==0 on both sides, symmetric across stages:
store refuses to commit a 0-byte temp before the key write, lookup treats a
0-byte cached artifact as a MISS so existing poison self-heals on re-derive.
A valid .wwi/.o is never 0 bytes, so the guard cannot misfire.
Regression: test/wcc/989_pkgcache_poison_run.c, table-driven over
{poison P.wwi | P.o | both}, non-vacuity proven by guard-neuter.
ww run / ww test created /tmp/ww_{run,test}_<pid>.sepwork/ but removed only
the built binary, leaking the scratch DIR every invocation — the tmpfs filler
(98,811 entries blocked the gate twice). Add a keepscratch param to
build_one_sep: a thin wrapper rm -rfs the impl scratch at the single
choke-point when keepscratch==0 AND the path ends ".sepwork" (covers every
return, success+error; fires post-link, pre-run). do_build keeps it (the
byte-id gates read <stem>.sepwork from ww build -o); do_run/do_test clean;
do_test no-o redirects scratch into /tmp. Both stages symmetric; reuses the
existing shell rm -rf idiom (lib/os.removeall = #109). Test 989_sepscratch_run
is self-scoped by child pid (non-flaky) with a KEEP control + revert-verified
non-vacuity. Full gate: 448 pass, zero new run/test leaks. (ww_d_* relic +
historical bulk = one-time sweep + agent-probe discipline, not code.)
The E3 flip (#87) made sep the sole compile path and deleted the
combined.ww writer, leaving the six committed *.combined.ww files dead.
Remove them and the last references to the retired amalgamator.
- rm the 6 tracked *.combined.ww (selfhost/cmd/{w6a,w6c,w6l,ww,wwdump}/
main.combined.ww + selfhost/test/smoke.combined.ww). Verified no live
build path or gate still feeds one as compiler INPUT.
- 901_asserttyped_gap: its 5 combined.ww gap fixtures were the last
combined.ww INPUT consumers (4 already missing/vacuous post-flip, only
smoke.combined.ww still fed). Retarget all 5 to sep-feed via a
resolveunit helper (whole-package-dir copy -> `ww build --sep` ->
<stem>.sepwork/__root.unit.ww), mirroring 990's #89 pattern; the 3
import-free test fixtures stay raw-fed. All 8 counts hold at 0 (A-D
coverage, vacuous since the flip, is live again).
- INV-2 (the driver's unresolvable-import-is-fatal guard) is KEPT; only
its "Mirrors the deleted expand" lineage tail is swept. The #110
combined_ww_fresh freshness gate was already removed in #89 (5f85852).
- Sweep dangling amalgamator lineage comments (build_one/buildone/expand/
peek_package/peekpackage + stale combined.ww/combined intermediates)
in cmd/ww/main.c + selfhost/cmd/ww/main.ww, symmetrically (rule-10),
and the stale Makefile combined.ww test-comments (enumcap bigmod.unit.ww
+ 784/787/792/794/848 sep .s cmp + make-smoke sep self-compile).
Closes M4 and epic #22. all 445 pass; 990/993/994/995 byte-id HOLD;
sizelint clean.
The out/.pkgcache content-keyed store copied each artifact IN-PLACE
(cp -f / copyfile) to the fixed paths P.wwi/P.o/P.key. Key-last gave
crash-consistency but NOT concurrent-read safety: two same-stage builds
of a shared lib pkg (rt/time/os) target one out/.pkgcache/<pkg>/P.{wwi,o};
once an early finisher writes P.key, a later build's cache_lookup copies
P.wwi/P.o while a mid-finisher is still mid-write -> torn read -> corrupt
link / cs!=ww. The key is content-only, so it is purely the non-atomic
write.
Fix (Go-build-cache pattern, both stages in lock-step, rule 10): write
each artifact to a per-pid same-dir temp (P.wwi.tmp.<pid> etc.) then
rename() into place. Same dir => rename is atomic (cross-fs is not);
per-pid temp => concurrent writers don't clobber each other mid-copy;
content-keyed => last-writer-wins is byte-identical. Key renamed LAST so
a reader that sees the new key always finds complete artifacts. On any
mid-store error the per-pid temps are unlinked so a failed store leaves
no litter (cstage goto cleanup; wwstage cachermtmp helper).
cstage cmd/ww/main.c cache_store: libc rename(2) + getpid().
wwstage selfhost/cmd/ww/main.ww cachestore: new os.rename + cachetmp.
lib/os/os.ww: add rename(2) (RENAME=82), ref/hare/os/os.ha:17 -- returns
raw i32 errno like sibling remove/mkdir/rmdir (ww's os is the flat
syscall floor, no fs:: layer); a second pathbuf2 slot holds newpath
since kpath's single pathbuf can't carry both paths.
cache_lookup is unchanged: it reads cache->private scratch, and an atomic
source is never torn.
The torn-read race is closed BY CONSTRUCTION; a deterministic behavioral
regression-guard isn't feasible through the product build path (content-
keying => concurrent COLD builds all MISS+STORE, never HIT-read a mid-store
entry; a warm cache is never re-stored). The deferred white-box guard is
TASK #105. A WHY-comment at both fix sites records this.
Tests: 989_sepbuild_run KEEPS its private per-pid WW_PKGCACHE -- the
comment is corrected: the pin is NOT a torn-read mask (closed by
construction) but cold-compile isolation for the test's INTERMEDIATE
(.s/.unit.ww) byte-id compare, which a cache HIT legitimately skips
producing. The former 989_pkgcache_atomic_run is renamed to
989_pkgcache_concurrent_run and HONESTLY relabeled: it is a concurrent
shared-cache build-correctness smoke (N concurrent --sep builds sharing
one cache -> every binary byte-identical to an isolated reference + correct
run, both stages), NOT a torn-read/atomicity proof (a review revert-
experiment proved the original claim vacuous). Shrunk to 4 concurrent
builds x 1 batch x both stages. COLD/dev-only, off every byte-id/bootstrap
gate.
selfhost/cmd/ww/main.combined.ww remains stale (its writer was deleted at
the M4 E3-C1 flip; #90 deletes the file) -- not regenerated.
make test: all 445 passed; make sizelint clean; 990-997 byte-id hold.
build_one_sep (per-package compile + .wwi interfaces + link) becomes the
sole build path. do_build/do_run/do_test and the ww twins all route
through it; --sep is now an accepted no-op and the run-rejects-sep guard
is removed.
Deleted the single-file amalgamator, both stages: build_one, expand,
expand_dir, peek_package (+ the wwstage twins + strictpkgmismatch).
unit_has_package is retained -- the sep scan loop's inline-package check
needs it. The sep-shared helpers (enumerate_dir_ww, locate_import*,
import_path_form, ImportSet, and ww counterparts) stay; they back the
surviving sep path.
Restores missing-package enforcement under sep by construction: the sep
scan loop loudly rejects an unresolvable import (cannot find package
<name>) unless the package is defined inline in the same unit -- matching
the deleted amalgamator and closing the silent-accept the flip would
otherwise introduce.
All 5 wwstage tools relink (each is built via the now-sep `ww build`);
emitted asm is byte-identical to the combined build per bootstrap input,
so the binary md5 delta is pure link layout, not codegen.
selfhost/cmd/ww/main.combined.ww is now stale and unregenerable (its
writer build_one is deleted); #90 deletes it next.
Test retargets folded in (rule-11 carve-out, #61/#133 precedent): each
asserts post-flip-only behavior, is un-pre-migratable unlike #93/#94/#103,
and splitting reddens one side. Closes#97.
- 989_slttypepref -> dir-package layout (xb imports xa so both same-leaf
`invalid` types are in scope at xb.f); inline-multipackage was the
amalgamator shape, deleted with the flip.
- 989_sepbuild_run -> run --sep now genuinely runs (exit 7), not the old
loud-reject (exit 2); + a private per-pid WW_PKGCACHE so the cs/ww
per-package byte-id compare on the shared real lib pkgs (rt/time/os) no
longer races concurrent siblings on the global out/.pkgcache (the flip
made sep the sole path, so every test now contends that cache).
- 737_direnum -> the deleted strictpkgmismatch "differs from" wording ->
sep's "does not match import path" (shared substring, wwstage terser #68).
- 989_lib_byteid -> corpus-completeness scan excludes generated .sepwork
scratch (the old `! -name '*.combined.ww'` exclude didn't cover the new
sep artifact).
The driver searchpath is srcd-first (srcd = the entry file's directory).
A co-located black-box test lib/<mod>/<mod>test.ww makes srcd=lib/<mod>,
so resolving `import <mod>` hit the sibling-FILE branch lib/<mod>/<mod>.ww
and folded it inline into the consumer unit under the wrong module tag
("package <mod> does not match import path <importer>") — 7 lib-run tests
fail under separate compilation. The combined amalgamator tolerated the
co-location; only sep surfaced it.
Resolve a package directory-first: walk ALL searchpath entries for a
directory match, and only fall back to a file match if no directory
exists anywhere. A dir-package now beats a same-named sibling file (fixes
the self-named shadow), while a leaf package with no directory (e.g.
lib/encoding/hex) still resolves via its file. This realizes the driver's
"a module is the directory" intent; the originally-specced per-directory
suppression was rejected because it broke leaf packages (rob-pike). Both
stages (cmd/ww/main.c + selfhost twin). The dir-beats-earlier-file
precedence change is latent and loud-failing (#101).
Move-set: ww + ww_ww (driver) only; w6c_ww/wwdump_ww/w6a_ww/w6l_ww HOLD.
Gate: test/wcc/989_coloimport_sep.c (table-driven, both stages).
