The out/.pkgcache content-keyed store copied each artifact IN-PLACE
(cp -f / copyfile) to the fixed paths P.wwi/P.o/P.key. Key-last gave
crash-consistency but NOT concurrent-read safety: two same-stage builds
of a shared lib pkg (rt/time/os) target one out/.pkgcache/<pkg>/P.{wwi,o};
once an early finisher writes P.key, a later build's cache_lookup copies
P.wwi/P.o while a mid-finisher is still mid-write -> torn read -> corrupt
link / cs!=ww. The key is content-only, so it is purely the non-atomic
write.
Fix (Go-build-cache pattern, both stages in lock-step, rule 10): write
each artifact to a per-pid same-dir temp (P.wwi.tmp.<pid> etc.) then
rename() into place. Same dir => rename is atomic (cross-fs is not);
per-pid temp => concurrent writers don't clobber each other mid-copy;
content-keyed => last-writer-wins is byte-identical. Key renamed LAST so
a reader that sees the new key always finds complete artifacts. On any
mid-store error the per-pid temps are unlinked so a failed store leaves
no litter (cstage goto cleanup; wwstage cachermtmp helper).
cstage cmd/ww/main.c cache_store: libc rename(2) + getpid().
wwstage selfhost/cmd/ww/main.ww cachestore: new os.rename + cachetmp.
lib/os/os.ww: add rename(2) (RENAME=82), ref/hare/os/os.ha:17 -- returns
raw i32 errno like sibling remove/mkdir/rmdir (ww's os is the flat
syscall floor, no fs:: layer); a second pathbuf2 slot holds newpath
since kpath's single pathbuf can't carry both paths.
cache_lookup is unchanged: it reads cache->private scratch, and an atomic
source is never torn.
The torn-read race is closed BY CONSTRUCTION; a deterministic behavioral
regression-guard isn't feasible through the product build path (content-
keying => concurrent COLD builds all MISS+STORE, never HIT-read a mid-store
entry; a warm cache is never re-stored). The deferred white-box guard is
TASK #105. A WHY-comment at both fix sites records this.
Tests: 989_sepbuild_run KEEPS its private per-pid WW_PKGCACHE -- the
comment is corrected: the pin is NOT a torn-read mask (closed by
construction) but cold-compile isolation for the test's INTERMEDIATE
(.s/.unit.ww) byte-id compare, which a cache HIT legitimately skips
producing. The former 989_pkgcache_atomic_run is renamed to
989_pkgcache_concurrent_run and HONESTLY relabeled: it is a concurrent
shared-cache build-correctness smoke (N concurrent --sep builds sharing
one cache -> every binary byte-identical to an isolated reference + correct
run, both stages), NOT a torn-read/atomicity proof (a review revert-
experiment proved the original claim vacuous). Shrunk to 4 concurrent
builds x 1 batch x both stages. COLD/dev-only, off every byte-id/bootstrap
gate.
selfhost/cmd/ww/main.combined.ww remains stale (its writer was deleted at
the M4 E3-C1 flip; #90 deletes the file) -- not regenerated.
make test: all 445 passed; make sizelint clean; 990-997 byte-id hold.
build_one_sep (per-package compile + .wwi interfaces + link) becomes the
sole build path. do_build/do_run/do_test and the ww twins all route
through it; --sep is now an accepted no-op and the run-rejects-sep guard
is removed.
Deleted the single-file amalgamator, both stages: build_one, expand,
expand_dir, peek_package (+ the wwstage twins + strictpkgmismatch).
unit_has_package is retained -- the sep scan loop's inline-package check
needs it. The sep-shared helpers (enumerate_dir_ww, locate_import*,
import_path_form, ImportSet, and ww counterparts) stay; they back the
surviving sep path.
Restores missing-package enforcement under sep by construction: the sep
scan loop loudly rejects an unresolvable import (cannot find package
<name>) unless the package is defined inline in the same unit -- matching
the deleted amalgamator and closing the silent-accept the flip would
otherwise introduce.
All 5 wwstage tools relink (each is built via the now-sep `ww build`);
emitted asm is byte-identical to the combined build per bootstrap input,
so the binary md5 delta is pure link layout, not codegen.
selfhost/cmd/ww/main.combined.ww is now stale and unregenerable (its
writer build_one is deleted); #90 deletes it next.
Test retargets folded in (rule-11 carve-out, #61/#133 precedent): each
asserts post-flip-only behavior, is un-pre-migratable unlike #93/#94/#103,
and splitting reddens one side. Closes#97.
- 989_slttypepref -> dir-package layout (xb imports xa so both same-leaf
`invalid` types are in scope at xb.f); inline-multipackage was the
amalgamator shape, deleted with the flip.
- 989_sepbuild_run -> run --sep now genuinely runs (exit 7), not the old
loud-reject (exit 2); + a private per-pid WW_PKGCACHE so the cs/ww
per-package byte-id compare on the shared real lib pkgs (rt/time/os) no
longer races concurrent siblings on the global out/.pkgcache (the flip
made sep the sole path, so every test now contends that cache).
- 737_direnum -> the deleted strictpkgmismatch "differs from" wording ->
sep's "does not match import path" (shared substring, wwstage terser #68).
- 989_lib_byteid -> corpus-completeness scan excludes generated .sepwork
scratch (the old `! -name '*.combined.ww'` exclude didn't cover the new
sep artifact).
The --sep producer compiled the ROOT build-target with the .wwi-producer
-I flag, so check_exported_type ran on the root and rejected a real-tool
root's legitimate `export fn f(a: *t)` over an unexported local type t
(the root is terminal — its interface is never imported, and its .wwi is
never consumed). The combined build never passes -I, so it built fine.
For pi==root, invoke w6c with -c -o only, no -I. Both stages (the wwstage
twin builds the shorter root argv). Gates that asserted __root.wwi exists
encoded the buggy behavior; updated to assert __root.s (the consumed
product) while deps' .wwi byte-id is retained.
M3-tail commit-6 prerequisite. Gate 989_seproot_export_run reproduces the
export-fn-over-unexported-type root + proves it sep-builds, with a
non-vacuity leg that the forced -I path still rejects.
build_one_sep (both stages + ww/main.combined.ww regen): discover_deps
(transitive directory-package set, dotted-path identity), tri-color
reverse-topo (cycle bails; loud reject is commit 4), the transitive
producer loop (one w6c -c -I pass per package, dep-first, each both
consumer and producer of its .wwi), and a flat w6l of the .o set.
combined.ww stays the DEFAULT live path; --sep is additive.
Every dep is tagged by its full dotted import path on prepend
(//ww:module <path>), so the definer's qualified symbol (#53) == the
consumer's qualified reference (#40) and the sep .o set links. The
prepend is the TRANSITIVE closure (lead-ratified, superseding
rob-c3-spec §1.3 direct-deps): a dep's interface can name a transitive
dep's type (os exposes time.instant), so the consuming unit needs the
whole closure for resolution — direct-deps-only does not type-check.
Consistent with the current flat-unit transitive-namespace model (the
visibility tighten is #45, post-M4).
Gate 989_sepbuild_run drives ww + ww_ww --sep on the real chain
root->os->{rt,time}: build+run (exit 7) + cs==ww per-pkg .s/.wwi/.unit
+ final binary + transitive-topo discovery + keystone bodies==.wwi
(os,root) through the real driver. Cold scratch; -o-redirected.
