wcc,ww,os: atomic pkgcache store via temp+rename, both stages (#104)
The out/.pkgcache content-keyed store copied each artifact IN-PLACE
(cp -f / copyfile) to the fixed paths P.wwi/P.o/P.key. Key-last gave
crash-consistency but NOT concurrent-read safety: two same-stage builds
of a shared lib pkg (rt/time/os) target one out/.pkgcache/<pkg>/P.{wwi,o};
once an early finisher writes P.key, a later build's cache_lookup copies
P.wwi/P.o while a mid-finisher is still mid-write -> torn read -> corrupt
link / cs!=ww. The key is content-only, so it is purely the non-atomic
write.
Fix (Go-build-cache pattern, both stages in lock-step, rule 10): write
each artifact to a per-pid same-dir temp (P.wwi.tmp.<pid> etc.) then
rename() into place. Same dir => rename is atomic (cross-fs is not);
per-pid temp => concurrent writers don't clobber each other mid-copy;
content-keyed => last-writer-wins is byte-identical. Key renamed LAST so
a reader that sees the new key always finds complete artifacts. On any
mid-store error the per-pid temps are unlinked so a failed store leaves
no litter (cstage goto cleanup; wwstage cachermtmp helper).
cstage cmd/ww/main.c cache_store: libc rename(2) + getpid().
wwstage selfhost/cmd/ww/main.ww cachestore: new os.rename + cachetmp.
lib/os/os.ww: add rename(2) (RENAME=82), ref/hare/os/os.ha:17 -- returns
raw i32 errno like sibling remove/mkdir/rmdir (ww's os is the flat
syscall floor, no fs:: layer); a second pathbuf2 slot holds newpath
since kpath's single pathbuf can't carry both paths.
cache_lookup is unchanged: it reads cache->private scratch, and an atomic
source is never torn.
The torn-read race is closed BY CONSTRUCTION; a deterministic behavioral
regression-guard isn't feasible through the product build path (content-
keying => concurrent COLD builds all MISS+STORE, never HIT-read a mid-store
entry; a warm cache is never re-stored). The deferred white-box guard is
TASK #105. A WHY-comment at both fix sites records this.
Tests: 989_sepbuild_run KEEPS its private per-pid WW_PKGCACHE -- the
comment is corrected: the pin is NOT a torn-read mask (closed by
construction) but cold-compile isolation for the test's INTERMEDIATE
(.s/.unit.ww) byte-id compare, which a cache HIT legitimately skips
producing. The former 989_pkgcache_atomic_run is renamed to
989_pkgcache_concurrent_run and HONESTLY relabeled: it is a concurrent
shared-cache build-correctness smoke (N concurrent --sep builds sharing
one cache -> every binary byte-identical to an isolated reference + correct
run, both stages), NOT a torn-read/atomicity proof (a review revert-
experiment proved the original claim vacuous). Shrunk to 4 concurrent
builds x 1 batch x both stages. COLD/dev-only, off every byte-id/bootstrap
gate.
selfhost/cmd/ww/main.combined.ww remains stale (its writer was deleted at
the M4 E3-C1 flip; #90 deletes the file) -- not regenerated.
make test: all 445 passed; make sizelint clean; 990-997 byte-id hold.
This commit is contained in:
@@ -236,14 +236,15 @@ main(void)
|
||||
runwait(cmd);
|
||||
mkdir(td, 0755);
|
||||
|
||||
/* E3-C1: with the flip making sep the sole build path, every concurrent
|
||||
* test now writes the global out/.pkgcache; this test's cs/ww per-package
|
||||
* byte-id compare on the heavily-shared real lib pkgs (rt/time/os) then
|
||||
* races a sibling building the same pkg. Pin a private per-pid cache so
|
||||
* the cs and ww legs are isolated (system() builds inherit this env).
|
||||
* pkgcache writes are non-atomic (cp -f), the real torn-read bug; this
|
||||
* hermetic isolation is correct standalone, pending #104 (atomic
|
||||
* temp+rename, both stages). */
|
||||
/* This test byte-id compares the per-package INTERMEDIATES (.s, .unit.ww)
|
||||
* between the cs and ww legs, which a cache HIT legitimately skips
|
||||
* producing (a HIT copies only .wwi/.o and bypasses compose+w6c). So the
|
||||
* compare requires every package to COLD-compile: pin a private per-pid
|
||||
* cache to force misses. This is orthogonal to #104 — the atomic temp+
|
||||
* rename closed the torn-read race (by construction; #105 is the deferred
|
||||
* white-box guard). Concurrent shared-cache build correctness is smoked by
|
||||
* 989_pkgcache_concurrent_run. Here the pin is for intermediate-compare
|
||||
* isolation, not a torn-read mask. */
|
||||
char cachedir[80];
|
||||
snprintf(cachedir, sizeof cachedir, "%s/pkgcache", td);
|
||||
setenv("WW_PKGCACHE", cachedir, 1);
|
||||
|
||||
Reference in New Issue
Block a user