Hojun-Cho/ww
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

wcc,ww,os: atomic pkgcache store via temp+rename, both stages (#104)

Browse Source 
The out/.pkgcache content-keyed store copied each artifact IN-PLACE
(cp -f / copyfile) to the fixed paths P.wwi/P.o/P.key. Key-last gave
crash-consistency but NOT concurrent-read safety: two same-stage builds
of a shared lib pkg (rt/time/os) target one out/.pkgcache/<pkg>/P.{wwi,o};
once an early finisher writes P.key, a later build's cache_lookup copies
P.wwi/P.o while a mid-finisher is still mid-write -> torn read -> corrupt
link / cs!=ww. The key is content-only, so it is purely the non-atomic
write.

Fix (Go-build-cache pattern, both stages in lock-step, rule 10): write
each artifact to a per-pid same-dir temp (P.wwi.tmp.<pid> etc.) then
rename() into place. Same dir => rename is atomic (cross-fs is not);
per-pid temp => concurrent writers don't clobber each other mid-copy;
content-keyed => last-writer-wins is byte-identical. Key renamed LAST so
a reader that sees the new key always finds complete artifacts. On any
mid-store error the per-pid temps are unlinked so a failed store leaves
no litter (cstage goto cleanup; wwstage cachermtmp helper).

  cstage cmd/ww/main.c cache_store: libc rename(2) + getpid().
  wwstage selfhost/cmd/ww/main.ww cachestore: new os.rename + cachetmp.
  lib/os/os.ww: add rename(2) (RENAME=82), ref/hare/os/os.ha:17 -- returns
    raw i32 errno like sibling remove/mkdir/rmdir (ww's os is the flat
    syscall floor, no fs:: layer); a second pathbuf2 slot holds newpath
    since kpath's single pathbuf can't carry both paths.

cache_lookup is unchanged: it reads cache->private scratch, and an atomic
source is never torn.

The torn-read race is closed BY CONSTRUCTION; a deterministic behavioral
regression-guard isn't feasible through the product build path (content-
keying => concurrent COLD builds all MISS+STORE, never HIT-read a mid-store
entry; a warm cache is never re-stored). The deferred white-box guard is
TASK #105. A WHY-comment at both fix sites records this.

Tests: 989_sepbuild_run KEEPS its private per-pid WW_PKGCACHE -- the
comment is corrected: the pin is NOT a torn-read mask (closed by
construction) but cold-compile isolation for the test's INTERMEDIATE
(.s/.unit.ww) byte-id compare, which a cache HIT legitimately skips
producing. The former 989_pkgcache_atomic_run is renamed to
989_pkgcache_concurrent_run and HONESTLY relabeled: it is a concurrent
shared-cache build-correctness smoke (N concurrent --sep builds sharing
one cache -> every binary byte-identical to an isolated reference + correct
run, both stages), NOT a torn-read/atomicity proof (a review revert-
experiment proved the original claim vacuous). Shrunk to 4 concurrent
builds x 1 batch x both stages. COLD/dev-only, off every byte-id/bootstrap
gate.

selfhost/cmd/ww/main.combined.ww remains stale (its writer was deleted at
the M4 E3-C1 flip; #90 deletes the file) -- not regenerated.

make test: all 445 passed; make sizelint clean; 990-997 byte-id hold.
This commit is contained in:
Hojun-Cho
2026-06-18 20:20:37 +09:00
parent 33edc386f1
commit a9778ec000
6 changed files with 383 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
Makefile
View File

@@ -570,6 +570,7 @@ TESTS = $(BIN)/test_smoke $(BIN)/test_lex $(BIN)/test_parse $(BIN)/test_check \
$(BIN)/test_sepcycle_dup \
$(BIN)/test_separchive_run \
$(BIN)/test_pkgcache_run \
$(BIN)/test_pkgcache_concurrent_run \
$(BIN)/test_c6soak_run \
$(BIN)/test_septest_run \
$(BIN)/test_coloimport_sep \
@@ -3243,6 +3244,17 @@ $(BIN)/test_pkgcache_run: test/wcc/989_pkgcache_run.c $(BIN)/ww $(BIN)/ww_ww \
$(BIN)/w6l $(BIN)/w6l_ww $(LIB)/libwwrt.a | $(BIN)
$(CC) $(CFLAGS) -o $@ $<
# 989_pkgcache_concurrent_run — #104 smoke: concurrent --sep builds sharing one
# out/.pkgcache all produce correct, byte-identical binaries (not a torn-read
# atomicity guard — that is closed by construction, #105). N concurrent builds
# share one cache + one lib pkg; each binary must be byte-identical to an
# isolated reference + run correctly. COLD/dev-only (off every byte-id/bootstrap
# gate). Needs both driver + compiler + asm + linker stages + libwwrt.a.
$(BIN)/test_pkgcache_concurrent_run: test/wcc/989_pkgcache_concurrent_run.c $(BIN)/ww $(BIN)/ww_ww \
$(BIN)/w6c $(BIN)/w6c_ww $(BIN)/w6a $(BIN)/w6a_ww \
$(BIN)/w6l $(BIN)/w6l_ww $(LIB)/libwwrt.a | $(BIN)
$(CC) $(CFLAGS) -o $@ $<
# 989_c6soak_run — M3-tail commit-6 dual-path soak gate, phase-1 leg (#46
# c6). Builds real lib chains (encoding.utf8 dotted + strings/bytes same-leaf
# `contains`) BOTH combined and --sep, asserts combined==sep run-equivalence