Continue fold-2: migrate the pointer-deref / narrow-load / pointer-array-stride
family from bespoke build+run C twins to test/lang @test, retiring each twin in
the same commit. Runtime coverage MOVES from $(TESTS) to test-lang (T1 runs +
asserts via `ww test`) + test-lang-byteid (T2 keeps cs==ww .s byte-id); the
$(TESTS) headline drops 5. Every assert is a primitive int/bool comparison with
a width-preserving (`==`) sink so a stale high half or wrong stride FAILS; the
narrow-signed rows route through an `: i32` cast against a 64B-widened literal
to force sign-extension onto the load. Each .c row maps to one inline @test fn
(50 cases total, strict superset of the C rows):
947_deref_narrow_run.c -> deref_narrow_test.ww (#116, 10 rows: *p reads pointee width not 8B MOVQ; i32/u32/u8/i8/i16/u16 + !i32-alias + enum-i8 + bool/i64 controls)
949_ptrarr_index_run.c -> ptrarr_index_test.ww (#61, 23 rows: p[i]/(&p[i])/(*p)[i] stride by size(T); {1,2,4,8}B, const+var idx, param/local/cast bases, neighbor guards, nested *[2][3], *[3]str header, siphash round())
949_dotbase_arr_run.c -> dotbase_arr_test.ww (#135, 3 rows: (*struct).arrayfield[i] read/write/compound addresses the field)
944_def_amp_idx_run.c -> def_amp_idx_test.ww (#94, 6 rows: &D[i] over a def-array; +plain/read/2D controls)
944_alias_amp_idx_run.c -> alias_amp_idx_test.ww (#5, 8 rows: &a[i] over an alias-typed base classifies off the chased type; local/global, narrow, fwd-ref, plain+str controls)
The two sibling .c (949_dotbase_addr_slice_run, 944_alias_def_addr_run) stay in
$(TESTS): the first is run-only byteid=0 (#254 cs!=ww rows), the second carries
a LOUD rule-7 reject row — both routed to fold-3 (task #7). Bump
LANGBYTEID_EXPECTED_MIN 38->43.
Continue fold-2: migrate the global-value read/store/addr-of family (non-tagged)
from bespoke build+run C twins to test/lang @test, retiring each twin in the
same commit. Runtime coverage MOVES from $(TESTS) to test-lang (T1 runs+asserts
via `ww test`) + test-lang-byteid (T2 keeps cs==ww .s byte-id); the $(TESTS)
headline drops 3 (421->418). All asserts are primitive int comparisons; fresh
globals are zeroed, so a store that misses the symbol reads back 0 and FAILS.
989_globptrfield_run.c -> glob_ptr_field_test.ww (scalar field store through a module-global *struct pointer loads the pointer from gp(SB), not saved BP; runtime gp=&backing form, static-init is #48-blocked)
989_globstructret_run.c -> glob_struct_ret_test.ww (returning a module-global struct ident by value copies g's bytes; rsz 16 + rsz 24, each field asserted)
989_dotbasehijack_run.c -> dotbase_hijack_test.ww (indexing an [N]T field of a module-global struct addresses the struct field, not an unrelated global sharing the field name; +typed-inner control +colliding-global-intact pin)
glob_ptr_field keeps only the .c's offset-0 row: an added non-zero-offset row
(gp.g=9) surfaced a cs!=ww divergence (cstage folds the offset into the store
displacement `MOVQ AX,8(BX)`; wwstage emits `ADDQ $8,BX; MOVQ AX,(BX)`) — a
latent global-*struct-pointer field-store divergence beyond the .c's coverage,
reported for the backlog, not carried here. Bump LANGBYTEID_EXPECTED_MIN 35->38.
Continue fold-2: migrate the #73 whole-struct field-copy ragged-tail test from a
bespoke build+run C twin to test/lang @test, retiring the twin in the same
commit. Runtime coverage MOVES from $(TESTS) to test-lang (T1 runs+asserts via
`ww test`) + test-lang-byteid (T2 keeps cs==ww .s byte-id); coverage is
preserved, the $(TESTS) headline drops 1 (422->421). All four rows are pure
value-rows (no reject rows): each poisons `mark` at the inner struct's natural
offset, copies, then asserts every field back with primitive int comparisons, so
an 8-byte MOVQ that over-writes the ragged-tail successor FAILS.
989_structcopytail_run.c -> structcopytail_test.ww (tail2/tail6/tail7 ragged-tail copy preserves mark; ctl8 8-aligned control)
This is the only Family-3 file with no build-must-fail rows; the three
compound-OP= twins (948_idx_compound, 949_dotfield_compound, 949_idxfield_compound)
carry loud //ww:error reject rows a runtime @test cannot replicate and are
DEFERRED to a fold-3 value-split + reject-carrier pass (they stay fully in
$(TESTS), no coverage lost).
Bump LANGBYTEID_EXPECTED_MIN 34->35 to ratchet the new corpus floor.
Continue fold-2: migrate drew's Family 2 (def-dimensioned array len/cap/slice
resolution) from bespoke build+run C twins to test/lang @test, retiring each
twin in the same commit. Runtime coverage MOVES from $(TESTS) to test-lang (T1
runs+asserts via `ww test`) + test-lang-byteid (T2 keeps cs==ww .s byte-id);
coverage is preserved, the $(TESTS) headline drops 3 (425->422). All asserts are
primitive int comparisons (no fmt/strconv in the assert path); the defcap rows
poison cap != len and assert both words so a dropped cap word FAILS.
989_defdim_field_run.c -> defdim_field_test.ww (`.len` field-read on a def-dim [MAX]T resolves from the type table: local/let-global/def cgdot arms + sum)
989_defdim_slice_run.c -> defdim_slice_test.ww (slicing a def-dim [MAX]T resolves len AND cap from the type table: default-hi + cgbasecap, local+global)
989_defdim_argslice_run.c -> defdim_argslice_test.ww (def-dim slice passed as a call arg resolves default-hi len in the N_SLICE arg-push arms; litctrl pins the N_INTLIT path)
Bump LANGBYTEID_EXPECTED_MIN 31->34 to ratchet the new corpus floor.
Continue fold-2 (after 374e97b): migrate the remaining Family-1 str/slice
global + literal + index + call-arg group from bespoke build+run C twins to
test/lang @test, retiring each twin in the same commit. Runtime coverage MOVES
from $(TESTS) to test-lang (T1 runs+asserts via `ww test`) + test-lang-byteid
(T2 keeps cs==ww .s byte-id); coverage is preserved, the $(TESTS) headline
drops 9 (434->425). All asserts are primitive int/u8/bool comparisons (no
fmt/strconv in the assert path); the slice-store/index rows reset the global
each fn and sum ADJACENT elements so a dropped/mis-strided/over-wide word FAILS.
989_globslicefield_run.c -> glob_slice_field_test.ww (slice field of a global struct: g.f=<slice> stores full 24B header; len/cap/non-zero-offset + str/scalar controls)
989_globstrslice_run.c -> glob_str_slice_arg_test.ww (global str sliced with default hi passed as call arg loads its len word; explicit-hi control)
989_trystr_run.c -> try_str_unwrap_test.ww (`!` unwrap of str-success tagged union shuffles the str header for ident-source/error-first/success-first)
797_len_strglobal_run.c -> len_str_global_test.ww (len(str-global) loads .len via name(SB); local-str control)
801_litstr_pseudo_run.c -> lit_str_pseudo_test.ww (string-literal .len/.ptr pseudo-field; empty/multibyte + arg-passthrough)
803_globalidx_run.c -> global_index_test.ww (global str/slice index read/addr-of/store/compound, esz 1/4; local regression pins)
903_tuple_elem_slice_len.c -> tuple_elem_slice_len_test.ww (len(t.N) of a slice/str tuple element loads .len at +8; 2/3-slice, str-slice both orders)
927_composite_call_arg_run.c-> composite_call_arg_test.ww (slice-returning CALL passed inline as a composite arg; canonical/letslice/two-call/middle/nested/scalar/tagged)
952_slicecopy_assign_run.c -> slice_copy_assign_test.ww (bulk slice-copy-assign `arr[lo:hi]=bs`, esz 1/4, field/via-ptr/local bases; reslice-read companion)
rd_reslice asserts the TRUE value 360 (the .c twin's want=104 was 360 & 0xFF,
an exit-code truncation). 723_composite_call_arg.c's comment repointed to the
new test/lang location. 802_lenidx_run.c is DEFERRED (it carries //ww:error
reject rows — needs a value-rows-only split + a slim reject carrier, a fold-3
pass). Bump LANGBYTEID_EXPECTED_MIN 22->31 to ratchet the new corpus floor.
Migrate the slice/str-header core of drew's Family 1 from bespoke
build+run C twins to test/lang @test, retiring each now-redundant twin in
the same commit. Runtime coverage MOVES from $(TESTS) to test-lang (T1
runs+asserts via `ww test`) + test-lang-byteid (T2 keeps cs==ww .s
byte-id); coverage is preserved, the $(TESTS) headline drops 6 (440->434).
All asserts are primitive int/u8/bool comparisons (no fmt/strconv in the
assert path); the slice-store families poison the slot (cap!=len) and read
it back so a dropped data word FAILS.
928_str_abi_run.c -> str_abi_test.ww (str 24B ABI: .len/.cap across literal/arg/return/field/tuple/deref/index/tagged)
941_slice_store_cap_run.c -> slice_store_cap_test.ww (slice value store through indexed/field/chained lhs writes full 24B header; cap==8)
942_subslice_cap_run.c -> subslice_cap_test.ww (sub-slice cap = base_cap-lo; array/slice/str/append-no-realloc/hi-default)
943_subslice_ptresz_run.c -> subslice_ptresz_test.ww (sub-slice ptr advances lo*esz bytes; esz 2/4/8, let + call-arg)
944_deref_slice_store_run.c-> deref_slice_store_test.ww(*p=sliceval whole-deref store writes 24B header; cap==8)
949_f6_header_run.c -> f6_header_test.ww (str/slice header partial load/store; .cap/.len after clobber)
Bump LANGBYTEID_EXPECTED_MIN 16->22 to ratchet the new corpus floor.
These test/wcc/*_run.c carriers were migrated to test/lang @test packages
(value-asserting under test-lang/T1) and now have a byte-id home under
test-lang-byteid/T2. With T1 wired into make test + test-commit (prior
commit), union(T1 runtime value-asserts + T2 .s byte-id) covers everything
each .c proved, so the C twins are pure-deletion redundant. Each retired
family below names the T1 twin that now carries its value obligation; every
twin was verified to READ the value/slot and ASSERT it (not merely compile),
including the pre-zeroed-slot families (it poisons cap!=len / spoils the base
so a wrong data word fails the read-back, which byte-id alone cannot catch).
989_gunsigned_run -> gunsigned_test (global unsigned opcode select)
989_chainidx_run -> chainidx_test (chained m[i][k] header load)
989_idxarg_run -> idxarg_test (indexed elem call-arg header push)
906_callret_unsigned_arith -> callret_unsigned_test (call-return unsigned arm)
912_sar_shr_run -> sar_shr_test (signed >> emits SAR)
793_widen_pad_zero_run -> widen_pad_test (#227 high-pad zeroing; reads pad)
932_str_elem_cap_run -> str_elem_cap_test (str-elem .cap full 24B load)
957_size_type_run -> size_type_test (`size` type-position bind)
933_str_field_cap_run -> str_field_cap_test (str field .cap read)
934_str_chained_field_cap_run-> str_chained_field_cap_test (chained field .cap)
935_str_tuple_elem_cap_run -> str_tuple_elem_cap_test (tuple-elem str .cap)
936_str_arrfield_cap_run -> str_arrfield_cap_test (array-field str .cap)
937_str_arrfield_store_cap_run -> str_arrfield_store_cap_test (array-field store)
938_str_chainfield_store_cap_run-> str_chainfield_store_cap_test (chain-field store)
939_str_massign_store_cap_run -> str_massign_store_cap_test (multi-assign store)
110_uniesc_run.c is KEPT: it carries a .wwi round-trip sep-build (exit 42)
the uniesc_test twin does not replicate (HAS-UNIQUE-MODE, deferred).
First piece of the Go-model test rebuild: runww.ww is a ww program (the
test/run.go analog) that drives compiler cases through BOTH stages and
asserts behavior/diagnostics. Directives //ww:run / //ww:run-exit N /
//ww:error "<substr>" / //ww:compile; spawn+stderr-capture lift the
driver's procrun (main.ww:160) + dup2(2). The //ww:error check requires
rc!=0 AND the diagnostic substring (the #20 non-vacuity guard -- a crash
can't pass), and a malformed error directive fails loudly.
Bulk corpus migration + ww test wiring are follow-up folds; case spawns
need os.envp() (#28) and per-pid /tmp paths (#29) first.
Reject helpers checked only rc!=0, so a SEGFAULT (exit 139, or the
driver's "w6c failed" exit 1) counted as a clean reject -- a wwstage
crash could pass vacuously (it did, latently, on bodied bare-... pre
#11). Every reject row now captures stderr and requires the actual
diagnostic substring (rc!=0 AND strstr) across all 16 reject tests; a
crash emits no diagnostic, so it now fails. This is the
differential-reject backstop (#15): both stages must cleanly reject
with the expected message.
Two genuine cstage/wwstage diagnostic-body divergences are documented
inline via per-stage substrings, not papered over (845 tuple parse,
catA_f2 tuple arity); catalogued in #21.
A bodied fn with a bare C-style `...` was silently accepted by cstage
and SEGFAULTED wwstage (resolvefnbody walked a typeless `...` param).
Gate it: bare C-`...` is allowed only on bodiless decls (extern /
@symbol prototypes), the real FFI path; Hare-style `T...` is unaffected.
ww restricts C-`...` to bodiless decls pending vastart/vaarg/vaend
builtins (#16); harec permits bodied C-variadic fns (check.c:3656) -- a
documented divergence, reopened when #16 lands.
Test 852 runs both stages; its reject rows require the gate's diagnostic
(not merely a nonzero exit), so a crash can't pass them vacuously.
Mirror cstage's C-variadic call handling in the ww self-host: parse a
bare `...` param (decl.ww), skip param-keyed desugar for it to avoid a
nil-deref (check.ww), and emit AL = XMM-reg count plus CVTSS2SD
promotion of f32 args in the variadic tail (cgenutil.ww, cgenexpr.ww).
Closes the cat-A wwstage silent miscompile (AL=0, unpromoted f32 tail).
Parse/check/cgen are one atomic align-up (parse alone miscompiles, so
not bisect-splittable). 989_ffivariadic now runs dual-stage (cstage ww
+ wwstage ww_ww), 12/12; w6c==w6c_ww byte-identical. Byte-id alone is
blind here (the bootstrap calls no float-bearing C variadic), so the
ww_ww runtime rows are the real net.
C99 6.5.2.2p6 requires float->double promotion for floats in the
variadic region of a call; w6c emitted MOVSS (4B), so a C variadic's
va_arg(double) read 8B of garbage. Promote at push (CVTSS2SD) so the
8B slot holds a real double, covering the XMM-reg and >8-float spill
cases uniformly; fi/AL XMM-count is unchanged.
cstage only; the wwstage twin rides C2 (#10), which adds wwstage's
C-variadic-call codegen.
SysV §3.5.7 requires a variadic call to set AL = number of vector (XMM) regs used for the variable float args; the C callee gates its xmm-save-area stores on `test %al,%al`, so the old hardcoded XORQ AX,AX (AL=0) made va_arg(double) read garbage for any float-bearing C variadic call. Emit MOVQ $fi,AX (fi = the in-scope XMM cursor, ≤8); w6a has no MOVL-immediate encoding so MOVQ is the assemblable form and sets AL=fi identically. fi==0 keeps XORQ → byte-identical to pre-fix for no-float variadic calls. Runtime test 989_ffivariadic links a cc-compiled va_arg(double) fixture (zero relocs/undefined, w6l-linkable) and sweeps N=3/5/8 floats (N=2 is vacuous via stale-stack aliasing). C1 of the C-FFI-variadic align-up (USER ruling); C2 wwstage + C3 bodiless gate follow. ref/qbe/amd64/sysv.c:384. 454 green.
Mirrors cstage check.c:1491-1495 (type_isint via the syntax.typeisint tinfo chaser, which chases TY_NAMED.under/TY_ENUM.sub — not the AST-keyed isinttypeast that would falsely reject an alias-int index). Record-and-continue, before the base-bail. Reject path emits no asm so cstage==wwstage byte-id holds (453 green). Pre-existing index double-emit deferred (#6).
The wwstage checker silently accepted enums with a non-integer storage type, duplicate members, or a non-constant member value; cstage already rejects all three (cmd/wcc/check.c:1000-1042). Add validateenummembers, a pure read-only diagnostic dispatched once per enum decl from resolvewalk's N_TENUM arm (check.ww:791, beside stampenumvals -- not the per-query recompute arms), mirroring the catB-7/14 validatestructfields pattern. Storage gate uses typeisint on the resolved tinfo (the exact type_isint mirror: chases TY_NAMED.under and TY_ENUM.sub, so an int-alias storage is accepted; raw-AST isinttypeast would not). Duplicate members: O(n^2) name walk. Unfoldable values reuse enumvalfold with until=member (forward-only). Emits via cerr + c.errs, no mutation, so valid-program codegen is unchanged and cstage==wwstage byte-id holds.
wwstage's value-fold message is intentionally generic where cstage's eval_enum_value gives per-reason text (enumvalfold returns a bool, not a reason); both reject. Documented at the site, filed as follow-up task #10.
Test: new table-driven both-stage reject test 850_enum_reject (non-int storage / duplicate member / forward-ref rows + a distinct-member control whose backward-ref value builds and runs). Full make test: 452 green incl. 990-997 byte-id.
The wwstage checker silently accepted a struct with repeated field names; cstage already rejects it (cmd/wcc/check.c:925-947). Add validatestructfields, dispatched once per struct decl from resolvewalk's eager type-decl arm (check.ww:792, sibling to the N_TENUM stampenumvals fire): a pure read-only O(n^2) named-field dup walk that emits "duplicate field 'X'" via cerr + c.errs, with no mutation -- valid-program codegen is unchanged so cstage==wwstage byte-id holds. Named fields only; ww has no struct embedding, so cstage's embed-collision arm is intentionally not ported (separate parser gap, catB-89).
Test: new table-driven both-stage reject test 849_dupfield_reject (adjacent / non-adjacent / different-type dup rows + a distinct-field control that builds and runs). Full make test: 451 green incl. 990-997 byte-id.
The depth->=2 chained `t.m.l = s` struct-ident field copy selected its tail with (tail==4)?MOVL:(tail==1?MOVB:MOVQ), so every tail in {2,3,5,6,7} fell to an 8-byte MOVQ that over-wrote past the destination field — clobbering a @packed neighbour (t.m.l=s wrote s's slop over t.m.z: cstage exit 0 vs wwstage's correct 222) and diverging cs!=ww gate-blind. Replace with a descending 4/2/1 MOVL/MOVW/MOVB ladder comparing to ssz (the field's natural type-table size), aligning cstage UP to wwstage's sized ladder + cg_aggcopy. New table-driven gate 949_chained_dot_struct_copy_run (7 sizes x runtime-exit oracle on both drivers + cs==ww byte-id; negative-control proven).
A struct/array sized {3,5,6,7} returned by value fell past every N_LET call-receive arm (the sized-tail arm handles {1,2,4}, the memcpy arm gates on sz>8), so cstage emitted nothing and silently DROPPED the CALL — garbage exit plus a gate-blind cs!=ww divergence (wwstage emitted CALL+MOVQ via its generic let fallback). Align cstage UP: emit the CALL + one MOVQ AX, the SysV single-eightbyte receive (ref/qbe/amd64/sysv.c retr). New table-driven gate 949 (runtime exit oracle on both drivers + cs==ww byte-id; negative-control proven).
ww run / ww test created /tmp/ww_{run,test}_<pid>.sepwork/ but removed only
the built binary, leaking the scratch DIR every invocation — the tmpfs filler
(98,811 entries blocked the gate twice). Add a keepscratch param to
build_one_sep: a thin wrapper rm -rfs the impl scratch at the single
choke-point when keepscratch==0 AND the path ends ".sepwork" (covers every
return, success+error; fires post-link, pre-run). do_build keeps it (the
byte-id gates read <stem>.sepwork from ww build -o); do_run/do_test clean;
do_test no-o redirects scratch into /tmp. Both stages symmetric; reuses the
existing shell rm -rf idiom (lib/os.removeall = #109). Test 989_sepscratch_run
is self-scoped by child pid (non-flaky) with a KEEP control + revert-verified
non-vacuity. Full gate: 448 pass, zero new run/test leaks. (ww_d_* relic +
historical bulk = one-time sweep + agent-probe discipline, not code.)
The E3 flip (#87) made sep the sole compile path and deleted the
combined.ww writer, leaving the six committed *.combined.ww files dead.
Remove them and the last references to the retired amalgamator.
- rm the 6 tracked *.combined.ww (selfhost/cmd/{w6a,w6c,w6l,ww,wwdump}/
main.combined.ww + selfhost/test/smoke.combined.ww). Verified no live
build path or gate still feeds one as compiler INPUT.
- 901_asserttyped_gap: its 5 combined.ww gap fixtures were the last
combined.ww INPUT consumers (4 already missing/vacuous post-flip, only
smoke.combined.ww still fed). Retarget all 5 to sep-feed via a
resolveunit helper (whole-package-dir copy -> `ww build --sep` ->
<stem>.sepwork/__root.unit.ww), mirroring 990's #89 pattern; the 3
import-free test fixtures stay raw-fed. All 8 counts hold at 0 (A-D
coverage, vacuous since the flip, is live again).
- INV-2 (the driver's unresolvable-import-is-fatal guard) is KEPT; only
its "Mirrors the deleted expand" lineage tail is swept. The #110
combined_ww_fresh freshness gate was already removed in #89 (5f85852).
- Sweep dangling amalgamator lineage comments (build_one/buildone/expand/
peek_package/peekpackage + stale combined.ww/combined intermediates)
in cmd/ww/main.c + selfhost/cmd/ww/main.ww, symmetrically (rule-10),
and the stale Makefile combined.ww test-comments (enumcap bigmod.unit.ww
+ 784/787/792/794/848 sep .s cmp + make-smoke sep self-compile).
Closes M4 and epic #22. all 445 pass; 990/993/994/995 byte-id HOLD;
sizelint clean.
The out/.pkgcache content-keyed store copied each artifact IN-PLACE
(cp -f / copyfile) to the fixed paths P.wwi/P.o/P.key. Key-last gave
crash-consistency but NOT concurrent-read safety: two same-stage builds
of a shared lib pkg (rt/time/os) target one out/.pkgcache/<pkg>/P.{wwi,o};
once an early finisher writes P.key, a later build's cache_lookup copies
P.wwi/P.o while a mid-finisher is still mid-write -> torn read -> corrupt
link / cs!=ww. The key is content-only, so it is purely the non-atomic
write.
Fix (Go-build-cache pattern, both stages in lock-step, rule 10): write
each artifact to a per-pid same-dir temp (P.wwi.tmp.<pid> etc.) then
rename() into place. Same dir => rename is atomic (cross-fs is not);
per-pid temp => concurrent writers don't clobber each other mid-copy;
content-keyed => last-writer-wins is byte-identical. Key renamed LAST so
a reader that sees the new key always finds complete artifacts. On any
mid-store error the per-pid temps are unlinked so a failed store leaves
no litter (cstage goto cleanup; wwstage cachermtmp helper).
cstage cmd/ww/main.c cache_store: libc rename(2) + getpid().
wwstage selfhost/cmd/ww/main.ww cachestore: new os.rename + cachetmp.
lib/os/os.ww: add rename(2) (RENAME=82), ref/hare/os/os.ha:17 -- returns
raw i32 errno like sibling remove/mkdir/rmdir (ww's os is the flat
syscall floor, no fs:: layer); a second pathbuf2 slot holds newpath
since kpath's single pathbuf can't carry both paths.
cache_lookup is unchanged: it reads cache->private scratch, and an atomic
source is never torn.
The torn-read race is closed BY CONSTRUCTION; a deterministic behavioral
regression-guard isn't feasible through the product build path (content-
keying => concurrent COLD builds all MISS+STORE, never HIT-read a mid-store
entry; a warm cache is never re-stored). The deferred white-box guard is
TASK #105. A WHY-comment at both fix sites records this.
Tests: 989_sepbuild_run KEEPS its private per-pid WW_PKGCACHE -- the
comment is corrected: the pin is NOT a torn-read mask (closed by
construction) but cold-compile isolation for the test's INTERMEDIATE
(.s/.unit.ww) byte-id compare, which a cache HIT legitimately skips
producing. The former 989_pkgcache_atomic_run is renamed to
989_pkgcache_concurrent_run and HONESTLY relabeled: it is a concurrent
shared-cache build-correctness smoke (N concurrent --sep builds sharing
one cache -> every binary byte-identical to an isolated reference + correct
run, both stages), NOT a torn-read/atomicity proof (a review revert-
experiment proved the original claim vacuous). Shrunk to 4 concurrent
builds x 1 batch x both stages. COLD/dev-only, off every byte-id/bootstrap
gate.
selfhost/cmd/ww/main.combined.ww remains stale (its writer was deleted at
the M4 E3-C1 flip; #90 deletes the file) -- not regenerated.
make test: all 445 passed; make sizelint clean; 990-997 byte-id hold.
build_one_sep (per-package compile + .wwi interfaces + link) becomes the
sole build path. do_build/do_run/do_test and the ww twins all route
through it; --sep is now an accepted no-op and the run-rejects-sep guard
is removed.
Deleted the single-file amalgamator, both stages: build_one, expand,
expand_dir, peek_package (+ the wwstage twins + strictpkgmismatch).
unit_has_package is retained -- the sep scan loop's inline-package check
needs it. The sep-shared helpers (enumerate_dir_ww, locate_import*,
import_path_form, ImportSet, and ww counterparts) stay; they back the
surviving sep path.
Restores missing-package enforcement under sep by construction: the sep
scan loop loudly rejects an unresolvable import (cannot find package
<name>) unless the package is defined inline in the same unit -- matching
the deleted amalgamator and closing the silent-accept the flip would
otherwise introduce.
All 5 wwstage tools relink (each is built via the now-sep `ww build`);
emitted asm is byte-identical to the combined build per bootstrap input,
so the binary md5 delta is pure link layout, not codegen.
selfhost/cmd/ww/main.combined.ww is now stale and unregenerable (its
writer build_one is deleted); #90 deletes it next.
Test retargets folded in (rule-11 carve-out, #61/#133 precedent): each
asserts post-flip-only behavior, is un-pre-migratable unlike #93/#94/#103,
and splitting reddens one side. Closes#97.
- 989_slttypepref -> dir-package layout (xb imports xa so both same-leaf
`invalid` types are in scope at xb.f); inline-multipackage was the
amalgamator shape, deleted with the flip.
- 989_sepbuild_run -> run --sep now genuinely runs (exit 7), not the old
loud-reject (exit 2); + a private per-pid WW_PKGCACHE so the cs/ww
per-package byte-id compare on the shared real lib pkgs (rt/time/os) no
longer races concurrent siblings on the global out/.pkgcache (the flip
made sep the sole path, so every test now contends that cache).
- 737_direnum -> the deleted strictpkgmismatch "differs from" wording ->
sep's "does not match import path" (shared substring, wwstage terser #68).
- 989_lib_byteid -> corpus-completeness scan excludes generated .sepwork
scratch (the old `! -name '*.combined.ww'` exclude didn't cover the new
sep artifact).
The last flip casualty in the test-migration class (the filed #66
"WW_W6C under sep" framing was a phantom — cstage honors the override;
see re-scoped #66 for the real wwstage align-up). 809_idx_structlit_store's
asm_byte_identical drove `bin/ww build` then read <src>.s next-to-source;
the flip relocates asm to .sepwork/, so it now drives `ww build --sep -o
<stem>` (cstage + WW_W6C=w6c_ww wwstage legs) and byte-diffs the
concatenated per-package .sepwork/*.s. callee_capture imports strings, so
concat-all is the faithful equivalent of the old whole-program compare.
Adding --sep is flip-invariant (green pre- and post-flip). 809_delete_range
is flip-invariant (direct w6c -o), left untouched. Test-only; all 5 pins HOLD.
94c, the last C-mig piece. The #65 enumeratedir-cap gate built a 300-file
bigmod dir (past the old 256 cap) and byte-compared cstage vs wwstage
.combined.ww to prove both enumerate the complete file set. The flip
deletes combined.ww, so the comparison now targets each stage's
<stem>.sepwork/bigmod.unit.ww (one `package bigmod;` clause per enrolled
file). count_pkgs asserts the full nfiles count and `cmp` byte-compares
the two stages' unit, so a silent cap-drop on either side reddens both
paths. >cap fixture unchanged. Test-only; all 5 binary pins HOLD.
94b of the C-mig2 split (own commit: lone phase-1 test-commit cache
gate). The corpus gate now drives two --sep builds (ww / ww_ww) per lib
unit and byte-diffs the concatenated per-package .sepwork/*.s. Keying on
every per-package .s — the imported lib body (e.g. log.s) and lib/test
(test.s), never collapsed to __root.s — preserves the cs!=ww lib-body
coverage the gate exists for. Tri-mode (ID/DIVERGE/WWREJECT) per module,
import-landed sentinel, and corpus_complete scan all preserved. 43/43
(net=M_ID via #102). Test-only; all 5 binary pins HOLD.
94a of the C-mig2 split (rob rule-11): 13 Pattern-A gates that fed a
.combined.ww to w6c/w6c_ww now drive two --sep builds (ww / ww_ww) and
byte-diff the concatenated per-package .sepwork/*.s, mirroring #93's
convention. Lands pre-flip while combined.ww still exists as the
reversible safety net. Test-only; all 5 binary pins HOLD.
989_enumcap_run deferred to 94b (its decisive assertion is a combined.ww
content diff, not a .s diff).
The bare-`main` carve-out (which keeps the link entry's main unmangled)
keyed on `leaf == "main" && imported == 0`. Under the combined path a
dependency's body folds in with imported==1, so only the root's main
stayed bare. Under separate compilation each package is its own unit and
a dependency's body carries a path-mangling module-reset but imported==0
(#57) — so an imported `fn main` matched the carve-out, emitted a bare
`TEXT main`, and collided with the root entry (`w6l: duplicate symbol
main`). The combined path was unaffected, so this only surfaced under sep.
Gate the carve-out with sep_isdep = (wwiout != NULL): the producer emits a
.wwi output only for dependency units, never for the root/link-entry unit
(root stripped, #69), symmetric on both stages. Only the root unit's main
now stays bare; an imported package's main mangles on its import path
(e.g. aa.bb.main). Both stages.
Gate: test/wcc/989_depmain_sep.c (table-driven, dotted + single-component
shapes, both stages; asserts the mangled dep main + a single bare root
main + cs==ww byte-id; combined path stays neutral).
The driver searchpath is srcd-first (srcd = the entry file's directory).
A co-located black-box test lib/<mod>/<mod>test.ww makes srcd=lib/<mod>,
so resolving `import <mod>` hit the sibling-FILE branch lib/<mod>/<mod>.ww
and folded it inline into the consumer unit under the wrong module tag
("package <mod> does not match import path <importer>") — 7 lib-run tests
fail under separate compilation. The combined amalgamator tolerated the
co-location; only sep surfaced it.
Resolve a package directory-first: walk ALL searchpath entries for a
directory match, and only fall back to a file match if no directory
exists anywhere. A dir-package now beats a same-named sibling file (fixes
the self-named shadow), while a leaf package with no directory (e.g.
lib/encoding/hex) still resolves via its file. This realizes the driver's
"a module is the directory" intent; the originally-specced per-directory
suppression was rejected because it broke leaf packages (rob-pike). Both
stages (cmd/ww/main.c + selfhost twin). The dir-beats-earlier-file
precedence change is latent and loud-failing (#101).
Move-set: ww + ww_ww (driver) only; w6c_ww/wwdump_ww/w6a_ww/w6l_ww HOLD.
Gate: test/wcc/989_coloimport_sep.c (table-driven, both stages).
The driver flip moves build artifacts from next-to-source <stem>.s to a
.sepwork/ scratch dir. 29 Pattern-B gates now build `ww build --sep -o <stem>`
and read <stem>.sepwork/__root.s (multi-package gates concat all
<stem>.sepwork/*.s, since cross-package labels live in per-package .s).
All intermediates redirect to /tmp (WW_PKGCACHE + -o), so the corpus runs
parallel-safe with no source-tree pollution. Tests pass now (--sep is live)
and survive the flip.
915 additionally retargeted off strconv's PRIVATE left_shift_table (a let,
not export) — which separate compilation correctly hides — onto a test-local
package that exports its own probe table (#96). The combined path only linked
it via a single-unit private leak; encapsulation is now honored under sep.
Test-only; all 5 *_ww binaries HOLD. 989_m1mangle_run deferred (blocked by
#99, imported-package fn main mangling under sep).
The N_TTAGGED serializer emitted each variant via wwi_type but never
re-emitted the `...` prefix for TK_ELLIPSIS spread variants, so an
exported `(...inner | str)` round-tripped through .wwi as `(inner | str)`.
The consumer's checker then could not flatten inner's members into the
alias and variadic-assignability rejected bare members — under separate
compilation this broke fmt/log/getopt. Re-emit `...` before the variant
type, both stages; the producer stays purely syntactic (flatten/dedup
remain the consumer's type-store job, per ref/hare/hare/unparse/type.ha:290-300).
Gate: test/wcc/989_wwispread_sep.c — table-driven (2-arm + 3-arm spreads)
x both stages, asserts the marker survives the .wwi, the consumer binds
bare members under --sep (exit 0), and cs==ww .wwi byte-identity.
910_at_test.c and 997_at_test_ww.c build via `ww build` then read
<stem>.combined.ww as a driver side-effect; the E3 flip removes that
side-effect, so they cannot pass post-flip. Their coverage was fully
re-hosted in E2-C2 -- every axis (run-exit, -T cs/ww byte-id, the
userman/badsig/undefbody/linkfail rejects, the dup-decl family, modfn
coexist, user `fn run` coexist) has a live home in 989_septest_run,
911_attest_drop, or 989_declns_sep, several strengthened. Deleting them
before the flip keeps `make test` bisect-clean.
Also retargets a stale 'Sibling of 910_at_test' doc comment in
911_attest_record.c to the surviving gates.
The E3 driver flip makes build_one_sep the sole path; it writes
.sepwork/*.unit.ww, not <stem>.combined.ww, so the flip stops producing
combined.ww. Migrate its consumers to sep-feed first, while both build
paths still exist (reversible; all 5 stage binaries stay byte-identical):
- 990_selfhost: resolveunit drives `ww build --sep` and reads
<stem>.sepwork/__root.unit.ww; probe_cgen_match gains err.ww (sep
inlines fmt's .wwi only, so cs==ww holds); probe_ww_links re-expressed
as `ww_ww build --sep`->run; probe_bootstrap_fixed_point removed (the
monolith path it tested is deleted by E4 -- the self-application axis
lives in `make bootstrap`, the cs==ww axis in 995).
- 994_w6c_ww: drop the combined-fed emit_eq + corpus; keep the live
per-package cs-sep==ww-sep oracle (cmp_sepwork + diff_one + bad[]).
- #110 combined_ww_fresh gate removed (its regen-vs-committed premise
dies with its producer).
Non-vacuity proven: a one-line emitter mutation reddens 994's per-package
cs!=ww check across 15 packages; revert restores green.
Final E2-C2 commit. New decl-namespace gate, both stages: dup fn/type/def/let build-FAIL with a right-reason "duplicate <kind> <name>" diag (shared substring; ww has no line:col); builtin_redecl ACCEPT exit 7 (shadowing a pre-seeded builtin is redeclarable); modfn coexist reshaped to a real directory package (696 precedent) via ww build --sep on two layouts (modfn_coexist + _vbu use-before-value) so both run to 6, cs==ww per-pkg .s/.wwi/.unit.ww, cross-order byte-id on .s/.wwi (.unit.ww excluded: verbatim source flips by construction, order-independence is codegen). __root.s carries both CALL aa.helper and CALL main.aa distinctly. Test-only; all 5 pins hold; 443 to 444.
New direct-w6c -T gate adopting the 3 import-free fixtures 910/997 own (attest_nondrop/undefbody/calldropped.ww) so they outlive the .c gates the E3 flip retires. 8-row nondrop table: compile attest_nondrop.ww four ways (cstage/wwstage x non-T/-T) to .s; non-T keeps only the live sym, -T keeps all @test bodies; cs==ww byte-id on both flags (rule 10). undefbody: both stages reject (body type-checked before the splice). linkfail: full triple link must fail on the dropped @test symbol, guarded by a stderr grep so it fails for the right reason. Test-only; all 5 pins hold; 442 to 443.
Extends the ww test --sep run-exit gate with 3 rows on both stages: userman and badsig REJECT (exit!=0; explicit main under -T, bad @test signature), collide_run ACCEPT exit==0 (user fn run + lib test.run coexist post #80/#84). Reuses the 910/997 fixtures unchanged (SSoT; they outlive the .c gates the E3 flip retires). Under sep a regressed run/test.run collision is a hard w6l duplicate-symbol link error so exit==0 is non-vacuous; collide_run also pins cs==ww byte-id (1 TEXT main, 1 CALL test.run) for the rule-10 stage-symmetry the combined coexist_run carried. Test-only; all 5 binary pins hold.
The -T harness synthesized `use test;` after name-binding, so the lib/test runner run keyed the bare scope and collided with a user-defined bare fn run — a spurious "duplicate fn run" reject (the E1 tolerance seam). Prepending the synth use before binding keys the runner as test.run in the test module namespace, distinct from the user bare run; the two coexist. Hare-faithful: the runner is its own test module (ref/hare/test/+test.ha:97). Inverts attest_userrun.ww from the #23-mandated reject to a coexist fixture; gate asserts exactly 1 TEXT run + 1 TEXT test.run on the -T asm (distinct symbols, not a dead-dup). Closes#80.
A package-less primary's bare fn (module="") whose leaf collided with an
imported module's same-leaf exported fn was mis-mangled to the imported
qualified name (a user `fn run` emitted as `test.run`), producing a
dead-duplicate symbol the linker silently shadowed -- a #263-class silent
miscompile, gate-blind and symmetric across both stages. cgen now registers
bare-module fns and resolves a bare-ident reference to its own bare leaf:
mod_lookup_for_fn prefers the bare entry when the call carries no module
hint and skips bare entries when it does, so the moduled-caller path stays
byte-identical. The moduled `main` entry carve-out is an orthogonal rule
(the linker entry is force-bared) and is retained. Prereq for the @test
user-`run` coexist (#80). The bare non-fn (let/def/type) sibling is the
same class but hint-less; deferred as #85, noted at the retained skip.
Make `ww test --sep` work the Hare +test way: the -T synth test-main emits a
qualified test.run, and the test package is injected as an ordinary
separately-compiled dependency instead of splicing lib/test source into a
flat unit. Additive — combined stays the default and 910/997 are untouched
(their migration is M4 E2).
- compiler synth (both stages): the -T main emits N_DOT test.run plus a
synthetic N_USE "test"; cmd/wcc/check.c + selfhost/cmd/wcc/check.ww.
- driver (both stages): build_one_sep gains is_test, injects the test package
as a root dep, and passes -T to the root; do_test --sep routes a single-file
test through the sep producer; cmd/ww/main.c + selfhost/cmd/ww/main.ww.
- 989_septest_run gate: ww test --sep on both stages, run-exit + cs==ww
byte-id of the sep .s, non-vacuous.
The synth's test.run is left ty_err by the checker in both regimes (lib/test's
run is scope-keyed under "" not "test"; cgen emits the correct CALL via run's
//ww:module test directive) — wwstage tolerates it like cstage (rule-10). The
genuine fix, module-keying run under sep so the call type-resolves, is #80.
w6c_ww/wwdump_ww/ww_ww move (their embedded source changed); w6a_ww/w6l_ww and
the combined codegen output are unchanged.
Before M4 deletes the combined.ww embedding (byte-id-moving), capture a
golden record of the combined codegen oracle while both build paths still
exist. The manifest pins the md5 of the combined-path .s emitted by the
pinned w6c_ww for the five bootstrap targets, plus the five combined-epoch
binary md5s, with a regen procedure. Re-derivable from the m4-combined-epoch
tag, so a future sep-path regression stays drift-checkable against
last-known-good combined output.
Data-only; the five wwstage binary pins are unchanged.
Make the sep-compile == combined-compile output equivalence a standing
regression test — the last criterion before M4 can delete the combined.ww
embedding model.
989_c6soak_run (phase-1): Tier-A/B + same-leaf collision soak. Two real
roots (utf8: encoding.utf8 dotted-path + strings; collide: strings.contains
+ bytes.contains) built both ways with isolated cold WW_PKGCACHE; asserts
combined-run-exit == sep-run-exit on both drivers, cs==ww per-pkg .s/.wwi,
and two distinct collision labels.
994_w6c_ww (phase-2): Tier-C folded into the existing w6c byte-id oracle.
A sep-built w6c emits byte-identical .s to a combined-built w6c on all five
bootstrap targets (w6c/wwdump/w6a/w6l/ww .combined.ww); .wwi is in the
compare set. The tool binaries themselves differ by a benign constant
link-layout delta — the oracle is tool OUTPUT, not tool-binary identity.
Test-only; the five wwstage binary pins are unchanged.
The parser folded a qualified type pkg.Type into two different node shapes by position: declaration position collapsed it into one N_TNAME (resolved via the strrchr-leaf path), but literal position left an N_DOT chain that the struct-literal typeref handoff had no resolver arm for, so pkg.Type{...} rejected with "expected type expression".
Normalize the literal-position N_DOT chain into the same source-order N_TNAME the declaration path emits, reusing the existing resolver; no new checker arm. cstage flattens at parseprimary struct-lit handoff; wwstage (no token peek) folds dots in parsepostfix and normalizes there, guarding numeric tuple components and staying in the postfix loop so trailing ops still chain. Both stages emit identical N_STRUCTLIT(N_TNAME). Prereq for qualifying wcc syntax refs (#75).
After the frontend consolidated into one syntax package, the wcc backend
imports syntax and calls its exported fns — whose signatures reference
types that were unexported. Producing syntax's .wwi interface re-triggered
check_exported_type ("exported declaration references unexported type"):
the residual of BUG-A at the one surviving syntax->wcc boundary. Export
the 16 types that appear in syntax's wcc-facing public surface (directly
in an exported signature, or via a recursively-referenced exported struct
field): nkind, node, lex, tok, tkind, parser, scope, sym, skind, tinfo,
tykind, tfield, tparam, ttupleelem, tctx, tinfocacheent. The set is
minimal (unexporting any one re-breaks the producer) and complete; pos
stays internal. Pure source change — exporting a type emits no code, so
the bootstrap binaries are byte-identical (verified against a clean base
build); only syntax's .wwi gains the type decls.
Post-frontend-reorg residual (#74). syntax now sep-produces clean both
stages. The separate concern of wcc's currently-unqualified refs to
syntax symbols (#75) is a distinct follow-up. Gate 989_syntaxexport_run.
The ww compiler frontend was split across packages lex (lex+tok), ww
(ast+sym+typ), and parse — mirroring Hare's ref/hare/hare/{ast,lex,parse}.
That split's only payoff is third-party reuse, which ww has zero of: the
frontend is consumed by exactly one client, the wcc backend. The split's
cost is a wide cross-package export surface — every fn over a sibling
package's type must export it, and under separate compilation that
re-triggers check_exported_type, plus a phantom `import tok;` (tok lives
in package lex). Consolidate into ONE package lib/ww/syntax/, modelled on
Go's cmd/compile/internal/syntax. The 9 files move in (package syntax);
the intra-frontend mutual references become same-package; wcc and the
tool mains import syntax. No cstage C change (the C frontend mangles from
the source package clause). Internal data shapes (AST kinds, token model,
lexer/parser state) still mirror ref/hare/hare per rule 6/12 — only the
module decomposition collapses; the stdlib is untouched.
USER-approved (#74); spec .ai/rob-frontend-reorg.md (drew2 fidelity-
confirmed). Rule-6 carve-out documented in CLAUDE.md. Dissolves the tok
phantom import; collapses the intra-frontend export sprawl. Byte-id
rebaseline (lex.X/parse.X/ww.X -> syntax.X); cs==ww held. The residual
syntax->wcc export surface (10 types) + the unqualified-ref question are
separate follow-ups (#72/#75).
The .wwi (separate-compile interface) producer could not serialize an
exported def whose initializer is a struct/array literal (N_STRUCTLIT/
N_ARRLIT) — `export def f64info: floatinfo = floatinfo{...}` aborted with
"unhandled const-expr node kind 15". Such a def is a DATA-global per the
#52 model, so its value lives once in the defining package's .o; the
interface needs only the type+symbol. Emit a value-less prototype
`export def X: T;` for aggregate-initializer defs; scalar fold-eligible
defs keep their value (the importer const-folds those). The parser gains
an optional-init arm so the importer can parse the prototype — value-less
`def X: T;` is now legal in any source, symmetric with the existing
bodyless-fn prototype `fn f();` (USER ruling: unconditional; a value-less
def with no defining .o is a loud undefined-symbol error at link, never
silent). Both stages; producer + parser fold into one commit (the
producer's output is unparseable without the parser arm).
M3-tail commit-6 prerequisite #2 (surfaced by the c6 scout). The
aggregate-def-field const-fold boundary is documented inline (#71). Gate
989_sepstructdef_run proves struct+array exported defs sep-build, link,
and run via external DATA refs, cs==ww, with a value-less .wwi.
The --sep producer compiled the ROOT build-target with the .wwi-producer
-I flag, so check_exported_type ran on the root and rejected a real-tool
root's legitimate `export fn f(a: *t)` over an unexported local type t
(the root is terminal — its interface is never imported, and its .wwi is
never consumed). The combined build never passes -I, so it built fine.
For pi==root, invoke w6c with -c -o only, no -I. Both stages (the wwstage
twin builds the shorter root argv). Gates that asserted __root.wwi exists
encoded the buggy behavior; updated to assert __root.s (the consumed
product) while deps' .wwi byte-id is retained.
M3-tail commit-6 prerequisite. Gate 989_seproot_export_run reproduces the
export-fn-over-unexported-type root + proves it sep-builds, with a
non-vacuity leg that the forced -I path still rejects.
A separately-compiled package's primary body was emitted under a bare
`//ww:module-reset`, so its own `package <leaf>;` clause set curmod to
the leaf (e.g. utf8) while the importer spliced the .wwi under the full
`//ww:module encoding.utf8` — definer mangled `utf8.X`, importer wanted
`encoding.utf8.X`, unresolved. Thread the dotted path through the
directive: `//ww:module-reset <path>` sets curmod to the dotted path
(imported stays 0, so the root `fn main` stays bare per #32), and the
body's package clause is demoted to a leaf==last-component assertion
instead of overwriting curmod. Aligns sep-build to the M1 path-mangle
model; only the SEP emitter changes (the combined build_one arm is
untouched, so all combined byte-id gates hold). Both stages mirrored.
Commit-6 broad-soak prerequisite. Gate 989_sepdotpath_run sep-builds a
2-level dotted package and proves definer==importer qualification +
single-component non-vacuity, cs==ww.
Per-package build cache for `ww build --sep`: before recompiling a
package, recompute a plain-text key manifest (md5sum-hex lines of the
package sources, each direct dep's .wwi, the w6c and w6a binaries, plus
the compile flags) and reuse the cached .o/.wwi on a byte-for-byte key
hit. Both stages shell the same host md5sum (no ww-side md5) so the
non-compiler key lines are byte-identical cstage==wwstage; the compiler
md5 lines differ per stage BY DESIGN, giving each stage its own cache
namespace so a hit can never reuse the other stage's .o and mask a
cs!=ww codegen divergence. Cached outputs (.o/.wwi) stay byte-identical
across stages. Root package never cached; cache lives under $(OUT)
(gitignored, wiped by make clean). Both stages mirrored.
Dev-only convenience, off every bootstrap/byte-id gate. Gate
989_pkgcache_run (cold) proves miss-compiles / hit-skips-byte-id /
independent key-bust per input class with non-vacuity rehit.
Per-package .a archives are written by a self-hosted deterministic ar
writer (zeroed mtime/uid/gid, fixed mode 100644, stable member order)
so cstage and wwstage emit byte-identical archives. The linker
force-loads the root .o positionally and pulls deps from .a; a
post-pull PASS-3 over unloaded members reports duplicate symbols
through the archive (#31). Both stages mirrored (cmd/ + selfhost/).
USER-ruled D2 (self-hosted ar writer); pike P1/P2 link model. Gate
989_separchive_run proves cs.a==ww.a byte-identity, 3x-determinism,
link-consumes-.a (exit 7), and the masked-dup-through-.a loud fire.
