A >32B tagged-union field (slice payload) read through a direct *struct
pointer byte-diverged: wwstage's cgloadtaggedfield always loaded R8@+24
before CX@+16, but cstage's direct-*struct-ptr arm (cgen.c ~11926) loads in
offset order CX@+16 then R8@+24. Both ran correct -- a pre-existing rule-10
asm divergence, for a local *struct ptr as well as a global one.
Thread a cxlast flag through cgloadtaggedfield: the direct-ptr site
(cgptrfieldload, the shared local+global chokepoint) passes cxlast=false to
match cstage's offset order; the other 5 callers keep cxlast=true (byte
unchanged). A global flip was rejected -- it would clobber the CX-base
callers (CX@+16 first destroys the base before the R8@+24 read), and the
chained-BX caller must stay R8-first to mirror cstage's chained twin
(cgen.c ~12021); the order is a genuine per-arm property of cstage, not
derivable from the base register.
Test: +2 rows (tagged_slice_field via global *struct ptr, _local via local
*struct ptr), runtime + byte-id; both proven to fail byte-id with only the
compiler files reverted.
wwstage's chained-N_DOT resolver (dotchainresolve) didn't resolve a global
*struct root (only local *T and global value-struct), so gp.sf.len / gp.x.y
bailed to an inner-dot load + shuffle, byte-diverging from cstage's offset-fold.
Both stages already ran correct after #15 (475c003) -- a pure rule-10 asm
divergence. cstage is untouched (the oracle); wwstage aligns up.
Resolve a global N_TPTR root, and extract emitchainbase for the viacx base-load
(byte-identical across the 5 read + 2 store sites it replaces). The chained
STORE caller declines the global-ptr root (yok=false) so it falls to cstage's
address-spine mirror -- matching the #6/#15 decline-to-resolver discipline;
local *T chained stores still fold.
Test: +2 chained rows (gp.sf.len, gp.x.q), runtime + byte-id; proven to fail
byte-id with only the compiler files reverted, pass with the fix.
Sibling follow-ups filed: #17 (>32B tagged word-order), #18 (chained read into
an i64 sink MOVSXD check).
Reading gp.f through a module-global pointer miscompiled in BOTH stages,
differently: cstage classified gp as a local at boff 0 and derefed BP
(MOVQ (BP),BX), wwstage collapsed gp.f to an undefined global symbol f
(MOVQ f(SB)). Both now load the pointer value from the global's data slot
before the field offset, converging on MOVQ gp(SB),BX; MOVQ off(BX),AX.
cstage mirrors the #6 store decline; wwstage gains a global-ptr arm and
shares a cgptrfieldload helper with the local arm.
Fused, not split: the two stages must emit byte-identical asm, so a
one-stage commit would fail the byte-id gate. Sibling byte-divergences
filed: #16 (chained-spine gp.x.y), #17 (>32B tagged word-order).
Test: table-driven 689_globptr_field_read_run (24 rows, runtime + byte-id).
The selfhost checker's resolvewalk had no loop-nesting guard and no
N_BREAK/N_CONTINUE arm, so `break`/`continue` outside any loop fell through
the generic child recursion and was silently accepted -- while cstage
(cmd/wcc/check.c) correctly rejects them. A cs!=ww checker divergence
(rule 10); cstage is correct (break/continue outside a loop is an error in
Hare/C/Go), so align wwstage DOWN, not cstage up.
Mirror cstage's mechanism exactly (check.c:598/2494/2529/2611): a `loops`
counter incremented around for and for-range bodies -- the for-`else` and
the init/cond/post walked OUTSIDE the count, since a break there targets an
enclosing loop -- rejecting break/continue when loops==0 with a
byte-identical `file:line:col: error: <kw> outside loop` diagnostic.
match/switch are not loop targets, matching cstage.
The divergence survived because 300_check.c only exercised the in-process C
checker, never w6c_ww; the fix adds 4 rows to the both-stage
989_catA_f2_reject carrier (break/continue outside loop, the for-else
els-outside-count edge, and an in-loop control). make clean && make test:
all 402 passed, byte-id self-compile gates 990-996 green.
wwi_emit took the .wwi `package` leaf from the first primary decl's module tag;
a fully empty primary module body (zero decls) had none, so the leaf stayed the
literal default "main" and the importer rejected it ("package main does not
match import path <leaf>"). The module identity is only available at parse time
(curmod is overwritten by imported //ww:module sections before emit), so stamp
the primary path onto the N_FILE node (TK_MODULE and TK_MODRESET rp!=NULL sites,
only-if-empty so a bare-reset `package main` root stays "main") and, when the
decl-scan finds no leaf, fall back to that stamped path. Symmetric cstage+
selfhost; both detect scan-miss via the same found-flag so the emitted .wwi
stays byte-identical.
Regression: test/wcc/989_wwileaf_run.c, table-driven over {empty body,
comment-only, nested a.b.c} decl-less shapes, non-vacuity proven.
A torn producer write (e.g. disk-full mid-copy) could leave a 0-byte P.wwi or
P.o in out/.pkgcache under a self-consistent key; cache_lookup checked only
existence, so every later build HIT and served the empty artifact forever
(silent serve-wrong). Reject size==0 on both sides, symmetric across stages:
store refuses to commit a 0-byte temp before the key write, lookup treats a
0-byte cached artifact as a MISS so existing poison self-heals on re-derive.
A valid .wwi/.o is never 0 bytes, so the guard cannot misfire.
Regression: test/wcc/989_pkgcache_poison_run.c, table-driven over
{poison P.wwi | P.o | both}, non-vacuity proven by guard-neuter.
cgassign had dedicated N_DOT-store arms for a local-ptr base, a global
value-struct, and chained bases, but none for a global-pointer scalar
field. That case fell through to the generic cgplaceaddr/dotchainaddr
route, which folds the field offset (ADDQ $foff,BX) then stores to (BX).
cstage emits a single displacement store (MOVQ AX,foff(BX)) via its
via_ptr global scalar arm, so the two stages diverged on asm shape
(rule 10). Both forms are runtime-correct here -- BX is a fresh throwaway
in the generic route -- so this was a byte-id divergence, not a
miscompile.
Add the missing displacement-store arm, predicate-mirroring cstage's
via_ptr global scalar arm exactly: plain assignment only, scalar field
only; non-scalar field types stay on the generic path (their global-ptr
deref is a separate deferred item). glob_ptr_field_test.ww gains an
off-8 row as the regression pin -- offset-0 cannot catch it because
ADDQ $0 is suppressed.
Surfaced by the fold-2 Fam-5 migration.
A bodied fn with a bare C-style `...` was silently accepted by cstage
and SEGFAULTED wwstage (resolvefnbody walked a typeless `...` param).
Gate it: bare C-`...` is allowed only on bodiless decls (extern /
@symbol prototypes), the real FFI path; Hare-style `T...` is unaffected.
ww restricts C-`...` to bodiless decls pending vastart/vaarg/vaend
builtins (#16); harec permits bodied C-variadic fns (check.c:3656) -- a
documented divergence, reopened when #16 lands.
Test 852 runs both stages; its reject rows require the gate's diagnostic
(not merely a nonzero exit), so a crash can't pass them vacuously.
Mirror cstage's C-variadic call handling in the ww self-host: parse a
bare `...` param (decl.ww), skip param-keyed desugar for it to avoid a
nil-deref (check.ww), and emit AL = XMM-reg count plus CVTSS2SD
promotion of f32 args in the variadic tail (cgenutil.ww, cgenexpr.ww).
Closes the cat-A wwstage silent miscompile (AL=0, unpromoted f32 tail).
Parse/check/cgen are one atomic align-up (parse alone miscompiles, so
not bisect-splittable). 989_ffivariadic now runs dual-stage (cstage ww
+ wwstage ww_ww), 12/12; w6c==w6c_ww byte-identical. Byte-id alone is
blind here (the bootstrap calls no float-bearing C variadic), so the
ww_ww runtime rows are the real net.
Mirrors cstage check.c:1491-1495 (type_isint via the syntax.typeisint tinfo chaser, which chases TY_NAMED.under/TY_ENUM.sub — not the AST-keyed isinttypeast that would falsely reject an alias-int index). Record-and-continue, before the base-bail. Reject path emits no asm so cstage==wwstage byte-id holds (453 green). Pre-existing index double-emit deferred (#6).
The wwstage checker silently accepted enums with a non-integer storage type, duplicate members, or a non-constant member value; cstage already rejects all three (cmd/wcc/check.c:1000-1042). Add validateenummembers, a pure read-only diagnostic dispatched once per enum decl from resolvewalk's N_TENUM arm (check.ww:791, beside stampenumvals -- not the per-query recompute arms), mirroring the catB-7/14 validatestructfields pattern. Storage gate uses typeisint on the resolved tinfo (the exact type_isint mirror: chases TY_NAMED.under and TY_ENUM.sub, so an int-alias storage is accepted; raw-AST isinttypeast would not). Duplicate members: O(n^2) name walk. Unfoldable values reuse enumvalfold with until=member (forward-only). Emits via cerr + c.errs, no mutation, so valid-program codegen is unchanged and cstage==wwstage byte-id holds.
wwstage's value-fold message is intentionally generic where cstage's eval_enum_value gives per-reason text (enumvalfold returns a bool, not a reason); both reject. Documented at the site, filed as follow-up task #10.
Test: new table-driven both-stage reject test 850_enum_reject (non-int storage / duplicate member / forward-ref rows + a distinct-member control whose backward-ref value builds and runs). Full make test: 452 green incl. 990-997 byte-id.
The wwstage checker silently accepted a struct with repeated field names; cstage already rejects it (cmd/wcc/check.c:925-947). Add validatestructfields, dispatched once per struct decl from resolvewalk's eager type-decl arm (check.ww:792, sibling to the N_TENUM stampenumvals fire): a pure read-only O(n^2) named-field dup walk that emits "duplicate field 'X'" via cerr + c.errs, with no mutation -- valid-program codegen is unchanged so cstage==wwstage byte-id holds. Named fields only; ww has no struct embedding, so cstage's embed-collision arm is intentionally not ported (separate parser gap, catB-89).
Test: new table-driven both-stage reject test 849_dupfield_reject (adjacent / non-adjacent / different-type dup rows + a distinct-field control that builds and runs). Full make test: 451 green incl. 990-997 byte-id.
ww run / ww test created /tmp/ww_{run,test}_<pid>.sepwork/ but removed only
the built binary, leaking the scratch DIR every invocation — the tmpfs filler
(98,811 entries blocked the gate twice). Add a keepscratch param to
build_one_sep: a thin wrapper rm -rfs the impl scratch at the single
choke-point when keepscratch==0 AND the path ends ".sepwork" (covers every
return, success+error; fires post-link, pre-run). do_build keeps it (the
byte-id gates read <stem>.sepwork from ww build -o); do_run/do_test clean;
do_test no-o redirects scratch into /tmp. Both stages symmetric; reuses the
existing shell rm -rf idiom (lib/os.removeall = #109). Test 989_sepscratch_run
is self-scoped by child pid (non-flaky) with a KEEP control + revert-verified
non-vacuity. Full gate: 448 pass, zero new run/test leaks. (ww_d_* relic +
historical bulk = one-time sweep + agent-probe discipline, not code.)
The E3 flip (#87) made sep the sole compile path and deleted the
combined.ww writer, leaving the six committed *.combined.ww files dead.
Remove them and the last references to the retired amalgamator.
- rm the 6 tracked *.combined.ww (selfhost/cmd/{w6a,w6c,w6l,ww,wwdump}/
main.combined.ww + selfhost/test/smoke.combined.ww). Verified no live
build path or gate still feeds one as compiler INPUT.
- 901_asserttyped_gap: its 5 combined.ww gap fixtures were the last
combined.ww INPUT consumers (4 already missing/vacuous post-flip, only
smoke.combined.ww still fed). Retarget all 5 to sep-feed via a
resolveunit helper (whole-package-dir copy -> `ww build --sep` ->
<stem>.sepwork/__root.unit.ww), mirroring 990's #89 pattern; the 3
import-free test fixtures stay raw-fed. All 8 counts hold at 0 (A-D
coverage, vacuous since the flip, is live again).
- INV-2 (the driver's unresolvable-import-is-fatal guard) is KEPT; only
its "Mirrors the deleted expand" lineage tail is swept. The #110
combined_ww_fresh freshness gate was already removed in #89 (5f85852).
- Sweep dangling amalgamator lineage comments (build_one/buildone/expand/
peek_package/peekpackage + stale combined.ww/combined intermediates)
in cmd/ww/main.c + selfhost/cmd/ww/main.ww, symmetrically (rule-10),
and the stale Makefile combined.ww test-comments (enumcap bigmod.unit.ww
+ 784/787/792/794/848 sep .s cmp + make-smoke sep self-compile).
Closes M4 and epic #22. all 445 pass; 990/993/994/995 byte-id HOLD;
sizelint clean.
The out/.pkgcache content-keyed store copied each artifact IN-PLACE
(cp -f / copyfile) to the fixed paths P.wwi/P.o/P.key. Key-last gave
crash-consistency but NOT concurrent-read safety: two same-stage builds
of a shared lib pkg (rt/time/os) target one out/.pkgcache/<pkg>/P.{wwi,o};
once an early finisher writes P.key, a later build's cache_lookup copies
P.wwi/P.o while a mid-finisher is still mid-write -> torn read -> corrupt
link / cs!=ww. The key is content-only, so it is purely the non-atomic
write.
Fix (Go-build-cache pattern, both stages in lock-step, rule 10): write
each artifact to a per-pid same-dir temp (P.wwi.tmp.<pid> etc.) then
rename() into place. Same dir => rename is atomic (cross-fs is not);
per-pid temp => concurrent writers don't clobber each other mid-copy;
content-keyed => last-writer-wins is byte-identical. Key renamed LAST so
a reader that sees the new key always finds complete artifacts. On any
mid-store error the per-pid temps are unlinked so a failed store leaves
no litter (cstage goto cleanup; wwstage cachermtmp helper).
cstage cmd/ww/main.c cache_store: libc rename(2) + getpid().
wwstage selfhost/cmd/ww/main.ww cachestore: new os.rename + cachetmp.
lib/os/os.ww: add rename(2) (RENAME=82), ref/hare/os/os.ha:17 -- returns
raw i32 errno like sibling remove/mkdir/rmdir (ww's os is the flat
syscall floor, no fs:: layer); a second pathbuf2 slot holds newpath
since kpath's single pathbuf can't carry both paths.
cache_lookup is unchanged: it reads cache->private scratch, and an atomic
source is never torn.
The torn-read race is closed BY CONSTRUCTION; a deterministic behavioral
regression-guard isn't feasible through the product build path (content-
keying => concurrent COLD builds all MISS+STORE, never HIT-read a mid-store
entry; a warm cache is never re-stored). The deferred white-box guard is
TASK #105. A WHY-comment at both fix sites records this.
Tests: 989_sepbuild_run KEEPS its private per-pid WW_PKGCACHE -- the
comment is corrected: the pin is NOT a torn-read mask (closed by
construction) but cold-compile isolation for the test's INTERMEDIATE
(.s/.unit.ww) byte-id compare, which a cache HIT legitimately skips
producing. The former 989_pkgcache_atomic_run is renamed to
989_pkgcache_concurrent_run and HONESTLY relabeled: it is a concurrent
shared-cache build-correctness smoke (N concurrent --sep builds sharing
one cache -> every binary byte-identical to an isolated reference + correct
run, both stages), NOT a torn-read/atomicity proof (a review revert-
experiment proved the original claim vacuous). Shrunk to 4 concurrent
builds x 1 batch x both stages. COLD/dev-only, off every byte-id/bootstrap
gate.
selfhost/cmd/ww/main.combined.ww remains stale (its writer was deleted at
the M4 E3-C1 flip; #90 deletes the file) -- not regenerated.
make test: all 445 passed; make sizelint clean; 990-997 byte-id hold.
build_one_sep (per-package compile + .wwi interfaces + link) becomes the
sole build path. do_build/do_run/do_test and the ww twins all route
through it; --sep is now an accepted no-op and the run-rejects-sep guard
is removed.
Deleted the single-file amalgamator, both stages: build_one, expand,
expand_dir, peek_package (+ the wwstage twins + strictpkgmismatch).
unit_has_package is retained -- the sep scan loop's inline-package check
needs it. The sep-shared helpers (enumerate_dir_ww, locate_import*,
import_path_form, ImportSet, and ww counterparts) stay; they back the
surviving sep path.
Restores missing-package enforcement under sep by construction: the sep
scan loop loudly rejects an unresolvable import (cannot find package
<name>) unless the package is defined inline in the same unit -- matching
the deleted amalgamator and closing the silent-accept the flip would
otherwise introduce.
All 5 wwstage tools relink (each is built via the now-sep `ww build`);
emitted asm is byte-identical to the combined build per bootstrap input,
so the binary md5 delta is pure link layout, not codegen.
selfhost/cmd/ww/main.combined.ww is now stale and unregenerable (its
writer build_one is deleted); #90 deletes it next.
Test retargets folded in (rule-11 carve-out, #61/#133 precedent): each
asserts post-flip-only behavior, is un-pre-migratable unlike #93/#94/#103,
and splitting reddens one side. Closes#97.
- 989_slttypepref -> dir-package layout (xb imports xa so both same-leaf
`invalid` types are in scope at xb.f); inline-multipackage was the
amalgamator shape, deleted with the flip.
- 989_sepbuild_run -> run --sep now genuinely runs (exit 7), not the old
loud-reject (exit 2); + a private per-pid WW_PKGCACHE so the cs/ww
per-package byte-id compare on the shared real lib pkgs (rt/time/os) no
longer races concurrent siblings on the global out/.pkgcache (the flip
made sep the sole path, so every test now contends that cache).
- 737_direnum -> the deleted strictpkgmismatch "differs from" wording ->
sep's "does not match import path" (shared substring, wwstage terser #68).
- 989_lib_byteid -> corpus-completeness scan excludes generated .sepwork
scratch (the old `! -name '*.combined.ww'` exclude didn't cover the new
sep artifact).
The bare-`main` carve-out (which keeps the link entry's main unmangled)
keyed on `leaf == "main" && imported == 0`. Under the combined path a
dependency's body folds in with imported==1, so only the root's main
stayed bare. Under separate compilation each package is its own unit and
a dependency's body carries a path-mangling module-reset but imported==0
(#57) — so an imported `fn main` matched the carve-out, emitted a bare
`TEXT main`, and collided with the root entry (`w6l: duplicate symbol
main`). The combined path was unaffected, so this only surfaced under sep.
Gate the carve-out with sep_isdep = (wwiout != NULL): the producer emits a
.wwi output only for dependency units, never for the root/link-entry unit
(root stripped, #69), symmetric on both stages. Only the root unit's main
now stays bare; an imported package's main mangles on its import path
(e.g. aa.bb.main). Both stages.
Gate: test/wcc/989_depmain_sep.c (table-driven, dotted + single-component
shapes, both stages; asserts the mangled dep main + a single bare root
main + cs==ww byte-id; combined path stays neutral).
The driver searchpath is srcd-first (srcd = the entry file's directory).
A co-located black-box test lib/<mod>/<mod>test.ww makes srcd=lib/<mod>,
so resolving `import <mod>` hit the sibling-FILE branch lib/<mod>/<mod>.ww
and folded it inline into the consumer unit under the wrong module tag
("package <mod> does not match import path <importer>") — 7 lib-run tests
fail under separate compilation. The combined amalgamator tolerated the
co-location; only sep surfaced it.
Resolve a package directory-first: walk ALL searchpath entries for a
directory match, and only fall back to a file match if no directory
exists anywhere. A dir-package now beats a same-named sibling file (fixes
the self-named shadow), while a leaf package with no directory (e.g.
lib/encoding/hex) still resolves via its file. This realizes the driver's
"a module is the directory" intent; the originally-specced per-directory
suppression was rejected because it broke leaf packages (rob-pike). Both
stages (cmd/ww/main.c + selfhost twin). The dir-beats-earlier-file
precedence change is latent and loud-failing (#101).
Move-set: ww + ww_ww (driver) only; w6c_ww/wwdump_ww/w6a_ww/w6l_ww HOLD.
Gate: test/wcc/989_coloimport_sep.c (table-driven, both stages).
The N_TTAGGED serializer emitted each variant via wwi_type but never
re-emitted the `...` prefix for TK_ELLIPSIS spread variants, so an
exported `(...inner | str)` round-tripped through .wwi as `(inner | str)`.
The consumer's checker then could not flatten inner's members into the
alias and variadic-assignability rejected bare members — under separate
compilation this broke fmt/log/getopt. Re-emit `...` before the variant
type, both stages; the producer stays purely syntactic (flatten/dedup
remain the consumer's type-store job, per ref/hare/hare/unparse/type.ha:290-300).
Gate: test/wcc/989_wwispread_sep.c — table-driven (2-arm + 3-arm spreads)
x both stages, asserts the marker survives the .wwi, the consumer binds
bare members under --sep (exit 0), and cs==ww .wwi byte-identity.
The -T harness synthesized `use test;` after name-binding, so the lib/test runner run keyed the bare scope and collided with a user-defined bare fn run — a spurious "duplicate fn run" reject (the E1 tolerance seam). Prepending the synth use before binding keys the runner as test.run in the test module namespace, distinct from the user bare run; the two coexist. Hare-faithful: the runner is its own test module (ref/hare/test/+test.ha:97). Inverts attest_userrun.ww from the #23-mandated reject to a coexist fixture; gate asserts exactly 1 TEXT run + 1 TEXT test.run on the -T asm (distinct symbols, not a dead-dup). Closes#80.
A package-less primary's bare fn (module="") whose leaf collided with an
imported module's same-leaf exported fn was mis-mangled to the imported
qualified name (a user `fn run` emitted as `test.run`), producing a
dead-duplicate symbol the linker silently shadowed -- a #263-class silent
miscompile, gate-blind and symmetric across both stages. cgen now registers
bare-module fns and resolves a bare-ident reference to its own bare leaf:
mod_lookup_for_fn prefers the bare entry when the call carries no module
hint and skips bare entries when it does, so the moduled-caller path stays
byte-identical. The moduled `main` entry carve-out is an orthogonal rule
(the linker entry is force-bared) and is retained. Prereq for the @test
user-`run` coexist (#80). The bare non-fn (let/def/type) sibling is the
same class but hint-less; deferred as #85, noted at the retained skip.
Make `ww test --sep` work the Hare +test way: the -T synth test-main emits a
qualified test.run, and the test package is injected as an ordinary
separately-compiled dependency instead of splicing lib/test source into a
flat unit. Additive — combined stays the default and 910/997 are untouched
(their migration is M4 E2).
- compiler synth (both stages): the -T main emits N_DOT test.run plus a
synthetic N_USE "test"; cmd/wcc/check.c + selfhost/cmd/wcc/check.ww.
- driver (both stages): build_one_sep gains is_test, injects the test package
as a root dep, and passes -T to the root; do_test --sep routes a single-file
test through the sep producer; cmd/ww/main.c + selfhost/cmd/ww/main.ww.
- 989_septest_run gate: ww test --sep on both stages, run-exit + cs==ww
byte-id of the sep .s, non-vacuous.
The synth's test.run is left ty_err by the checker in both regimes (lib/test's
run is scope-keyed under "" not "test"; cgen emits the correct CALL via run's
//ww:module test directive) — wwstage tolerates it like cstage (rule-10). The
genuine fix, module-keying run under sep so the call type-resolves, is #80.
w6c_ww/wwdump_ww/ww_ww move (their embedded source changed); w6a_ww/w6l_ww and
the combined codegen output are unchanged.
After the frontend consolidated into one syntax package (#74), wcc still referenced syntax symbols unqualified — residue of the old flat combined namespace, where bare refs resolved by accident. Under separate compilation Hare and Go both require the package qualifier, so those bare refs would not sep-resolve.
Qualify every wcc reference to a syntax type, function, or enum member as syntax.X across the seven syntax-importing files. Resolution-only: the resolved symbol and emitted code are unchanged, so the two combined.ww regenerate textually but all five _ww binaries hold byte-for-byte. The struct-literal sites resolve via #76. This makes w6c fully separate-compilable.
The parser folded a qualified type pkg.Type into two different node shapes by position: declaration position collapsed it into one N_TNAME (resolved via the strrchr-leaf path), but literal position left an N_DOT chain that the struct-literal typeref handoff had no resolver arm for, so pkg.Type{...} rejected with "expected type expression".
Normalize the literal-position N_DOT chain into the same source-order N_TNAME the declaration path emits, reusing the existing resolver; no new checker arm. cstage flattens at parseprimary struct-lit handoff; wwstage (no token peek) folds dots in parsepostfix and normalizes there, guarding numeric tuple components and staying in the postfix loop so trailing ops still chain. Both stages emit identical N_STRUCTLIT(N_TNAME). Prereq for qualifying wcc syntax refs (#75).
After the frontend consolidated into one syntax package, the wcc backend
imports syntax and calls its exported fns — whose signatures reference
types that were unexported. Producing syntax's .wwi interface re-triggered
check_exported_type ("exported declaration references unexported type"):
the residual of BUG-A at the one surviving syntax->wcc boundary. Export
the 16 types that appear in syntax's wcc-facing public surface (directly
in an exported signature, or via a recursively-referenced exported struct
field): nkind, node, lex, tok, tkind, parser, scope, sym, skind, tinfo,
tykind, tfield, tparam, ttupleelem, tctx, tinfocacheent. The set is
minimal (unexporting any one re-breaks the producer) and complete; pos
stays internal. Pure source change — exporting a type emits no code, so
the bootstrap binaries are byte-identical (verified against a clean base
build); only syntax's .wwi gains the type decls.
Post-frontend-reorg residual (#74). syntax now sep-produces clean both
stages. The separate concern of wcc's currently-unqualified refs to
syntax symbols (#75) is a distinct follow-up. Gate 989_syntaxexport_run.
The ww compiler frontend was split across packages lex (lex+tok), ww
(ast+sym+typ), and parse — mirroring Hare's ref/hare/hare/{ast,lex,parse}.
That split's only payoff is third-party reuse, which ww has zero of: the
frontend is consumed by exactly one client, the wcc backend. The split's
cost is a wide cross-package export surface — every fn over a sibling
package's type must export it, and under separate compilation that
re-triggers check_exported_type, plus a phantom `import tok;` (tok lives
in package lex). Consolidate into ONE package lib/ww/syntax/, modelled on
Go's cmd/compile/internal/syntax. The 9 files move in (package syntax);
the intra-frontend mutual references become same-package; wcc and the
tool mains import syntax. No cstage C change (the C frontend mangles from
the source package clause). Internal data shapes (AST kinds, token model,
lexer/parser state) still mirror ref/hare/hare per rule 6/12 — only the
module decomposition collapses; the stdlib is untouched.
USER-approved (#74); spec .ai/rob-frontend-reorg.md (drew2 fidelity-
confirmed). Rule-6 carve-out documented in CLAUDE.md. Dissolves the tok
phantom import; collapses the intra-frontend export sprawl. Byte-id
rebaseline (lex.X/parse.X/ww.X -> syntax.X); cs==ww held. The residual
syntax->wcc export surface (10 types) + the unqualified-ref question are
separate follow-ups (#72/#75).
The .wwi (separate-compile interface) producer could not serialize an
exported def whose initializer is a struct/array literal (N_STRUCTLIT/
N_ARRLIT) — `export def f64info: floatinfo = floatinfo{...}` aborted with
"unhandled const-expr node kind 15". Such a def is a DATA-global per the
#52 model, so its value lives once in the defining package's .o; the
interface needs only the type+symbol. Emit a value-less prototype
`export def X: T;` for aggregate-initializer defs; scalar fold-eligible
defs keep their value (the importer const-folds those). The parser gains
an optional-init arm so the importer can parse the prototype — value-less
`def X: T;` is now legal in any source, symmetric with the existing
bodyless-fn prototype `fn f();` (USER ruling: unconditional; a value-less
def with no defining .o is a loud undefined-symbol error at link, never
silent). Both stages; producer + parser fold into one commit (the
producer's output is unparseable without the parser arm).
M3-tail commit-6 prerequisite #2 (surfaced by the c6 scout). The
aggregate-def-field const-fold boundary is documented inline (#71). Gate
989_sepstructdef_run proves struct+array exported defs sep-build, link,
and run via external DATA refs, cs==ww, with a value-less .wwi.
The --sep producer compiled the ROOT build-target with the .wwi-producer
-I flag, so check_exported_type ran on the root and rejected a real-tool
root's legitimate `export fn f(a: *t)` over an unexported local type t
(the root is terminal — its interface is never imported, and its .wwi is
never consumed). The combined build never passes -I, so it built fine.
For pi==root, invoke w6c with -c -o only, no -I. Both stages (the wwstage
twin builds the shorter root argv). Gates that asserted __root.wwi exists
encoded the buggy behavior; updated to assert __root.s (the consumed
product) while deps' .wwi byte-id is retained.
M3-tail commit-6 prerequisite. Gate 989_seproot_export_run reproduces the
export-fn-over-unexported-type root + proves it sep-builds, with a
non-vacuity leg that the forced -I path still rejects.
A separately-compiled package's primary body was emitted under a bare
`//ww:module-reset`, so its own `package <leaf>;` clause set curmod to
the leaf (e.g. utf8) while the importer spliced the .wwi under the full
`//ww:module encoding.utf8` — definer mangled `utf8.X`, importer wanted
`encoding.utf8.X`, unresolved. Thread the dotted path through the
directive: `//ww:module-reset <path>` sets curmod to the dotted path
(imported stays 0, so the root `fn main` stays bare per #32), and the
body's package clause is demoted to a leaf==last-component assertion
instead of overwriting curmod. Aligns sep-build to the M1 path-mangle
model; only the SEP emitter changes (the combined build_one arm is
untouched, so all combined byte-id gates hold). Both stages mirrored.
Commit-6 broad-soak prerequisite. Gate 989_sepdotpath_run sep-builds a
2-level dotted package and proves definer==importer qualification +
single-component non-vacuity, cs==ww.
Per-package build cache for `ww build --sep`: before recompiling a
package, recompute a plain-text key manifest (md5sum-hex lines of the
package sources, each direct dep's .wwi, the w6c and w6a binaries, plus
the compile flags) and reuse the cached .o/.wwi on a byte-for-byte key
hit. Both stages shell the same host md5sum (no ww-side md5) so the
non-compiler key lines are byte-identical cstage==wwstage; the compiler
md5 lines differ per stage BY DESIGN, giving each stage its own cache
namespace so a hit can never reuse the other stage's .o and mask a
cs!=ww codegen divergence. Cached outputs (.o/.wwi) stay byte-identical
across stages. Root package never cached; cache lives under $(OUT)
(gitignored, wiped by make clean). Both stages mirrored.
Dev-only convenience, off every bootstrap/byte-id gate. Gate
989_pkgcache_run (cold) proves miss-compiles / hit-skips-byte-id /
independent key-bust per input class with non-vacuity rehit.
Per-package .a archives are written by a self-hosted deterministic ar
writer (zeroed mtime/uid/gid, fixed mode 100644, stable member order)
so cstage and wwstage emit byte-identical archives. The linker
force-loads the root .o positionally and pulls deps from .a; a
post-pull PASS-3 over unloaded members reports duplicate symbols
through the archive (#31). Both stages mirrored (cmd/ + selfhost/).
USER-ruled D2 (self-hosted ar writer); pike P1/P2 link model. Gate
989_separchive_run proves cs.a==ww.a byte-identity, 3x-determinism,
link-consumes-.a (exit 7), and the masked-dup-through-.a loud fire.
Promote commit-3's tri-color topo bail into a loud dep_cycle error that names the full import cycle, byte-identical both stages (cmd/ww/main.c + selfhost/cmd/ww/main.ww, deps.ha:243 parity).
Add a non-vacuous negative gate (989_sepcycle_dup) proving the pre-existing w6l duplicate-symbol reject fires loud + non-zero on a cross-package collision; no new linker code.
#58(b)(c) link-arg parity deferred (system()-string vs procrun()-argv is structurally un-unifiable in this scope); #61 filed for the byte-id-blind w6l_ww dup-message divergence.
build_one_sep (both stages + ww/main.combined.ww regen): discover_deps
(transitive directory-package set, dotted-path identity), tri-color
reverse-topo (cycle bails; loud reject is commit 4), the transitive
producer loop (one w6c -c -I pass per package, dep-first, each both
consumer and producer of its .wwi), and a flat w6l of the .o set.
combined.ww stays the DEFAULT live path; --sep is additive.
Every dep is tagged by its full dotted import path on prepend
(//ww:module <path>), so the definer's qualified symbol (#53) == the
consumer's qualified reference (#40) and the sep .o set links. The
prepend is the TRANSITIVE closure (lead-ratified, superseding
rob-c3-spec §1.3 direct-deps): a dep's interface can name a transitive
dep's type (os exposes time.instant), so the consuming unit needs the
whole closure for resolution — direct-deps-only does not type-check.
Consistent with the current flat-unit transitive-namespace model (the
visibility tighten is #45, post-M4).
Gate 989_sepbuild_run drives ww + ww_ww --sep on the real chain
root->os->{rt,time}: build+run (exit 7) + cs==ww per-pkg .s/.wwi/.unit
+ final binary + transitive-topo discovery + keystone bodies==.wwi
(os,root) through the real driver. Cold scratch; -o-redirected.
Under M1 mangling, EXPORTED non-fn decls (let/def/type) skipped path-
qualification and emitted a BARE symbol (`types.I64_MAX` -> `I64_MAX`).
Under separate compilation two packages exporting the same data leaf
would then collide at w6l. Masked in-tree only because no two packages
export the same non-fn leaf.
§7-A (USER-locked, harec's model): path-qualify EVERY exported decl
(fn AND data) at the single mangle choke-point — mod_collect /
collectmods. Retire the `!isfn && d->export` (cstage) and `exported==0`
(wwstage) skips: every decl with a module now mangles `<mod>.<name>`.
The ONLY bare symbols left are @symbol FFI overrides (ffi_resolve at
emit) and the ROOT unit's `main` — both already carved out before the
map insert.
With exported decls in the map the exact-(name,hint)-or-bare value
dance is dead — its sole purpose was the bare-exported case. Delete
mod_lookup_value / mod_mangle_value / mahint (cstage) and
modlookupvalue / emitsymnamehint (wwstage); the value-global sites now
route through the same hint-aware-with-fallback lookup as fns
(mod_mangle_fn/mafn, emitfnname). Net negative LOC in the mangler.
Transparent rename on the live combined path: ref and def move in
lockstep, so cs==ww byte-id holds and the self-host still builds + runs
(fixed-point/995). Byte-id REBASELINE — all 5 ww binaries shift. The
w6c/wwdump combined.ww embed wcc cgen and are regenerated.
Strlit labels were emitted as `_S_<n>` from a global counter with no
per-unit prefix (cgen.c intern_strlit + wwstage internstrlit twin).
Under separate compilation two str-bearing packages both emit `_S_0`..
-> w6l link collision.
Prefix the label with the owning package PATH (`<module>._S_<n>`,
matching mklabel's spelling). The prefix is c->cur_mod, set per-fn by
cgfn and now per-decl by let_pre_intern (save/restore so the later
emit passes, which read cur_mod for fn-ptr relocs, are unaffected).
Pure function of the module path — NOT a build-nonce — so the
self-host fixed-point holds across ww2/ww3/ww4. Both stages, symmetric.
Transparent rename on the live combined path: the label is interned
once and shared by every reference, so ref and def move in lockstep.
cs==ww byte-id holds; the w6c/wwdump combined.ww embed wcc/cgen.ww and
are regenerated.
746_strdef_inline: the strdef-inline sentinel pinned the bare
`LEAQ\t_S_` shape; update to the module-prefixed form (alpha._S_ for
the in-module def, beta._S_ for the use-site-interned cross-module
inline).
989_m3sep_run: add the #49 LINK leg. The str sub-fixture (sleaf+smid)
was keystone-only — never linked — precisely because the global
counter made both emit `_S_0`. With the prefix, compile both `-c`
separately, link (w6l) + run (sroot reads a distinguishing byte through
each string's .ptr, so a collided label would corrupt the exit), both
stages + cs==ww final exe.
lib/types limit consts were bare `def`s, so the .wwi (sep-compile's
interface) correctly omitted them while the flat combined.ww let a
cross-package user (lib/strings splitn → types.I32_MAX) reach the
private def — sep-compile then failed (wwstage `asserttyped: dot
'I32_MAX'`; cstage undefined-ref). Hare exports types::I32_MAX and the
whole limit family (ref/hare/types/limits.ha, arch+x86_64.ha); ww not
exporting them was the divergence.
export the 24 existing limit defs ({I,U}{8,16,32,64}_{MIN,MAX},
INT/UINT/SIZE/UINTPTR_{MIN,MAX}) and the existing RUNE_MIN, and add
exported RUNE_MAX. ww's derived machine-word int/uint/size/uintptr
VALUES are kept verbatim (user-ratified 64-bit-int divergence); fidelity
here is the NAME SET + export-visibility, not the values. RUNE_MAX is
written `0x10ffff: rune` — same codepoint as Hare's '\U0010ffff', forced
because ww's lexer has no \u/\U escape (#50).
Exporting the consts made `w6c -I` walk them and fatal on RUNE_MIN
('\0'): the .wwi const-expr unparser had no N_RUNELIT arm. Add one,
both stages (wwi_rune / wwirune), rendering a \xHH-escaped rune literal
(>0xFF fails loud, #50). Const casts need no arm — the checker folds
them to integer literals before the producer runs. 989_m2wwi_run gains
a types.wwi gate (byte-id + re-parse + asserts export def I32_MAX and
RUNE_MAX reach the interface). byte-id-neutral: a def emits no symbol.
The M2 .wwi producer rendered an exported fn carrying @symbol("...")
as a bare prototype, dropping the FFI link-symbol binding. A
sep-compiled consumer reading the .wwi then emitted `CALL malloc`
instead of `CALL rt_malloc` for `@symbol("rt_malloc") export fn
malloc`, breaking the bodies-vs-.wwi byte-id and the link. Affects
every package whose closure reaches rt/os.
Emit codegen/link-relevant attributes through a single named
predicate (wwi_attr_relevant / wwiattrrelevant), today true iff the
name is "symbol" — the only such attribute that exists. @align/@offset
are NOT field attributes in ww (the parser parses no field attrs, the
N_TFIELD node has no attr slot); the predicate is named for the class
so they slot in if ww ever grows them (#51). attr is assigned at
exactly one site per stage (parse.c:1351 / decl.ww:168), both inside
parsefn, so only N_FNDECL carries attrs and the fn-decl render path
covers the whole class.
Both stages, byte-identical (rule 10). 989_m2wwi_run synth gate gains
an @symbol fn + a content assertion that the .wwi carries it verbatim.
New `w6c -c` (both stages): separate-compile / primary-only codegen.
Emit code+DATA ONLY for a package's own (imported==0) decls; treat every
`.wwi`-sourced (imported==1) dep decl as an external. Pure addition behind
the flag — combined.ww stays the LIVE path, `-c` is off on every existing
invocation, so the 990-997 byte-id gates + all prior tests are unperturbed.
The keystone (rob): a `.wwi` is body-less/init-less prototype source, and
cgen already skips body-less fns as externs, so dep fns/types/defs emit
NOTHING for free. The single genuinely-new guard is an imported value-
global (`export let`): its DATAW would DUPLICATE the dep's own definition
(link collision), so it is skipped. The `imported==0` gate is applied at
all top-level emit sites for uniformity (close-by-construction): the fn
loop, emit_lets/emitletdataw, emit_defs/emitdefconstants, and
let_pre_intern/letpreintern — that last one because an imported dep's body
initializer interns strlits while its rhs-stripped `.wwi` does not, which
would shift the _S_ sequence; gating it keeps the strlit table a pure
function of the package's own decls. EXACTLY symmetric with M2's producer
`imported==0` filter — same predicate both directions.
Driver `--sep` build_one_sep + per-package archives + multi-.a link +
cache + BROAD real-target dual-path soak are M3-tail (#46, rob ruling B):
M3-core ships the codegen spine + a self-contained gate that proves all
codegen correctness without a production driver.
Gate 989_m3sep_run: a synth leaf->mid->root fixture carrying all four
cross-boundary fact-classes (fn signature, struct LAYOUT, `def` const
VALUE, `export let` value-global). Per package, holding `-c` constant:
`w6c -c` of (deps-as-bodies) == (deps-as-.wwi) byte-for-byte (the .wwi
conveys exactly the dep facts P's codegen needs); cs==ww at the .s AND
final-exe level (rule 10); sep-path determinism; the value-global guard
(imported origin_tag never re-emits DATAW); and behavioral identity (the
linked program's exit code is the real cross-boundary computation). COLD:
.wwi materialized fresh every run (no warm cache).
combined.ww regen'd for wwdump + w6c (both embed cgen.ww); diff is exactly
the four guards + the flag wiring, nothing spurious.
New `w6c -I <out.wwi>` flag (both stages) writes a re-parseable
ww-prototype rendering of a package's EXPORTED surface. M2 dead-code:
nothing consumes .wwi yet (combined.ww stays the live path); the flag is
off on every existing invocation, so the 990-997 byte-id gates and all
prior tests are unperturbed.
The unparse walks the AST type-expr subtree (N_T* nodes), not the
tinfo Type* (which collapses nominal pkg.Name identity). Deterministic
output: package line, byte-sorted imports, byte-sorted decls — a pure
function of the exported API. cmd/w6c/wwi.c + selfhost/cmd/wcc/wwi.ww
emit byte-identical .wwi (new cross-stage byte-id substrate, rule 10).
check_exported_type (drew) rides the producer entry, flag-gated: an
exported signature naming a non-exported nominal is loud-rejected before
any byte is written, identically on both stages. Ports harec
check.c:4092-4168, recursing the type-AST and gating on the resolved
SK_TYPE sym's decl export flag (Sym.exported is vestigial in both
stages; the predeclared synthetic `nomem` decl carries no source
position and is treated as a builtin leaf — cstage parity).
Two wwstage checker AST-mutations are normalized to cstage's pristine
view for byte-id: the N_TPARAM tuple-element wrapper (unwrapped) and the
variadic `T...`→`[]T` param desugar (peeled).
Gate 989_m2wwi_run: ascii/strings/getopt each produce a .wwi that
re-parses (wwdump -a) and is cs==ww byte-identical; a private-type-leak
fixture is rejected identically by both stages (non-vacuous check).
use_hint/usehint were unit-global first-leaf-match: two directory-
packages exporting the same fn leaf, each imported by a different module
aliasing the same bareword, mis-routed every qualified call to whichever
use was collected first. Identically wrong on both stages (byte-id-green
#263-class). Key the hint on (owner-module, alias) and prefer cur_mod,
mirroring the checker's use_path curmod-preference (55f54fb).
989_m1usehint_run: two same-leaf pick() across a.math/b.math, each
module's call routes to its own import (111/222) + cs.s==ww.s.
Switch symbol mangling from the import leaf clause to the full dotted import path for directory packages; single-file imports keep package-clause mangling (isdir-gate: imported<=>directory-import). The root build unit's fn main stays bare, every other top-level decl mangles, closing #31's duplicate-main hazard by construction (#32). Both stages, byte-identical.
Single commit, not split: the bare rename (f244af3) is red on its own because it unmasks cross-module resolution gaps that do not reproduce pre-M1, so the fixes are intrinsic to making the rename correct. Included: wwstage fnret/fnparamslookupmod map import alias->path (#199b cross-module union-variant scrutinee resolved the wrong fn's union); cstage use_path prefers the referencing module's import for an ambiguous leaf alias (sha256 crypto.math vs strconv math). Tests table-driven: 989_m1mangle_run/_sym, 989_m1union_run (gate-visible per-arm exit codes + cs==ww byte-id).
The sym carried an is_const flag (lib/ww/sym.ww) but wwstage never
set it at the let-install nor consumed it at assignment, so mutating
a `const` slipped through silently. Mirror cstage's two sites: set
is_const when n.op == TK_CONST at the local let-install (cmd/wcc/
check.c:2408) and reject an N_ASSIGN whose lhs ident resolves to an
is_const sym (cmd/wcc/check.c:1889-1896). cstage already rejected;
this aligns wwstage's w6c_ww UP. A bare `_` discard lvalue (empty
str) is skipped.
Regen w6c/wwdump combined.ww (checker embeds in both). Valid-program
codegen unchanged → cs==ww byte-id gate stays green.
wwstage's checkretassign short-circuited on a value-less `return;`
("skip flagging for now"), so `fn f() i32 = { return; }` built and
RET'd a garbage register. Mirror cstage cmd/wcc/check.c:2428-2439:
the no-value return has type void, then run isassignable(c.fnret,
void) — void→void and void→(T|void) accept, void→i32 is a confident
reject. cstage already rejected; this aligns wwstage's w6c_ww UP.
Regen w6c/wwdump combined.ww (checker embeds in both). Valid-program
codegen unchanged → cs==ww byte-id gate stays green.
wwstage's tuple-parse loop checked the RPAREN-break at the top, so
`(a,)` parsed as a 1-element N_TUPLE and reached cgen — a silent
wrong-accept. A trailing comma is legal only after >=2 elements.
Align the loop order to cstage cmd/wcc/parse.c:552-558 (parse each
element before the RPAREN-break); `(a,)` now errors at the next
parseexpr, `(a, b)` / `(a, b,)` are unchanged. cstage already
rejected; this brings wwstage's w6c_ww parser into agreement.
Regen w6c/wwdump combined.ww (parser embeds in both). Valid-program
codegen unchanged → cs==ww byte-id gate stays green.
cstage rejects a binop whose two integer operands have different
types (e.g. int vs i32 from len()); wwstage accepted it, miscompiling
under no-implicit-promotion. Align wwstage UP: unifyarith now chases
aliases and loud-rejects an integer-type mismatch, routing the
ordered-comparison ops through the same path with the error message
threaded on `e`. Per the user's no-implicit-promotion decision.
Scope carve-outs: EQ/NEQ stay out of the reject (#34, the comparison
operators keep their own widening rule) and a rune literal is exempt
(#35, N_RUNELIT is still untyped at this point). Adds the 29-case
test/wcc/949_intbinop_mismatch.c and its Makefile wiring.
wwstage's typeeqast lacked the identity short-circuit cstage type_eq
opens with (cmd/wcc/type.c:250 `if (a == b) return 1`). Enum/struct/
array type nodes are shared from their decl, so two references to the
same type resolve to one node; without the fast-path the catch-all
returns false. Exposed by #26's integer-mismatch reject, which fired
on a same-enum binop like w6l's `os.flag.WRONLY|CREATE|TRUNC` that
cstage accepts via this check. Corpus output unchanged (the w6c_ww/
wwdump_ww binaries move because check.ww regenerates combined.ww).
memio.fixedwrite returned a successful 0-byte write once the sink
filled, so an overflowing fprintf/bsprintf surfaced a truncated prefix
as a successful str instead of an error. Hare's fixed_write returns
nomem there (ref/hare/memio/stream.ha:161); the bsprintf/fprintf
io.error arm already forwards it, so the prefix-on-overflow path is the
only divergence.
Mirror Hare's full guard order: an empty input buf short-circuits to 0
(stream.ha:157) before the full-sink nomem guard, so a 0-byte write to
a full sink stays 0 (no new divergence). fmt.bsprintf/formatone keep
their logic; only their now-stale WHY-comments are rewritten, and
formatone's tail-pad counter is left as-is (the width-form restore is a
deferred follow-up, out of F-R scope). memio's own `fixed` doc comment,
which still claimed ww surfaces 0 on a full buffer, is corrected to the
new nomem contract.
Tests: flip the two fmt rows that pinned the prefix bug (bsprintf_trunc,
bsprintf_width_trunc) plus memiotest fixedwritecases' overflow row to
assert `is nomem`; add positive controls (bsprintf_exact must still
succeed) + an empty-sink discriminator (bsprintf_empty) + a dedicated
fixedwritefull unit pinning the memio.ww:190 contract.
Regenerates the w6c and wwdump combined.ww (memio's fixedwrite change
and `fixed` doc comment are the only embedded changes; fmt is
dead-code-eliminated from both).
