The E3 flip (#87) made sep the sole compile path and deleted the
combined.ww writer, leaving the six committed *.combined.ww files dead.
Remove them and the last references to the retired amalgamator.
- rm the 6 tracked *.combined.ww (selfhost/cmd/{w6a,w6c,w6l,ww,wwdump}/
main.combined.ww + selfhost/test/smoke.combined.ww). Verified no live
build path or gate still feeds one as compiler INPUT.
- 901_asserttyped_gap: its 5 combined.ww gap fixtures were the last
combined.ww INPUT consumers (4 already missing/vacuous post-flip, only
smoke.combined.ww still fed). Retarget all 5 to sep-feed via a
resolveunit helper (whole-package-dir copy -> `ww build --sep` ->
<stem>.sepwork/__root.unit.ww), mirroring 990's #89 pattern; the 3
import-free test fixtures stay raw-fed. All 8 counts hold at 0 (A-D
coverage, vacuous since the flip, is live again).
- INV-2 (the driver's unresolvable-import-is-fatal guard) is KEPT; only
its "Mirrors the deleted expand" lineage tail is swept. The #110
combined_ww_fresh freshness gate was already removed in #89 (5f85852).
- Sweep dangling amalgamator lineage comments (build_one/buildone/expand/
peek_package/peekpackage + stale combined.ww/combined intermediates)
in cmd/ww/main.c + selfhost/cmd/ww/main.ww, symmetrically (rule-10),
and the stale Makefile combined.ww test-comments (enumcap bigmod.unit.ww
+ 784/787/792/794/848 sep .s cmp + make-smoke sep self-compile).
Closes M4 and epic #22. all 445 pass; 990/993/994/995 byte-id HOLD;
sizelint clean.
The out/.pkgcache content-keyed store copied each artifact IN-PLACE
(cp -f / copyfile) to the fixed paths P.wwi/P.o/P.key. Key-last gave
crash-consistency but NOT concurrent-read safety: two same-stage builds
of a shared lib pkg (rt/time/os) target one out/.pkgcache/<pkg>/P.{wwi,o};
once an early finisher writes P.key, a later build's cache_lookup copies
P.wwi/P.o while a mid-finisher is still mid-write -> torn read -> corrupt
link / cs!=ww. The key is content-only, so it is purely the non-atomic
write.
Fix (Go-build-cache pattern, both stages in lock-step, rule 10): write
each artifact to a per-pid same-dir temp (P.wwi.tmp.<pid> etc.) then
rename() into place. Same dir => rename is atomic (cross-fs is not);
per-pid temp => concurrent writers don't clobber each other mid-copy;
content-keyed => last-writer-wins is byte-identical. Key renamed LAST so
a reader that sees the new key always finds complete artifacts. On any
mid-store error the per-pid temps are unlinked so a failed store leaves
no litter (cstage goto cleanup; wwstage cachermtmp helper).
cstage cmd/ww/main.c cache_store: libc rename(2) + getpid().
wwstage selfhost/cmd/ww/main.ww cachestore: new os.rename + cachetmp.
lib/os/os.ww: add rename(2) (RENAME=82), ref/hare/os/os.ha:17 -- returns
raw i32 errno like sibling remove/mkdir/rmdir (ww's os is the flat
syscall floor, no fs:: layer); a second pathbuf2 slot holds newpath
since kpath's single pathbuf can't carry both paths.
cache_lookup is unchanged: it reads cache->private scratch, and an atomic
source is never torn.
The torn-read race is closed BY CONSTRUCTION; a deterministic behavioral
regression-guard isn't feasible through the product build path (content-
keying => concurrent COLD builds all MISS+STORE, never HIT-read a mid-store
entry; a warm cache is never re-stored). The deferred white-box guard is
TASK #105. A WHY-comment at both fix sites records this.
Tests: 989_sepbuild_run KEEPS its private per-pid WW_PKGCACHE -- the
comment is corrected: the pin is NOT a torn-read mask (closed by
construction) but cold-compile isolation for the test's INTERMEDIATE
(.s/.unit.ww) byte-id compare, which a cache HIT legitimately skips
producing. The former 989_pkgcache_atomic_run is renamed to
989_pkgcache_concurrent_run and HONESTLY relabeled: it is a concurrent
shared-cache build-correctness smoke (N concurrent --sep builds sharing
one cache -> every binary byte-identical to an isolated reference + correct
run, both stages), NOT a torn-read/atomicity proof (a review revert-
experiment proved the original claim vacuous). Shrunk to 4 concurrent
builds x 1 batch x both stages. COLD/dev-only, off every byte-id/bootstrap
gate.
selfhost/cmd/ww/main.combined.ww remains stale (its writer was deleted at
the M4 E3-C1 flip; #90 deletes the file) -- not regenerated.
make test: all 445 passed; make sizelint clean; 990-997 byte-id hold.
build_one_sep (per-package compile + .wwi interfaces + link) becomes the
sole build path. do_build/do_run/do_test and the ww twins all route
through it; --sep is now an accepted no-op and the run-rejects-sep guard
is removed.
Deleted the single-file amalgamator, both stages: build_one, expand,
expand_dir, peek_package (+ the wwstage twins + strictpkgmismatch).
unit_has_package is retained -- the sep scan loop's inline-package check
needs it. The sep-shared helpers (enumerate_dir_ww, locate_import*,
import_path_form, ImportSet, and ww counterparts) stay; they back the
surviving sep path.
Restores missing-package enforcement under sep by construction: the sep
scan loop loudly rejects an unresolvable import (cannot find package
<name>) unless the package is defined inline in the same unit -- matching
the deleted amalgamator and closing the silent-accept the flip would
otherwise introduce.
All 5 wwstage tools relink (each is built via the now-sep `ww build`);
emitted asm is byte-identical to the combined build per bootstrap input,
so the binary md5 delta is pure link layout, not codegen.
selfhost/cmd/ww/main.combined.ww is now stale and unregenerable (its
writer build_one is deleted); #90 deletes it next.
Test retargets folded in (rule-11 carve-out, #61/#133 precedent): each
asserts post-flip-only behavior, is un-pre-migratable unlike #93/#94/#103,
and splitting reddens one side. Closes#97.
- 989_slttypepref -> dir-package layout (xb imports xa so both same-leaf
`invalid` types are in scope at xb.f); inline-multipackage was the
amalgamator shape, deleted with the flip.
- 989_sepbuild_run -> run --sep now genuinely runs (exit 7), not the old
loud-reject (exit 2); + a private per-pid WW_PKGCACHE so the cs/ww
per-package byte-id compare on the shared real lib pkgs (rt/time/os) no
longer races concurrent siblings on the global out/.pkgcache (the flip
made sep the sole path, so every test now contends that cache).
- 737_direnum -> the deleted strictpkgmismatch "differs from" wording ->
sep's "does not match import path" (shared substring, wwstage terser #68).
- 989_lib_byteid -> corpus-completeness scan excludes generated .sepwork
scratch (the old `! -name '*.combined.ww'` exclude didn't cover the new
sep artifact).
The last flip casualty in the test-migration class (the filed #66
"WW_W6C under sep" framing was a phantom — cstage honors the override;
see re-scoped #66 for the real wwstage align-up). 809_idx_structlit_store's
asm_byte_identical drove `bin/ww build` then read <src>.s next-to-source;
the flip relocates asm to .sepwork/, so it now drives `ww build --sep -o
<stem>` (cstage + WW_W6C=w6c_ww wwstage legs) and byte-diffs the
concatenated per-package .sepwork/*.s. callee_capture imports strings, so
concat-all is the faithful equivalent of the old whole-program compare.
Adding --sep is flip-invariant (green pre- and post-flip). 809_delete_range
is flip-invariant (direct w6c -o), left untouched. Test-only; all 5 pins HOLD.
94c, the last C-mig piece. The #65 enumeratedir-cap gate built a 300-file
bigmod dir (past the old 256 cap) and byte-compared cstage vs wwstage
.combined.ww to prove both enumerate the complete file set. The flip
deletes combined.ww, so the comparison now targets each stage's
<stem>.sepwork/bigmod.unit.ww (one `package bigmod;` clause per enrolled
file). count_pkgs asserts the full nfiles count and `cmp` byte-compares
the two stages' unit, so a silent cap-drop on either side reddens both
paths. >cap fixture unchanged. Test-only; all 5 binary pins HOLD.
94b of the C-mig2 split (own commit: lone phase-1 test-commit cache
gate). The corpus gate now drives two --sep builds (ww / ww_ww) per lib
unit and byte-diffs the concatenated per-package .sepwork/*.s. Keying on
every per-package .s — the imported lib body (e.g. log.s) and lib/test
(test.s), never collapsed to __root.s — preserves the cs!=ww lib-body
coverage the gate exists for. Tri-mode (ID/DIVERGE/WWREJECT) per module,
import-landed sentinel, and corpus_complete scan all preserved. 43/43
(net=M_ID via #102). Test-only; all 5 binary pins HOLD.
connect/bind export `*sockaddrin`, so under --sep the .wwi producer's
check_exported_type (both stages, correctly) rejects an exported decl
referencing the unexported type. Invisible on the old combined path
(net inlined, fed to w6c without -I). Hare exports sockaddr_in
(ref/hare/sys/+linux/socket.ha:11) and lib/net plays Hare's sys role,
so the caller must be able to name it. Same class as #48. No codegen
change (export is a checker property); byte-id holds.
94a of the C-mig2 split (rob rule-11): 13 Pattern-A gates that fed a
.combined.ww to w6c/w6c_ww now drive two --sep builds (ww / ww_ww) and
byte-diff the concatenated per-package .sepwork/*.s, mirroring #93's
convention. Lands pre-flip while combined.ww still exists as the
reversible safety net. Test-only; all 5 binary pins HOLD.
989_enumcap_run deferred to 94b (its decisive assertion is a combined.ww
content diff, not a .s diff).
The bare-`main` carve-out (which keeps the link entry's main unmangled)
keyed on `leaf == "main" && imported == 0`. Under the combined path a
dependency's body folds in with imported==1, so only the root's main
stayed bare. Under separate compilation each package is its own unit and
a dependency's body carries a path-mangling module-reset but imported==0
(#57) — so an imported `fn main` matched the carve-out, emitted a bare
`TEXT main`, and collided with the root entry (`w6l: duplicate symbol
main`). The combined path was unaffected, so this only surfaced under sep.
Gate the carve-out with sep_isdep = (wwiout != NULL): the producer emits a
.wwi output only for dependency units, never for the root/link-entry unit
(root stripped, #69), symmetric on both stages. Only the root unit's main
now stays bare; an imported package's main mangles on its import path
(e.g. aa.bb.main). Both stages.
Gate: test/wcc/989_depmain_sep.c (table-driven, dotted + single-component
shapes, both stages; asserts the mangled dep main + a single bare root
main + cs==ww byte-id; combined path stays neutral).
The driver searchpath is srcd-first (srcd = the entry file's directory).
A co-located black-box test lib/<mod>/<mod>test.ww makes srcd=lib/<mod>,
so resolving `import <mod>` hit the sibling-FILE branch lib/<mod>/<mod>.ww
and folded it inline into the consumer unit under the wrong module tag
("package <mod> does not match import path <importer>") — 7 lib-run tests
fail under separate compilation. The combined amalgamator tolerated the
co-location; only sep surfaced it.
Resolve a package directory-first: walk ALL searchpath entries for a
directory match, and only fall back to a file match if no directory
exists anywhere. A dir-package now beats a same-named sibling file (fixes
the self-named shadow), while a leaf package with no directory (e.g.
lib/encoding/hex) still resolves via its file. This realizes the driver's
"a module is the directory" intent; the originally-specced per-directory
suppression was rejected because it broke leaf packages (rob-pike). Both
stages (cmd/ww/main.c + selfhost twin). The dir-beats-earlier-file
precedence change is latent and loud-failing (#101).
Move-set: ww + ww_ww (driver) only; w6c_ww/wwdump_ww/w6a_ww/w6l_ww HOLD.
Gate: test/wcc/989_coloimport_sep.c (table-driven, both stages).
The driver flip moves build artifacts from next-to-source <stem>.s to a
.sepwork/ scratch dir. 29 Pattern-B gates now build `ww build --sep -o <stem>`
and read <stem>.sepwork/__root.s (multi-package gates concat all
<stem>.sepwork/*.s, since cross-package labels live in per-package .s).
All intermediates redirect to /tmp (WW_PKGCACHE + -o), so the corpus runs
parallel-safe with no source-tree pollution. Tests pass now (--sep is live)
and survive the flip.
915 additionally retargeted off strconv's PRIVATE left_shift_table (a let,
not export) — which separate compilation correctly hides — onto a test-local
package that exports its own probe table (#96). The combined path only linked
it via a single-unit private leak; encapsulation is now honored under sep.
Test-only; all 5 *_ww binaries HOLD. 989_m1mangle_run deferred (blocked by
#99, imported-package fn main mangling under sep).
The N_TTAGGED serializer emitted each variant via wwi_type but never
re-emitted the `...` prefix for TK_ELLIPSIS spread variants, so an
exported `(...inner | str)` round-tripped through .wwi as `(inner | str)`.
The consumer's checker then could not flatten inner's members into the
alias and variadic-assignability rejected bare members — under separate
compilation this broke fmt/log/getopt. Re-emit `...` before the variant
type, both stages; the producer stays purely syntactic (flatten/dedup
remain the consumer's type-store job, per ref/hare/hare/unparse/type.ha:290-300).
Gate: test/wcc/989_wwispread_sep.c — table-driven (2-arm + 3-arm spreads)
x both stages, asserts the marker survives the .wwi, the consumer binds
bare members under --sep (exit 0), and cs==ww .wwi byte-identity.
910_at_test.c and 997_at_test_ww.c build via `ww build` then read
<stem>.combined.ww as a driver side-effect; the E3 flip removes that
side-effect, so they cannot pass post-flip. Their coverage was fully
re-hosted in E2-C2 -- every axis (run-exit, -T cs/ww byte-id, the
userman/badsig/undefbody/linkfail rejects, the dup-decl family, modfn
coexist, user `fn run` coexist) has a live home in 989_septest_run,
911_attest_drop, or 989_declns_sep, several strengthened. Deleting them
before the flip keeps `make test` bisect-clean.
Also retargets a stale 'Sibling of 910_at_test' doc comment in
911_attest_record.c to the surviving gates.
The E3 driver flip makes build_one_sep the sole path; it writes
.sepwork/*.unit.ww, not <stem>.combined.ww, so the flip stops producing
combined.ww. Migrate its consumers to sep-feed first, while both build
paths still exist (reversible; all 5 stage binaries stay byte-identical):
- 990_selfhost: resolveunit drives `ww build --sep` and reads
<stem>.sepwork/__root.unit.ww; probe_cgen_match gains err.ww (sep
inlines fmt's .wwi only, so cs==ww holds); probe_ww_links re-expressed
as `ww_ww build --sep`->run; probe_bootstrap_fixed_point removed (the
monolith path it tested is deleted by E4 -- the self-application axis
lives in `make bootstrap`, the cs==ww axis in 995).
- 994_w6c_ww: drop the combined-fed emit_eq + corpus; keep the live
per-package cs-sep==ww-sep oracle (cmp_sepwork + diff_one + bad[]).
- #110 combined_ww_fresh gate removed (its regen-vs-committed premise
dies with its producer).
Non-vacuity proven: a one-line emitter mutation reddens 994's per-package
cs!=ww check across 15 packages; revert restores green.
Final E2-C2 commit. New decl-namespace gate, both stages: dup fn/type/def/let build-FAIL with a right-reason "duplicate <kind> <name>" diag (shared substring; ww has no line:col); builtin_redecl ACCEPT exit 7 (shadowing a pre-seeded builtin is redeclarable); modfn coexist reshaped to a real directory package (696 precedent) via ww build --sep on two layouts (modfn_coexist + _vbu use-before-value) so both run to 6, cs==ww per-pkg .s/.wwi/.unit.ww, cross-order byte-id on .s/.wwi (.unit.ww excluded: verbatim source flips by construction, order-independence is codegen). __root.s carries both CALL aa.helper and CALL main.aa distinctly. Test-only; all 5 pins hold; 443 to 444.
New direct-w6c -T gate adopting the 3 import-free fixtures 910/997 own (attest_nondrop/undefbody/calldropped.ww) so they outlive the .c gates the E3 flip retires. 8-row nondrop table: compile attest_nondrop.ww four ways (cstage/wwstage x non-T/-T) to .s; non-T keeps only the live sym, -T keeps all @test bodies; cs==ww byte-id on both flags (rule 10). undefbody: both stages reject (body type-checked before the splice). linkfail: full triple link must fail on the dropped @test symbol, guarded by a stderr grep so it fails for the right reason. Test-only; all 5 pins hold; 442 to 443.
Extends the ww test --sep run-exit gate with 3 rows on both stages: userman and badsig REJECT (exit!=0; explicit main under -T, bad @test signature), collide_run ACCEPT exit==0 (user fn run + lib test.run coexist post #80/#84). Reuses the 910/997 fixtures unchanged (SSoT; they outlive the .c gates the E3 flip retires). Under sep a regressed run/test.run collision is a hard w6l duplicate-symbol link error so exit==0 is non-vacuous; collide_run also pins cs==ww byte-id (1 TEXT main, 1 CALL test.run) for the rule-10 stage-symmetry the combined coexist_run carried. Test-only; all 5 binary pins hold.
The -T harness synthesized `use test;` after name-binding, so the lib/test runner run keyed the bare scope and collided with a user-defined bare fn run — a spurious "duplicate fn run" reject (the E1 tolerance seam). Prepending the synth use before binding keys the runner as test.run in the test module namespace, distinct from the user bare run; the two coexist. Hare-faithful: the runner is its own test module (ref/hare/test/+test.ha:97). Inverts attest_userrun.ww from the #23-mandated reject to a coexist fixture; gate asserts exactly 1 TEXT run + 1 TEXT test.run on the -T asm (distinct symbols, not a dead-dup). Closes#80.
A package-less primary's bare fn (module="") whose leaf collided with an
imported module's same-leaf exported fn was mis-mangled to the imported
qualified name (a user `fn run` emitted as `test.run`), producing a
dead-duplicate symbol the linker silently shadowed -- a #263-class silent
miscompile, gate-blind and symmetric across both stages. cgen now registers
bare-module fns and resolves a bare-ident reference to its own bare leaf:
mod_lookup_for_fn prefers the bare entry when the call carries no module
hint and skips bare entries when it does, so the moduled-caller path stays
byte-identical. The moduled `main` entry carve-out is an orthogonal rule
(the linker entry is force-bared) and is retained. Prereq for the @test
user-`run` coexist (#80). The bare non-fn (let/def/type) sibling is the
same class but hint-less; deferred as #85, noted at the retained skip.
Rewrite `make bootstrap` to the sep build path: sep-build w6c, orchestrated
by the cstage ww driver chaining the frontend via WW_W6C (stage2 w6c_ww ->
ww2; stage3 ww2 -> ww3; stage4 ww3 -> ww4; cmp ww2 == ww3 == ww4). wwdump is
unchainable under sep (no -o/-I, writes asm to stdout), so the chainable w6c
is the bootstrapped target (test 994 pins w6c == wwdump -c).
Honest boundary, no overclaim: cstage ww runs the sep glue only
(dep-discovery / per-package w6c -c / archive / link) and is NOT part of the
fixed point; the fixed point proves the wwstage compiler (w6c, genuinely
chained) plus assembler (w6a_ww) and linker (w6l_ww) reach a byte-stable
fixed point. The wwstage driver ww_ww is not yet in the loop (it can't swap
its frontend) -- that closer is #66.
Makefile only; all five wwstage binary pins unchanged. bootstrap is a
standalone dev target, not part of make test.
Make `ww test --sep` work the Hare +test way: the -T synth test-main emits a
qualified test.run, and the test package is injected as an ordinary
separately-compiled dependency instead of splicing lib/test source into a
flat unit. Additive — combined stays the default and 910/997 are untouched
(their migration is M4 E2).
- compiler synth (both stages): the -T main emits N_DOT test.run plus a
synthetic N_USE "test"; cmd/wcc/check.c + selfhost/cmd/wcc/check.ww.
- driver (both stages): build_one_sep gains is_test, injects the test package
as a root dep, and passes -T to the root; do_test --sep routes a single-file
test through the sep producer; cmd/ww/main.c + selfhost/cmd/ww/main.ww.
- 989_septest_run gate: ww test --sep on both stages, run-exit + cs==ww
byte-id of the sep .s, non-vacuous.
The synth's test.run is left ty_err by the checker in both regimes (lib/test's
run is scope-keyed under "" not "test"; cgen emits the correct CALL via run's
//ww:module test directive) — wwstage tolerates it like cstage (rule-10). The
genuine fix, module-keying run under sep so the call type-resolves, is #80.
w6c_ww/wwdump_ww/ww_ww move (their embedded source changed); w6a_ww/w6l_ww and
the combined codegen output are unchanged.
Before M4 deletes the combined.ww embedding (byte-id-moving), capture a
golden record of the combined codegen oracle while both build paths still
exist. The manifest pins the md5 of the combined-path .s emitted by the
pinned w6c_ww for the five bootstrap targets, plus the five combined-epoch
binary md5s, with a regen procedure. Re-derivable from the m4-combined-epoch
tag, so a future sep-path regression stays drift-checkable against
last-known-good combined output.
Data-only; the five wwstage binary pins are unchanged.
Make the sep-compile == combined-compile output equivalence a standing
regression test — the last criterion before M4 can delete the combined.ww
embedding model.
989_c6soak_run (phase-1): Tier-A/B + same-leaf collision soak. Two real
roots (utf8: encoding.utf8 dotted-path + strings; collide: strings.contains
+ bytes.contains) built both ways with isolated cold WW_PKGCACHE; asserts
combined-run-exit == sep-run-exit on both drivers, cs==ww per-pkg .s/.wwi,
and two distinct collision labels.
994_w6c_ww (phase-2): Tier-C folded into the existing w6c byte-id oracle.
A sep-built w6c emits byte-identical .s to a combined-built w6c on all five
bootstrap targets (w6c/wwdump/w6a/w6l/ww .combined.ww); .wwi is in the
compare set. The tool binaries themselves differ by a benign constant
link-layout delta — the oracle is tool OUTPUT, not tool-binary identity.
Test-only; the five wwstage binary pins are unchanged.
After the frontend consolidated into one syntax package (#74), wcc still referenced syntax symbols unqualified — residue of the old flat combined namespace, where bare refs resolved by accident. Under separate compilation Hare and Go both require the package qualifier, so those bare refs would not sep-resolve.
Qualify every wcc reference to a syntax type, function, or enum member as syntax.X across the seven syntax-importing files. Resolution-only: the resolved symbol and emitted code are unchanged, so the two combined.ww regenerate textually but all five _ww binaries hold byte-for-byte. The struct-literal sites resolve via #76. This makes w6c fully separate-compilable.
The parser folded a qualified type pkg.Type into two different node shapes by position: declaration position collapsed it into one N_TNAME (resolved via the strrchr-leaf path), but literal position left an N_DOT chain that the struct-literal typeref handoff had no resolver arm for, so pkg.Type{...} rejected with "expected type expression".
Normalize the literal-position N_DOT chain into the same source-order N_TNAME the declaration path emits, reusing the existing resolver; no new checker arm. cstage flattens at parseprimary struct-lit handoff; wwstage (no token peek) folds dots in parsepostfix and normalizes there, guarding numeric tuple components and staying in the postfix loop so trailing ops still chain. Both stages emit identical N_STRUCTLIT(N_TNAME). Prereq for qualifying wcc syntax refs (#75).
After the frontend consolidated into one syntax package, the wcc backend
imports syntax and calls its exported fns — whose signatures reference
types that were unexported. Producing syntax's .wwi interface re-triggered
check_exported_type ("exported declaration references unexported type"):
the residual of BUG-A at the one surviving syntax->wcc boundary. Export
the 16 types that appear in syntax's wcc-facing public surface (directly
in an exported signature, or via a recursively-referenced exported struct
field): nkind, node, lex, tok, tkind, parser, scope, sym, skind, tinfo,
tykind, tfield, tparam, ttupleelem, tctx, tinfocacheent. The set is
minimal (unexporting any one re-breaks the producer) and complete; pos
stays internal. Pure source change — exporting a type emits no code, so
the bootstrap binaries are byte-identical (verified against a clean base
build); only syntax's .wwi gains the type decls.
Post-frontend-reorg residual (#74). syntax now sep-produces clean both
stages. The separate concern of wcc's currently-unqualified refs to
syntax symbols (#75) is a distinct follow-up. Gate 989_syntaxexport_run.
The ww compiler frontend was split across packages lex (lex+tok), ww
(ast+sym+typ), and parse — mirroring Hare's ref/hare/hare/{ast,lex,parse}.
That split's only payoff is third-party reuse, which ww has zero of: the
frontend is consumed by exactly one client, the wcc backend. The split's
cost is a wide cross-package export surface — every fn over a sibling
package's type must export it, and under separate compilation that
re-triggers check_exported_type, plus a phantom `import tok;` (tok lives
in package lex). Consolidate into ONE package lib/ww/syntax/, modelled on
Go's cmd/compile/internal/syntax. The 9 files move in (package syntax);
the intra-frontend mutual references become same-package; wcc and the
tool mains import syntax. No cstage C change (the C frontend mangles from
the source package clause). Internal data shapes (AST kinds, token model,
lexer/parser state) still mirror ref/hare/hare per rule 6/12 — only the
module decomposition collapses; the stdlib is untouched.
USER-approved (#74); spec .ai/rob-frontend-reorg.md (drew2 fidelity-
confirmed). Rule-6 carve-out documented in CLAUDE.md. Dissolves the tok
phantom import; collapses the intra-frontend export sprawl. Byte-id
rebaseline (lex.X/parse.X/ww.X -> syntax.X); cs==ww held. The residual
syntax->wcc export surface (10 types) + the unqualified-ref question are
separate follow-ups (#72/#75).
The .wwi (separate-compile interface) producer could not serialize an
exported def whose initializer is a struct/array literal (N_STRUCTLIT/
N_ARRLIT) — `export def f64info: floatinfo = floatinfo{...}` aborted with
"unhandled const-expr node kind 15". Such a def is a DATA-global per the
#52 model, so its value lives once in the defining package's .o; the
interface needs only the type+symbol. Emit a value-less prototype
`export def X: T;` for aggregate-initializer defs; scalar fold-eligible
defs keep their value (the importer const-folds those). The parser gains
an optional-init arm so the importer can parse the prototype — value-less
`def X: T;` is now legal in any source, symmetric with the existing
bodyless-fn prototype `fn f();` (USER ruling: unconditional; a value-less
def with no defining .o is a loud undefined-symbol error at link, never
silent). Both stages; producer + parser fold into one commit (the
producer's output is unparseable without the parser arm).
M3-tail commit-6 prerequisite #2 (surfaced by the c6 scout). The
aggregate-def-field const-fold boundary is documented inline (#71). Gate
989_sepstructdef_run proves struct+array exported defs sep-build, link,
and run via external DATA refs, cs==ww, with a value-less .wwi.
The --sep producer compiled the ROOT build-target with the .wwi-producer
-I flag, so check_exported_type ran on the root and rejected a real-tool
root's legitimate `export fn f(a: *t)` over an unexported local type t
(the root is terminal — its interface is never imported, and its .wwi is
never consumed). The combined build never passes -I, so it built fine.
For pi==root, invoke w6c with -c -o only, no -I. Both stages (the wwstage
twin builds the shorter root argv). Gates that asserted __root.wwi exists
encoded the buggy behavior; updated to assert __root.s (the consumed
product) while deps' .wwi byte-id is retained.
M3-tail commit-6 prerequisite. Gate 989_seproot_export_run reproduces the
export-fn-over-unexported-type root + proves it sep-builds, with a
non-vacuity leg that the forced -I path still rejects.
A separately-compiled package's primary body was emitted under a bare
`//ww:module-reset`, so its own `package <leaf>;` clause set curmod to
the leaf (e.g. utf8) while the importer spliced the .wwi under the full
`//ww:module encoding.utf8` — definer mangled `utf8.X`, importer wanted
`encoding.utf8.X`, unresolved. Thread the dotted path through the
directive: `//ww:module-reset <path>` sets curmod to the dotted path
(imported stays 0, so the root `fn main` stays bare per #32), and the
body's package clause is demoted to a leaf==last-component assertion
instead of overwriting curmod. Aligns sep-build to the M1 path-mangle
model; only the SEP emitter changes (the combined build_one arm is
untouched, so all combined byte-id gates hold). Both stages mirrored.
Commit-6 broad-soak prerequisite. Gate 989_sepdotpath_run sep-builds a
2-level dotted package and proves definer==importer qualification +
single-component non-vacuity, cs==ww.
Per-package build cache for `ww build --sep`: before recompiling a
package, recompute a plain-text key manifest (md5sum-hex lines of the
package sources, each direct dep's .wwi, the w6c and w6a binaries, plus
the compile flags) and reuse the cached .o/.wwi on a byte-for-byte key
hit. Both stages shell the same host md5sum (no ww-side md5) so the
non-compiler key lines are byte-identical cstage==wwstage; the compiler
md5 lines differ per stage BY DESIGN, giving each stage its own cache
namespace so a hit can never reuse the other stage's .o and mask a
cs!=ww codegen divergence. Cached outputs (.o/.wwi) stay byte-identical
across stages. Root package never cached; cache lives under $(OUT)
(gitignored, wiped by make clean). Both stages mirrored.
Dev-only convenience, off every bootstrap/byte-id gate. Gate
989_pkgcache_run (cold) proves miss-compiles / hit-skips-byte-id /
independent key-bust per input class with non-vacuity rehit.
Per-package .a archives are written by a self-hosted deterministic ar
writer (zeroed mtime/uid/gid, fixed mode 100644, stable member order)
so cstage and wwstage emit byte-identical archives. The linker
force-loads the root .o positionally and pulls deps from .a; a
post-pull PASS-3 over unloaded members reports duplicate symbols
through the archive (#31). Both stages mirrored (cmd/ + selfhost/).
USER-ruled D2 (self-hosted ar writer); pike P1/P2 link model. Gate
989_separchive_run proves cs.a==ww.a byte-identity, 3x-determinism,
link-consumes-.a (exit 7), and the masked-dup-through-.a loud fire.
Promote commit-3's tri-color topo bail into a loud dep_cycle error that names the full import cycle, byte-identical both stages (cmd/ww/main.c + selfhost/cmd/ww/main.ww, deps.ha:243 parity).
Add a non-vacuous negative gate (989_sepcycle_dup) proving the pre-existing w6l duplicate-symbol reject fires loud + non-zero on a cross-package collision; no new linker code.
#58(b)(c) link-arg parity deferred (system()-string vs procrun()-argv is structurally un-unifiable in this scope); #61 filed for the byte-id-blind w6l_ww dup-message divergence.
build_one_sep (both stages + ww/main.combined.ww regen): discover_deps
(transitive directory-package set, dotted-path identity), tri-color
reverse-topo (cycle bails; loud reject is commit 4), the transitive
producer loop (one w6c -c -I pass per package, dep-first, each both
consumer and producer of its .wwi), and a flat w6l of the .o set.
combined.ww stays the DEFAULT live path; --sep is additive.
Every dep is tagged by its full dotted import path on prepend
(//ww:module <path>), so the definer's qualified symbol (#53) == the
consumer's qualified reference (#40) and the sep .o set links. The
prepend is the TRANSITIVE closure (lead-ratified, superseding
rob-c3-spec §1.3 direct-deps): a dep's interface can name a transitive
dep's type (os exposes time.instant), so the consuming unit needs the
whole closure for resolution — direct-deps-only does not type-check.
Consistent with the current flat-unit transitive-namespace model (the
visibility tighten is #45, post-M4).
Gate 989_sepbuild_run drives ww + ww_ww --sep on the real chain
root->os->{rt,time}: build+run (exit 7) + cs==ww per-pkg .s/.wwi/.unit
+ final binary + transitive-topo discovery + keystone bodies==.wwi
(os,root) through the real driver. Cold scratch; -o-redirected.
Under M1 mangling, EXPORTED non-fn decls (let/def/type) skipped path-
qualification and emitted a BARE symbol (`types.I64_MAX` -> `I64_MAX`).
Under separate compilation two packages exporting the same data leaf
would then collide at w6l. Masked in-tree only because no two packages
export the same non-fn leaf.
§7-A (USER-locked, harec's model): path-qualify EVERY exported decl
(fn AND data) at the single mangle choke-point — mod_collect /
collectmods. Retire the `!isfn && d->export` (cstage) and `exported==0`
(wwstage) skips: every decl with a module now mangles `<mod>.<name>`.
The ONLY bare symbols left are @symbol FFI overrides (ffi_resolve at
emit) and the ROOT unit's `main` — both already carved out before the
map insert.
With exported decls in the map the exact-(name,hint)-or-bare value
dance is dead — its sole purpose was the bare-exported case. Delete
mod_lookup_value / mod_mangle_value / mahint (cstage) and
modlookupvalue / emitsymnamehint (wwstage); the value-global sites now
route through the same hint-aware-with-fallback lookup as fns
(mod_mangle_fn/mafn, emitfnname). Net negative LOC in the mangler.
Transparent rename on the live combined path: ref and def move in
lockstep, so cs==ww byte-id holds and the self-host still builds + runs
(fixed-point/995). Byte-id REBASELINE — all 5 ww binaries shift. The
w6c/wwdump combined.ww embed wcc cgen and are regenerated.
Strlit labels were emitted as `_S_<n>` from a global counter with no
per-unit prefix (cgen.c intern_strlit + wwstage internstrlit twin).
Under separate compilation two str-bearing packages both emit `_S_0`..
-> w6l link collision.
Prefix the label with the owning package PATH (`<module>._S_<n>`,
matching mklabel's spelling). The prefix is c->cur_mod, set per-fn by
cgfn and now per-decl by let_pre_intern (save/restore so the later
emit passes, which read cur_mod for fn-ptr relocs, are unaffected).
Pure function of the module path — NOT a build-nonce — so the
self-host fixed-point holds across ww2/ww3/ww4. Both stages, symmetric.
Transparent rename on the live combined path: the label is interned
once and shared by every reference, so ref and def move in lockstep.
cs==ww byte-id holds; the w6c/wwdump combined.ww embed wcc/cgen.ww and
are regenerated.
746_strdef_inline: the strdef-inline sentinel pinned the bare
`LEAQ\t_S_` shape; update to the module-prefixed form (alpha._S_ for
the in-module def, beta._S_ for the use-site-interned cross-module
inline).
989_m3sep_run: add the #49 LINK leg. The str sub-fixture (sleaf+smid)
was keystone-only — never linked — precisely because the global
counter made both emit `_S_0`. With the prefix, compile both `-c`
separately, link (w6l) + run (sroot reads a distinguishing byte through
each string's .ptr, so a collided label would corrupt the exit), both
stages + cs==ww final exe.
lib/types limit consts were bare `def`s, so the .wwi (sep-compile's
interface) correctly omitted them while the flat combined.ww let a
cross-package user (lib/strings splitn → types.I32_MAX) reach the
private def — sep-compile then failed (wwstage `asserttyped: dot
'I32_MAX'`; cstage undefined-ref). Hare exports types::I32_MAX and the
whole limit family (ref/hare/types/limits.ha, arch+x86_64.ha); ww not
exporting them was the divergence.
export the 24 existing limit defs ({I,U}{8,16,32,64}_{MIN,MAX},
INT/UINT/SIZE/UINTPTR_{MIN,MAX}) and the existing RUNE_MIN, and add
exported RUNE_MAX. ww's derived machine-word int/uint/size/uintptr
VALUES are kept verbatim (user-ratified 64-bit-int divergence); fidelity
here is the NAME SET + export-visibility, not the values. RUNE_MAX is
written `0x10ffff: rune` — same codepoint as Hare's '\U0010ffff', forced
because ww's lexer has no \u/\U escape (#50).
Exporting the consts made `w6c -I` walk them and fatal on RUNE_MIN
('\0'): the .wwi const-expr unparser had no N_RUNELIT arm. Add one,
both stages (wwi_rune / wwirune), rendering a \xHH-escaped rune literal
(>0xFF fails loud, #50). Const casts need no arm — the checker folds
them to integer literals before the producer runs. 989_m2wwi_run gains
a types.wwi gate (byte-id + re-parse + asserts export def I32_MAX and
RUNE_MAX reach the interface). byte-id-neutral: a def emits no symbol.
The M2 .wwi producer rendered an exported fn carrying @symbol("...")
as a bare prototype, dropping the FFI link-symbol binding. A
sep-compiled consumer reading the .wwi then emitted `CALL malloc`
instead of `CALL rt_malloc` for `@symbol("rt_malloc") export fn
malloc`, breaking the bodies-vs-.wwi byte-id and the link. Affects
every package whose closure reaches rt/os.
Emit codegen/link-relevant attributes through a single named
predicate (wwi_attr_relevant / wwiattrrelevant), today true iff the
name is "symbol" — the only such attribute that exists. @align/@offset
are NOT field attributes in ww (the parser parses no field attrs, the
N_TFIELD node has no attr slot); the predicate is named for the class
so they slot in if ww ever grows them (#51). attr is assigned at
exactly one site per stage (parse.c:1351 / decl.ww:168), both inside
parsefn, so only N_FNDECL carries attrs and the fn-decl render path
covers the whole class.
Both stages, byte-identical (rule 10). 989_m2wwi_run synth gate gains
an @symbol fn + a content assertion that the .wwi carries it verbatim.
New `w6c -c` (both stages): separate-compile / primary-only codegen.
Emit code+DATA ONLY for a package's own (imported==0) decls; treat every
`.wwi`-sourced (imported==1) dep decl as an external. Pure addition behind
the flag — combined.ww stays the LIVE path, `-c` is off on every existing
invocation, so the 990-997 byte-id gates + all prior tests are unperturbed.
The keystone (rob): a `.wwi` is body-less/init-less prototype source, and
cgen already skips body-less fns as externs, so dep fns/types/defs emit
NOTHING for free. The single genuinely-new guard is an imported value-
global (`export let`): its DATAW would DUPLICATE the dep's own definition
(link collision), so it is skipped. The `imported==0` gate is applied at
all top-level emit sites for uniformity (close-by-construction): the fn
loop, emit_lets/emitletdataw, emit_defs/emitdefconstants, and
let_pre_intern/letpreintern — that last one because an imported dep's body
initializer interns strlits while its rhs-stripped `.wwi` does not, which
would shift the _S_ sequence; gating it keeps the strlit table a pure
function of the package's own decls. EXACTLY symmetric with M2's producer
`imported==0` filter — same predicate both directions.
Driver `--sep` build_one_sep + per-package archives + multi-.a link +
cache + BROAD real-target dual-path soak are M3-tail (#46, rob ruling B):
M3-core ships the codegen spine + a self-contained gate that proves all
codegen correctness without a production driver.
Gate 989_m3sep_run: a synth leaf->mid->root fixture carrying all four
cross-boundary fact-classes (fn signature, struct LAYOUT, `def` const
VALUE, `export let` value-global). Per package, holding `-c` constant:
`w6c -c` of (deps-as-bodies) == (deps-as-.wwi) byte-for-byte (the .wwi
conveys exactly the dep facts P's codegen needs); cs==ww at the .s AND
final-exe level (rule 10); sep-path determinism; the value-global guard
(imported origin_tag never re-emits DATAW); and behavioral identity (the
linked program's exit code is the real cross-boundary computation). COLD:
.wwi materialized fresh every run (no warm cache).
combined.ww regen'd for wwdump + w6c (both embed cgen.ww); diff is exactly
the four guards + the flag wiring, nothing spurious.
New `w6c -I <out.wwi>` flag (both stages) writes a re-parseable
ww-prototype rendering of a package's EXPORTED surface. M2 dead-code:
nothing consumes .wwi yet (combined.ww stays the live path); the flag is
off on every existing invocation, so the 990-997 byte-id gates and all
prior tests are unperturbed.
The unparse walks the AST type-expr subtree (N_T* nodes), not the
tinfo Type* (which collapses nominal pkg.Name identity). Deterministic
output: package line, byte-sorted imports, byte-sorted decls — a pure
function of the exported API. cmd/w6c/wwi.c + selfhost/cmd/wcc/wwi.ww
emit byte-identical .wwi (new cross-stage byte-id substrate, rule 10).
check_exported_type (drew) rides the producer entry, flag-gated: an
exported signature naming a non-exported nominal is loud-rejected before
any byte is written, identically on both stages. Ports harec
check.c:4092-4168, recursing the type-AST and gating on the resolved
SK_TYPE sym's decl export flag (Sym.exported is vestigial in both
stages; the predeclared synthetic `nomem` decl carries no source
position and is treated as a builtin leaf — cstage parity).
Two wwstage checker AST-mutations are normalized to cstage's pristine
view for byte-id: the N_TPARAM tuple-element wrapper (unwrapped) and the
variadic `T...`→`[]T` param desugar (peeled).
Gate 989_m2wwi_run: ascii/strings/getopt each produce a .wwi that
re-parses (wwdump -a) and is cs==ww byte-identical; a private-type-leak
fixture is rejected identically by both stages (non-vacuous check).
use_hint/usehint were unit-global first-leaf-match: two directory-
packages exporting the same fn leaf, each imported by a different module
aliasing the same bareword, mis-routed every qualified call to whichever
use was collected first. Identically wrong on both stages (byte-id-green
#263-class). Key the hint on (owner-module, alias) and prefer cur_mod,
mirroring the checker's use_path curmod-preference (55f54fb).
989_m1usehint_run: two same-leaf pick() across a.math/b.math, each
module's call routes to its own import (111/222) + cs.s==ww.s.
Switch symbol mangling from the import leaf clause to the full dotted import path for directory packages; single-file imports keep package-clause mangling (isdir-gate: imported<=>directory-import). The root build unit's fn main stays bare, every other top-level decl mangles, closing #31's duplicate-main hazard by construction (#32). Both stages, byte-identical.
Single commit, not split: the bare rename (f244af3) is red on its own because it unmasks cross-module resolution gaps that do not reproduce pre-M1, so the fixes are intrinsic to making the rename correct. Included: wwstage fnret/fnparamslookupmod map import alias->path (#199b cross-module union-variant scrutinee resolved the wrong fn's union); cstage use_path prefers the referencing module's import for an ambiguous leaf alias (sha256 crypto.math vs strconv math). Tests table-driven: 989_m1mangle_run/_sym, 989_m1union_run (gate-visible per-arm exit codes + cs==ww byte-id).
Dotted import paths lengthen mangled symbols; the assembler's
fixed symbol/TEXT/DATA name buffers must not silently truncate a
mangled name into a collision. Byte-id-neutral: no current symbol
reaches 63 chars. wwstage w6a already parses symbols via dynamic
dupstr, so no mirror cap exists there.
The sym carried an is_const flag (lib/ww/sym.ww) but wwstage never
set it at the let-install nor consumed it at assignment, so mutating
a `const` slipped through silently. Mirror cstage's two sites: set
is_const when n.op == TK_CONST at the local let-install (cmd/wcc/
check.c:2408) and reject an N_ASSIGN whose lhs ident resolves to an
is_const sym (cmd/wcc/check.c:1889-1896). cstage already rejected;
this aligns wwstage's w6c_ww UP. A bare `_` discard lvalue (empty
str) is skipped.
Regen w6c/wwdump combined.ww (checker embeds in both). Valid-program
codegen unchanged → cs==ww byte-id gate stays green.
wwstage's checkretassign short-circuited on a value-less `return;`
("skip flagging for now"), so `fn f() i32 = { return; }` built and
RET'd a garbage register. Mirror cstage cmd/wcc/check.c:2428-2439:
the no-value return has type void, then run isassignable(c.fnret,
void) — void→void and void→(T|void) accept, void→i32 is a confident
reject. cstage already rejected; this aligns wwstage's w6c_ww UP.
Regen w6c/wwdump combined.ww (checker embeds in both). Valid-program
codegen unchanged → cs==ww byte-id gate stays green.
wwstage's tuple-parse loop checked the RPAREN-break at the top, so
`(a,)` parsed as a 1-element N_TUPLE and reached cgen — a silent
wrong-accept. A trailing comma is legal only after >=2 elements.
Align the loop order to cstage cmd/wcc/parse.c:552-558 (parse each
element before the RPAREN-break); `(a,)` now errors at the next
parseexpr, `(a, b)` / `(a, b,)` are unchanged. cstage already
rejected; this brings wwstage's w6c_ww parser into agreement.
Regen w6c/wwdump combined.ww (parser embeds in both). Valid-program
codegen unchanged → cs==ww byte-id gate stays green.
