ww/PLAN.md

PLAN

A phased plan for ww. Each phase ends with a green make test and a tagged checkpoint. Don't start phase N+1 until phase N is green.

CLAUDE.md is the rulebook. This file is the schedule.

Phase 0 — Skeleton (C bootstrap scaffold)

Goal: a repo that builds, has a Makefile, and has a ww driver that prints its version. No compilation yet.

Deliverables:

• Makefile (POSIX) with all, test, clean, install targets.
• cmd/wwc/ C library skeleton (builds libwwc.a):
  • ww.h — central typedefs (Node, Sym, Type, Lex)
  • mem.c — bump arena allocator (no malloc in hot paths)
  • err.c — errorf, fatal, warn
  • (lex/parse/check stubs added in later phases)
• cmd/ww/ driver skeleton: argv parsing, version print.
• test/run — shell test harness.
• test/wwc/000_smoke.c — smoke test that asserts ww -V prints the version.

Per-target binaries (6c, 6a, 6l) are NOT created here. They ship in their own phases (4, 5, 6).

Exit criteria:

• make produces out/bin/ww and out/lib/libwwc.a.
• make test runs the smoke test and passes.

Phase 1 — Lexer (in libwwc.a)

Goal: tokenize ww source into a stream of Tok structs.

Deliverables:

• cmd/wwc/lex.c — hand-rolled DFA. Handles:
  • identifiers, keywords (fn, let, def, if, else, for, switch, case, return, use, type, struct, defer, break, continue, export, proc, chan, nil, true, false)
  • integer literals (dec, hex, oct, bin) with suffixes (u8, i32...)
  • float literals
  • rune literals 'x' and string literals "..." (UTF-8)
  • operators and punctuation, including @ for attributes (@symbol("name"), etc.)
  • explicit ; terminators — no automatic insertion (Hare rule)
  • // and /* */ comments
• cmd/wwc/tok.c — token names, debug printer.
• test/wwc/100_lex.c — table-driven: input → token sequence.
• test/lang/lex/*.ww — small files exercised via a tiny test harness that links against libwwc.a.

Exit criteria:

• A lextool test binary dumps a representative sample's token stream deterministically.
• All lex tests green.

Phase 2 — Parser & AST (in libwwc.a)

Goal: parse the full grammar into an AST. No types yet.

Deliverables:

• cmd/wwc/parse.c — recursive descent. Pratt expression parser for precedence. No yacc.
• cmd/wwc/ast.c — Node constructor helpers; printer.
• Grammar coverage:
  • use imports (paths use . not ::)
  • type declarations (struct, alias, fn type) with trailing =
  • def constants
  • let declarations (top-level and local)
  • fn definitions with = { ... }; body. No methods; receivers are just first arguments.
  • export visibility marker
  • all expressions, statements, control flow
  • defer
  • body-less fn declarations and @symbol("name") attribute for FFI imports
• test/wwc/200_parse.c — table-driven AST shape tests.
• test/lang/parse/*.ww — programs that should parse but not yet typecheck; harness only checks parser exits 0.

Exit criteria:

• 50+ parse tests green, drawing shapes from ref/hare/cmd/hare/main.ha.
• AST printer output is deterministic.

Phase 3 — Type checker (in libwwc.a)

Goal: resolve names, infer/check types, produce a typed AST.

Deliverables:

• cmd/wwc/sym.c — symbol table. Lexical scopes. Plan 9 style hash.
• cmd/wwc/type.c — type representation, equality, conversion rules.
• cmd/wwc/check.c — type checking pass over the AST. Errors carry source locations.
• Built-in types: all integer widths, bool, f32, f64, rune, str, void, pointers, slices []T, fixed arrays [N]T, function types.
• Slice semantics: { ptr, len, cap }, indexing bounds-checked at runtime (debug builds; release may elide via flag).
• test/wwc/300_check.c — table-driven: src → expected error or OK.
• test/lang/check/*.ww — both passing and failing cases.

Exit criteria:

• All built-in types correctly checked.
• Negative tests produce stable, useful diagnostics.

Phase 4 — 6c amd64 compiler

Goal: turn typed AST into Plan 9-style amd64 assembly text. The binary 6c is a Plan 9 cc analogue for amd64. There is no separate SSA IR file; the only on-disk intermediate is .s.

Deliverables:

• cmd/6c/ (links against libwwc.a):
  • 6.out.h — amd64 opcode enum, register names, addressing modes. Mirror ref/plan9front/sys/src/cmd/6c/ shape.
  • gc.h — Prog, Adr, scratch regs, stack layout.
  • cgen.c — typed AST → Prog list (walking, not SSA).
  • txt.c — Prog list → textual Plan 9 amd64 asm.
  • swt.c, peep.c, reg.c — switch lowering, peephole pass, naive register allocation (linear scan or spill-everywhere first; tighten later).
  • mkfile — Plan 9 mkfile next to the Makefile entries.
• Output suffix .s. Plan 9 amd64 asm syntax (not GAS).
• test/lang/6c/*.ww — golden tests: src → expected .s text.

Exit criteria:

• 6c hello.ww produces hello.s.
• 20+ programs round-trip identically across runs.
• The output is consumable by phase 5's 6a.

Phase 5 — 6a amd64 assembler

Goal: assemble Plan 9 amd64 asm into ELF object files. ELF (not Plan 9 a.out) so we can interop with C archives in phase 8.

Deliverables:

• cmd/6a/:
  • lex.c — tokenize Plan 9 asm.
  • parse.c — hand-rolled grammar (Plan 9 uses yacc; we go hand-rolled to keep rule #6 honest).
  • asm.c — encode amd64 instructions (REX, ModR/M, SIB).
  • obj.c — emit ELF64 relocatable objects.
  • mkfile
• Pseudoregisters supported: SP, FP, SB (static base) per Plan 9 convention, on top of AX, BX, ..., R8-R15.
• test/wwc/500_asm.c — round-trip known asm to known bytes.

Exit criteria:

• 6a hello.s -o hello.o produces a valid ELF amd64 object.
• objdump -d hello.o matches expectations across a 20-program corpus.

Phase 6 — 6l amd64 linker

Goal: link ELF objects into a static ELF executable.

Deliverables:

• cmd/6l/:
  • obj.c — load ELF .o files and .a archives.
  • sym.c — global symbol table; resolution.
  • pass.c — section layout, relocation application.
  • out.c — emit static ELF executable.
  • mkfile
• Static linking only. No dynamic loader. No PT_INTERP segment.
• lib/libwwrt.a (built later in phase 7) is auto-included.

Exit criteria:

• 6l -o hello hello.o libwwrt.a produces a static ELF binary.
• ldd hello reports "not a dynamic executable".
• The binary exits 0 with the program's intended semantics.

Phase 7 — Runtime + driver wiring

Goal: a tiny static runtime, plus the ww driver wired to call 6c → 6a → 6l end to end.

Deliverables:

• rt/start.s — entry, sets up argc/argv, calls main.
• rt/syscall.s — Linux syscall trampoline.
• rt/panic.c — panic, abort, stack unwind for defer.
• rt/mem.c — small page allocator built on mmap. Users get raw pages; higher-level allocators ship in lib/.
• rt/slice.c — slice helpers used by codegen (bounds_check, slice_grow for explicit user calls).
• Build artifact: out/lib/libwwrt.a.
• cmd/ww/ driver: reads ww build foo.ww, runs 6c then 6a then 6l, links libwwrt.a into the output.

Exit criteria:

• A ww program with no libc dependency runs.
• Adding use os pulls in syscall wrappers; still static.

Phase 8 — C FFI + static linking C libs

Goal: bind to C libraries; produce static binaries containing them.

Deliverables:

• Body-less fn declarations + @symbol("name") attribute parser and checker rules already in phases 2/3 — finalize codegen.
• amd64 SysV ABI: formalize struct-by-value and varargs in 6c.
• 6l learns to slurp static archives from a search path.
• lib/c/libc/ — minimal libc bindings (malloc, free, printf, read, write, open, close).
• lib/c/tls/ — bindings to libtls or BearSSL (decide; BearSSL is more aligned with our static-linking story).
• lib/c/crypto/ — bindings to libcrypto subset (sha256, aes, hmac).
• lib/c/curses/ — ncurses bindings (initscr, getch, mvprintw, ...).
• ww driver: ww build -lcrypto -ltls -lncurses resolves these via a pkg-config-style probe, hands .a paths to 6l.

Exit criteria:

• examples/tlsclient.ww connects to https://example.org and prints the response. Statically linked. No .so deps in ldd.
• examples/menu.ww runs an ncurses TUI.

Phase 9 — Standard library (Hare-shaped)

Goal: a usable stdlib. Each module ships with tests.

Order of build (each is its own milestone):

1. types limits, int helpers
2. bytes slice ops on []u8
3. strings ops on str
4. io stream struct (function-pointer vtable, no interface); eof sentinel value
5. bufio buffered reader/writer (Plan 9 bio analogue)
6. fmt printf-family writing through io.stream
7. os argv, env, stdin/out/err, file ops
8. errors error = str and sentinel constants
9. strconv itoa, atoi, parse float
10. sort sort.slice, binary search
11. path filepath ops
12. encoding/utf8, encoding/hex, encoding/base64
13. hash/crc32, hash/fnv, hash/sha256 (pure ww)
14. time monotonic, wall clock, sleep
15. net dial/listen TCP, UDP

Each module follows the Hare layout: one directory, multiple files, +linux.ww, +test.ww overlays where useful.

Exit criteria per module:

• Module compiles standalone with ww build ./lib/<mod>.
• Module tests pass: ww test ./lib/<mod>.
• Public API documented in README inside the module dir.

Phase 10 — Self-host

Goal: rewrite libwwc, 6c, 6a, 6l, and ww in ww. Drop the C bootstrap.

Steps:

1. Translate ww.h and Node/Sym/Type/Prog/Adr to ww structs.
2. Port lex.c → lex.ww. Diff token streams against C output.
3. Port parse.c → parse.ww. Diff ASTs.
4. Port check.c. Diff typed ASTs.
5. Port 6c (cgen, txt, peep, reg, swt). Diff .s output byte for byte.
6. Port 6a and 6l. Diff resulting .o and final binaries.
7. Three-stage bootstrap: Cstage (gcc-built tools) → ww1 → ww2 → ww3, with cmp ww2 ww3 OK for each tool.

Exit criteria:

• make bootstrap runs the three-stage build green.
• The C cmd/wwc/, cmd/6c/, cmd/6a/, cmd/6l/ C trees are deleted in this phase's final commit.

Status (2026-05)

Steps 1–6 done. The four ww-side tools live in selfhost/cmd/:

• wwc/ — ww-native lex/parse/check/cgen
• 6a/ — ww-native amd64 assembler
• 6l/ — ww-native amd64 linker
• ww/ — ww-native driver

Step 7 reaches its fixed point: make bootstrap produces ww1 → ww2 → ww3 with cmp ww2 ww3 byte-identical. Tests 990–993 confirm each selfhost tool is byte-identical to its Cstage counterpart.

Exit criterion 1 (bootstrap green) is met. Exit criterion 2 (delete the C trees) is deferred to v1.0: removing cmd/wwc/, cmd/6c/, cmd/6a/, cmd/6l/, cmd/ww/ is a one-way door. Until the compiler stops churning we keep Cstage as the canonical fresh-checkout entry point and treat selfhost/cmd/* as the path forward. At v1.0, the plan is to ship a checked-in stage-0 binary archive (bootstrap/<arch>/{ww,6c,6a,6l}), delete the C trees, and promote selfhost/cmd/* to cmd/*.

See BOOTSTRAP.md for the build flow and make cstage/make wwstage targets.

Phase 11 — CSP

Goal: channels and lightweight processes, without GC.

Deliverables:

• Runtime scheduler: cooperative, M:N, with stack-per-proc (start with fixed-size stacks; segmented stacks are a research item).
• proc f(args) statement: spawns a process. Fire-and-forget by default. To keep a handle, use the stdlib spawner (sched.spawn(f, args) returns a procval). No GC means proc lifetimes must be explicit.
• chan T type. c <- v send; let v = <-c receive; select { ... }.
• Channel ownership: closing a channel is a single-writer responsibility (Go rule). Buffered and unbuffered both supported.
• sync package: mutex, waitgroup, once.

Concurrency-safety constraints (because no GC):

• Sending a pointer over a channel transfers ownership unless the type is explicitly marked shared. The checker enforces this.
• No data races by construction in the safe subset; unsafe pkg exists for power users.

Exit criteria:

• Classic CSP examples (ping-pong, prime sieve, fan-in/fan-out) compile and run.
• Race detector under -race (later sub-phase) catches a planted race in tests.

Test cadence

Every phase has its own test directory (test/wwc/<phase>_*.c, test/lang/<phase-area>/*.ww). make test runs them all in order of phase. CI is just make test in a clean tree.

Versioning checkpoints

Tag the tree at each phase boundary:

• v0.0 end of phase 0
• v0.1 end of phase 1
• ...
• v0.10 end of phase 10 (self-hosted)
• v1.0 end of phase 11 (CSP merged, ABI declared stable enough to start caring)

Risks and mitigations

• Prog/Adr design churn. Mitigate by reading Plan 9 cc and per-target compilers before we write our own. Don't invent until you've copied.
• Writing a linker is hard. ELF is documented, static linking is the small subset. Read ref/plan9front/sys/src/cmd/8l/ and the ELF spec before writing 6l. Start with hello-world and grow.
• Static linking against C libs. OpenSSL is a pain to build static; BearSSL or LibreTLS are more pleasant. Pick early.
• Self-host bootstrap divergence. Diff outputs at every stage, not just the final binary. Token streams, ASTs, and .s text must match.
• CSP without GC. Channel-of-pointer ownership rules are the hard part. Prototype the checker with unsafe-allowed first; tighten later.

Non-goals (restated)

• No package manager.
• No generics, no interfaces, no tagged unions, no closures, no lambdas.
• No async/await.
• No map.
• No GC.
• No LLVM, no QBE, no external compiler infrastructure. The toolchain is cc/8c/8a/8l-shaped (Plan 9), per target.