ww/PLAN.md

# PLAN

A phased plan for `ww`. Each phase ends with a green `make test` and
a tagged checkpoint. Don't start phase N+1 until phase N is green.

CLAUDE.md is the rulebook. This file is the schedule.

## Phase 0 — Skeleton (C bootstrap scaffold)

Goal: a repo that builds, has a Makefile, and has a `ww` driver
that prints its version. No compilation yet.

Deliverables:

- `Makefile` (POSIX) with `all`, `test`, `clean`, `install` targets.
- `cmd/wcc/` C library skeleton (builds `libwcc.a`):
  - `ww.h`     — central typedefs (`Node`, `Sym`, `Type`, `Lex`)
  - `mem.c`    — bump arena allocator (no `malloc` in hot paths)
  - `err.c`    — `errorf`, `fatal`, `warn`
  - (lex/parse/check stubs added in later phases)
- `cmd/ww/` driver skeleton: argv parsing, version print.
- `test/run` — shell test harness.
- `test/wcc/000_smoke.c` — smoke test that asserts `ww -V` prints
  the version.

Per-target binaries (`w6c`, `w6a`, `w6l`) are NOT created here. They
ship in their own phases (4, 5, 6).

Exit criteria:

- `make` produces `out/bin/ww` and `out/lib/libwcc.a`.
- `make test` runs the smoke test and passes.

## Phase 1 — Lexer (in `libwcc.a`)

Goal: tokenize ww source into a stream of `Tok` structs.

Deliverables:

- `cmd/wcc/lex.c` — hand-rolled DFA. Handles:
  - identifiers, keywords (`fn`, `let`, `def`, `if`, `else`, `for`,
    `switch`, `case`, `return`, `use`, `type`, `struct`, `defer`,
    `break`, `continue`, `export`, `proc`, `chan`, `nil`, `true`,
    `false`)
  - integer literals (dec, hex, oct, bin) with suffixes (`u8`, `i32`...)
  - float literals
  - rune literals `'x'` and string literals `"..."` (UTF-8)
  - operators and punctuation, including `@` for attributes
    (`@symbol("name")`, etc.)
  - explicit `;` terminators — no automatic insertion (Hare rule)
  - `//` and `/* */` comments
- `cmd/wcc/tok.c` — token names, debug printer.
- `test/wcc/100_lex.c` — table-driven: input → token sequence.
- `test/lang/lex/*.ww` — small files exercised via a tiny test
  harness that links against `libwcc.a`.

Exit criteria:

- A `lextool` test binary dumps a representative sample's token
  stream deterministically.
- All lex tests green.

## Phase 2 — Parser & AST (in `libwcc.a`)

Goal: parse the full grammar into an AST. No types yet.

Deliverables:

- `cmd/wcc/parse.c` — recursive descent. Pratt expression parser for
  precedence. No yacc.
- `cmd/wcc/ast.c` — `Node` constructor helpers; printer.
- Grammar coverage:
  - `use` imports (paths use `.` not `::`)
  - `type` declarations (struct, alias, fn type) with trailing `=`
  - `def` constants
  - `let` declarations (top-level and local)
  - `fn` definitions with `= { ... };` body. No methods; receivers
    are just first arguments.
  - `export` visibility marker
  - all expressions, statements, control flow
  - `defer`
  - body-less `fn` declarations and `@symbol("name")` attribute for
    FFI imports
- `test/wcc/200_parse.c` — table-driven AST shape tests.
- `test/lang/parse/*.ww` — programs that should parse but not yet
  typecheck; harness only checks parser exits 0.

Exit criteria:

- 50+ parse tests green, drawing shapes from
  `ref/hare/cmd/hare/main.ha`.
- AST printer output is deterministic.

## Phase 3 — Type checker (in `libwcc.a`)

Goal: resolve names, infer/check types, produce a typed AST.

Deliverables:

- `cmd/wcc/sym.c` — symbol table. Lexical scopes. Plan 9 style hash.
- `cmd/wcc/type.c` — type representation, equality, conversion rules.
- `cmd/wcc/check.c` — type checking pass over the AST. Errors carry
  source locations.
- Built-in types: all integer widths, `bool`, `f32`, `f64`, `rune`,
  `str`, `void`, pointers, slices `[]T`, fixed arrays `[N]T`,
  function types.
- Slice semantics: `{ ptr, len, cap }`, indexing bounds-checked at
  runtime (debug builds; release may elide via flag).
- `test/wcc/300_check.c` — table-driven: src → expected error or OK.
- `test/lang/check/*.ww` — both passing and failing cases.

Exit criteria:

- All built-in types correctly checked.
- Negative tests produce stable, useful diagnostics.

## Phase 4 — `w6c` amd64 compiler

Goal: turn typed AST into Plan 9-style amd64 assembly text. The
binary `w6c` is a Plan 9 cc analogue for amd64. There is no separate
SSA IR file; the only on-disk intermediate is `.s`.

Deliverables:

- `cmd/w6c/` (links against `libwcc.a`):
  - `6.out.h`  — amd64 opcode enum, register names, addressing
    modes. Mirror `ref/plan9front/sys/src/cmd/w6c/` shape.
  - `gc.h`     — `Prog`, `Adr`, scratch regs, stack layout.
  - `cgen.c`   — typed AST → `Prog` list (walking, not SSA).
  - `txt.c`    — `Prog` list → textual Plan 9 amd64 asm.
  - `swt.c`, `peep.c`, `reg.c` — switch lowering, peephole pass,
    naive register allocation (linear scan or spill-everywhere
    first; tighten later).
  - `mkfile`   — Plan 9 mkfile next to the Makefile entries.
- Output suffix `.s`. Plan 9 amd64 asm syntax (not GAS).
- `test/lang/w6c/*.ww` — golden tests: src → expected `.s` text.

Exit criteria:

- `w6c hello.ww` produces `hello.s`.
- 20+ programs round-trip identically across runs.
- The output is consumable by phase 5's `w6a`.

## Phase 5 — `w6a` amd64 assembler

Goal: assemble Plan 9 amd64 asm into ELF object files. ELF (not
Plan 9 a.out) so we can interop with C archives in phase 8.

Deliverables:

- `cmd/w6a/`:
  - `lex.c`    — tokenize Plan 9 asm.
  - `parse.c`  — hand-rolled grammar (Plan 9 uses yacc; we go
    hand-rolled to keep rule #6 honest).
  - `asm.c`    — encode amd64 instructions (REX, ModR/M, SIB).
  - `obj.c`    — emit ELF64 relocatable objects.
  - `mkfile`
- Pseudoregisters supported: `SP`, `FP`, `SB` (static base) per
  Plan 9 convention, on top of `AX`, `BX`, ..., `R8`-`R15`.
- `test/wcc/500_asm.c` — round-trip known asm to known bytes.

Exit criteria:

- `w6a hello.s -o hello.o` produces a valid ELF amd64 object.
- `objdump -d hello.o` matches expectations across a 20-program
  corpus.

## Phase 6 — `w6l` amd64 linker

Goal: link ELF objects into a static ELF executable.

Deliverables:

- `cmd/w6l/`:
  - `obj.c`    — load ELF `.o` files and `.a` archives.
  - `sym.c`    — global symbol table; resolution.
  - `pass.c`   — section layout, relocation application.
  - `out.c`    — emit static ELF executable.
  - `mkfile`
- Static linking only. No dynamic loader. No PT_INTERP segment.
- `lib/libwwrt.a` (built later in phase 7) is auto-included.

Exit criteria:

- `w6l -o hello hello.o libwwrt.a` produces a static ELF binary.
- `ldd hello` reports "not a dynamic executable".
- The binary exits 0 with the program's intended semantics.

## Phase 7 — Runtime + driver wiring

Goal: a tiny static runtime, plus the `ww` driver wired to call
`w6c → w6a → w6l` end to end.

Deliverables:

- `rt/start.s`     — entry, sets up argc/argv, calls `main`.
- `rt/syscall.s`   — Linux syscall trampoline.
- `rt/panic.c`     — `panic`, `abort`, stack unwind for `defer`.
- `rt/mem.c`       — small page allocator built on `mmap`. Users get
  raw pages; higher-level allocators ship in `lib/`.
- `rt/slice.c`     — slice helpers used by codegen
  (`bounds_check`, `slice_grow` for explicit user calls).
- Build artifact: `out/lib/libwwrt.a`.
- `cmd/ww/` driver: reads `ww build foo.ww`, runs `w6c` then `w6a`
  then `w6l`, links `libwwrt.a` into the output.

Exit criteria:

- A ww program with no libc dependency runs.
- Adding `use os` pulls in syscall wrappers; still static.

## Phase 8 — C FFI + static linking C libs

Goal: bind to C libraries; produce static binaries containing them.

Deliverables:

- Body-less `fn` declarations + `@symbol("name")` attribute parser
  and checker rules already in phases 2/3 — finalize codegen.
- amd64 SysV ABI: formalize struct-by-value and varargs in `w6c`.
- `w6l` learns to slurp static archives from a search path.
- `lib/c/libc/` — minimal libc bindings (`malloc`, `free`, `printf`,
  `read`, `write`, `open`, `close`).
- `lib/c/tls/`  — bindings to libtls or BearSSL (decide; BearSSL is
  more aligned with our static-linking story).
- `lib/c/crypto/` — bindings to libcrypto subset (sha256, aes, hmac).
- `lib/c/curses/` — ncurses bindings (initscr, getch, mvprintw, ...).
- `ww` driver: `ww build -lcrypto -ltls -lncurses` resolves these
  via a `pkg-config`-style probe, hands `.a` paths to `w6l`.

Exit criteria:

- `examples/tlsclient.ww` connects to https://example.org and
  prints the response. Statically linked. No `.so` deps in `ldd`.
- `examples/menu.ww` runs an ncurses TUI.

## Phase 9 — Standard library (Hare-shaped)

Goal: a usable stdlib. Each module ships with tests.

Order of build (each is its own milestone):

1. `types`     limits, int helpers
2. `bytes`     slice ops on `[]u8`
3. `strings`   ops on `str`
4. `io`        `stream` struct (function-pointer vtable, no interface);
               `eof` sentinel value
5. `bufio`     buffered reader/writer (Plan 9 `bio` analogue)
6. `fmt`       printf-family writing through `io.stream`
7. `os`        argv, env, stdin/out/err, file ops
8. `errors`    `error = str` and sentinel constants
9. `strconv`   itoa, atoi, parse float
10. `sort`     `sort.slice`, binary search
11. `path`     filepath ops
12. `encoding/utf8`, `encoding/hex`, `encoding/base64`
13. `hash/crc32`, `hash/fnv`, `hash/sha256` (pure ww)
14. `time`     monotonic, wall clock, sleep
15. `net`      dial/listen TCP, UDP

Each module follows the Hare layout: one directory, multiple files,
`+linux.ww`, `+test.ww` overlays where useful.

Exit criteria per module:

- Module compiles standalone with `ww build ./lib/<mod>`.
- Module tests pass: `ww test ./lib/<mod>`.
- Public API documented in `README` inside the module dir.

## Phase 10 — Self-host

Goal: rewrite `libwcc`, `w6c`, `w6a`, `w6l`, and `ww` in ww. Drop the
C bootstrap.

Steps:

1. Translate `ww.h` and `Node`/`Sym`/`Type`/`Prog`/`Adr` to ww
   structs.
2. Port `lex.c` → `lex.ww`. Diff token streams against C output.
3. Port `parse.c` → `parse.ww`. Diff ASTs.
4. Port `check.c`. Diff typed ASTs.
5. Port `w6c` (cgen, txt, peep, reg, swt). Diff `.s` output byte
   for byte.
6. Port `w6a` and `w6l`. Diff resulting `.o` and final binaries.
7. Three-stage bootstrap:
   `Cstage (gcc-built tools) → ww1 → ww2 → ww3`, with
   `cmp ww2 ww3` OK for each tool.

Exit criteria:

- `make bootstrap` runs the three-stage build green.
- The C `cmd/wcc/`, `cmd/w6c/`, `cmd/w6a/`, `cmd/w6l/` C trees are
  deleted in this phase's final commit.

### Status (2026-05)

Steps 1–6 done. The five ww-side tools live in `selfhost/cmd/`:

- `wwc/` — ww-native lex/parse/check/cgen (the frontend library)
- `w6c/`  — ww-native compiler binary; thin driver over wwc's cgen
- `w6a/`  — ww-native amd64 assembler
- `w6l/`  — ww-native amd64 linker
- `ww/`  — ww-native driver, shells to `w6c_ww/w6a_ww/w6l_ww`

Step 7 reaches its fixed point: `make bootstrap` produces ww1 → ww2 →
ww3 with `cmp ww2 ww3` byte-identical. Tests 990–994 confirm each
selfhost tool is byte-identical to its Cstage counterpart on the
wwdump-corpus (and `ww_ww build` is byte-identical to `ww build` on
test 993's hello + wwdump corpus, despite using a different
toolchain underneath).

`ww_ww build foo.ww` is now C-free at runtime: the driver invokes
the wwstage tools end-to-end. Cstage is still required at first-
checkout time (no checked-in stage-0 binary yet). ET_DYN dynamic
linking is also fully ported to the ww side (test 996 confirms
`w6l_ww -L ... -l ...` is byte-identical to `w6l` on snake); the one
remaining gap is that `ww_ww build`'s argv parser does not yet
forward `-L`/`-l` to the linker — `w6l_ww` accepts them when invoked
directly, just not through the ww-side driver. Snake's Makefile
still drives the C `ww` for that reason; switching it to `w6l_ww` +
explicit args works today.

Exit criterion 1 (bootstrap green) is met. Exit criterion 2 (delete
the C trees) is **deferred to v1.0**: removing `cmd/wcc/`, `cmd/w6c/`,
`cmd/w6a/`, `cmd/w6l/`, `cmd/ww/` is a one-way door. Until the compiler
stops churning we keep Cstage as the canonical fresh-checkout entry
point and treat `selfhost/cmd/*` as the path forward. At v1.0, the
plan is to ship a checked-in stage-0 binary archive
(`bootstrap/<arch>/{ww,w6c,w6a,w6l}`), delete the C trees, and promote
`selfhost/cmd/*` to `cmd/*`.

See `BOOTSTRAP.md` for the build flow and `make cstage`/`make wwstage`
targets.

## Phase 11 — CSP

Goal: channels and lightweight processes, without GC.

Deliverables:

- Runtime scheduler: cooperative, M:N, with stack-per-proc (start
  with fixed-size stacks; segmented stacks are a research item).
- `proc f(args)` statement: spawns a process. Fire-and-forget by
  default. To keep a handle, use the stdlib spawner
  (`sched.spawn(f, args)` returns a `procval`). No GC means proc
  lifetimes must be explicit.
- `chan T` type. `c <- v` send; `let v = <-c` receive;
  `select { ... }`.
- Channel ownership: closing a channel is a single-writer
  responsibility (Go rule). Buffered and unbuffered both supported.
- `sync` package: `mutex`, `waitgroup`, `once`.

Concurrency-safety constraints (because no GC):

- Sending a pointer over a channel transfers ownership unless the
  type is explicitly marked `shared`. The checker enforces this.
- No data races by construction in the safe subset; `unsafe` pkg
  exists for power users.

Exit criteria:

- Classic CSP examples (ping-pong, prime sieve, fan-in/fan-out)
  compile and run.
- Race detector under `-race` (later sub-phase) catches a planted
  race in tests.

## Test cadence

Every phase has its own test directory (`test/wcc/<phase>_*.c`,
`test/lang/<phase-area>/*.ww`). `make test` runs them all in order
of phase. CI is just `make test` in a clean tree.

## Versioning checkpoints

Tag the tree at each phase boundary:

- `v0.0` end of phase 0
- `v0.1` end of phase 1
- ...
- `v0.10` end of phase 10 (self-hosted)
- `v1.0` end of phase 11 (CSP merged, ABI declared stable enough
  to start caring)

## Risks and mitigations

- **`Prog`/`Adr` design churn.** Mitigate by reading Plan 9 cc and
  per-target compilers before we write our own. Don't invent until
  you've copied.
- **Writing a linker is hard.** ELF is documented, static linking
  is the small subset. Read `ref/plan9front/sys/src/cmd/8l/` and
  the ELF spec before writing `w6l`. Start with hello-world and grow.
- **Static linking against C libs.** OpenSSL is a pain to build
  static; BearSSL or LibreTLS are more pleasant. Pick early.
- **Self-host bootstrap divergence.** Diff outputs at every stage,
  not just the final binary. Token streams, ASTs, and `.s` text
  must match.
- **CSP without GC.** Channel-of-pointer ownership rules are the
  hard part. Prototype the checker with `unsafe`-allowed first;
  tighten later.

## Non-goals (restated)

- No package manager.
- No generics, no interfaces, no tagged unions, no closures, no
  lambdas.
- No async/await.
- No `map`.
- No GC.
- No LLVM, no QBE, no external compiler infrastructure. The
  toolchain is `cc/8c/8a/8l`-shaped (Plan 9), per target.