first commit

PLAN.md

@@ -0,0 +1,369 @@
+# PLAN
+
+A phased plan for `ww`. Each phase ends with a green `make test` and
+a tagged checkpoint. Don't start phase N+1 until phase N is green.
+
+CLAUDE.md is the rulebook. This file is the schedule.
+
+## Phase 0 — Skeleton (C bootstrap scaffold)
+
+Goal: a repo that builds, has a Makefile, and has a `ww` driver
+that prints its version. No compilation yet.
+
+Deliverables:
+
+- `Makefile` (POSIX) with `all`, `test`, `clean`, `install` targets.
+- `cmd/wwc/` C library skeleton (builds `libwwc.a`):
+  - `ww.h`     — central typedefs (`Node`, `Sym`, `Type`, `Lex`)
+  - `mem.c`    — bump arena allocator (no `malloc` in hot paths)
+  - `err.c`    — `errorf`, `fatal`, `warn`
+  - (lex/parse/check stubs added in later phases)
+- `cmd/ww/` driver skeleton: argv parsing, version print.
+- `test/run` — shell test harness.
+- `test/wwc/000_smoke.c` — smoke test that asserts `ww -V` prints
+  the version.
+
+Per-target binaries (`6c`, `6a`, `6l`) are NOT created here. They
+ship in their own phases (4, 5, 6).
+
+Exit criteria:
+
+- `make` produces `out/bin/ww` and `out/lib/libwwc.a`.
+- `make test` runs the smoke test and passes.
+
+## Phase 1 — Lexer (in `libwwc.a`)
+
+Goal: tokenize ww source into a stream of `Tok` structs.
+
+Deliverables:
+
+- `cmd/wwc/lex.c` — hand-rolled DFA. Handles:
+  - identifiers, keywords (`fn`, `let`, `def`, `if`, `else`, `for`,
+    `switch`, `case`, `return`, `use`, `type`, `struct`, `defer`,
+    `break`, `continue`, `export`, `proc`, `chan`, `nil`, `true`,
+    `false`)
+  - integer literals (dec, hex, oct, bin) with suffixes (`u8`, `i32`...)
+  - float literals
+  - rune literals `'x'` and string literals `"..."` (UTF-8)
+  - operators and punctuation, including `@` for attributes
+    (`@symbol("name")`, etc.)
+  - explicit `;` terminators — no automatic insertion (Hare rule)
+  - `//` and `/* */` comments
+- `cmd/wwc/tok.c` — token names, debug printer.
+- `test/wwc/100_lex.c` — table-driven: input → token sequence.
+- `test/lang/lex/*.ww` — small files exercised via a tiny test
+  harness that links against `libwwc.a`.
+
+Exit criteria:
+
+- A `lextool` test binary dumps a representative sample's token
+  stream deterministically.
+- All lex tests green.
+
+## Phase 2 — Parser & AST (in `libwwc.a`)
+
+Goal: parse the full grammar into an AST. No types yet.
+
+Deliverables:
+
+- `cmd/wwc/parse.c` — recursive descent. Pratt expression parser for
+  precedence. No yacc.
+- `cmd/wwc/ast.c` — `Node` constructor helpers; printer.
+- Grammar coverage:
+  - `use` imports (paths use `.` not `::`)
+  - `type` declarations (struct, alias, fn type) with trailing `=`
+  - `def` constants
+  - `let` declarations (top-level and local)
+  - `fn` definitions with `= { ... };` body. No methods; receivers
+    are just first arguments.
+  - `export` visibility marker
+  - all expressions, statements, control flow
+  - `defer`
+  - body-less `fn` declarations and `@symbol("name")` attribute for
+    FFI imports
+- `test/wwc/200_parse.c` — table-driven AST shape tests.
+- `test/lang/parse/*.ww` — programs that should parse but not yet
+  typecheck; harness only checks parser exits 0.
+
+Exit criteria:
+
+- 50+ parse tests green, drawing shapes from
+  `ref/hare/cmd/hare/main.ha`.
+- AST printer output is deterministic.
+
+## Phase 3 — Type checker (in `libwwc.a`)
+
+Goal: resolve names, infer/check types, produce a typed AST.
+
+Deliverables:
+
+- `cmd/wwc/sym.c` — symbol table. Lexical scopes. Plan 9 style hash.
+- `cmd/wwc/type.c` — type representation, equality, conversion rules.
+- `cmd/wwc/check.c` — type checking pass over the AST. Errors carry
+  source locations.
+- Built-in types: all integer widths, `bool`, `f32`, `f64`, `rune`,
+  `str`, `void`, pointers, slices `[]T`, fixed arrays `[N]T`,
+  function types.
+- Slice semantics: `{ ptr, len, cap }`, indexing bounds-checked at
+  runtime (debug builds; release may elide via flag).
+- `test/wwc/300_check.c` — table-driven: src → expected error or OK.
+- `test/lang/check/*.ww` — both passing and failing cases.
+
+Exit criteria:
+
+- All built-in types correctly checked.
+- Negative tests produce stable, useful diagnostics.
+
+## Phase 4 — `6c` amd64 compiler
+
+Goal: turn typed AST into Plan 9-style amd64 assembly text. The
+binary `6c` is a Plan 9 cc analogue for amd64. There is no separate
+SSA IR file; the only on-disk intermediate is `.s`.
+
+Deliverables:
+
+- `cmd/6c/` (links against `libwwc.a`):
+  - `6.out.h`  — amd64 opcode enum, register names, addressing
+    modes. Mirror `ref/plan9front/sys/src/cmd/6c/` shape.
+  - `gc.h`     — `Prog`, `Adr`, scratch regs, stack layout.
+  - `cgen.c`   — typed AST → `Prog` list (walking, not SSA).
+  - `txt.c`    — `Prog` list → textual Plan 9 amd64 asm.
+  - `swt.c`, `peep.c`, `reg.c` — switch lowering, peephole pass,
+    naive register allocation (linear scan or spill-everywhere
+    first; tighten later).
+  - `mkfile`   — Plan 9 mkfile next to the Makefile entries.
+- Output suffix `.s`. Plan 9 amd64 asm syntax (not GAS).
+- `test/lang/6c/*.ww` — golden tests: src → expected `.s` text.
+
+Exit criteria:
+
+- `6c hello.ww` produces `hello.s`.
+- 20+ programs round-trip identically across runs.
+- The output is consumable by phase 5's `6a`.
+
+## Phase 5 — `6a` amd64 assembler
+
+Goal: assemble Plan 9 amd64 asm into ELF object files. ELF (not
+Plan 9 a.out) so we can interop with C archives in phase 8.
+
+Deliverables:
+
+- `cmd/6a/`:
+  - `lex.c`    — tokenize Plan 9 asm.
+  - `parse.c`  — hand-rolled grammar (Plan 9 uses yacc; we go
+    hand-rolled to keep rule #6 honest).
+  - `asm.c`    — encode amd64 instructions (REX, ModR/M, SIB).
+  - `obj.c`    — emit ELF64 relocatable objects.
+  - `mkfile`
+- Pseudoregisters supported: `SP`, `FP`, `SB` (static base) per
+  Plan 9 convention, on top of `AX`, `BX`, ..., `R8`-`R15`.
+- `test/wwc/500_asm.c` — round-trip known asm to known bytes.
+
+Exit criteria:
+
+- `6a hello.s -o hello.o` produces a valid ELF amd64 object.
+- `objdump -d hello.o` matches expectations across a 20-program
+  corpus.
+
+## Phase 6 — `6l` amd64 linker
+
+Goal: link ELF objects into a static ELF executable.
+
+Deliverables:
+
+- `cmd/6l/`:
+  - `obj.c`    — load ELF `.o` files and `.a` archives.
+  - `sym.c`    — global symbol table; resolution.
+  - `pass.c`   — section layout, relocation application.
+  - `out.c`    — emit static ELF executable.
+  - `mkfile`
+- Static linking only. No dynamic loader. No PT_INTERP segment.
+- `lib/libwwrt.a` (built later in phase 7) is auto-included.
+
+Exit criteria:
+
+- `6l -o hello hello.o libwwrt.a` produces a static ELF binary.
+- `ldd hello` reports "not a dynamic executable".
+- The binary exits 0 with the program's intended semantics.
+
+## Phase 7 — Runtime + driver wiring
+
+Goal: a tiny static runtime, plus the `ww` driver wired to call
+`6c → 6a → 6l` end to end.
+
+Deliverables:
+
+- `rt/start.s`     — entry, sets up argc/argv, calls `main`.
+- `rt/syscall.s`   — Linux syscall trampoline.
+- `rt/panic.c`     — `panic`, `abort`, stack unwind for `defer`.
+- `rt/mem.c`       — small page allocator built on `mmap`. Users get
+  raw pages; higher-level allocators ship in `lib/`.
+- `rt/slice.c`     — slice helpers used by codegen
+  (`bounds_check`, `slice_grow` for explicit user calls).
+- Build artifact: `out/lib/libwwrt.a`.
+- `cmd/ww/` driver: reads `ww build foo.ww`, runs `6c` then `6a`
+  then `6l`, links `libwwrt.a` into the output.
+
+Exit criteria:
+
+- A ww program with no libc dependency runs.
+- Adding `use os` pulls in syscall wrappers; still static.
+
+## Phase 8 — C FFI + static linking C libs
+
+Goal: bind to C libraries; produce static binaries containing them.
+
+Deliverables:
+
+- Body-less `fn` declarations + `@symbol("name")` attribute parser
+  and checker rules already in phases 2/3 — finalize codegen.
+- amd64 SysV ABI: formalize struct-by-value and varargs in `6c`.
+- `6l` learns to slurp static archives from a search path.
+- `lib/c/libc/` — minimal libc bindings (`malloc`, `free`, `printf`,
+  `read`, `write`, `open`, `close`).
+- `lib/c/tls/`  — bindings to libtls or BearSSL (decide; BearSSL is
+  more aligned with our static-linking story).
+- `lib/c/crypto/` — bindings to libcrypto subset (sha256, aes, hmac).
+- `lib/c/curses/` — ncurses bindings (initscr, getch, mvprintw, ...).
+- `ww` driver: `ww build -lcrypto -ltls -lncurses` resolves these
+  via a `pkg-config`-style probe, hands `.a` paths to `6l`.
+
+Exit criteria:
+
+- `examples/tlsclient.ww` connects to https://example.org and
+  prints the response. Statically linked. No `.so` deps in `ldd`.
+- `examples/menu.ww` runs an ncurses TUI.
+
+## Phase 9 — Standard library (Hare-shaped)
+
+Goal: a usable stdlib. Each module ships with tests.
+
+Order of build (each is its own milestone):
+
+1. `types`     limits, int helpers
+2. `bytes`     slice ops on `[]u8`
+3. `strings`   ops on `str`
+4. `io`        `stream` struct (function-pointer vtable, no interface);
+               `eof` sentinel value
+5. `bufio`     buffered reader/writer (Plan 9 `bio` analogue)
+6. `fmt`       printf-family writing through `io.stream`
+7. `os`        argv, env, stdin/out/err, file ops
+8. `errors`    `error = str` and sentinel constants
+9. `strconv`   itoa, atoi, parse float
+10. `sort`     `sort.slice`, binary search
+11. `path`     filepath ops
+12. `encoding/utf8`, `encoding/hex`, `encoding/base64`
+13. `hash/crc32`, `hash/fnv`, `hash/sha256` (pure ww)
+14. `time`     monotonic, wall clock, sleep
+15. `net`      dial/listen TCP, UDP
+
+Each module follows the Hare layout: one directory, multiple files,
+`+linux.ww`, `+test.ww` overlays where useful.
+
+Exit criteria per module:
+
+- Module compiles standalone with `ww build ./lib/<mod>`.
+- Module tests pass: `ww test ./lib/<mod>`.
+- Public API documented in `README` inside the module dir.
+
+## Phase 10 — Self-host
+
+Goal: rewrite `libwwc`, `6c`, `6a`, `6l`, and `ww` in ww. Drop the
+C bootstrap.
+
+Steps:
+
+1. Translate `ww.h` and `Node`/`Sym`/`Type`/`Prog`/`Adr` to ww
+   structs.
+2. Port `lex.c` → `lex.ww`. Diff token streams against C output.
+3. Port `parse.c` → `parse.ww`. Diff ASTs.
+4. Port `check.c`. Diff typed ASTs.
+5. Port `6c` (cgen, txt, peep, reg, swt). Diff `.s` output byte
+   for byte.
+6. Port `6a` and `6l`. Diff resulting `.o` and final binaries.
+7. Three-stage bootstrap:
+   `Cstage (gcc-built tools) → ww1 → ww2 → ww3`, with
+   `cmp ww2 ww3` OK for each tool.
+
+Exit criteria:
+
+- `make bootstrap` runs the three-stage build green.
+- The C `cmd/wwc/`, `cmd/6c/`, `cmd/6a/`, `cmd/6l/` C trees are
+  deleted in this phase's final commit.
+
+## Phase 11 — CSP
+
+Goal: channels and lightweight processes, without GC.
+
+Deliverables:
+
+- Runtime scheduler: cooperative, M:N, with stack-per-proc (start
+  with fixed-size stacks; segmented stacks are a research item).
+- `proc f(args)` statement: spawns a process. Fire-and-forget by
+  default. To keep a handle, use the stdlib spawner
+  (`sched.spawn(f, args)` returns a `procval`). No GC means proc
+  lifetimes must be explicit.
+- `chan T` type. `c <- v` send; `let v = <-c` receive;
+  `select { ... }`.
+- Channel ownership: closing a channel is a single-writer
+  responsibility (Go rule). Buffered and unbuffered both supported.
+- `sync` package: `mutex`, `waitgroup`, `once`.
+
+Concurrency-safety constraints (because no GC):
+
+- Sending a pointer over a channel transfers ownership unless the
+  type is explicitly marked `shared`. The checker enforces this.
+- No data races by construction in the safe subset; `unsafe` pkg
+  exists for power users.
+
+Exit criteria:
+
+- Classic CSP examples (ping-pong, prime sieve, fan-in/fan-out)
+  compile and run.
+- Race detector under `-race` (later sub-phase) catches a planted
+  race in tests.
+
+## Test cadence
+
+Every phase has its own test directory (`test/wwc/<phase>_*.c`,
+`test/lang/<phase-area>/*.ww`). `make test` runs them all in order
+of phase. CI is just `make test` in a clean tree.
+
+## Versioning checkpoints
+
+Tag the tree at each phase boundary:
+
+- `v0.0` end of phase 0
+- `v0.1` end of phase 1
+- ...
+- `v0.10` end of phase 10 (self-hosted)
+- `v1.0` end of phase 11 (CSP merged, ABI declared stable enough
+  to start caring)
+
+## Risks and mitigations
+
+- **`Prog`/`Adr` design churn.** Mitigate by reading Plan 9 cc and
+  per-target compilers before we write our own. Don't invent until
+  you've copied.
+- **Writing a linker is hard.** ELF is documented, static linking
+  is the small subset. Read `ref/plan9front/sys/src/cmd/8l/` and
+  the ELF spec before writing `6l`. Start with hello-world and grow.
+- **Static linking against C libs.** OpenSSL is a pain to build
+  static; BearSSL or LibreTLS are more pleasant. Pick early.
+- **Self-host bootstrap divergence.** Diff outputs at every stage,
+  not just the final binary. Token streams, ASTs, and `.s` text
+  must match.
+- **CSP without GC.** Channel-of-pointer ownership rules are the
+  hard part. Prototype the checker with `unsafe`-allowed first;
+  tighten later.
+
+## Non-goals (restated)
+
+- No package manager.
+- No generics, no interfaces, no tagged unions, no closures, no
+  lambdas.
+- No async/await.
+- No `map`.
+- No GC.
+- No LLVM, no QBE, no external compiler infrastructure. The
+  toolchain is `cc/8c/8a/8l`-shaped (Plan 9), per target.