ww/examples/lisp/CLAUDE.md

examples/lisp — tiny Lisp interpreter in pure ww. Demo program; not a
production interpreter. Treat the files below as a worked example of
"what does and doesn't lower cleanly through the wwstage cgen today",
not as a reference Lisp implementation.

## Layout

- `lispcore.ww`     interpreter module: types, lexer, parser, env,
                    eval, apply, printer, REPL. Everything the entry
                    point and the test driver consume is `export`-ed.
- `lisp.ww`         entry point; `use lispcore;` + `main()`.
- `lisp_test.ww`    in-process test driver (72 probes). Built as a
                    standalone binary, exec'd directly — `ww test`
                    drops `-I` in single-file mode, so the Makefile
                    runs the binary itself.
- `test_*.lisp`     demo source files (`cat test_X.lisp | ./lisp`).

The lispcore.ww + lisp.ww split exists so the tests can `use` the
interpreter functions. Both files combine under the same module name
(the directory's basename, `lisp`), because `ww`'s module resolver
takes the *containing-directory* basename when finding a sibling.
Cross-module type prefixes like `lispcore.value` don't resolve in the
test — use the bare names.

## wwstage cgen workarounds at play

Every workaround below is documented inline at its use site. The
shape of the bug is what matters, not the specific symptom — the
same bug class shows up in any new code that hits the same pattern.

1. **Top-level `[N]T` arrays mis-address inside functions.**
   `globalarr[i] = …` lowers to `LEAQ (BP), BX` instead of
   `LEAQ globalarr(SB), BX` — the global address gets treated as a
   stack frame, corrupting both. Heap-allocate via `os.alloc` into a
   `*T` and alias to a local at the top of each function before
   indexing. See `sym_off` / `sym_blob` / `obuf`.

2. **`global_ptr[i]` mis-addresses too.** Even after switching to a
   pointer global, `MOVQ global_ptr(SB), BX` is replaced with
   `MOVQ (BP), BX`. Alias `let p = global_ptr;` at function entry,
   then `p[i]`. See every helper that touches the symbol interner.

3. **Two-level field write through a non-pointer sub-struct.**
   `L.cur.kind = k` or `L.src.ptr = buf.ptr` (where `cur`/`src` is a
   non-pointer struct field of a struct reached through a pointer) is
   silently dropped — the function body emits no store. Either
   flatten the sub-struct (see `lexer.curkind`/`curival`/…) or build
   the whole sub-struct as a local and do a single whole-struct
   assign (`let s: str = …; L.src = s;`).

4. **f64 through every boundary is unreliable.** The cgen routes f64
   stores via integer registers; in most paths AX gets stored where
   X0 should have been written.
   - `alloc(value{ fval = v })`: writes the kind enum (AX still
     holds it) instead of the f64.
   - `p.fval = v` through a `*T`: same — stores AX.
   - `*p = v` for `*f64`: stores AX.
   - `func(f64_arg)` where the source is a struct field: cgen does
     `MOVQ off(BX), AX` (integer load) and never puts the bits in X0.

   What does work: `MOVSD X0, global(SB)` (a top-level f64 global)
   and `MOVSD X0, off(BP)` (a local f64 slot). The interpreter
   threads f64 through a scratch global (`fbuf`) and byte-copies
   8 bytes wherever a `*f64` would normally suffice. See `vfloat`,
   `to_f64`, `copybytes`.

5. **`alloc(value{ text = s })` writes only `s.ptr`.** The cgen sets
   up `s.ptr` in AX, sets the new-pointer in BX, then needs `s.len`
   in another reg — but the same BX gets clobbered by the new-pointer
   reload, so the `.len` store never happens. Manual init via
   `p.text = s` writes both halves correctly.

6. **`(slice | E)` tagged-union returns drop `slice.len`.** The
   wwstage return convention is AX=tag, DX=payload1, CX=payload2. A
   slice header is 24 bytes (ptr/len/cap); only ptr and cap come
   through in DX/CX. The intermediate `BX = s.len` is loaded but
   never moved into a return register. Inline the slice-building
   loop into the caller instead of factoring it into a helper. See
   `eval`'s argument-eval inline.

7. **`xs[i].kind` drops the trailing field load.** Field access on a
   slice element gives back only the bytes at `&xs[i]` — the cgen
   doesn't chain the dereference. Bind to a local first:
   `let p = xs[i]; if (p.kind …)`. See every builtin.

8. **f64 compound assigns are mis-lowered to `acc = d` (no OP).**
   `acc += f` / `acc /= f` etc. on f64 locals drop the operator.
   Write the explicit form: `acc = acc + f` / `acc = acc / f`.
   Integer compound assigns work fine, so `acc += i` on `i64`
   stays as-is.

9. **`let r = call(); foreign_call(); return r?;` corrupts `r` when
   the call returned a tagged union.** The (`tag`, `payload1`,
   `payload2`) triple sits in AX/DX/CX after the call, and the
   foreign call between capture and `?`-unwrap clobbers at least one
   register before the cgen has spilled it to the local slot.
   Symptom: a `(*value | rterror)` whose `rterror` carries a string
   literal prints with a `str.len` of tens of thousands. Workaround:
   `match` the union inline before the foreign call and let each arm
   return its own typed result. See eval's BUILTIN apply path
   (where we free the args slice after `apply_builtin`).

If a new function "should work but acts weird", the bug is almost
always one of the above and shows up under valgrind/gdb the same
way it did the first time: silently dropped store, missing field
read, garbage payload after a tagged-union return.

## Tail-call optimization

`eval` is a single `for(true)` trampoline; a tail position rewrites
`v` (current expr) and `cure` (current env) in place and `continue`s
instead of recursing. Tail positions are:

- the chosen branch of `if`
- the last form of `begin` / `let` / a lambda body
- a direct call in any of the above

A `tailed` flag flips on the first jump into a fresh scope (LET
binding, lambda body) so subsequent `define`/`set!` mutate the local
`cure` chain rather than the caller's `ein` slot — same scoping the
old recursive `run_body(_, &pe)` path gave.

Mutual recursion still doesn't work — that's a `define` tie-back
limitation, not a TCO one. Defining `evn?` before `od?` captures an
env where `od?` is unbound; the tie-back only adds the self-binding.

## Interpreter limitations (design, not bug)

- **Arena, not GC.** Cells live in two bump-pointer arenas: `perm`
  (top-level definitions and the value graph each one pins) and
  `trans` (parse cells, intermediate evals, the form's printed
  result). `repl()` calls `arena_reset_trans()` between top-level
  forms; a Cheney-style `promote_v` deep-copies the value graph at
  every top-level `define`/`set!` boundary so no perm cell ever
  points into trans. Forwarding markers (`pin = -1` plus the new
  perm ptr stashed in `.car`/`.val`) break cycles. Detached trans
  chunks go onto a per-arena free list, so peak virtual address
  space is bounded by the largest form's working set. Builtin
  args-slice headers in eval's apply path are released with
  `os.free` per dispatch — without that, `rt_ensure`'s page-per-
  call allocation dominates the profile. test_huge peaks at ~2.6 MB
  under `--pages-as-heap=yes`, down from ~525 MB pre-arena (~200×).
  No real GC inside a single form, so a pathological one-shot like
  `(fib 25)` would still grow trans linearly until the form returns.
- **No bigints.** `i64` wraps silently on overflow. `(fact 21)`
  rolls over.
- **Float printing is fixed `%.6f`.** `1.0` prints as `1.000000`.
  Strconv has no `ftos` yet.
- **String escapes are accepted but not translated.** `"\n"` in
  source lands as the two bytes `\\` + `n`, not a newline.
- **No `(load)` / file I/O builtins.** Programs come in via stdin.

## Latent issues — not observed, but the surface to watch

- Other i64 compound assigns in the cgen aren't audited site-by-site.
  Workaround #8 covers f64; if you add new arithmetic on integers
  inside a hot loop, scan the asm.
- The interactive REPL's `shift_buf(L.curstart)` assumes `parse_expr`
  always primes exactly one token of lookahead after returning success.
  A future parser change that skips that prime silently eats input.
- The `define` tie-back prepends a self-binding frame onto
  `bv.envp`. It assumes constructors always initialize `envp`. They
  do today.
- Cross-module type names (`lispcore.value`) don't resolve; we
  side-step by using bare names. If `ww`'s module resolver starts
  honoring the qualified form, the tests still work — but if it
  starts *rejecting* unqualified cross-module references, both
  files break.

## Build / test / profile

```
make            # build ./lisp
make test       # build + run lisp_test (72 in-process probes)
make demo       # cat each test_*.lisp through ./lisp
make clean
```

Demo files are pipe-driven: `cat test_arith.lisp | ./lisp`.

Valgrind:

```
valgrind --error-exitcode=2 ./lisp < test_huge.lisp
valgrind --tool=massif --pages-as-heap=yes ./lisp < test_huge.lisp
```

The memcheck leak counter reads "0 allocs / 0 frees" because
`rt_alloc` is `mmap`, not `malloc`. Memcheck still catches uninit
reads, bad accesses, and stack issues. For real heap-shape analysis
use massif with `--pages-as-heap=yes`.